Releases·Tool Release
Quixotic AI launches Jinfer, a JVM-native AI inference engine with no Python, ONNX, or Docker
Quixotic AI released Jinfer, an early-stage AI inference engine for the JVM covering chat, vision, audio, embeddings, reranking, and TTS without Python, ONNX, Docker, or sidecar processes. It is built from native JVM components and targets Java, C, CUDA, HIP, Metal, OpenCL, and Mojo backends. This gives Java developers a self-contained deployment path compatible with Spring AI, LangChain4j, and GraalVM Native Image.
93 engagement2 sourcesTue, Sep 15, 2026, 06:42 AM
AI Overview
- Quixotic AI released Jinfer, a JVM inference engine supporting chat, vision, audio, embeddings, reranking, and TTS with no Python, ONNX, Docker, or sidecar processes
- The stack includes pure-Java toknroll tokenizers, gguf/safetensors support for llama.cpp and HuggingFace formats, jam quantized matmul kernels, and a jota Tensor API targeting CUDA, HIP, Metal, OpenCL, and Mojo
- Jinfer ships Spring AI and LangChain4j integrations and GraalVM Native Image compatibility; the early release is CPU-only, with the author seeking API feedback
Related
Releases·Tool Release
GitHub releases framework-agnostic Copilot SDK for Java with BYOK support
GitHub has released the GitHub Copilot SDK for Java, a framework-agnostic way to drive AI from Java enterprise apps. It removes dependencies on LangChain4j or Spring AI and, with BYOK support, is AI vendor neutral.
GitHub has released the GitHub Copilot SDK for Java, a framework-agnostic way to drive AI from Java enterprise apps. It removes dependencies on LangChain4j or Spring AI and, with BYOK support, is AI vendor neutral.
0 engagement1 sourcerssMon, Aug 10, 2026, 07:30 PM
Releases·Tool Release
Developers launch CPU-first inference server with 4B chat+vision, ASR, TTS
A team built a CPU-first inference server that runs a 4B chat+vision model, ASR, and TTS behind one OpenAI-compatible endpoint. It is free to run, targeting developers who want to avoid GPU costs.
A team built a CPU-first inference server that runs a 4B chat+vision model, ASR, and TTS behind one OpenAI-compatible endpoint. It is free to run, targeting developers who want to avoid GPU costs.
1 engagement1 sourceredditThu, Aug 13, 2026, 01:51 AM
Releases·Tool Release
Libargus: Low-latency local LLM runner via OpenJDK Panama FFM (Java 22)
A developer built Libargus, a low-latency local LLM runner that uses OpenJDK's Project Panama Foreign Function & Memory API (Java 22) to interface directly with llama.cpp, whisper.cpp, and ggml compute graphs, eliminating the REST sidecar. The project achieves zero-allocation on hot paths by allocating memory segments once inside confined Arenas.
A developer built Libargus, a low-latency local LLM runner that uses OpenJDK's Project Panama Foreign Function & Memory API (Java 22) to interface directly with llama.cpp, whisper.cpp, and ggml compute graphs, eliminating the REST sidecar. The project achieves zero-allocation on hot paths by allocating memory segments once inside confined Arenas.
49 engagement2 sourceshackernewsTue, Jul 14, 2026, 02:40 PM
Products·launched
Java AI agent tutorial using LangChain4j
A tutorial and guide for Java developers to build AI agents without Python, using the LangChain4j library. It demonstrates creating a working agent in about 30 minutes, addressing the problem of Python-centric AI resources for Java/Spring Boot developers.
A tutorial and guide for Java developers to build AI agents without Python, using the LangChain4j library. It demonstrates creating a working agent in about 30 minutes, addressing the problem of Python-centric AI resources for Java/Spring Boot developers.
13 engagement1 sourceredditMon, Aug 3, 2026, 10:57 AM
Developments·Paper
MetaInfer: LLM-as-Compiler approach generates inference engines from model knowledge only
MetaInfer, an 'LLM-as-Compiler' system, generates inference engines using only model knowledge, eliminating the need for hand-coded optimization kernels. It addresses the growing complexity of supporting diverse model families, hardware, and quantization schemes in general-purpose frameworks.
MetaInfer, an 'LLM-as-Compiler' system, generates inference engines using only model knowledge, eliminating the need for hand-coded optimization kernels. It addresses the growing complexity of supporting diverse model families, hardware, and quantization schemes in general-purpose frameworks.
0 engagement1 sourcearxivTue, Jul 14, 2026, 03:22 PM