Releases·Tool Release
PXA ships as set-and-forget LLM engine for Pascal/Volta GPUs with PXQ codec, auto-tuning, 2.3× prefill gain
The PXA project released v2026.09.07-rc1, an auto-tuned LLM inference engine targeting Pascal and Volta GPUs. It uses a PXQ codec plus per-card tuned kernels and claims 2.3× the long-prompt prefill speed of the next-best Volta engine on that engine's own hardware. The release demonstrates a 177B-class MoE model with 150k context running on four Tesla P100s.
28 engagement1 sourceMon, Sep 7, 2026, 08:18 PM
AI Overview
- PXA v2026.09.07-rc1 auto-tunes per-card for Pascal and Volta GPUs, requiring no manual flags like -b, -ub, -sm, or cache-type
- Benchmarks against llama.cpp, ik_llama.cpp, and 1Cat vLLM claim 2.3× the long-prompt prefill of the next-best Volta engine on that engine's hardware
- Runs a 177B-class MoE model with 150k context on four Tesla P100s
- Prefill performance roughly doubled since the developer's last public release, with zero flags required
Related
Releases·Tool Release
2x Tesla p100s, q6_k quant, Qwen 3.8 27B ~60tps V3.0
A developer released V3.0 of llama.cpp kernel optimizations for Tesla P100 GPUs, lifting Qwen 3.8 27B Q6_K decode from 7-15 tps to 50-60 tps on a two-card $160 setup. The work, largely AI-assisted, was merged upstream on Sept 22 with strict regression testing.
A developer released V3.0 of llama.cpp kernel optimizations for Tesla P100 GPUs, lifting Qwen 3.8 27B Q6_K decode from 7-15 tps to 50-60 tps on a two-card $160 setup. The work, largely AI-assisted, was merged upstream on Sept 22 with strict regression testing.
40 engagement1 sourceredditSun, Sep 27, 2026, 08:42 PM
Releases·Tool Release
Proxima vLLM plugin implements Star-KV, claims 4x throughput
A developer released Proxima, a vLLM plugin implementing the Star-KV paper, claiming it serves 4x more requests without hardware changes. The plugin includes a decode kernel faster than FlashAttention-2 at higher batch sizes, potentially improving inference efficiency.
A developer released Proxima, a vLLM plugin implementing the Star-KV paper, claiming it serves 4x more requests without hardware changes. The plugin includes a decode kernel faster than FlashAttention-2 at higher batch sizes, potentially improving inference efficiency.
4 engagement1 sourcehackernewsTue, Aug 11, 2026, 03:04 PM
Releases·Tool Release
Turing Engine serves 70B models on a single 24GB GPU at 3,064 tok/s
A new inference engine, Turing Engine, claims to serve 70B-parameter models like LLaMA-3.1-70B and Qwen-2.5-72B on a single 24GB GPU, achieving 3,064 tokens per second with 75% KV cache compression. It supports Unsloth checkpoints, potentially enabling high-throughput inference on consumer hardware.
A new inference engine, Turing Engine, claims to serve 70B-parameter models like LLaMA-3.1-70B and Qwen-2.5-72B on a single 24GB GPU, achieving 3,064 tokens per second with 75% KV cache compression. It supports Unsloth checkpoints, potentially enabling high-throughput inference on consumer hardware.
1 engagement1 sourceredditTue, Aug 25, 2026, 10:34 AM
Releases·Tool Release
Truespar open-sources Paddock, a Rust/C++ inference engine with custom CUDA kernels, under MIT/Apache-2.0
The developers of Paddock, a Rust/C++ inference engine with its own CUDA kernels, released the code as open source on schedule, fulfilling an August commitment to ship in September. The public repo is their internal repo, including kernels, and the engine already serves roughly 300B tokens per year in production at their company. Practitioners get a single binary exposing OpenAI- and Anthropic-style APIs that loads GGUF and safetensors weights.
The developers of Paddock, a Rust/C++ inference engine with its own CUDA kernels, released the code as open source on schedule, fulfilling an August commitment to ship in September. The public repo is their internal repo, including kernels, and the engine already serves roughly 300B tokens per year in production at their company. Practitioners get a single binary exposing OpenAI- and Anthropic-style APIs that loads GGUF and safetensors weights.
92 engagement1 sourceredditFri, Sep 4, 2026, 09:15 AM
Social·Community
User runs Qwen3.6:35B UD Q4_K_M at 80 tok/s on single Nvidia P40
A Reddit user reports achieving 80 tok/s burst speed running Unsloth's Qwen3.6:35B UD Q4_K_M quantized model with 100k context on a single Nvidia P40 (24GB) using TheTom's TurboQuant fork of llama.cpp. The setup disables reasoning to avoid thinking loops and omits the vision component to save memory.
A Reddit user reports achieving 80 tok/s burst speed running Unsloth's Qwen3.6:35B UD Q4_K_M quantized model with 100k context on a single Nvidia P40 (24GB) using TheTom's TurboQuant fork of llama.cpp. The setup disables reasoning to avoid thinking loops and omits the vision component to save memory.
148 engagement2 sourcesredditWed, Jul 15, 2026, 11:10 PM