Releases·Tool Release
SGLang v0.5.19 ships beam search, DeepEP v2, and Qwen3.8 support across 786 PRs
The SGLang project released v0.5.19, a large community-driven update adding support for new models including Qwen3.8 (2.4T-A95B) and Qwen3.8-27B. The release aggregates 786 pull requests from 214 contributors and points users to the project cookbook for the full supported-model list.
0 engagement1 sourceSat, Sep 5, 2026, 02:27 AM
AI Overview
- v0.5.19 merges 786 PRs from 214 contributors, adding Qwen3.8 (2.4T-A95B), Qwen3.8-27B, dots3.note, Ling-3.0-flash/tiny, Spark2.5, and Granite 4.2
- Beam search via beam_width returns n best sequences; unsupported with speculative decoding, disaggregation, DP attention, or HiCache
- DeepEP v2 ElasticBuffer (--moe-a2a-backend deepep_v2) enables CUDA-graph decode across nodes for DeepSeek-V3/V4 and Qwen3-MoE FP8
- LayerNorm sequence parallelism (--enable-layernorm-sp) cuts Qwen3-8B prefill 3.5% on H100 and 5.6% on B200
- W4A8 MoE FP8 activation quantization on Hopper lifts DeepSeek-V4-Flash output throughput ~12% with no GSM8K accuracy change, needing FlashInfer 0.6.18
- Unified radix tree becomes the default cache for all models; supports SWA prefix reuse, runtime L3 attach/detach, and rank-consistent PP HiCache
- Lean attention kernel on MI300X/MI355X delivers up to 1.52x throughput and 3.62x lower inter-token latency on MI355X, on by default
- DSA models on ROCm: GLM-5.2 decode TPOT drops from 23 ms to 8 ms on eight MI355Xs; DeepSeek-V4 top-k kernel up to 3x faster
Entities
Related
Releases·Tool Release
SGLang v0.5.18 adds Muse Glimmer, Intern-S2-Mobius, SANA-Video support
SGLang v0.5.18 is released with support for new models including Muse Glimmer, Intern-S2-Mobius, and SANA-Video. The release includes 710 PRs from 212 contributors, expanding the inference engine's model coverage.
SGLang v0.5.18 is released with support for new models including Muse Glimmer, Intern-S2-Mobius, and SANA-Video. The release includes 710 PRs from 212 contributors, expanding the inference engine's model coverage.
0 engagement1 sourcegithubSat, Aug 22, 2026, 12:09 AM
Releases·Tool Release
SGLang v0.5.21 ships 779 PRs, PD hot-switching, Rust prefix cache, and DeepSeek-V4.1 support
The SGLang project shipped v0.5.21, adding support for new LLM, VLM, and diffusion models including DeepSeek-V4.1 Flash, GigaChat 3.5, and DiffusionGemma. Key features include on-the-fly prefill/decode switching, a Rust-based prefix cache by default, new Decisions and Score APIs, and throughput gains for DeepSeek and Kimi models.
The SGLang project shipped v0.5.21, adding support for new LLM, VLM, and diffusion models including DeepSeek-V4.1 Flash, GigaChat 3.5, and DiffusionGemma. Key features include on-the-fly prefill/decode switching, a Rust-based prefix cache by default, new Decisions and Score APIs, and throughput gains for DeepSeek and Kimi models.
0 engagement1 sourcegithubFri, Oct 2, 2026, 01:09 AM
Releases·Tool Release
llama.cpp adds support for IBM Granite SWA and MoE models via PR #25505
A pull request by gabe-l-hart adds support for IBM's Granite models with sliding window attention (SWA) and mixture-of-experts (MoE) architectures to llama.cpp. This enables local inference of these new Granite variants.
A pull request by gabe-l-hart adds support for IBM's Granite models with sliding window attention (SWA) and mixture-of-experts (MoE) architectures to llama.cpp. This enables local inference of these new Granite variants.
3 engagement1 sourceredditWed, Aug 19, 2026, 10:13 PM
Releases·Tool Release
SGLang v0.5.16 introduces DSpark speculative decoding achieving 383.7 tok/s on DeepSeek-V4-Pro
SGLang v0.5.16 ships DSpark, a confidence-driven speculative decoding algorithm that drafts semi-autoregressively in blocks and sizes verify windows based on draft confidence. On DeepSeek-V4-Pro with TP8 on B300 (batch size 1), it reaches 383.7 tok/s at accept length ~5. The release includes 574 PRs from 169 contributors.
SGLang v0.5.16 ships DSpark, a confidence-driven speculative decoding algorithm that drafts semi-autoregressively in blocks and sizes verify windows based on draft confidence. On DeepSeek-V4-Pro with TP8 on B300 (batch size 1), it reaches 383.7 tok/s at accept length ~5. The release includes 574 PRs from 169 contributors.
0 engagement1 sourcegithubSat, Jul 25, 2026, 12:13 AM
Releases·Model Release
Hugging Face Transformers v5.16.0 adds Qwen4-Exp with GatedResidual, QSA, PLE
Hugging Face released Transformers v5.16.0, adding support for Qwen4-Exp, a hybrid text and multimodal architecture built on Qwen3.5. The model introduces three novel components: GatedResidual (GR), Qwen Sparse Attention (QSA), and Per-Layer Embedding (PLE). These innovations aim to improve efficiency and performance in large-scale models.
Hugging Face released Transformers v5.16.0, adding support for Qwen4-Exp, a hybrid text and multimodal architecture built on Qwen3.5. The model introduces three novel components: GatedResidual (GR), Qwen Sparse Attention (QSA), and Per-Layer Embedding (PLE). These innovations aim to improve efficiency and performance in large-scale models.
0 engagement1 sourcegithubWed, Aug 26, 2026, 12:35 PM