Releases·Tool Release
Transformers v5.17.0 adds HYV4 Hy4-Preview, VibeVoice, NeoMME, Fun-ASR-Nano, Kimi Linear, Canary-1B-v2, NeuCodec
Hugging Face released transformers v5.17.0, adding support for the new HYV4 Hy4-Preview model, a 780B-parameter mixture-of-experts LLM that activates 49B parameters per token. The release gives practitioners library-level support for the architecture's 1M-token context window and MLA-based attention.
0 engagement1 sourceWed, Sep 9, 2026, 03:42 PM
AI Overview
- Transformers v5.17.0 adds HYV4 Hy4-Preview, a 780B-parameter MoE (49B active, 256 routed experts, 1M context) combining MLA, DSA, gated MLA with attention sinks, and independent hyper-connections.
- Release adds VibeVoice, a next-token diffusion LLM for multi-speaker long-form speech, and Kimi Linear, Moonshot AI's hybrid architecture with per-channel-gated Kimi Delta Attention.
- v5.17.0 adds NeoMME, H Company's 260M/800M multimodal-native multilingual encoders, Fun-ASR-Nano, Alibaba's 800M ASR model, and NVIDIA's Canary-1B-v2 ASR/AST model.
- Adds NeuCodec, an XCodec2-based neural codec using single-codebook finite scalar quantization at 0.8kbps with no non-commercial data restrictions.
- Breaking change standardizes vision 2D/3D rotary embeddings into a unified RoPE module in modeling_rope_utils.py, requiring custom vision models to migrate.
- Generation improvements cut per-step accelerator synchronization overhead, and fixes address quantized cache, VoxtralRealtime static-cache rejection, FP8 Qwen embeddings, and kernel fallbacks.
Related
Releases·Model Release
Hugging Face Transformers v5.16.0 adds Qwen4-Exp with GatedResidual, QSA, PLE
Hugging Face released Transformers v5.16.0, adding support for Qwen4-Exp, a hybrid text and multimodal architecture built on Qwen3.5. The model introduces three novel components: GatedResidual (GR), Qwen Sparse Attention (QSA), and Per-Layer Embedding (PLE). These innovations aim to improve efficiency and performance in large-scale models.
Hugging Face released Transformers v5.16.0, adding support for Qwen4-Exp, a hybrid text and multimodal architecture built on Qwen3.5. The model introduces three novel components: GatedResidual (GR), Qwen Sparse Attention (QSA), and Per-Layer Embedding (PLE). These innovations aim to improve efficiency and performance in large-scale models.
0 engagement1 sourcegithubWed, Aug 26, 2026, 12:35 PM
Releases·Model Release
Tencent Hy Team releases Hy3, a 295B-parameter MoE model
Tencent's Hy Team released Hy3, a 295-billion-parameter Mixture-of-Experts (MoE) language model with 21B active parameters and a 3.8B MTP layer. The model outperforms similar-sized models and rivals open-source models with 2-5x parameters. It is available on Hugging Face in full (598GB) and FP8 quantized (300GB) versions.
Tencent's Hy Team released Hy3, a 295-billion-parameter Mixture-of-Experts (MoE) language model with 21B active parameters and a 3.8B MTP layer. The model outperforms similar-sized models and rivals open-source models with 2-5x parameters. It is available on Hugging Face in full (598GB) and FP8 quantized (300GB) versions.
0 engagement1 sourcerssMon, Jul 6, 2026, 11:57 PM
Releases·Tool Release
Hugging Face releases transformers v5.13.1 patch to enable vLLM compatibility
Hugging Face released transformers v5.13.1, a patch focused on enabling compatibility with the latest vLLM release. The update includes defensive handling of legacy layer types, fixes for custom code with new linear layer names, and a fix for _LazyAutoMapping.register with string keys.
Hugging Face released transformers v5.13.1, a patch focused on enabling compatibility with the latest vLLM release. The update includes defensive handling of legacy layer types, fixes for custom code with new linear layer names, and a fix for _LazyAutoMapping.register with string keys.
0 engagement1 sourcegithubSat, Jul 11, 2026, 09:15 AM
Social·Community
Community speculates on Qwen 4 architecture
A Reddit user speculates about the architecture of the upcoming Qwen 4 model, proposing several possible designs including embedding-offloaded linear layers with 51b n-grams, sparse full-attention with dense routing, and a multi-head latent attention hybrid. The post reflects community interest and speculation ahead of an official release.
A Reddit user speculates about the architecture of the upcoming Qwen 4 model, proposing several possible designs including embedding-offloaded linear layers with 51b n-grams, sparse full-attention with dense routing, and a multi-head latent attention hybrid. The post reflects community interest and speculation ahead of an official release.
1.2k engagement2 sourcesredditTue, Aug 25, 2026, 01:15 PM
Releases·Model Releasefeatured
Z.ai releases GLM-5.3-Flash, first open-weight glm5_next model
Z.ai released GLM-5.3-Flash, the first natively multimodal model in the GLM-5 series and the first open-weight release of the glm5_next architecture. The model is positioned as outperforming GLM-5.2 at one-tenth the price while approaching Claude Opus 4.8 on coding and agentic benchmarks. It introduces a hybrid sparse and linear architecture.
Z.ai released GLM-5.3-Flash, the first natively multimodal model in the GLM-5 series and the first open-weight release of the glm5_next architecture. The model is positioned as outperforming GLM-5.2 at one-tenth the price while approaching Claude Opus 4.8 on coding and agentic benchmarks. It introduces a hybrid sparse and linear architecture.
42028.4k engagement6 sourcesreddit, x, githubWed, Aug 26, 2026, 03:17 PM