Releases·Model Release
Agnes-AI releases Agnes-3.0-Flash 33B open-weights multimodal model with 262K context and hybrid attention
Agnes-AI published Agnes-3.0-Flash, a 33B open multimodal model with a 262,144-token context window, adjustable reasoning effort, tool calling, and text, image, and video understanding. It uses a hybrid architecture where only 18 of 72 layers run global attention, keeping most state recurrent. Artificial Analysis lists it publicly despite the org labeling similar models proprietary.
298 engagement1 sourceSat, Sep 12, 2026, 08:05 AM
AI Overview
- Agnes-3.0-Flash (33B) pairs text, image, and video understanding with a 262,144-token context window, adjustable reasoning effort, and tool calling.
- Its 72-layer hybrid decoder runs gated delta-rule recurrence in 54 layers and global attention in 18 (3:1), leaving only 18 KV-cache layers.
- Global attention layers use 24 query heads / 4 KV heads (6:1 GQA); delta-rule layers use 16 key / 48 value heads with fp32 recurrent state.
- Each layer combines SwiGLU feed-forward (17,408) with a parallel 2,048 SwiGLU branch; vocab size is 248,320; a 27-layer vision tower uses patch 16.
- Artificial Analysis lists the model as Agnes AI's 'Proprietary model' despite open weights; AA score is 36.
Related
Releases·Model Release
inclusionAI releases Ling-3.0-flash-VL: 124B-parameter sparse MoE vision-language model with 1M-token context
inclusionAI released Ling-3.0-flash-VL on Hugging Face, a vision-language model adding native image and video understanding to Ling-3.0-flash's language and long-context capabilities. It uses a sparse MoE design (124B total, 5.5B active per token) with a 1M-token context window, aimed at visual reasoning and agentic workflows.
inclusionAI released Ling-3.0-flash-VL on Hugging Face, a vision-language model adding native image and video understanding to Ling-3.0-flash's language and long-context capabilities. It uses a sparse MoE design (124B total, 5.5B active per token) with a 1M-token context window, aimed at visual reasoning and agentic workflows.
183 engagement2 sourcesredditTue, Sep 8, 2026, 03:57 PM
Social·Community
Community speculates on Qwen 4 architecture
A Reddit user speculates about the architecture of the upcoming Qwen 4 model, proposing several possible designs including embedding-offloaded linear layers with 51b n-grams, sparse full-attention with dense routing, and a multi-head latent attention hybrid. The post reflects community interest and speculation ahead of an official release.
A Reddit user speculates about the architecture of the upcoming Qwen 4 model, proposing several possible designs including embedding-offloaded linear layers with 51b n-grams, sparse full-attention with dense routing, and a multi-head latent attention hybrid. The post reflects community interest and speculation ahead of an official release.
1.2k engagement2 sourcesredditTue, Aug 25, 2026, 01:15 PM
Releases·Model Releasefeatured
Z.ai releases GLM-5.3-Flash, first open-weight glm5_next model
Z.ai released GLM-5.3-Flash, the first natively multimodal model in the GLM-5 series and the first open-weight release of the glm5_next architecture. The model is positioned as outperforming GLM-5.2 at one-tenth the price while approaching Claude Opus 4.8 on coding and agentic benchmarks. It introduces a hybrid sparse and linear architecture.
Z.ai released GLM-5.3-Flash, the first natively multimodal model in the GLM-5 series and the first open-weight release of the glm5_next architecture. The model is positioned as outperforming GLM-5.2 at one-tenth the price while approaching Claude Opus 4.8 on coding and agentic benchmarks. It introduces a hybrid sparse and linear architecture.
42028.4k engagement6 sourcesreddit, x, githubWed, Aug 26, 2026, 03:17 PM
Products·launched
Interactive attention architecture lab covering MLA, DSA, KDA, Gated DeltaNet and more
A browser-based reference for understanding attention and memory mechanisms in modern LLM architectures. It visualizes trade-offs like memory growth, exactness, and cost reductions, helping researchers and engineers compare designs.
A browser-based reference for understanding attention and memory mechanisms in modern LLM architectures. It visualizes trade-offs like memory growth, exactness, and cost reductions, helping researchers and engineers compare designs.
2 engagement1 sourceredditMon, Aug 3, 2026, 12:24 PM
Developments·Paper
Researchers propose training-free attention-guided switching between explicit and latent thoughts for MLLMs
A new arXiv paper introduces a training-free method for multimodal LLMs that switches between explicit text-based Chain-of-Thought and latent reasoning based on attention signals, aiming to balance computational efficiency and reasoning accuracy. The approach addresses the instability of adapting training-free LLM reasoning to multimodal settings, which stems from token-level entropy conflating perceptual ambiguity.
A new arXiv paper introduces a training-free method for multimodal LLMs that switches between explicit text-based Chain-of-Thought and latent reasoning based on attention signals, aiming to balance computational efficiency and reasoning accuracy. The approach addresses the instability of adapting training-free LLM reasoning to multimodal settings, which stems from token-level entropy conflating perceptual ambiguity.
0 engagement1 sourcearxivTue, Aug 4, 2026, 10:46 AM