Releases·Tool Release
Volunteer releases Sharp-Spark-X2.5-4B GGUF quant for agentic coding on ≤16GB RAM hardware
A volunteer released Sharp-Spark-X2.5-4B, a custom GGUF quant of Spark-X2.5-4B on Hugging Face, targeting users with small GPUs or under 16GB RAM. The quant combines a revised chat template, a coding/cybersecurity-weighted importance matrix, and non-standard per-tensor bit allocation optimized toward SWE-bench-Live rather than KL-divergence. It aims to bring autonomous agentic coding to hardware that cannot run 9B-dense or 35B-A3B models.
342 engagement1 sourceMon, Sep 21, 2026, 03:52 PM
AI Overview
- Reddit user peculiar-ragdoll released Sharp-Spark-X2.5-4B-GGUF, a quant of Spark-X2.5-4B targeting small-GPU or ≤16GB-RAM machines, runnable on smartphones or older gaming laptops
- The quant fixes Spark-4B's chat template and swaps in a system prompt tuned for agentic coding behavior, token use, and correctness
- A custom importance matrix calibrated on a corpus weighted toward agentic coding and cybersecurity relocates bit precision within tensors
- Sharp-Spark uses a non-standard per-tensor bit allocation optimized with a custom KL-divergence and long-context retrieval tool, favoring SWE-bench-Live over fidelity metrics like KL-divergence
- The creator, a non-profit volunteer, calls the benchmarks limited in scope and positions SWE-bench-Live as a genuinely difficult real-codebase metric
- The release page is on Hugging Face at peculiar-ragdoll/Sharp-Spark-X2.5-4B-GGUF
Related
Social·Community
Community discovers Spark-X2.5-4B and 1.7B small LLMs with native 1M context on Hugging Face
A Reddit user browsing Hugging Face found Spark-X2.5-4B and Spark-X2.5-1.7B from XHToken, small models with their own architecture rather than fine-tunes. The 4B reportedly benchmarks neck and neck with Qwen 3.5 9B, and both claim native 1M context. They do not yet run out of the box on llama.cpp.
A Reddit user browsing Hugging Face found Spark-X2.5-4B and Spark-X2.5-1.7B from XHToken, small models with their own architecture rather than fine-tunes. The 4B reportedly benchmarks neck and neck with Qwen 3.5 9B, and both claim native 1M context. They do not yet run out of the box on llama.cpp.
242 engagement1 sourceredditTue, Sep 1, 2026, 02:35 PM
Releases·Tool Release
Unsloth ships large performance release: Vulkan on AMD, MLX speedups, PyTorch 2.11
Unsloth added local-run support for Qwen3.8-Flash-Next and GLM-5.3-Flash with GGUF weights on Hugging Face, then shipped follow-ups enabling multi-token prediction for up to 2x faster inference and a major performance/bugfix release. The latest release switches AMD to Vulkan by default for a 20% prefill/decoding boost over ROCm and includes 200+ bug fixes. Practitioners gain faster local GGUF inference, one-click Hermes support, and MLX fine-tuning on Apple Silicon.
Unsloth added local-run support for Qwen3.8-Flash-Next and GLM-5.3-Flash with GGUF weights on Hugging Face, then shipped follow-ups enabling multi-token prediction for up to 2x faster inference and a major performance/bugfix release. The latest release switches AMD to Vulkan by default for a 20% prefill/decoding boost over ROCm and includes 200+ bug fixes. Practitioners gain faster local GGUF inference, one-click Hermes support, and MLX fine-tuning on Apple Silicon.
Updated Wed, Sep 9, 2026, 03:18 PM — September 8 release defaults AMD to Vulkan for 20% perf boost, fixes AMD gibberish issues, and adds 200+ bug fixes
1601.8k engagement7 sourcesgithub, xThu, Aug 27, 2026, 01:09 PM
Social·Community
Users run Qwen 3.8 27B locally on consumer GPUs: 75 tok/s on 16GB, agentic coding on 12GB, AMD slowdowns
Local LLM users are sharing configurations for running Qwen 3.8 27B on consumer GPUs. An RTX 5080 setup reaches ~75 tok/s decode using a custom hybrid GGUF quant with multi-token prediction, while a 12GB laptop GPU runs it for agentic coding at 9–11 tok/s with Unsloth Dynamic 3.0 quants. An AMD RX 9060 XT user finds the 27B too slow and switches to Ornith 1.5 9B.
Local LLM users are sharing configurations for running Qwen 3.8 27B on consumer GPUs. An RTX 5080 setup reaches ~75 tok/s decode using a custom hybrid GGUF quant with multi-token prediction, while a 12GB laptop GPU runs it for agentic coding at 9–11 tok/s with Unsloth Dynamic 3.0 quants. An AMD RX 9060 XT user finds the 27B too slow and switches to Ornith 1.5 9B.
Updated Mon, Aug 31, 2026, 09:34 AM — An RTX 5080 user reports ~75 tok/s decode, sometimes 100+ tok/s with good MTP acceptance, on the jrell hybrid quant.
210952 engagement4 sourcesredditThu, Aug 20, 2026, 11:54 PM
Products·building
Local coding agent for small models with context pruning and schema repair
A tool that makes small local LLMs (2B-7B) viable for coding tasks on consumer GPUs by pruning context to prevent KV-cache bloat and repairing tool-call schemas to avoid loops. It targets developers with limited VRAM who want to avoid cloud API costs.
A tool that makes small local LLMs (2B-7B) viable for coding tasks on consumer GPUs by pruning context to prevent KV-cache bloat and repairing tool-call schemas to avoid loops. It targets developers with limited VRAM who want to avoid cloud API costs.
34 engagement1 sourceredditSun, Aug 16, 2026, 04:11 PM
Products·building
Mference: lightweight local reasoning model with web search and tool calling
Mference is a local AI assistant that runs a small, fine-tuned reasoning model (Qwen3-0.6B) with minimal memory usage, using web search and tool calling to compensate for its small knowledge base. It solves the problem of running capable models on edge devices without consuming excessive RAM, targeting users who want privacy and efficiency.
Mference is a local AI assistant that runs a small, fine-tuned reasoning model (Qwen3-0.6B) with minimal memory usage, using web search and tool calling to compensate for its small knowledge base. It solves the problem of running capable models on edge devices without consuming excessive RAM, targeting users who want privacy and efficiency.
11 engagement1 sourceredditMon, Aug 10, 2026, 10:03 AM