Releases·Tool Release
Do you need some extra memory on your DGX Spark?
A developer released a repo helping DGX Spark users with a spare 10-24 GB GPU move the spec-decode draft model off the Spark onto that GPU. The freed memory can be used for larger context or better quantization quality. It works on a single Spark or a Spark cluster and ships as eugr-vllm-compatible mods.
101 engagement1 sourceTue, Sep 29, 2026, 05:13 AM
AI Overview
- Reddit user ships a repo offloading DGX Spark spec-decode draft models to a spare 10-24 GB GPU via TCP or RDMA, freeing memory for context or better quantization
- Tool ships as eugr-vllm compatible mods supporting single Sparks or Spark clusters
Related
Social·Community
Users report DSpark draft model slow with DeepSeek-V4-Flash on llama-server
Users testing DeepSeek-V4-Flash-0731 with llama-server report that the DSpark draft model delivers only 1-2 tokens per second, far slower than the MTP draft model's 30-40 t/s. They seek advice on configuring speculative decoding correctly, while another user asks how to free OS RAM when serving the model on 2x DGX Spark systems.
Users testing DeepSeek-V4-Flash-0731 with llama-server report that the DSpark draft model delivers only 1-2 tokens per second, far slower than the MTP draft model's 30-40 t/s. They seek advice on configuring speculative decoding correctly, while another user asks how to free OS RAM when serving the model on 2x DGX Spark systems.
030129 engagement3 sourcesredditSat, Aug 8, 2026, 12:00 AM
Social·Community
Qwen 3.8 27B UD-IQ4_XS even faster on 16GB CUDA via hot-swappable speculative decoding
A developer released a fork of Raymond's llama.cpp KV cache streaming project that adds hot-swappable speculative decoding, letting the draft model (MTP or DFlash2) be ejected from VRAM and reloaded as context grows. It runs Qwen 3.8 27B UD-IQ4_XS on a 16GB RTX 5060 Ti, raising token generation speed versus standard llama.cpp host-RAM offloading.
A developer released a fork of Raymond's llama.cpp KV cache streaming project that adds hot-swappable speculative decoding, letting the draft model (MTP or DFlash2) be ejected from VRAM and reloaded as context grows. It runs Qwen 3.8 27B UD-IQ4_XS on a 16GB RTX 5060 Ti, raising token generation speed versus standard llama.cpp host-RAM offloading.
Updated Sun, Sep 13, 2026, 03:50 PM — Fork now supports hot-swappable speculative decoding (MTP/DFlash2) alongside KV streaming on 16GB VRAM
35 engagement1 sourceredditSun, Sep 13, 2026, 03:50 PM
Social·Community
Users run DeepSeek V4 Flash on single GH200 with DSpark UVA offload
A user shares a reference setup for running DeepSeek-V4-Flash-0731 on a single NVIDIA GH200 using DSpark and UVA offload, since the 284B MoE model does not fit in 144GB HBM. The post credits prior work by u/TimAndTimi and u/Reddactor's blog series, and details the hardware and model specs.
A user shares a reference setup for running DeepSeek-V4-Flash-0731 on a single NVIDIA GH200 using DSpark and UVA offload, since the 284B MoE model does not fit in 144GB HBM. The post credits prior work by u/TimAndTimi and u/Reddactor's blog series, and details the hardware and model specs.
14 engagement1 sourceredditMon, Aug 17, 2026, 02:38 PM
Social·Community
Got an old slow low vram GPU laying around? Might be worth it to use for Just Vision mmproj llama.cpp
A Reddit user shares a llama.cpp configuration tip: instead of using --no-mmproj-offload to save VRAM, which is slow, users can dedicate a secondary low-VRAM GPU to the vision projector (mmproj) with --mmdev. This keeps inference speed unaffected while making vision processing roughly an order of magnitude faster, especially useful for agentic coding workflows.
A Reddit user shares a llama.cpp configuration tip: instead of using --no-mmproj-offload to save VRAM, which is slow, users can dedicate a secondary low-VRAM GPU to the vision projector (mmproj) with --mmdev. This keeps inference speed unaffected while making vision processing roughly an order of magnitude faster, especially useful for agentic coding workflows.
42 engagement1 sourceredditSat, Sep 12, 2026, 01:18 AM
Releases·Model Release
Ling-3.0-flash on DGX Spark hits 38.7 tok/s with two flags; 15k-token response sustains 35.68 tok/s
Community tests show Ling-3.0-flash INT4 on a single DGX Spark can be nearly doubled in speed from 20.8 to 38.7 tok/s by disabling eager execution and enabling MTP speculative decoding. A separate run generated 15,128 tokens in one response at a steady 35.68 tok/s, demonstrating sustained throughput for local inference.
Community tests show Ling-3.0-flash INT4 on a single DGX Spark can be nearly doubled in speed from 20.8 to 38.7 tok/s by disabling eager execution and enabling MTP speculative decoding. A separate run generated 15,128 tokens in one response at a steady 35.68 tok/s, demonstrating sustained throughput for local inference.
Updated Thu, Aug 13, 2026, 04:12 PM — A later run on the same setup generated 15,128 tokens in one response at 35.68 tok/s, confirming sustained throughput.
120104 engagement3 sourcesredditSun, Aug 9, 2026, 04:10 PM