A developer extended their Ollama fork that clusters two NVIDIA DGX Sparks (128 GB each) over a 200 Gb/s RDMA cable to serve multiple models on the worker machine. The fix replaces the single-client ggml-rpc-server with one RPC server slot per model, since each Ollama model runs its own llama-server and previously fell back silently to the local GPU.
A developer wired llama.cpp's recently merged tensor-parallel-over-RPC (PR #26610, October) into their Ollama fork, letting one Ollama instance drive two DGX Sparks (128 GB each, 200 Gb/s RDMA cable) simultaneously instead of sequentially. gpt-oss:120b hit 58.1 tok/s versus 42.7 on a single node, matching vLLM TP=2 (55.7 tok/s), and models like DeepSeek-V4-Flash (156 GB) now run at all.
A developer wired llama.cpp's recently merged tensor-parallel-over-RPC (PR #26610, October) into their Ollama fork, letting one Ollama instance drive two DGX Sparks (128 GB each, 200 Gb/s RDMA cable) simultaneously instead of sequentially. gpt-oss:120b hit 58.1 tok/s versus 42.7 on a single node, matching vLLM TP=2 (55.7 tok/s), and models like DeepSeek-V4-Flash (156 GB) now run at all.
A user modded a GMKtec EVO-X2 (Strix Halo) mini PC to run two Radeon AI Pro R9700 GPUs through its NVMe slots using PCIe adapters, connected via USB4 and limited to PCIe 4.0 x4 each. llama.cpp benchmarks with a Qwen 27B model show a 32% token-generation speedup over a single card, after fixing tensor parallelism issues in the Vulkan build and enabling RCCL in the ROCm build.
A user modded a GMKtec EVO-X2 (Strix Halo) mini PC to run two Radeon AI Pro R9700 GPUs through its NVMe slots using PCIe adapters, connected via USB4 and limited to PCIe 4.0 x4 each. llama.cpp benchmarks with a Qwen 27B model show a 32% token-generation speedup over a single card, after fixing tensor parallelism issues in the Vulkan build and enabling RCCL in the ROCm build.
A user with a 4x32GB AMD R9700 GPU setup asks whether llama.cpp can spread requests across multiple cards to run Qwen3.8-27B. They note that splitting a single request across cards causes PCIe bottlenecks and no speedup, and they want to run parallel requests instead. The main issue is that multiple llama-server processes cannot share the cache.
A user with a 4x32GB AMD R9700 GPU setup asks whether llama.cpp can spread requests across multiple cards to run Qwen3.8-27B. They note that splitting a single request across cards causes PCIe bottlenecks and no speedup, and they want to run parallel requests instead. The main issue is that multiple llama-server processes cannot share the cache.
A Reddit user patched llama.cpp to run a Qwen3.8 DFlash2 setup with main weights in tensor parallel on two RTX 4060s while offloading draft model layers over RPC to an AMD Strix machine. Mixing vendors normally forces split-mode layer, but the patches keep tensor parallel intact, raising context from 150k to ~195k tokens at no speed cost. The post doubles as a recipe: a known-good commit, two cherry-picked PRs, and a local patch others can rebuild with an agent.
A Reddit user patched llama.cpp to run a Qwen3.8 DFlash2 setup with main weights in tensor parallel on two RTX 4060s while offloading draft model layers over RPC to an AMD Strix machine. Mixing vendors normally forces split-mode layer, but the patches keep tensor parallel intact, raising context from 150k to ~195k tokens at no speed cost. The post doubles as a recipe: a known-good commit, two cherry-picked PRs, and a local patch others can rebuild with an agent.
A community audit of 443 GGUF quants across 25 repos found 64 files whose contents don't match their filename's claimed quant type. The cause: k-quants and i-quants need tensor rows divisible by 256, and when they aren't, llama-quantize silently substitutes a ~4.5 bpw fallback while keeping the low-bit name. Follow-up posts show AtomicChat quants affected and a patched-llama.cpp workaround producing a true 3.07 bpw, 11.77 GiB Nemotron-3.5-Lightning file that runs 262K context on 16 GB.
A community audit of 443 GGUF quants across 25 repos found 64 files whose contents don't match their filename's claimed quant type. The cause: k-quants and i-quants need tensor rows divisible by 256, and when they aren't, llama-quantize silently substitutes a ~4.5 bpw fallback while keeping the low-bit name. Follow-up posts show AtomicChat quants affected and a patched-llama.cpp workaround producing a true 3.07 bpw, 11.77 GiB Nemotron-3.5-Lightning file that runs 262K context on 16 GB.