Show HN: Janus – Go binary that runs GGUF models via Vulkan on AMD/Intel/Nvidia

Early adopters of Intel's B70 GPUs are encountering problems running llama.cpp across multiple cards. While prefill speed scales with additional GPUs, the output becomes garbled when using the CYSL backend with Intel driver 2026.1. The Vulkan backend shows 30% lower prefill performance and degrades further with more GPUs.

Social·Communityfeatured

Users report DeepSeek-V4-Flash-0731 speeds on varied GPUs, 11–59.5 tok/s

Reddit users are sharing benchmark results for running DeepSeek-V4-Flash-0731 GGUF locally on consumer and workstation GPUs, with speeds ranging from 11 tok/s on a mixed AMD setup to 59.5 tok/s on an RTX PRO 6000 Max-Q eGPU. Quantization, multi-GPU setups, and DSpark drafter support are common themes, with some users noting improved performance from newer llama.cpp builds.

Reddit users are sharing benchmark results for running DeepSeek-V4-Flash-0731 GGUF locally on consumer and workstation GPUs, with speeds ranging from 11 tok/s on a mixed AMD setup to 59.5 tok/s on an RTX PRO 6000 Max-Q eGPU. Quantization, multi-GPU setups, and DSpark drafter support are common themes, with some users noting improved performance from newer llama.cpp builds.

Releases·Tool Release

Transformers now runs llama.cpp quants

Hugging Face announced that transformers can now load and run llama.cpp GGUF quantized checkpoints directly via from_pretrained, reusing ggml's Metal kernels through the kernels library to approach llama.cpp performance. Initial support targets Apple Silicon Macs, starting with the Qwen3.5 architecture. This bridges llama.cpp's local inference foundation with transformers' PyTorch model-definition ecosystem, enabling experimentation, evaluation, fine-tuning, and conversion validation on GGUF checkpoints.

Hugging Face announced that transformers can now load and run llama.cpp GGUF quantized checkpoints directly via from_pretrained, reusing ggml's Metal kernels through the kernels library to approach llama.cpp performance. Initial support targets Apple Silicon Macs, starting with the Qwen3.5 architecture. This bridges llama.cpp's local inference foundation with transformers' PyTorch model-definition ecosystem, enabling experimentation, evaluation, fine-tuning, and conversion validation on GGUF checkpoints.

Releases·Model Release

ISTA-DASLab releases GSQ-RCO GGUFs for Qwen3.8-Flash-Next plus 50% expert-pruned Coder build at ~1.89 bpw

ISTA Deep Algorithms and Systems Lab released GGUF quantizations of the 176.9B-parameter sparse MoE model Qwen3.8-Flash-Next using two new methods, GSQ and RCO. Four quantized builds run from 2.40 to 3.50 bpw, and a Coder build removes half the routed experts, yielding an effective ~1.89 bits per original parameter and a 29.6 GB resident footprint that fits a single 32 GB accelerator. Benchmarks show the 3.50 bpw build matching BF16 on every evaluated benchmark.

ISTA Deep Algorithms and Systems Lab released GGUF quantizations of the 176.9B-parameter sparse MoE model Qwen3.8-Flash-Next using two new methods, GSQ and RCO. Four quantized builds run from 2.40 to 3.50 bpw, and a Coder build removes half the routed experts, yielding an effective ~1.89 bits per original parameter and a 29.6 GB resident footprint that fits a single 32 GB accelerator. Benchmarks show the 3.50 bpw build matching BF16 on every evaluated benchmark.

Releases·Tool Release

Dev open-sources Vison, an MIT-licensed Vulkan desktop app running FLUX.1 Schnell, Z-Image and Wan locally via stable-diffusion.cpp on 6GB GPUs

A developer released Vison, an MIT-licensed Windows desktop app for image and video generation that runs FLUX.1 Schnell, Z-Image and Wan entirely on local GPUs via stable-diffusion.cpp, ggml and Vulkan. It targets low-VRAM hardware (a 6GB laptop GPU) with no CUDA and no cloud service, keeping prompts on-device. Practitioners get a fully local, permissively licensed frontend, though model weights carry their own licenses.

A developer released Vison, an MIT-licensed Windows desktop app for image and video generation that runs FLUX.1 Schnell, Z-Image and Wan entirely on local GPUs via stable-diffusion.cpp, ggml and Vulkan. It targets low-VRAM hardware (a 6GB laptop GPU) with no CUDA and no cloud service, keeping prompts on-device. Practitioners get a fully local, permissively licensed frontend, though model weights carry their own licenses.