Releases·Tool Release
shardr: open-source Go daemon gives models a SHA-256 content-addressed store with BitTorrent v2 sync and OpenAI-compatible llama.cpp serving
A developer released shardr, an Apache-2.0 Go daemon that gives local machines a single SHA-256 content-addressed store for model weights. It pulls models via BitTorrent v2, deduplicates disk copies, and serves them through llama-server with an OpenAI-compatible API. It targets duplicate 40 GB downloads, models vanishing from Hugging Face, and per-tool weight copies.
0 engagement1 sourceFri, Oct 2, 2026, 02:40 PM
AI Overview
- shardr (Apache-2.0) stores all models in one SHA-256-verified content-addressed store, eliminating duplicate weight copies on disk
- The daemon speaks BitTorrent v2, pulling models from peers and seeding held weights back into the swarm
- Its runner launches llama-server with an OpenAI-compatible API and mmaps weights directly from the store — no staging copies
- Runtimes are lockfile-pinned to digest-verified upstream llama.cpp release binaries; updates happen via PRs to the lockfile
- Pulls anchor to Hugging Face checksums for the exact revision via pirateface.co catalog; rescuing deleted models requires explicit --trust-catalog opt-in
- End-to-end support covers macOS arm64 and Linux amd64 with CI-built releases; no UI, Windows, or non-llama-server runtimes yet
Related
Social·Community
Hugging Face's Merve shares llama.cpp conference talk covering prefill vs decode and speculative decoding
Merve Noyan of Hugging Face gave a talk at a developer conference covering llama.cpp and core local inference concepts, including prefill vs decode, memory types, and speculative decoding. She shared the slides publicly on Reddit, inviting reuse with attribution.
Merve Noyan of Hugging Face gave a talk at a developer conference covering llama.cpp and core local inference concepts, including prefill vs decode, memory types, and speculative decoding. She shared the slides publicly on Reddit, inviting reuse with attribution.
44 engagement1 sourceredditTue, Oct 6, 2026, 06:02 PM
Releases·Tool Release
Transformers now runs llama.cpp quants
Hugging Face announced that transformers can now load and run llama.cpp GGUF quantized checkpoints directly via from_pretrained, reusing ggml's Metal kernels through the kernels library to approach llama.cpp performance. Initial support targets Apple Silicon Macs, starting with the Qwen3.5 architecture. This bridges llama.cpp's local inference foundation with transformers' PyTorch model-definition ecosystem, enabling experimentation, evaluation, fine-tuning, and conversion validation on GGUF checkpoints.
Hugging Face announced that transformers can now load and run llama.cpp GGUF quantized checkpoints directly via from_pretrained, reusing ggml's Metal kernels through the kernels library to approach llama.cpp performance. Initial support targets Apple Silicon Macs, starting with the Qwen3.5 architecture. This bridges llama.cpp's local inference foundation with transformers' PyTorch model-definition ecosystem, enabling experimentation, evaluation, fine-tuning, and conversion validation on GGUF checkpoints.
295 engagement2 sourcesrss, redditTue, Sep 22, 2026, 12:00 AM
Products·launched
Decentralized LLM distribution via torrents
A project that uses BitTorrent to distribute large language model files, making downloads faster and more decentralized. It resolves Hugging Face CDN URLs to actual file locations and uses web seeds for initial seeding, then relies on peer-to-peer sharing. Aimed at users who need to download large models efficiently.
A project that uses BitTorrent to distribute large language model files, making downloads faster and more decentralized. It resolves Hugging Face CDN URLs to actual file locations and uses web seeds for initial seeding, then relies on peer-to-peer sharing. Aimed at users who need to download large models efficiently.
153 engagement1 sourceredditTue, Jul 21, 2026, 12:10 PM
Releases·Tool Release
llama.cpp team ships llama.app Mac app and llama serve
The llama.cpp team has released llama.app, an official Mac app, and llama serve, making the project more approachable. The app is distributed via DMG and aims to provide a user-friendly interface similar to Ollama.
The llama.cpp team has released llama.app, an official Mac app, and llama serve, making the project more approachable. The app is distributed via DMG and aims to provide a user-friendly interface similar to Ollama.
64 engagement1 sourceredditSun, Aug 2, 2026, 08:44 PM
Releases·Tool Release
Unsloth ships large performance release: Vulkan on AMD, MLX speedups, PyTorch 2.11
Unsloth added local-run support for Qwen3.8-Flash-Next and GLM-5.3-Flash with GGUF weights on Hugging Face, then shipped follow-ups enabling multi-token prediction for up to 2x faster inference and a major performance/bugfix release. The latest release switches AMD to Vulkan by default for a 20% prefill/decoding boost over ROCm and includes 200+ bug fixes. Practitioners gain faster local GGUF inference, one-click Hermes support, and MLX fine-tuning on Apple Silicon.
Unsloth added local-run support for Qwen3.8-Flash-Next and GLM-5.3-Flash with GGUF weights on Hugging Face, then shipped follow-ups enabling multi-token prediction for up to 2x faster inference and a major performance/bugfix release. The latest release switches AMD to Vulkan by default for a 20% prefill/decoding boost over ROCm and includes 200+ bug fixes. Practitioners gain faster local GGUF inference, one-click Hermes support, and MLX fine-tuning on Apple Silicon.
Updated Wed, Sep 9, 2026, 03:18 PM — September 8 release defaults AMD to Vulkan for 20% perf boost, fixes AMD gibberish issues, and adds 200+ bug fixes
1601.8k engagement7 sourcesgithub, xThu, Aug 27, 2026, 01:09 PM