Releases·Tool Release
DeltaTensors compresses fine-tune checkpoints as weight deltas, cutting Qwen2.5-0.5B checkpoint from 953 MB to 294 MB
A developer released DeltaTensors, an open-source tool that stores fine-tuned model checkpoints as weight differences from the base model instead of full copies. On a Qwen2.5-0.5B fine-tune, a 953 MB model compressed to a 294 MB delta with minimal perplexity change. Practitioners keeping many fine-tunes of one base model can store the base once plus small .wdelta files.
14 engagement1 sourceTue, Sep 22, 2026, 04:24 AM
AI Overview
- DeltaTensors stores fine-tuned checkpoints as weight differences from the base model, compressing a Qwen2.5-0.5B fine-tune from 953 MB to 294 MB
- Reconstructing a compressed delta shifted perplexity from 19.11 to 19.22
- It applies post-training to existing full fine-tunes without LoRA, and supports chained deltas, Hugging Face Trainer checkpoints, and streaming reconstruction
- GitHub: https://github.com/AaravGaurdev/deltatensors
Related
Developments·Paper
Arxiv paper proposes code surrogate method for fine-tuning low-bit quantized models
An arxiv paper (2026-08-31) proposes a method for fine-tuning quantized models while keeping deployed checkpoints in low-bit form. It frames adaptation as optimization over quantization codes and scales, aiming to combine the efficiency of continuous low-bit training with the deployment fidelity of discrete search.
An arxiv paper (2026-08-31) proposes a method for fine-tuning quantized models while keeping deployed checkpoints in low-bit form. It frames adaptation as optimization over quantization codes and scales, aiming to combine the efficiency of continuous low-bit training with the deployment fidelity of discrete search.
0 engagement1 sourcearxivMon, Aug 31, 2026, 02:54 PM
Developments·Paper
ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization
ReQuant introduces a backpropagation-free, fixed-grid discrete refinement stage for post-training quantization (PTQ) of large language models. It keeps quantized weights improvable after an executable quantized model is produced, preserving the quantized format. This complements existing PTQ methods by reducing memory and computational costs further.
ReQuant introduces a backpropagation-free, fixed-grid discrete refinement stage for post-training quantization (PTQ) of large language models. It keeps quantized weights improvable after an executable quantized model is produced, preserving the quantized format. This complements existing PTQ methods by reducing memory and computational costs further.
0 engagement1 sourcearxivFri, Aug 7, 2026, 09:27 AM
Developments·Paper
NeMo-DCR: Bit-Exact Delta-Compressed Refit for Scalable Agentic RL at Trillion-Parameter Scale
An arXiv paper introduces NeMo-DCR (Delta-Compressed Refit), a bit-exact method for synchronizing policy weights from training to rollout clusters by sending only changed values. It cuts a 1T-parameter cross-region refit from 87.5 minutes to 150 seconds, a 35x speedup, making disaggregated agentic RL at trillion-parameter scale practical.
An arXiv paper introduces NeMo-DCR (Delta-Compressed Refit), a bit-exact method for synchronizing policy weights from training to rollout clusters by sending only changed values. It cuts a 1T-parameter cross-region refit from 87.5 minutes to 150 seconds, a 35x speedup, making disaggregated agentic RL at trillion-parameter scale practical.
0 engagement1 sourcearxivTue, Oct 6, 2026, 02:30 PM
Releases·Tool Release
Post-training delta compression stores 10 fine-tunes in size of ~4
A developer created a post-training delta compression tool that stores only the weight differences between a fine-tuned model and its base, reducing storage for 10 fine-tunes to roughly the size of 4 full checkpoints. The tool works on any trained model without requiring changes during training, unlike LoRA.
A developer created a post-training delta compression tool that stores only the weight differences between a fine-tuned model and its base, reducing storage for 10 fine-tunes to roughly the size of 4 full checkpoints. The tool works on any trained model without requiring changes during training, unlike LoRA.
6 engagement1 sourceredditWed, Jul 15, 2026, 03:31 PM
Developments·Paper
Super-Tuning: Activation-Aware Pruning Reused for Sparse Fine-Tuning
A new arxiv paper proposes Super, a sparse PEFT method that reuses Wanda-style activation-weighted magnitude scores from pruning to select a small trainable support, and Supra, a hybrid adapter combining sparse updates with LoRA. This approach reduces memory and compute for fine-tuning LLMs while maintaining performance.
A new arxiv paper proposes Super, a sparse PEFT method that reuses Wanda-style activation-weighted magnitude scores from pruning to select a small trainable support, and Supra, a hybrid adapter combining sparse updates with LoRA. This approach reduces memory and compute for fine-tuning LLMs while maintaining performance.
0 engagement1 sourcearxivFri, Jul 10, 2026, 10:55 AM