Releases·Tool Release
Prevent CUDA OOM in PyTorch with MEM v3, a memory governor that dynamically adjusts batch size mid-run
A developer released MEM v3, a PyTorch memory governor that watches VRAM and throughput in real time and dynamically adjusts batch size and gradient accumulation to avoid CUDA out-of-memory crashes during long training and fine-tuning runs. It targets the common tradeoff between restarting on memory spikes and conservatively undersizing batches. The tool ships with a Colab demo and open-source repo for community testing.
2 engagement1 sourceSun, Sep 27, 2026, 01:12 PM
AI Overview
- MEM v3 monitors VRAM and throughput in real time, scaling batch size and gradient accumulation on the fly without restarting the process
- Dynamic lane switching rescales batch size in milliseconds based on actual GPU memory pressure
- Handles +10 GB VRAM allocation shocks without crashing
- Uses atomic file replacement with SHA-256 checks across rotating checkpoint slots to survive power outages
- Ships a local web dashboard tracking loss, throughput, and lane switches; demo runs on a free Colab GPU
- Creator nobazzy released the code on GitHub at nobazzy/mem-llm-orchestrator
Related
Products·building
MEM Orchestrator: adaptive memory governor for PyTorch training on small GPUs
A lightweight runtime control layer around PyTorch that adapts to memory pressure during training to avoid CUDA OOM crashes. The author tested it on an RTX 5060 Ti (8GB) while training small local models; AI was used only to help translate the writeup, not the code. It targets hobbyists and researchers training models on consumer GPUs with limited VRAM.
A lightweight runtime control layer around PyTorch that adapts to memory pressure during training to avoid CUDA OOM crashes. The author tested it on an RTX 5060 Ti (8GB) while training small local models; AI was used only to help translate the writeup, not the code. It targets hobbyists and researchers training models on consumer GPUs with limited VRAM.
8 engagement2 sourcesredditWed, Sep 9, 2026, 09:11 PM
Social·Community
Guide explains how to fix CUDA OOM errors on local LLMs
A practical guide argues that CUDA out-of-memory errors on local LLM setups are often caused by inefficient memory usage rather than insufficient VRAM. It identifies bad batch sizes, overlapping processes, and forgotten cache as common culprits and offers tested fixes, challenging the assumption that larger GPUs are always necessary.
A practical guide argues that CUDA out-of-memory errors on local LLM setups are often caused by inefficient memory usage rather than insufficient VRAM. It identifies bad batch sizes, overlapping processes, and forgotten cache as common culprits and offers tested fixes, challenging the assumption that larger GPUs are always necessary.
2 engagement1 sourceredditTue, Aug 25, 2026, 05:08 PM
Releases·Tool Release
Show HN: UATC-Closed-loop VRAM control and dynamic data pruning for LLM training
A GitHub project introduces UATC, a closed-loop VRAM control and dynamic data pruning system for LLM training. The tool aims to optimize memory usage and training efficiency by dynamically adjusting VRAM allocation and pruning data during training.
A GitHub project introduces UATC, a closed-loop VRAM control and dynamic data pruning system for LLM training. The tool aims to optimize memory usage and training efficiency by dynamically adjusting VRAM allocation and pruning data during training.
1 engagement1 sourcehackernewsMon, Jul 13, 2026, 11:06 AM
Social·Community
How to automatically find the batch size when using Accelerate with FSDP2? [D]
A Reddit user asks how to replicate Hugging Face SFTTrainer's auto_find_batch_size behavior when multi-GPU training with accelerate launch and FSDP2 on a single node. The post highlights an open question for distributed-training practitioners: whether Accelerate can recover a distributed run after one process hits CUDA OOM, or whether automatic batch-size reduction must be implemented externally.
A Reddit user asks how to replicate Hugging Face SFTTrainer's auto_find_batch_size behavior when multi-GPU training with accelerate launch and FSDP2 on a single node. The post highlights an open question for distributed-training practitioners: whether Accelerate can recover a distributed run after one process hits CUDA OOM, or whether automatic batch-size reduction must be implemented externally.
3 engagement1 sourceredditMon, Sep 14, 2026, 05:24 PM
Developments·Paper
Deterministic Regime Switching and Feasibility Inversion in Dynamic Tensor Rematerialization
A preprint reports deterministic instabilities in Dynamic Tensor Rematerialization (DTR), an online eviction policy for memory-constrained DNN training, using the reference simrd simulator and public traces. On an LSTM trace, tiny budget changes switch execution regimes with overheads differing by up to 7.3x; on ResNet-32, a feasibility inversion makes budgets 0.102-0.106 OOM while adjacent budgets run. The authors argue these are distinct budget-sensitive pathologies and release code and results.
A preprint reports deterministic instabilities in Dynamic Tensor Rematerialization (DTR), an online eviction policy for memory-constrained DNN training, using the reference simrd simulator and public traces. On an LSTM trace, tiny budget changes switch execution regimes with overheads differing by up to 7.3x; on ResNet-32, a feasibility inversion makes budgets 0.102-0.106 OOM while adjacent budgets run. The authors argue these are distinct budget-sensitive pathologies and release code and results.
0 engagement1 sourcearxivFri, Sep 25, 2026, 01:26 PM