Releases·Model Release
Dev trains 348M model from scratch on 22.7B tokens to 99.4% on GPT-3 arithmetic tasks, adds up to 14 digits
A developer shared their fifth small language model, trained from scratch on 22.7B tokens and fine-tuned for arithmetic. The 348M-parameter model solves 14-digit problems by showing intermediate work — column addition with carries, borrow chains, and partial-product multiplication — rather than guessing answers. It improves on the author's previous 326M model trained on 10B tokens.
0 engagement1 sourceThu, Sep 10, 2026, 03:28 AM
AI Overview
- Reddit user nkthebass trains a 348M-parameter, 22.7B-token LLaMA-architecture model that scores 99.4% average on nine GPT-3 arithmetic sub-tasks, hitting 100% on five-digit addition versus GPT-3 175B's 9.3%.
- The model shows column-by-column work; extending its place-value list from 6 names to 19 lifted its clean addition ceiling from 8 digits to 14, since 'millions' and 'ten-millions' appeared in zero training examples.
- 3×3 multiplication reaches 98% and negative-result questions 85%, but GSM8K is 4% and ASDiv 16.5% — operation selection fails, not arithmetic; division and 4×4 multiplication are unsupported.
- Greedy decoding is required; sampling corrupts the column routine. Valid working predicts correctness 95.3% of the time, versus 0.7% valid-working-wrong-answer.
- The math model took 10 full fine-tuning rounds plus 3 LoRA adapters, with the LoRAs (~1% of post-training tokens) doing most useful work; instruction tuning lowers the V3 family's MC average from 43.2 to 41.8.
- An independent tester found two published numbers wrong, one understating the model by 40 points due to n=24 measurement; both are corrected on the Hugging Face model card.
Entities
Related
Developments·Paper
Developer hand-codes transformer weights to multiply with 100% accuracy
A developer implemented the grade-school multiplication algorithm as a computation graph and compiled it into a stock Phi-3 checkpoint using his own compiler, Torchwright, achieving 100% accuracy on all 3,000,000 supported three-digit expressions without any training. He published checkpoints supporting up to 12-digit by 12-digit multiplication and tested six frontier models for comparison.
A developer implemented the grade-school multiplication algorithm as a computation graph and compiled it into a stock Phi-3 checkpoint using his own compiler, Torchwright, achieving 100% accuracy on all 3,000,000 supported three-digit expressions without any training. He published checkpoints supporting up to 12-digit by 12-digit multiplication and tested six frontier models for comparison.
250 engagement1 sourceredditMon, Aug 10, 2026, 05:37 PM
Developments·Paper
Survey proposes Numerical Grounding Framework for LLM numeracy limits
A new arXiv survey argues that LLMs remain unreliable at elementary numerical tasks—magnitude comparison, large-integer arithmetic, fractions, scientific notation—despite strong math reasoning benchmarks. It introduces the Numerical Grounding Framework (NGF), decomposing numeracy into Representational Grounding (RG) and Procedural Grounding (PG), as a path to improvement.
A new arXiv survey argues that LLMs remain unreliable at elementary numerical tasks—magnitude comparison, large-integer arithmetic, fractions, scientific notation—despite strong math reasoning benchmarks. It introduces the Numerical Grounding Framework (NGF), decomposing numeracy into Representational Grounding (RG) and Procedural Grounding (PG), as a path to improvement.
0 engagement1 sourcearxivThu, Aug 13, 2026, 12:01 PM
Social·Community
Qwen3.8 27B addition in words: Simon Willison replicates GPT-4o word-sum experiment on DGX Spark
Inspired by Colin Fraser's two-year-old GPT-4o experiment on adding large numbers and writing answers in words, Simon Willison reran the test locally on a DGX Spark with Qwen3.8-27B. With reasoning enabled, the model answered correctly in 167 of 169 one-shot attempts. The result offers a controlled look at how reasoning modes affect LLM arithmetic reliability.
Inspired by Colin Fraser's two-year-old GPT-4o experiment on adding large numbers and writing answers in words, Simon Willison reran the test locally on a DGX Spark with Qwen3.8-27B. With reasoning enabled, the model answered correctly in 167 of 169 one-shot attempts. The result offers a controlled look at how reasoning modes affect LLM arithmetic reliability.
0 engagement1 sourcerssSun, Oct 4, 2026, 11:34 PM
Releases·Model Release
Developer pre-trains Shibai-700M-Base on 18B tokens for Python and Wikitext
A developer released Shibai-700M-Base, a 700M parameter model pre-trained on 18B tokens of Python and Wikitext. The model is not fine-tuned for chat and is intended for continued pre-training on doc string Python. The author acknowledges it likely underperforms Qwen 3 0.6B and Qwen 3.5 0.8B but claims it is an order of magnitude better than GPT-2.
A developer released Shibai-700M-Base, a 700M parameter model pre-trained on 18B tokens of Python and Wikitext. The model is not fine-tuned for chat and is intended for continued pre-training on doc string Python. The author acknowledges it likely underperforms Qwen 3 0.6B and Qwen 3.5 0.8B but claims it is an order of magnitude better than GPT-2.
61 engagement1 sourceredditWed, Jul 29, 2026, 07:29 PM
Developments·Paper
20B looping model matches Qwen3 Coder 30B at 10% of pre-training tokens
A new paper introduces a 20B parameter looping model that matches or beats Qwen3 Coder 30B while using only 3.5 trillion pre-training tokens (10% of the typical 35 trillion). The model was trained for hundreds of thousands of dollars, far less than the millions usually required, raising hopes for hobbyist-level pretraining. No weights have been released yet.
A new paper introduces a 20B parameter looping model that matches or beats Qwen3 Coder 30B while using only 3.5 trillion pre-training tokens (10% of the typical 35 trillion). The model was trained for hundreds of thousands of dollars, far less than the millions usually required, raising hopes for hobbyist-level pretraining. No weights have been released yet.
80 engagement1 sourceredditTue, Jul 21, 2026, 03:55 PM