AxiomicLabs' new benchmark, Tiny Theory of Mind, meant to gauge small models' theory of mind capabilities, has made it to the front page of Hugging Face's Datasets

Developments·Benchmark

pdx-experimental releases SLM benchmark suite for task governance and deterministic orchestration

The pdx-experimental project published pdx-benchmarks, a GitHub repository offering empirical evaluation frameworks and reproducible datasets for Small Language Models. It targets Task Governance and Deterministic Orchestration, giving practitioners tooling to evaluate SLM behavior in structured pipelines.

The pdx-experimental project published pdx-benchmarks, a GitHub repository offering empirical evaluation frameworks and reproducible datasets for Small Language Models. It targets Task Governance and Deterministic Orchestration, giving practitioners tooling to evaluate SLM behavior in structured pipelines.

A Reddit user benchmarked 23 Gemma 4 E4B models from Hugging Face using the abliterlitics gauntlet, finding that the most downloaded model is also the most broken. The comparison includes new benchmarks and tensor comparisons against the base model, aiming to expose bogus claims about model performance.

Developments·Benchmark

DeepSWE-mini a 16 instance subset of DeepSWE that replicates the rankings of the leaderboard

A community member released deepswe-mini, a 16-instance subset of the DeepSWE benchmark, then followed with terminal-bench-mini, a 14-instance subset of Terminal-Bench, both hosted on Hugging Face. Each subset was selected by analyzing the full benchmark to replicate the leaderboard's relative model rankings while running far faster on local hardware. The datasets let developers quickly benchmark new local LLMs without lengthy full agentic eval runs.

A community member released deepswe-mini, a 16-instance subset of the DeepSWE benchmark, then followed with terminal-bench-mini, a 14-instance subset of Terminal-Bench, both hosted on Hugging Face. Each subset was selected by analyzing the full benchmark to replicate the leaderboard's relative model rankings while running far faster on local hardware. The datasets let developers quickly benchmark new local LLMs without lengthy full agentic eval runs.

Show HN: OmnisBench, a re-gradable, open LLM routing benchmark on fresh tasks

Releases·Model Release

Pestle 27B Ternary: The First Medical Ternary Model Apache 2.0

Pestle 27B Ternary is released as the first medical ternary model under Apache 2.0. It uses ternary weights to reduce memory footprint while maintaining medical domain performance.

Pestle 27B Ternary is released as the first medical ternary model under Apache 2.0. It uses ternary weights to reduce memory footprint while maintaining medical domain performance.