Releases·Model Release
Show HN: 188M Hindi encoder, 28B tokens, 8K context, 1× RTX 4090
A developer released a Hindi-first ModernBERT encoder trained from scratch on 28.5B Hindi tokens, with an 8,192-token context length. The model targets Hindi retrieval quality on long documents, where existing architectures lack long context. After DPR fine-tuning it achieves SOTA on the Hindi retrieval benchmarks the author evaluated, including mMARCO Hindi.
1 engagement1 sourceSun, Sep 6, 2026, 02:57 PM
AI Overview
- Dev trained a 188M-parameter Hindi ModernBERT from scratch on ~28.5B tokens with 8,192-token context using one RTX 4090 (24GB) over ~5 days.
- After DPR fine-tuning, the model hits 0.2825 nDCG@10 on mMARCO Hindi and 0.2635 nDCG@10 on long-document MLDR Hindi, claiming SOTA.
- The model scores 0.8001 F1 on Hindi NER and 0.4731 Macro-F1 on MASSIVE Hindi intent classification.
- Weights live on Hugging Face (kkkamur07/hindi-modernbert) with code on GitHub; the dev seeks collaboration to extend the approach to other low-resource languages.
Related
Developments·Paper
Researchers release Polish ModernBERT, a family of four Polish BERT encoders
A new arXiv paper introduces Polish ModernBERT, four Polish encoder-only Transformers at Base and Large scales, each in 512-token and 8K-context variants. It adapts the ModernBERT pretraining recipe to Polish, where encoders have remained BERT/RoBERTa-style, and releases a Polish long-context benchmark. Practitioners gain modern, open encoder options for discriminative and representation-learning tasks in Polish.
A new arXiv paper introduces Polish ModernBERT, four Polish encoder-only Transformers at Base and Large scales, each in 512-token and 8K-context variants. It adapts the ModernBERT pretraining recipe to Polish, where encoders have remained BERT/RoBERTa-style, and releases a Polish long-context benchmark. Practitioners gain modern, open encoder options for discriminative and representation-learning tasks in Polish.
0 engagement1 sourcearxivTue, Sep 1, 2026, 03:13 PM
Releases·Model Release
Indie developer releases 3.7B multilingual reasoning MoE trained from scratch on a single RTX 4090
A solo developer uploaded a checkpoint of Polanka 3.7B, a multilingual reasoning mixture-of-experts model pretrained, mid-trained, and fine-tuned from scratch on one consumer-grade RTX 4090 over many months. The checkpoint is public on Hugging Face as a research artifact. It shows small-scale from-scratch pretraining remains feasible on consumer hardware.
A solo developer uploaded a checkpoint of Polanka 3.7B, a multilingual reasoning mixture-of-experts model pretrained, mid-trained, and fine-tuned from scratch on one consumer-grade RTX 4090 over many months. The checkpoint is public on Hugging Face as a research artifact. It shows small-scale from-scratch pretraining remains feasible on consumer hardware.
57 engagement1 sourceredditTue, Sep 1, 2026, 03:26 PM
Developments·Paper
MoganBert-TR: Turkish encoder model trained from scratch with CLM-to-MLM curriculum
Researchers introduce MoganBert-TR, a 149M-parameter Turkish encoder foundation model trained from scratch on a language-specific corpus using a two-stage curriculum: causal language modeling first, then masked language modeling. The paper also presents MoganBert-Embed, an embedding model derived from it.
Researchers introduce MoganBert-TR, a 149M-parameter Turkish encoder foundation model trained from scratch on a language-specific corpus using a two-stage curriculum: causal language modeling first, then masked language modeling. The paper also presents MoganBert-Embed, an embedding model derived from it.
0 engagement1 sourcearxivWed, Aug 26, 2026, 01:12 PM
Releases·Model Release
Liquid AI releases LFM2.5-Encoder family for fast long-context CPU inference
Liquid AI released LFM2.5-Encoder, a family of multilingual bidirectional encoders built on the LFM2 architecture, available in 230M and 350M parameter sizes. They are designed for fast long-context inference on CPU, claiming about 3.7x speedup over ModernBERT-base at long context. The models match or beat larger encoders on GLUE, SuperGLUE, and multilingual tasks.
Liquid AI released LFM2.5-Encoder, a family of multilingual bidirectional encoders built on the LFM2 architecture, available in 230M and 350M parameter sizes. They are designed for fast long-context inference on CPU, claiming about 3.7x speedup over ModernBERT-base at long context. The models match or beat larger encoders on GLUE, SuperGLUE, and multilingual tasks.
5 engagement2 sourcesreddit, rssTue, Jul 28, 2026, 03:01 PM
Developments·Paper
DenseOn and LateOn: fully open dense and late-interaction retrieval models released
Researchers released DenseOn and LateOn, two 149M-parameter open retrieval models trained on 665M contrastive pairs and 1.88M supervised pairs from public data. The models aim to close the reproducibility gap in state-of-the-art retrieval by providing an end-to-end open recipe, including translate-train for multilingual transfer.
Researchers released DenseOn and LateOn, two 149M-parameter open retrieval models trained on 665M contrastive pairs and 1.88M supervised pairs from public data. The models aim to close the reproducibility gap in state-of-the-art retrieval by providing an end-to-end open recipe, including translate-train for multilingual transfer.
0 engagement1 sourcearxivWed, Jul 29, 2026, 05:50 PM