Seznam introduced HTML-LM, a 154-million-parameter foundation model built on a ModernBERT-based architecture with HTML-aware training, released on Hugging Face under CC BY-NC 4.0. Trained on 100 million web documents with masked language modeling, bag-of-words prediction, and contrastive distillation from LLMs, it sets a new state of the art for classification and regression on the Czech Internet domain. The model is already in production at Seznam, processing thousands of web documents per second.
An arXiv paper evaluates LLM-distilled student models for multi-label topic assignment on e-commerce UGC, comparing causal generative SLMs (1B, 4B, 8B) against discriminative baselines (DeBERTa-V3, ModernBERT). It finds a data-dependent trade-off: discriminative models win on product reviews, while even 1B generative models beat them on multi-turn conversations and scale to 112 topics without the 35% Macro-F1 drop discriminative baselines suffer. The authors report production deployment across both domains with strict latency compliance at a global marketplace.
An arXiv paper evaluates LLM-distilled student models for multi-label topic assignment on e-commerce UGC, comparing causal generative SLMs (1B, 4B, 8B) against discriminative baselines (DeBERTa-V3, ModernBERT). It finds a data-dependent trade-off: discriminative models win on product reviews, while even 1B generative models beat them on multi-turn conversations and scale to 112 topics without the 35% Macro-F1 drop discriminative baselines suffer. The authors report production deployment across both domains with strict latency compliance at a global marketplace.
A Reddit user asked the community what specific use case a 300M parameter model should be optimized for to consistently outperform larger models (2B–20B). Suggestions included narrow-domain code generation, structured data extraction, document classification, workflow/agent planning, and log analysis.
A Reddit user asked the community what specific use case a 300M parameter model should be optimized for to consistently outperform larger models (2B–20B). Suggestions included narrow-domain code generation, structured data extraction, document classification, workflow/agent planning, and log analysis.
A new arXiv paper introduces Polish ModernBERT, four Polish encoder-only Transformers at Base and Large scales, each in 512-token and 8K-context variants. It adapts the ModernBERT pretraining recipe to Polish, where encoders have remained BERT/RoBERTa-style, and releases a Polish long-context benchmark. Practitioners gain modern, open encoder options for discriminative and representation-learning tasks in Polish.
A new arXiv paper introduces Polish ModernBERT, four Polish encoder-only Transformers at Base and Large scales, each in 512-token and 8K-context variants. It adapts the ModernBERT pretraining recipe to Polish, where encoders have remained BERT/RoBERTa-style, and releases a Polish long-context benchmark. Practitioners gain modern, open encoder options for discriminative and representation-learning tasks in Polish.
A Medium article discusses the dual challenge of training and deploying large language models and massive multilayer perceptrons, focusing on scaling and optimization techniques. The piece appears to be a general overview rather than a specific product or research announcement.
A Medium article discusses the dual challenge of training and deploying large language models and massive multilayer perceptrons, focusing on scaling and optimization techniques. The piece appears to be a general overview rather than a specific product or research announcement.
A developer released BetterGPT-150M, a compact causal language model with ~152 million parameters trained on 15B tokens. Benchmarks show it outperforms GPT-2 Small while matching models trained on larger token budgets. A live Hugging Face Space demo is available.
A developer released BetterGPT-150M, a compact causal language model with ~152 million parameters trained on 15B tokens. Benchmarks show it outperforms GPT-2 Small while matching models trained on larger token budgets. A live Hugging Face Space demo is available.