Releases·Model Release
Korean LLM v4: developer builds 1.09B-parameter Korean LLM from scratch with pretraining and SFT pipeline
A GitHub project released Korean LLM v4, a 1.09B-parameter decoder-style Transformer pretrained from scratch for Korean. It bundles pretraining and SFT pipelines with BF16 training, 8-bit AdamW, KV cache, and Tkinter/Matplotlib monitoring. It targets hobbyists wanting to build a Korean LLM on personal hardware.
23 engagement1 sourceWed, Sep 9, 2026, 02:41 PM
AI Overview
- GitHub project Korean LLM v4 trains a 1.09B-parameter decoder-only Transformer from scratch: 20 layers, hidden dim 1,920, 10 heads, RoPE, SwiGLU, RMSNorm
- Model tokenizes with beomi/Llama-3-Open-Ko-8B and defaults to BF16 precision with 8-bit AdamW support
- Training pipeline runs 100,000 pretraining steps plus 10,000 SFT steps, with Tkinter + Matplotlib monitoring and dataset caching
Related
Releases·Model Release
Korean LLM Advanced v3 released as 1.09B-parameter PyTorch project
Korean LLM Advanced v3, a PyTorch-based Korean language model implementation and training project, has been released. The model has 1.09 billion parameters and requires CUDA 11.8+ and PyTorch 2.0+.
Korean LLM Advanced v3, a PyTorch-based Korean language model implementation and training project, has been released. The model has 1.09 billion parameters and requires CUDA 11.8+ and PyTorch 2.0+.
43 engagement1 sourcegithubSun, Aug 23, 2026, 06:24 AM
Releases·Tool Release
OpenLanguageModel: A Readable PyTorch Library for Small Language Model Pretraining
OpenLanguageModel (OLM) is an open-source PyTorch library designed for building and pretraining small language models with transparent, readable code. The library uses modular components like Block, Residual, Repeat, and Parallel to make model architectures explicit, and supports moving from teaching notebooks to full pretraining runs. It integrates tokenizers, datasets, optimization, mixed precision, and hardware-aware execution for CPU, single-GPU, and multi-GPU setups.
OpenLanguageModel (OLM) is an open-source PyTorch library designed for building and pretraining small language models with transparent, readable code. The library uses modular components like Block, Residual, Repeat, and Parallel to make model architectures explicit, and supports moving from teaching notebooks to full pretraining runs. It integrates tokenizers, datasets, optimization, mixed precision, and hardware-aware execution for CPU, single-GPU, and multi-GPU setups.
0 engagement1 sourcearxivSat, Jul 18, 2026, 06:55 AM
Releases·Model Release
WicaraLLM: Indonesian 56M-parameter SLM trained from scratch on 1.3B tokens
WicaraLLM is a 56M-parameter Indonesian small language model built from scratch and trained on a 1.3B-token Indonesian corpus. It runs on a single laptop GPU with 6 GB VRAM, combining GQA, RoPE, SwiGLU, and RMSNorm with supervised fine-tuning for conversational use.
WicaraLLM is a 56M-parameter Indonesian small language model built from scratch and trained on a 1.3B-token Indonesian corpus. It runs on a single laptop GPU with 6 GB VRAM, combining GQA, RoPE, SwiGLU, and RMSNorm with supervised fine-tuning for conversational use.
14 engagement1 sourcegithubFri, Aug 21, 2026, 07:02 PM
Releases·Tool Release
Dr. Kobkrit Viriyayudhakorn releases Thai LLM Tutorials — 10-part series with notebooks
Dr. Kobkrit Viriyayudhakorn, creator of OpenThaiGPT and CEO of iApp Technology, released a 10-part Thai LLM tutorial series on kobkrit.com. The companion notebooks cover building and fine-tuning Thai LLMs from equations to runnable code, and include the KobEval-TH v1 benchmark with 120 items. The tutorials are licensed under Apache 2.0.
Dr. Kobkrit Viriyayudhakorn, creator of OpenThaiGPT and CEO of iApp Technology, released a 10-part Thai LLM tutorial series on kobkrit.com. The companion notebooks cover building and fine-tuning Thai LLMs from equations to runnable code, and include the KobEval-TH v1 benchmark with 120 items. The tutorials are licensed under Apache 2.0.
27 engagement1 sourcegithubSun, Jul 19, 2026, 04:35 PM
Developments·Paper
Controlled fine-tuning study compares SFT, LoRA, and DeepSpeed across 15 experiments
A GitHub study reports 15 controlled experiments comparing full-parameter SFT, LoRA, and DeepSpeed distributed training on Qwen3-0.6B for keyword extraction. It uses ablation analysis and statistical tests to explain method choices, with unified seed, sequence length, and training skeleton.
A GitHub study reports 15 controlled experiments comparing full-parameter SFT, LoRA, and DeepSpeed distributed training on Qwen3-0.6B for keyword extraction. It uses ablation analysis and statistical tests to explain method choices, with unified seed, sequence length, and training skeleton.
16 engagement1 sourcegithubMon, Aug 3, 2026, 12:43 AM