AI
Turkish NLP Gets a Modern Upgrade: TabiBERT Encoder Outperforms Predecessors on New Benchmark
Researchers introduce TabiBERT, a large-scale Turkish language model built on the latest ModernBERT architecture, along with TabiBench—a standardized evaluation suite of 27 datasets to measure progress in Turkish language understanding.
Researchers have unveiled TabiBERT, a newly built encoder-only transformer model designed specifically for Turkish-language tasks. The model incorporates architectural improvements from ModernBERT—including rotary positional embeddings, FlashAttention, and refined normalization—that have become standard in contemporary language model development.
Training and Capabilities#
TabiBERT was trained from scratch on one trillion tokens sourced from a diverse 86.58-billion-token corpus spanning web text (72%), academic papers (19%), source code (6%), and mathematical content (0.3%). The model supports sequences up to 8,192 tokens—sixteen times longer than previous Turkish BERT variants—and maintains computational efficiency even with this expanded context window.
A Standardized Evaluation Framework#
To enable rigorous comparison, the researchers also introduced TabiBench, a unified benchmark comprising 27 datasets organized into eight task categories. The framework uses standardized data splits and evaluation protocols, aggregated into a GLUE-style score on a 0–100 scale. This structured approach addresses the fragmentation that has long complicated Turkish NLP research.
Performance Results#
TabiBERT achieves a benchmark average of 77.28, surpassing BERTurk (the previous leading Turkish-specific model at 75.66). The new model shows particularly strong performance in question-answering tasks (+9.55 F1 improvement) and code retrieval (+2.41 NDCG@10), while demonstrating near-saturation on shorter text classification tasks. When compared to the multilingual mmBERT, TabiBERT falls slightly behind (78.98 vs. 77.28) but does so with half the parameters and one-third the training data, when accounting for Turkish-specific token efficiency.
Open Release and Impact#
The team has made model weights, training configurations, and evaluation code publicly available, establishing a transparent foundation for advancing Turkish language processing research in the future.
