diacnet-2.0
Put the accents and tone marks back, in 11 languages.
diacnet-2.0 takes text typed without its marks and returns it written properly: Yorùbá tone marks, Vietnamese accents, Polish and Turkish letters, Arabic vowels. It is the most accurate model in the DiacNet line and our first to handle Arabic.
The problem it solves
Most people type without diacritics because phone keyboards make them slow. Old databases strip them and OCR drops them. In tonal and accented languages that is not cosmetic: the same unmarked spelling can be several different words.
Fixing it by hand needs fluent proofreaders in every language. diacnet-2.0 does the first pass for you, and an alignment step makes sure it only adds marks and never rewrites your text.
The numbers
diacbench, 1,000 sentences per language
Error rate per language for the whole DiacNet line against diacnet-1.1 (log scale, lower is better). diacnet-2.0 has the lowest error on 8 of 10 languages.
Meaning hints
A short hint about the intended meaning lifts accuracy on ambiguous words: Vietnamese 0.63 → 0.90, Turkish 0.75 → 0.84, Polish 0.93 → 0.98.
Highlights
Yorùbá, rebuilt
The Yorùbá error rate falls from 0.174 to 0.070 against diacnet-1.1, and 12% of sentences now come back exactly right, up from 0.8%.
Arabic in two styles
Full vowelling with case endings for teaching and text-to-speech, or the lighter no-case-endings style most modern Arabic uses.
Hints for ambiguous words
Tell it which meaning you want ([g: word=meaning]) and Vietnamese ambiguous-word accuracy rises from 0.63 to 0.90.
Never rewrites your text
With alignment on (the default in the SDK), every letter you typed is kept and only the marks are taken from the model.
Run it in a few lines
from olaverse.nlp import Diacritizer
d = Diacritizer(model="diacnet-2.0", lang="yor", device="cuda")
d.restore("O so fun ara re pe oun ko ni isoro kankan.")
# → 'Ó sọ fún ara rẹ̀ pé òun kò ní ìṣòro kankan.'
# Arabic without case endings
d.restore("وهذا قول مرغوب عنه .", lang="ara", case_endings=False)
# Don't know the language? Leave it out.
Diacritizer(model="diacnet-2.0").restore("No pude sujetarme a la cuerda mas tiempo.")
# → 'No pude sujetarme a la cuerda más tiempo.'Which DiacNet model?
Smaller budget? diacnet-mini-2.0 does the same at half the size. Need a CPU model that can never touch your letters? Use diactag-2.0.
| diacnet-2.0THIS PAGE | diacnet-mini-2.0 → | diactag-2.0 → | |
|---|---|---|---|
| Parameters | 582M | 300M | 37.9M |
| How it works | Rewrites the text | Rewrites the text | Tags each character |
| Mean error rate, 10 languages | 0.0117 | 0.0141 | 0.0130 |
| Yorùbá error rate | 0.0698 | 0.0815 | 0.0787 |
| Strongest on | Yorùbá and European languages | Close to 2.0 at half the size | Igbo, Vietnamese, Modern Standard Arabic |
| Extras | Meaning hints, typo correction | Meaning hints, typo correction | CPU-ready ONNX, never alters letters |
Best for
- Publishing-quality restoration in Yorùbá and the European languages, where accuracy matters most.
- Text with ambiguous words, where a meaning hint settles the right marks.
- Messy input where you also want typos corrected along the way.
- Mixed-language batches, with the language detected for you.
Use cases
Diacritic restoration
Restore diacritics and tone marks on text that was typed, scraped, or keyed in without them, across 11 languages from Yorùbá and Igbo to Vietnamese, Polish and Arabic.
/ Text quality · ArabicArabic diacritization
Restore Arabic tashkīl, with or without case endings, for text-to-speech, teaching material and readable copy, from models that also cover ten other languages.
More production models
All modelsPurpleMIST-Flash-1.0
typed-decisions 0.720 · African TD 0.668
PurpleMISTPurpleMIST-Mini-1.0
typed-decisions 0.601 · 1.9B
MIST-ExtractMIST-Extract-1.0
91.8% docs fully right · 4B
MIST-ExtractMIST-Extract-Flash-1.0
9.7 docs/s · 2B
LID-608lid-neural-608
0.867 acc · 140M
LID-608lid-lite-608
37 MB · 6,800 texts/s
MIST QGmist-qg-1.5b
0.955 keep-rate · 25 langs