Restore diacritics and tone marks on text that was typed, scraped, or keyed in without them — across 10 languages, from Yorùbá and Igbo to Vietnamese and Polish.
Almost nobody types diacritics. Phone keyboards make them slow, legacy databases strip them, and OCR of older print drops them entirely. The result is an archive where "ọkọ" (husband), "oko" (farm), and "òkò" (stone) all collapse into the same string.
For a publisher that means unreadable copy. For an archive it means text that no longer says what the author wrote. For any search index built on top, it means queries that silently fail to match.
Restoring the marks by hand is a proofreading job measured in person-months, and it needs speakers of every language involved.
Tell the model which of its ten languages the text is in — yo, vi, ig, ha, pl, tr, pt, es, fr, it. One joint model covers all of them, so there is no per-language model to load or route between. Pass model="auto" and the SDK detects the language and routes for you.
diacnet-1.0diacnet-1.0 is fine-tuned from google/byt5-small, so it works on bytes rather than words. Yorùbá tone marks, Vietnamese combining diacritics, and Polish special characters all go through the same mechanism, and unseen words get handled instead of skipped.
diacnet-1.0The model is trained on sentence-level examples with a 256-byte window. Split long documents on sentence boundaries and batch them — a ~300M-parameter seq2seq model is cheap enough to sweep an entire back catalogue on one GPU.
The Olaverse SDK wraps the models with sane defaults. If you would rather not add a dependency, the second tab is the same pipeline in plain transformers.
from olaverse.nlp import Diacritizer
# One joint model, ten languages — name the one your text is in
d = Diacritizer(model="diacnet-1.0", lang="yo")
d.restore("se eranko naa si gbo o?")
# → 'ṣé ẹranko náà sì gbọ́ ọ?'
# Or let the SDK detect the language and route for you
Diacritizer(model="auto").restore("Ojo lo si oja lana")
# → 'Òjó lọ sí ọjà lana'
# lang= accepts: yo vi ig ha pl tr pt es fr itEvery language the models in this pipeline were trained on, with the codes you pass and the codes you get back.
Scoped deliberately to languages where diacritics are lexically meaningful. The other 15 languages in the source corpus (Swahili, Zulu, Amharic, Japanese and the rest) were left out because restoration either does not apply or is not the right frame for the script.
Every figure on this page comes from the published model cards, and every limitation is one the cards state themselves. Download the weights and check us.
Restore the marks your documents lost, then match on an accent-folded key — so a user typing "oko" finds "ọkọ", and still sees it written properly.
/ Routing & moderationDetect the language of support tickets, chat messages, queries, and documents across 25 languages — including the African languages most detectors get wrong.
The weights are open, so you can build it yourself this afternoon. If you would rather we tuned it to your domain, evaluated it properly, and handed it over, tell us what you are working on.