Restore diacritics and tone marks on text that was typed, scraped, or keyed in without them — across 10 languages, from Yorùbá and Igbo to Vietnamese and Polish.
Almost nobody types diacritics. Phone keyboards make them slow, legacy databases strip them, and OCR of older print drops them entirely. The result is an archive where "ọkọ" (husband), "oko" (farm), and "òkò" (stone) all collapse into the same string.
For a publisher that means unreadable copy. For an archive it means text that no longer says what the author wrote. For any search index built on top, it means queries that silently fail to match.
Restoring the marks by hand is a proofreading job measured in person-months, and it needs speakers of every language involved.
Handing the job to a generative model trades one problem for another: anything that writes the sentence out again can also reword it, and you only find out by diffing every output. On three languages a frontier LLM returns text worse than doing nothing at all — Hausa is 97.6% correct if left alone, and comes back 35% wrong.
Pass one of the ten — yo, vi, ig, ha, pl, tr, pt, es, fr, it, in ISO 639-1 or 639-3. One joint model covers all of them, so there is no per-language checkpoint to load or route between. Leave lang= out entirely and diactag-1.0 uses its own built-in LID head instead.
diactag-1.0Every character factorises into base + SHAPE + TONE, and only the two marks are predicted — the base letter is copied straight through. The output therefore has the same characters in the same order as the input, with the marks changed and nothing else. Structural compliance is 1.0000 by construction, not by measurement, and it holds under int8 quantisation too.
diactag-1.0Confidence is per character and temperature-calibrated (T = 1.14). Gate at 0.90 and you restore 97.1% of characters at DER 0.0039, with the remainder flagged for review; at 0.99 it is 91.9% at DER 0.0008. The threshold is per request, so one loaded model serves a CMS pre-fill and an archival pipeline at different points on the same curve.
diactag-1.0At 37.6M parameters the int8 ONNX export is a 38MB artefact running 244 chars/s with a 200ms median, for +0.03pp DER. That takes the GPU off the serving bill entirely. Use restore_batch for throughput work — the p95 tail is length-driven, so quantisation does not shorten it.
diactag-1.0The Olaverse SDK wraps the models with sane defaults. If you would rather not add a dependency, the second tab is the same pipeline in plain transformers.
from olaverse.nlp import Diacritizer
# One joint model, ten languages — name the one your text is in
d = Diacritizer(model="diactag-1.0", lang="yo")
d.restore("se eranko naa si gbo o?")
# → 'ṣé ẹranko náà sì gbọ́ ọ?'
# Leave lang= out and the model's own LID head routes for you
d = Diacritizer(model="diactag-1.0")
d.restore("Co ay rat dam dang") # → 'Cô ấy rất đảm đang'
d.detect_language("Lodz jest piekna") # → ('pol', 0.9999)
# Per-character confidence: commit the sure marks, review the rest
text, details = d.restore(src, return_details=True)
review = [c for c in details if c.confidence < 0.9]
# CPU serving — pip install olaverse[onnx] → 38MB, 244 chars/s on one core
d = Diacritizer(model="diactag-1.0", lang="yor", onnx=True)
# lang= accepts: yo vi ig ha pl tr pt es fr it (ISO 639-1 or 639-3)Every language the models in this pipeline were trained on, with the codes you pass and the codes you get back.
Scoped deliberately to languages where diacritics are lexically meaningful. The other 15 languages in the source corpus (Swahili, Zulu, Amharic, Japanese and the rest) were left out because restoration either does not apply or is not the right frame for the script.
Every figure on this page comes from the published model cards, and every limitation is one the cards state themselves. Download the weights and check us.
Restore the marks your documents lost, then match on an accent-folded key — so a user typing "oko" finds "ọkọ", and still sees it written properly.
/ Routing & moderationDetect the language of support tickets, chat messages, queries, and documents across 25 languages — including the African languages most detectors get wrong.
The weights are open, so you can build it yourself this afternoon. If you would rather we tuned it to your domain, evaluated it properly, and handed it over, tell us what you are working on.