All models
    Specialist · Language identification

    The LID-608 family

    One model line in two sizes, sharing 608 languages, the same labels and the same two modes. Neural for accuracy on short text, lite for raw speed on a CPU.

    Meet the models Compare
    / The lineup

    2 models, one interface

    LANGUAGE IDMOST ACCURATE

    lid-neural-608

    Know the language of any text, even when it is three words long.

    608
    Languages
    0.867
    Held-out accuracy
    0.906
    1-5 words, traffic mode
    140M
    Parameters
    Best for
    • Routing chat messages, search queries and support tickets, where text is short and typed by real users.
    • Products for African users that need to tell Yorùbá, Hausa, Igbo, Swahili and Pidgin apart reliably.
    • Accuracy-first pipelines with a GPU, with lid-lite-608 in front as a fast first pass at scale.
    Learn more 🤗 Weights
    LANGUAGE ID

    lid-lite-608

    608 languages in 37 MB, on a single CPU thread.

    608
    Languages
    37 MB
    Model size
    6,800/s
    Texts per CPU thread
    0.839
    Avg F1, 20 African languages
    Best for
    • Filtering web crawls and building language corpora at scale on ordinary CPUs.
    • Edge and serverless deployments, where a 37 MB model with no deep-learning stack matters.
    • Finding low-resource African languages that other detectors misfile.
    Learn more 🤗 Weights
    / Side by side

    Compare the LID-608 models

    lid-neural-608 →lid-lite-608 →
    Size140M parameters37 MB
    Speed~1,100 texts/s on an A100~6,800 texts/s on one CPU thread
    Held-out accuracy0.8670.858
    1-5 words, traffic mode0.9060.829
    Everyday phrases, traffic mode0.9430.838
    Best forUser input and short textBulk filtering, CPU and edge
    / Choosing

    Which one should you use?

    • Language ID on user input and short textlid-neural-608 →

      Traffic mode: 0.960 web-weighted accuracy, 0.906 on 1-5 word inputs.

    • Language ID at volume, CPU-only or at the edgelid-lite-608 →

      37 MB, about 6,800 texts/s on one CPU thread. Same 608 labels and modes.

    Other model families

    PurpleMIST

    Decision models · 2 models

    MIST-Extract

    Document extraction · 2 models

    DiacNet

    Diacritic restoration · 3 models

    MIST QG

    Question generation · 1 model