Three Kyrgyz clips, three transcripts
Clips come from the open FLEURS dataset. Model output is pasted as returned, Cyrillic untouched. English translations are FLEURS' own parallel sentences.
- Reference
- Эртерээк келип, кемпингдеги иш-чарага жана музыкага жакын жайгашкан орунду ээлесеңиз болот.
- Scribeword errors: 0 of 12
- Эртерээк келип, кемпингдеги иш-чарага жана музыкага жакын жайгашкан орунду ээлесеңиз болот.
- Whisper (Groq)word errors: 12 of 12
- Әртірек келіп кемпингтеге ішчараға, жанам музыкаға жақын жайғашқан орынды елесеңіз болады.
- Reference
- Жаракат алгандардын он тогузу, губернатордун кеңсесинин билдирүүсү боюнча полиция кызматкерлери болгон.
- Scribeword errors: 2 of 11
- Жаракат алгандардын 19-у губернатордун кеңсесинин билдирүүсү боюнча полиция кызматкерлери болгон.
- Whisper (Groq)word errors: 9 of 11
- жарақат алғандардың он тоғызы губернатордың кеңсесінің білдерусы бойынша полиция қызметкерлері болған
- Reference
- Асман күнү-түнү бир аз караңгы болгондуктан, түндүк жарыктарын көрүүгө жакшы мүмкүнчүлүк бар.
- Scribeword errors: 0 of 13
- Асман күнү-түнү бир аз караңгы болгондуктан, түндүк жарыктарын көрүүгө жакшы мүмкүнчүлүк бар.
- Whisper (Groq)word errors: 12 of 13
- асмаң гүні-түні бираз қараңы болғандықтан түндік жарықтарын гөруге жақшы мүмкіншілік бар
Audio and reference text: FLEURS (Google), CC BY 4.0; clips converted to MP3. Dataset card.
Word errors by model and language
| Model | Kyrgyz (28 clips) | Kazakh (10) | Russian (10) |
|---|---|---|---|
| Whisper large-v3-turbo via Groq | 85.7% | 24.2% | 3.1% |
| ElevenLabs Scribe v1 | 10.2% | 6.5% | 3.6% |
| ElevenLabs Scribe v2 | 10.0% | 5.9% | 3.6% |
| Language detected correctly: Groq / Scribe | 0 of 28 / 28 of 28 | 10 of 10 / 10 of 10 | 10 of 10 / 10 of 10 |
Put simply: on Russian Whisper is fine, even slightly ahead; on Kazakh it falls behind; on Kyrgyz it breaks down. Scribe v1 and v2 are indistinguishable on a sample this small.
Does Whisper support Kyrgyz?
No, and that explains the whole picture. Kyrgyz is not in Whisper's language list (tokenizer source: 100 languages, Kazakh included, Kyrgyz not). So the model decides it is hearing Kazakh, a close relative, and spells everything with Kazakh letters — қ, ғ, і, ә. In clip 2 the sounds are mostly right, "он тоғызы" is "он тогузу" heard correctly, but as Kyrgyz text it is wrong, and search over it finds nothing.
Groq returned "kazakh" as the detected language on 28 clips out of 28, and every transcript had letters that don't exist in Kyrgyz.
We tested openai/whisper-large-v3-turbo through Groq's API. Groq also serves the full whisper-large-v3; we didn't test it, but its language list is the same.
Scribe is the opposite case. ElevenLabs' own documentation puts Kyrgyz in its "Good" tier, 10–20% word error rate, and our 10% lands at the better end of that.
Transcribing Kyrgyz voice messages in Telegram
@smolevich_voice_bot uses Whisper via Groq by default ("🚀 Fast" mode), so a Kyrgyz voice message will come back in Kazakh spelling. Switch the mode: "🎤 Transcription" → the button with your current mode → "🎯 Accurate". That is Scribe v1, the same model as in the table. Accurate costs 25 credits a minute instead of 5; the 200 free monthly credits cover about 8 minutes of Kyrgyz audio.
FAQ
Can I force Whisper to transcribe Kyrgyz by setting the language?
No. Kyrgyz has no language code in Whisper, so there is nothing to set it to. The nearest option is Kazakh, and Kazakh spelling is exactly the problem.
How accurate is ElevenLabs Scribe on Kyrgyz?
About 10% of words wrong on 28 FLEURS clips. Part of that is formal: numbers come out as digits while the reference writes them as words.
Does passing the language code to Scribe help?
Barely. Scribe v2 with the language set scored 9.4% instead of 10.0%, within noise. On auto it identified Kyrgyz on every clip anyway.
How we measured
- Data: FLEURS by Google, test split, ky_kg — 28 clips with distinct sentences, 6 to 19 seconds each, about 5 minutes of audio in total. For comparison, 10 clips each from kk_kz (Kazakh) and ru_ru (Russian). We took the first clips in the archive in order, no cherry-picking.
- Models: Groq
whisper-large-v3-turbowith no language hint (the way our bot calls it), ElevenLabsscribe_v1andscribe_v2, also without a language hint, plusscribe_v2with languageky. - Metric:word error rate (WER): words the model substituted, dropped or added, divided by the number of words in the reference. That is why it can go above 100% on a single clip. Before comparing, text is lower-cased and punctuation removed, nothing else: "19" vs "он тогузу" counts as an error.
- Date: October 1, 2026.
- Caveats: 28 clips is a small sample; a half-point difference between Scribe versions means nothing. FLEURS sentences are prepared text read aloud (from the FLoRes translation benchmark); real voice messages with noise and fast speech may score differently.
- Audio and reference text: FLEURS (Conneau et al., 2022, arXiv:2205.12446), licensed CC BY 4.0; clips in the examples converted from WAV to MP3.