← BlogBlog · How it works

Speaker diarization: transcription that labels who said what

When several people talk in one recording, a plain transcript is a wall of text where you cannot tell who said what. Here is the same recording with and without speaker labels — audio included, plus a measured accuracy check.

Published By Stanislav Shupilkin7 min read

Turning speech into text is a solved problem — words come out almost perfect. What stays unsolved is that when several people are talking, the transcript loses who said each line. For an interview, a meeting or a consultation, that is half the value of the recording.

Splitting a recording by voice is called speaker diarization. In the bot it is one switch, and it works on any uploaded file. Here is what it sounds like and what it changes.

Listen to the recording

Ten seconds, three people rescheduling a call. Try to keep track by ear of who proposed what.

Three people talking
10.5 seconds · 3 voices · 6 turns

What a plain transcript gives you

Every word is correct. Reconstructing who agreed to what is impossible:

Let's move the call to Tuesday. Won't work, I'm in Yerevan on Tuesday. Wednesday then? On Wednesday we have the client demo. Thursday morning then. Thursday at eleven, noted.

What speaker labels give you

  • Speaker 1:Let's move the call to Tuesday.
  • Speaker 2:Won't work, I'm in Yerevan on Tuesday.
  • Speaker 1: Wednesday then?
  • Speaker 3: On Wednesday we have the client demo.
  • Speaker 2: Thursday morning, then.
  • Speaker 1: Thursday at eleven. Noted.

Same file, same recognition. The only difference is that a second system ran alongside — the one that tells voices apart.

What actually comes back

The speaker label is attached not to a turn but to every word, each with its own timestamps:

{ "text": "Let's",   "start": 0.079, "end": 0.199, "speaker_id": "speaker_0" }
{ "text": "move",    "start": 0.239, "end": 0.359, "speaker_id": "speaker_0" }
{ "text": "Won't",   "start": 1.700, "end": 1.859, "speaker_id": "speaker_1" }

Turns are assembled by gluing together consecutive words that share a label. That is exactly why an interruption is caught at all: the boundary falls where the voice actually changed, not at the end of a sentence.

How accurate is it

I made the recording myself, so I know the correct answer second by second. That means the error can be counted rather than judged by ear. Result: 3 of 3 voices found, 6 of 6 turns labelled correctly.

Now the hard case

The first example was polite: people take turns, voices differ. Real recordings are not like that. Here is a harder one — two similar female voices, short turns, a one-word «No» and a real interruption: the second line starts before the first has finished.

Talking over each other
8.6 seconds · 2 similar voices · overlapping speech

Without labels the interruption vanishes from the text entirely: «The estimate went up by 20%. Hold on. The main part is logistics.» — three sentences in a row, and nothing tells you a second person cut in.

With labels: 2 of 2 voices found, and 5 of 6 turns correct. The one it got wrong is the short «Hold on» — the interjection that gets buried under the next line. That is exactly where diarization is hardest, and I am leaving the miss in rather than picking a friendlier example: the Russian version of this same recording came out 6 of 6, so on short overlapping interjections the result is a coin toss rather than a guarantee.

How accurate is speaker detection

The first question anyone evaluating a tool will ask. I pooled all four recordings into one measurement — not per turn, but per word: was it attributed to the right speaker.

RecordingWordsCorrectTurns
Three voices, taking turns29100%6/6
Two similar voices, interruption2391.3%5/6
Three voices, Russian27100%6/6
Interruption, Russian1894.4%6/6

Pooled: 96.9% of words labelled correctly — 94 out of 97 — and 23 turns out of 24. The industry metric for this is DER, diarization error rate; mine is a simplified version of it, without treating pauses and overlapped speech as separate categories.

More useful than the average is where the errors sit. Every single one is in a recording with an interruption. Where people take turns, both recordings scored exactly 100%. Accuracy is decided not by language or by the number of speakers, but by whether voices overlap.

How much to trust these numbers. 97 words is a small sample, the recordings are short and generated with speech synthesis. This is a best case, not a benchmark. Accuracy drops with overlapping speech, similar timbres, more than four participants, background noise and longer recordings. On an hour-long meeting expect noticeably worse — and test it on your own file.

Why some models can do this and others cannot

This is not a forgotten checkbox. A speech recognition model turns sound into letters and is optimised for exactly one thing — guessing the words. Timbre, pitch and delivery are interferenceit must remove, so that «hello» in a deep voice and «hello» in a high one produce the same answer.

In other words, such a model deliberately throws away the very information that distinguishes one person from another. Asking it for speaker labels is like asking a simultaneous interpreter to describe the speaker's voice: they were carrying the meaning, not the sound.

Speaker labels come from a second systemrunning in parallel: it computes a voice «fingerprint» for each segment — much like a fingerprint, but for sound — and groups the similar ones. However many groups appear, that is how many speakers there were. If a provider ships that system, you get labels. If not, you never will.

Which engines in the bot support diarization

EngineSpeakersWhy
🎯 AccurateYesThe model returns a speaker label on every word: a voice-separation system runs alongside recognition
🚀 FastNoThe model does not tell voices apart at all — timbre is interference to it. In exchange this is the fastest and cheapest option
🪄 ExclusiveNoThe model returns plain text only, without speaker labels or timestamps

That is why the Diarization switch only appears once you pick «Accurate»: it is a property of the models, not a paywall we invented.

How to check the quality on your own recording

Do not take anyone's percentages on faith, mine included — you can check this in ten minutes with no tools at all.

  1. Take a recording where you already know who spoke. Any will do: a stand-up, a call, an interview you ran yourself.
  2. Transcribe it with diarization on.
  3. Walk through the turns and mark the ones with the wrong label. The simple score is the share of wrong turns. A finer one counts words: labels are attached per word, so an interruption can be cut in the middle of a sentence.

Pick not a convenient sample but your worst one: where people cut in, where voices are alike, where it is noisy. A tidy two-person dialogue will score 100% and teach you nothing; a hard one tells you immediately whether the tool fits your work.

The industry metric is DER, diarization error rate. It is stricter: it counts not only misattributed speech but missed and invented speech too, and penalises overlaps separately. For choosing a tool, counting wrong turns is enough.

Who actually needs this

  • Journalists. Transcribe an interview and immediately see which line is your question and which is the answer — no re-listening to work out whose sentence you are about to quote.
  • Anyone running meetings. Who promised what and by when is the entire point of the minutes. Flat text loses it.
  • Researchers and recruiters.Ten interviews in a row, and from each you need the other person's words, not your own.
  • Lawyers and clinicians. Where it matters not only what was said, but by whom.

How to turn it on

  1. Open the bot settings and tap «Recognition».
  2. Pick «🎯 Accurate» — the button also shows its price per minute.
  3. A Diarization switch appears next to it. Turn it on.
  4. Send a file: audio, video or a voice message. The transcript arrives already split into turns.

It works on uploaded files of any length, not just voice messages in chat. Format does not matter: mp3, wav, m4a, ogg, video. Open the bot — 200 free credits every month.

Honest limitations

The recording above is short and generated with speech synthesis, which is a favourable case, and the 6-of-6 figure comes from exactly that. Synthetic audio is not there to flatter the result: it is the only way I know the correct answer second by second and can count the error instead of guessing it.

On an hour-long meeting with four people, room noise and a phone on the table, accuracy will be lower. Test it on your own recording — one transcript is cheap, and whether it is good enough can only be answered on your own material.