A Speaker Label Is a Number, Not a Name
Zoom puts a name on every caption line because the roster knows who is talking, not because the software recognises a voice. Take the roster away and one mixed track leaves only a number.
You are watching a two-person interview in a language you do not speak. The captions arrive translated, and every line begins the same way: Speaker 1, then Speaker 2, then Speaker 1 again. It reads as though the software has worked out who is talking. It has — but not along the route you are picturing.
A name in a meeting app and a number in a browser extension come from two different places, and only one of them involves listening. The name is usually read out of the participant list. The number is inferred from the audio. Both end up in the same slot at the start of a caption line, which is why they get mistaken for the same feature.
The difference decides what a label can ever promise. A name claims to know a person. A number claims only that the voice changed. Everything below follows from that.
The name arrives with the account
Zoom states the mechanism in one sentence, in a help article about viewing captions: “When viewing captions, the spoken line is labeled with the speaker’s meeting display name.” That is all of it. The app is not comparing voices. It is reading a string that the participant’s client sent when they joined, then attaching that string to the audio the same client is sending.
Google Meet reaches the same place through different plumbing. Third-party tools that read Meet’s caption overlay describe their source as “participant names and active-speaker information” — a roster entry plus a signal about which connection is currently loudest. A comparison table NYU publishes for staff choosing a captioning option, written for administrators rather than engineers, sums it up: Zoom can indicate a change in speaker in the transcript window, and Meet “automatically labels speakers by name.” Neither says the software recognises anybody.
The consequence shows up exactly where the roster stops being per-person. Put four people in a conference room, join them through one laptop, and the roster has one entry for the room. Every line comes out under the laptop’s name. Earmark, a Mac app that reads Zoom’s interface to label its own transcripts, lists this as a limitation in its documentation: “Shared rooms show one name. If several people join through one Zoom connection, like a conference room, they all get that connection’s name.” Granola’s help centre says the same about Meet — when “multiple people use one room device”, the label “may not identify the individual correctly.”
Zoom also sells a fix for exactly that room, and the way it is described is the tell. Smart name tags for voice “can highlight the speaker in the room, and the resulting speaker tag can be used to provide attribution in meetings-related features like captions, transcripts, and meeting summaries.” Read that as an admission: the moment identity stops arriving with the connection, it has to be manufactured by something else.
With no roster, only a count remains
A browser extension that captures a tab is in a different position, and it is worth being concrete about why. It is not a participant. It has no roster, no display names, no per-person audio streams — only what the tab sends to the speakers, mixed down before it ever leaves the machine. The identity information was dropped at the point of capture, and no amount of modelling gets it back.
The only thing left to do with one track is sort it. The vendor’s own definition is careful about what that means: “Speaker diarization answers the question: Who spoke when?” Not who is this. Each token comes back tagged with a speaker field; group the tokens and you have segments, and the segments are what becomes a label.
In Tablingo the switch is off by default. Turn it on and the connection request carries a diarization flag, tokens arrive tagged, and the overlay prints “Speaker N:” when the id changes. The ids are small integers handed out for the length of one session, capped by the vendor at fifteen speakers, and similar-sounding voices degrade the grouping. They are not saved, they do not survive the session, and nothing in the pipeline maps them to a person.
| Source of the label | Can say | Cannot say |
|---|---|---|
| The account | A name | Two people, one room |
| The audio channel | Who is speaking | Which name that is |
| Clustering | That the voice changed | Who it belonged to |
The middle row is the one people forget. A separate audio channel is itself an identity source — not because it sounds like anyone, but because a channel is owned by whoever opened it. A caption built on per-person channels is reading the ownership, not the voice.
The cost is paid in latency
Sorting a single stream while it is still arriving is harder than sorting it afterwards, and the vendor says so directly. Soniox’s documentation on real-time diarization warns of “higher speaker attribution errors compared to async mode” and of “temporary speaker switches that stabilize as more context is available” — meaning an early guess can be revised, but only by the tokens that come after it.
Then comes the sentence that matters most for anything with a subtitle box on screen: “Endpoint detection and manual finalization force tokens to finalize early, which reduces diarization accuracy. For the highest diarization accuracy, do not use endpoint detection.”
Endpoint detection is what decides a sentence has ended. It is also the reason a caption appears while the person is still talking, instead of after they have stopped. Tablingo turns it on, because a caption delivered after the pause is not a caption. So the setting that makes the text timely is the very setting the vendor names as the one that makes the label less reliable. Timeliness and attribution are not two budgets. They are one budget, spent twice.
There is a second cost, smaller and easier to miss. The diarization flag travels in the connection configuration, and that configuration is sent once, when the socket opens. Flipping the switch mid-conversation does not adjust the running session — it closes it and opens another. A sentence or two leaves with it. Turning labels on is not free, which is part of why it is not the default.
The standards already allow “we don’t know”
After all that machinery, the professional answer to an unidentified speaker is not a guess. It is a number, and the rulebooks say so out loud.
US captioning rules treat speaker identity as information a viewer cannot observe for themselves. Title 47 CFR 79.1(j)(2)(i) requires captions to convey “nonverbal information that is not observable, such as the identity of speakers, the existence of music (whether or not there are also lyrics to be captioned), sound effects, and audience reaction, to the greatest extent possible, given the nature of the program.” Note the company identity keeps in that sentence: it sits in a list with music and sound effects, as one more thing a hearing viewer absorbs without being told.
Netflix’s style guide turns that obligation into a rule about when to say it. “Only use speaker IDs or sound effects when they cannot be visually identified.” Labels go in brackets, in lowercase. When two speakers share a subtitle, a hyphen opens each line, with “a maximum of one speaker per line.” And for a character who has not been named yet: “use [man] or [woman], or [male voice] or [female voice], so as not to provide information that is not yet present in the narrative. If the same identifier is used multiple times in one scene, numbers should be added to distinguish them, for example [man 1].”
[man 1] is a cluster id, written by hand. A subtitler with the picture, the script and the whole story in front of them can still do no better than a generic noun and a count.
One difference remains, and it is the honest one. [man 1] is a placeholder that gets resolved — later in the film the man is named and the label becomes [Henry]. A diarization id has no later. Nothing in the audio turns Speaker 2 into a name, because a name was never in the audio to begin with.
A test that takes a minute
Three checks, no tools, on any two-person conversation you can find.
First, look at the start of each subtitle line rather than the middle. Every line will carry a label even when nobody has changed speaker, because the label hangs on the line, and each new line starts fresh.
Second, wait for the moment two people talk over each other, or laugh at the same time. Two voices arriving as one signal is the case a single-track model has least to work with. Expect a missing label there, or an id that jumps to a new number and then back.
Third, turn the switch off and on again. The session reconnects, and the numbering starts over at 1 — same two people, same conversation, a minute later. Nothing was remembered, because nothing was ever known.
The third check is the one that settles it. A system that recognised people would not need to count them twice.
What the number is for
If a number is not a name, what does it buy? One bit per line: this is a different person from the last line. When the faces are on screen, that bit is redundant — you can already see it, which is precisely why Netflix tells subtitlers not to spend line space on it. When the faces are not on screen, or when two voices in an unfamiliar language sound alike to you, it is the only structure left in the stream.
That is also the honest reason to leave it off until you need it. It costs a reconnect, it inherits the error rate the vendor documents, and on a single speaker it produces nothing but the same two words down the page.
What it does not do is tell you who is talking. It cannot — and the platforms that appear to are not listening either. They are reading a name that arrived with the connection and setting it beside the words. Take the connection away, and what remains of a person inside an audio stream is a number.