Why Two “Real-Time” Caption Tools Can Sit Four Seconds Apart
Both of them will call themselves real-time, and neither is lying. The four seconds between them come from four decisions made before the audio ever leaves your browser — and only the smallest of those is the network.
Someone finishes a sentence on the call. You wait. About three seconds later the words appear — and by then they are already saying the next thing, which means you are now reading the past while listening to the present.
The instinct is to blame the connection. It is almost never the connection. Push a caption system onto a gigabit line and the delay barely moves, because the delay is not a bandwidth problem. It is the shape of the pipeline.
"Real-time" is not a specification. Nobody certifies it, no standards body defines how many milliseconds it allows, and both a tool that answers in a third of a second and one that answers in four can print it on the same store listing without either of them lying. Both of our longer guides quote that second number — meetings and podcasts both sit "roughly 2–4 seconds behind the audio" — and neither says what sets it. What sets it is four decisions stacked in order, and only the smallest of them is the network. Change one of those decisions and the same audio, on the same machine, lands in about a third of a second instead of three.
Stage one: waiting for a boundary
Speech recognition does not run on single words. To send anything to a model you first have to decide where a piece of audio starts and where it ends, and that decision is the first and usually the largest source of delay.
Our pipeline cuts at silence. Half a second of quiet means the sentence is probably over, so the chunk closes there. Two more rules sit on top of that:
- A minimum length. A chunk shorter than one second is not worth sending — there is not enough context for a model to be confident about anything. So the cutter waits until at least a second has accumulated.
- A hard ceiling. If someone never stops talking, the chunk cannot grow forever. At fifteen seconds it is cut regardless.
Read those three rules together and an uncomfortable consequence falls out: the half second of silence has to happen after the speaker stops. The captions cannot begin to move until the talking stops. That is why a caption appears to "burst" into existence the instant someone pauses — the pause is the trigger, not a coincidence.
And the corollary, which surprises people: the faster and less pausy the speaker, the later the captions. Someone who talks for twelve seconds without a breath gets no text at all for the first twelve seconds, because the chunk has not closed yet. It is not that the model is slow. It is that nobody told it where the sentence ended.
Stage two: getting the audio up there
We capture at 16 kHz, mono, 16-bit. That is the standard input for speech recognition, and it is not a bandwidth compromise — a 44.1 kHz stereo stream carries nothing extra that helps a transcription model, it only makes the upload bigger.
At that rate the audio is 32 KB per second. A fifteen-second chunk is just under half a megabyte, sent as one request to one domain. One round trip.
This is the smallest of the four stages. If your delay lives mostly here, your network genuinely is the problem — but upgrading the network will not buy back the missing three seconds, because three seconds is not what this stage costs.
Stage three: the model, whole or streaming
Here the pipeline forks, and this fork matters more than everything else in this article.
Whisper — the engine behind our standard mode — is not a streaming model. A whole chunk goes in, one finished block of text comes out. It gives you nothing until the chunk closes. So its floor is stage one's wait plus inference, and no amount of engineering removes the wait, because the wait is the input.
A streaming engine — what our real-time mode uses — is fed continuously. Audio goes up in micro-frames of about 296 milliseconds and tokens come back in batches. It does not need a sentence to finish before it starts producing words. That is a difference of an order of magnitude, not a tuning difference: fifteen seconds of waiting against roughly a third of a second.
Most of what people describe as "caption lag" is really this fork, and it is where the four seconds in the title come from. One tool closes a chunk and sends it; the other keeps a stream open and renders tokens as the model firms up. Both are real-time transcription by any honest reading of the phrase. They are not in the same speed class, and the difference never shows up on a spec sheet — it shows up the moment somebody talks for ten seconds without breathing.
Which is also why this is the only decision worth choosing by. Stages one, two and four are tuning: cut a little earlier, upload a little less, translate a little sooner. Each buys a fraction of a second. Stage three changes the unit of the wait — from the length of a sentence to the length of a syllable.
Stage four: and then the translation
In the standard path, translation is a second request. The transcript comes back, we post that text to a translation model, and the translated line renders when it is done. Two round trips, two models, one after the other.
One detail worth being honest about: that translation stream is delivered incrementally, but we hold it back and swap in the finished sentence rather than letting half-translated text wobble on screen. A translation that rewrites itself three times is harder to read than one that arrives a beat late. That is a deliberate choice, and it costs a few hundred milliseconds.
The real-time path has no second hop. Transcription and translation come out of the same upstream stream, translated tokens alongside original ones, so there is no queue to join.
What the four stages add up to
| Stage | What it waits for | Order of magnitude |
|---|---|---|
| Boundary | Half a second of silence after the speaker stops; fifteen seconds if they never do | 0.5 s, up to 15 s |
| Upload | One request, one round trip | Tens to a few hundred ms |
| Recognition | The whole chunk, on a non-streaming model | A few hundred ms |
| Translation | A second request, whole sentence | A few hundred ms |
Those are orders of magnitude, not promises. Accents, language pairs, network weather and how the speaker breathes all move them around. But the shape holds: one stage is measured in seconds, three are measured in fractions of one. That is why the single most effective thing anyone can change is the engine, and it is why "we have fast servers" is not an answer to this question.
Four checks that separate the two
You do not have to read a spec sheet, or trust a comparison table, to find out which side of the fork a tool sits on. A few minutes with any video does it.
- Say one long sentence without pausing. If ten seconds pass with no text at all, the tool is waiting for a boundary. If words start arriving before you finish, it is not.
- Watch the instant you stop. Text that appears as a whole paragraph was assembled from a closed chunk. Text that had already started, and merely settled, came out of a stream.
- Watch whether words get replaced. A streaming engine publishes provisional tokens and revises them, so an occasional word changes under your eyes. A tool that never corrects itself is either very accurate or not streaming at all.
- Count the hops to translation. If the original line lands and the translation follows a beat later, that is a second request. If both arrive together, they came out of one stream.
Run those four and the label stops mattering, because you have measured the thing the label was standing in for. It is also why we describe our own two modes differently instead of calling both of them instant: the real-time path sits on the streaming side — micro-frames of about 296 ms, tokens revised as the model firms up — and the standard path sits on the other, because Whisper is not a streaming model and no amount of tuning makes it one. Same product, two honest speeds, and now a way to tell which one you are getting.
Zero delay is not a feature, it's a trade
Every stage can be squeezed. None can be squeezed to nothing, and each squeeze is paid for with one of two currencies.
Accuracy. Recognition needs context. In the first two hundred milliseconds, "I scream" and "ice cream" are the same sound. The longer the system listens before committing, the more certain it is. Cut the wait and you do not get the same answer sooner; you get a worse answer sooner.
Stability. The only way to beat the boundary wait is to publish guesses and correct them. Our streaming path does exactly this: provisional tokens land in a buffer, get replaced when the model firms up, and the line you are reading can change under you. Done sparingly it feels instant. Done aggressively it feels broken, and readers would rather wait.
So a product advertising zero delay is usually doing the second thing and not saying so. The honest version is to name your side of the trade and show the number — which is exactly where the two engines inside our own pipeline diverge, and why "real-time" is a claim worth checking rather than a spec you can trust.
The comparison that reframes it: broadcast captions are slower
Broadcast captioning targets have historically run from three seconds to "under ten," with human stenography averaging over four. Against that, two to four seconds is unremarkable. The comparison is worth making because it reframes the problem: this is not a category that is failing to hit zero, it is a category where nobody has ever hit zero.
But there is a sharper lesson in how broadcast systems hide their delay.
A live stream has a buffer. The video already arrives at the viewer several seconds late, so caption processing can be absorbed into that gap and come out looking perfectly in sync. The delay never disappeared. It moved somewhere nobody notices.
A meeting has no such buffer. The video has to be live, because the other person is waiting for your reply — you cannot slow their face down by five seconds to give the caption time to catch up. A recorded video has no buffer either: the picture is already sitting there, so the only thing that can wait is the text.
That is the whole difference. The same two to four seconds of processing reads as "in sync" on a broadcast and "half a beat behind" in a call, not because the technology differs, but because one of them had somewhere to put the latency and the other did not.
The same two seconds weighs differently
| Setting | What the delay costs | What actually helps |
|---|---|---|
| Recorded podcast | Nothing — you can pause and rewind | Nothing |
| Online course | Little — lecturers repeat themselves | Pause and re-read |
| Live stream | Some — chat is already ahead of you | Watch chat for reaction, captions for content |
| Meeting | A lot — someone is waiting for your answer | Read keywords, not whole lines |
| Voice room | The worst — three people have moved on by the time you read | Its own problem, its own article |
This table is the reason the same tool can feel excellent in one place and useless in another. The pipeline is identical. The budget is not.
Using the delay instead of fighting it
In a meeting: stop reading every line. Captions in a call are for reconnecting when you lose the thread, not for following along word by word — the audio is still faster than your eyes plus the delay. Glance when you drop, look up when you catch. And say it out loud once at the start ("I'm reading captions, so if I don't react, please repeat") — that sentence prevents most of the misunderstandings a delay would otherwise cause.
For the things that matter, get them in writing. Numbers, names, deadlines and prices are exactly what a model is most likely to mishear, and exactly what a chat box will get right. Ask for those in the chat rather than trusting the caption line.
In a podcast: nothing. Drag back ten seconds and the captions come with you. Single-direction audio with a rewind button is the one setting where two to four seconds is not a cost at all, which is why the podcast case reads so differently from the meeting one.
Ask the speaker to breathe. Genuinely the most overlooked lever. A pause closes a chunk, a closed chunk produces text. Nobody thinks of pausing as a caption feature, and it is a larger factor than most settings.
Two things no tool can sell you
You cannot remove the last few hundred milliseconds, and you should be suspicious of anything that claims to. You cannot have captions ahead of the sound — a transcript that precedes the speech is a prediction, and predictions about what someone is about to say are wrong often enough to be worse than a delay.
And you cannot treat a caption line as a transcript. On a meeting it is a moving stream that is gone when you close the tab; on a recording it is a reading aid, not a document. If you need the record, take the record — a caption overlay is the wrong tool for that job, which is a separate article on its own.
Bottom line
Two tools can share a label and still sit four seconds apart, because the label describes an intention rather than a measurement. What you can measure is four decisions, in order: where the audio gets cut, how it is sent, whether the model streams, and whether translation is a second hop. Swap the third and nothing else has to change — same network, same machine, a third of a second instead of three.
That is the whole reason the two engines in our own pipeline behave differently, and the reason we name the mode you are in instead of selling one word for both. "Real-time" is worth verifying rather than accepting — and the four checks above are all it takes, on a video you already have open.
If you want to try ours, it's called Tablingo; the real-time mode is the streaming side of the fork described above. The meeting case is written up here, the podcast case here, and how the two capture strategies differ here.