How does an app know who's speaking? Inside automatic full-cast narration

To give each character in a book their own voice, you first have to answer a question that sounds trivial and is not: who is saying this line? A human reader answers it without noticing. Software has to be taught, and the teaching is the interesting part. It is also what makes automatic full-cast narration possible at all, so it is worth a close look.

The easy cases, and why they lull you

Some dialogue tells you exactly who is speaking.

“You have been in Afghanistan, I perceive,” said Holmes.

The tag is right there: a speech verb and a name, next to the quote. A simple program can find the quotation marks, look for a nearby “said” plus a proper noun, and get this right most of the time. If every line looked like this, the problem would be solved in an afternoon.

They do not.

Where it gets hard

Real prose is full of dialogue that hides its speaker, and each way of hiding is a different problem.

Unattributed lines. Once a conversation is going, authors drop the tags, because the reader is expected to keep count.

“You have been in Afghanistan.” “How on earth did you know that?” “Never mind. The question is one of the utmost importance.”

Three lines, no names. To assign them you have to track that two people are talking and that they are alternating, then carry that thread until something breaks it. Miscount once and every following line flips to the wrong person.

Pronoun tags. “He said” and “she asked” attribute the line to a person the sentence does not name. You have to know who “he” is right now, which means tracking who was last mentioned and who is even in the room.

Action beats instead of tags. Often there is no speech verb at all, just an action next to the quote.

Holmes leaned back and steepled his fingers. “This is a three-pipe problem.”

Nothing “said” anything. A person infers Holmes is speaking because he is the one acting. Software has to make the same leap.

Naming the cast at all. Before you can assign a line to Holmes, you have to know Holmes exists, and that “Sherlock,” “Mr. Holmes,” “Holmes,” and, in his friend’s narration, “he” can all be the same man. Building that list of characters and their aliases from the text, without a cast sheet, is its own step.

And then there is the ambiguous. First-person narration, letters quoted inside the story, a crowd where five people speak at once, a line an author left deliberately unclear. Sometimes the honest answer is that even a careful human reader would pause.

How the approaches evolved

Early attempts were rule-based: find the quotes, look for “said X” nearby, fall back to alternating speakers. Fast, and brittle exactly where prose gets interesting. Then came statistical models trained on annotated books. They handled context better but still leaned on surface patterns.

What changed recently is that large language models read a passage more like a person does. Given a chunk of a book, a capable model holds the scene in mind: who is present, who spoke last, whose action sits beside the quote. It uses all of that to attribute a line, including the untagged ones. It is not magic and it is not perfect, but it is a big step past matching “said” to the nearest name.

What BookNPC does with it

BookNPC runs this analysis over a book page by page. It works out the cast, resolves the aliases, and attributes each line, including the untagged runs. Then it assigns the narrator and every character a distinct voice. This is the one-time step behind casting a book, and it is why casting has a small cost: the work is in the reading, not the playback.

No attribution system is flawless on hard prose, so the app keeps you in control. As it plays, the current line is tinted in its speaker’s color and the speaker is named under the page, so you can see who it thinks is talking. The full cast list is right there, and you can swap any voice. If it mishears a tricky exchange, the miss is visible and fixable, not baked in silently.

Why this is the hard part, and the valuable one

Recording voices is the easy half of a full-cast audiobook. The expensive, skilled half has always been the direction: deciding who speaks each line, and how. Automatic speaker attribution is software doing that direction. It is now good enough to make a full cast out of any book you bring, not just the handful a studio chose to produce.

That is the machinery under a feature that, from the outside, just sounds like “every character has a voice.” To see it work, cast a dialogue-heavy book in BookNPC and watch the speaker names appear as it reads. The full-cast versus single-narrator guide covers what that does to the listening itself.