How the machine writes your page

What actually happens between COMPILE and your finished page — and the three ways we can make it fast.

A model only does one thing

Give it some text, and it predicts the next token — the next little chunk of text (a word, part of a word, a symbol).

That’s the whole trick. To write a page, it predicts one token, sticks it on the end, and predicts again. Over and over, thousands of times. This is called autoregressive generation — each token feeds back in to help pick the next.

How it picks a token

The text runs through the model once (a forward pass). Out comes a probability for every token it knows — then it takes one of the likeliest.

Then it appends “moves” and runs the whole thing again for the next token.

Why one-at-a-time is slow

Every single token means loading all 9 billion of the model’s numbers from memory and running a full pass — just to produce one token.

And it’s strictly in order: token #5 can’t start until #4 exists. A page is thousands of these, one after another.

The sneaky part: loading that giant model is the expensive step, and you pay it in full for each token. The GPU spends most of its time fetching weights, not doing math. That waste is exactly the opening the next two methods exploit.

Checking is cheaper than writing

A model can verify a whole batch of tokens in a single forward pass — all at once, in parallel — even though it can only generate one at a time.

It’s faster to grade a finished answer than to write one from scratch. Both fast methods below are built on this one fact.

Guess ahead, then check

Let something fast guess the next few tokens. The big model then checks all the guesses in one pass and keeps the ones it agrees with.

Think of a fast intern drafting the next few words in pencil, and the expert reading them in a single glance — checking off the right ones and crossing out from the first mistake. Reading is quicker than writing, so the expert gets ahead.

Crucially: the big model still approves every token. Same output as plain — just fewer slow passes to get there.

Accept the streak, stop at the first miss

the draft guessed 5 tokens:

the big model checks them all in one pass:

It accepts the matching streak (the water cycle), rejects at the first mismatch, and adds its own token there (moves). That’s 4 tokens from one pass instead of 1. How often guesses land is the acceptance rate — higher means faster.

A separate draft model

The classic setup: pair a small draft model with the big target. The little one guesses; the big one verifies.

The catch: you now load and run two models, and the draft is a different network — it can guess things the target wouldn’t, and every wrong guess is wasted work.

MTP: the model guesses for itself

Instead of a second model, give the model extra output heads — small add-ons trained to predict token +2, +3 straight from the thinking it already did for the current token.

The heads read the model’s own internal state, so drafting is nearly free — no second model to load — and the guesses agree with the target far more often, because they come from the same brain. MTP = Multi-Token Prediction.

External draft → self-draft

This is where frontier open models are heading (DeepSeek-V3, Qwen 3.5). Not the final word — other approaches like EAGLE-3 are right behind — but the current direction.

We didn’t guess. We measured.

Same model — Qwen3.5-9B — run three ways, on our real job: turn a spec into a full HTML page, with all five build groups generating at once. One RTX 4090, same prompt.

Tokens per second

MTP is nearly 2× plain. And the “clever” separate-draft method? The shortest bar on the chart.

The proven trick made it slower

The separate draft guessed right ~86% of the time — almost as often as MTP’s ~88%. So why was it slower than doing nothing?

Because the little draft model has to run first, every token — and that extra work cost more than it saved. High accuracy couldn’t rescue a second model. MTP’s heads are free; a separate model never is.

All five groups at once

MTP’s lead actually widened under load. Each group generates as if it had the whole GPU to itself.

Newer isn’t automatically better. Measure.

MTP won — but not because it was the newest. It won because on our hardware, our model, and our workload, the numbers said so. The older, “safer” method would have made things worse.

You only ever know which one wins by testing it on the thing you actually do.

That’s the real skill — not picking the newest trick, but measuring it.

Three ways to run one model

muggsofcode runs Qwen3.5-9B with MTP.