What you actually need to understand · 06
The nine ideas behind every number on this page
You do not need to know how a transformer works. You need to know what you are buying.
Tokenthe unit you buy
A chunk of text, usually part of a word. Roughly four characters, or about 0.75 words in English. "Understanding" might be one token. A rare surname might be four. Code and non-English languages break into more tokens per idea, so they cost more to say the same thing.
Input and outputtwo different prices
Everything sent to the model is input. Everything it writes back is output. Output costs four to six times more because generating text takes far more computation than reading it. Long answers, not long questions, are what empty a budget.
Context windowthe model's desk, not its memory
The maximum text a model can hold at once, now commonly one million tokens. It is not memory. It is cleared every request, which is why the whole conversation is re-sent every turn. A bigger window does not make anything cheaper. It makes it possible to spend more.
Reasoning tokensthe draft you never see
Some models write private working-out before answering. It improves hard tasks and is billed as output. On a maths problem it earns its keep. On a tweet it is pure loss.
Prompt cachinga discount for holding still
Providers will store a repeated prefix and re-read it at about a tenth of the input price. Writing it costs about 1.25 times. It pays back after one or two reuses, and it only works if the prefix is byte-identical every time.
Batch processinghalf price for patience
Submit work asynchronously and accept a slower turnaround, and both major providers halve the rate. Made for overnight jobs, bulk classification and anything you will read tomorrow.
Parameters and tierswhy one model costs ten times another
Vendors ship a big model, a middle model and a small one. The small one is not a worse product. It is a different tool. Most work is a screwdriver job, and people keep reaching for the power drill because it is the one on the front page.
Seat versus metertwo ways to be charged
A subscription buys capacity with soft limits. An API key buys exact usage with no ceiling. Subscriptions win for humans typing. API wins for automation you can measure. Copilot's 2026 shift to credits shows the line between them is moving toward the meter.
ROMTIthe only number that decides anything
Return on Model Token Investment. The value produced, minus the time you spent producing it, over the tokens it burned. It is the reason "which model is cheapest" is the wrong question, and "which model finishes the job in one pass" is the right one.
