What Are Tokens? LLM Token Costs Explained for Non-Engineers

Language models read text in chunks called tokens. A token sits somewhere between a character and a word: common short words like 'the' are one token, longer words split into two or three, and punctuation usually gets its own.

For ordinary English prose the working rule is about four characters per token, or roughly 0.75 words per token. A thousand words comes to around 1,300 tokens. That ratio is stable enough to budget with.

It stops holding for anything unusual. Code runs denser because brackets, semicolons and underscores each tend to be separate tokens. Non-English text costs more, sometimes dramatically — languages that do not use the Latin alphabet can run several times the token count for the same meaning. Long strings of digits, hashes and base64 are the worst case.

APIs charge per million tokens, and they charge separately for what you send and what comes back. Output is usually billed at three to five times the input rate, because generating text costs the provider more than reading it.

So a single call has two costs. The prompt you send — including the system message, any examples, and any document you pasted in — is billed at the input rate. The model's reply is billed at the output rate.

At small volumes this is invisible. At scale it is the whole budget. A feature handling ten thousand calls a day with a bloated system prompt can cost several times what the same feature costs after someone spends an afternoon trimming it.

Conversation history is the big one. In a chat interface, every turn typically resends the entire conversation so the model has context. Turn ten costs far more than turn one, because you are paying to re-read everything that came before it. Long chats grow quadratically in cost, not linearly.

System prompts are the quiet one. A carefully engineered instruction block might run 2,000 tokens, and it is sent with every single request. That is fine for a hundred calls and expensive for a hundred thousand.

Pasted documents are the obvious one, but people still underestimate them. A 20-page PDF is roughly 10,000 tokens before the model has produced a single word of reply.

Retries are the invisible one. Failed calls, timeouts and automatic retries are billed like any other request.

Separate from cost, every model caps how many tokens it will consider at once. That cap covers your prompt and its reply together, and it is called the context window.

Windows in common use range from 8,000 tokens at the small end to a million at the large end, with 128,000 and 200,000 being the most typical for current general-purpose models.

Exceeding the window is worse than it sounds. Some setups return a clear error, but plenty silently truncate the input instead — the model answers based on the part of your document it could see, and nothing tells you the rest was dropped. If an AI feature gives confident answers that ignore the end of long inputs, this is usually why.

Measure before you optimise. Estimate the token count of your system prompt, a typical user message and a typical reply, then multiply by expected volume. Most teams discover the cost sits somewhere they did not expect.

Trim the system prompt hardest, because it is paid on every call. Examples are usually the biggest chunk and often the most redundant — three good ones typically beat ten mediocre ones.

Cap output length explicitly. Output is the expensive side, and models will happily produce four paragraphs where one would do unless told otherwise.

Truncate or summarise conversation history rather than resending it whole. Keep the last few turns plus a short running summary.

Send the relevant excerpt rather than the entire document. Retrieving the three paragraphs that matter is both cheaper and usually produces a better answer than pasting fifty pages.

The four-characters-per-token rule gets you within about fifteen per cent for English prose, which is close enough for budgeting and for checking a document will fit a context window.

For an exact count you need the specific tokeniser the provider publishes for that model, and the number still varies by a few per cent between providers because each uses a different one.

For most decisions — will this fit, roughly what will this cost, is my system prompt too long — the approximation is all you need, and it takes seconds rather than an engineering task.