LLM APIs and the economics of tokens

An LLM API bill is one number multiplied across two prices: tokens in, tokens out. Get used to that and the bill stops being a surprise. Miss it, and you'll get a $4,000 invoice for a feature you assumed cost twenty bucks.

This is post 6 of 7 in the Running series. Previous posts covered the local-runtime side. This one is the cloud-API side: how token billing actually works, why output is more expensive than input, and how prompt caching changes the math by a factor of ten.

What a token actually is

A token is a chunk of text the model processes as one unit. Roughly:

• 1 token ≈ 4 characters of English.
• 1 token ≈ 0.75 of an English word.
• 1,000 tokens ≈ 750 words ≈ a page of text.
• 1M tokens ≈ a 750-page book.

The exact mapping is the model's tokenizer (BPE, SentencePiece, etc.). Different models tokenize the same text differently. Claude Opus 4.7 ships with a new tokenizer that produces up to 35% more tokens than 4.6 for the same input, meaning the same prompt costs 35% more even though the per-token price didn't change. Always compare like-for-like on the same tokenizer.

For practical purposes:

• A short chat message is 10–50 tokens.
• A typical prompt with system instructions is 500–2,000 tokens.
• A document you're summarizing is usually 5,000–50,000 tokens.
• A full codebase loaded into context is 100,000–1,000,000 tokens.

Input vs output: why output costs more

Every API charges different prices for input and output. Output is always more expensive. This catches everyone the first time.

A representative table for 2026, per million tokens:

Output is more expensive because of how inference works. Input tokens get processed in a single forward pass through the model: the GPU does it all at once, in parallel, in milliseconds. Output tokens are generated one at a time, autoregressively. Each output token requires a fresh forward pass over the entire context. A 1,000-token output runs the model 1,000 times.

The 5x or 6x ratio reflects that asymmetry, not arbitrary vendor pricing.

What this means for your bills

Two patterns surprise people:

Pattern 1: long context, short answers. You stuff 50,000 tokens of documentation into the prompt and ask for a one-line answer. Looks expensive (50K input) but is actually cheap. 50,000 input tokens at $5/M = $0.25. The 50-token answer is negligible.

Pattern 2: short prompts, long answers. You ask a model to write a 5,000-token report from a 200-token brief. The math flips. 200 input + 5,000 output at $5/$25 per M = $0.001 + $0.125 = $0.126. The output dominates. Across 1,000 reports, that's $126.

Code generation, document drafting, agentic loops, all output-heavy. Q&A over a knowledge base is input-heavy. Watch which mode your workload is in.

Prompt caching: the discount nobody applies

The biggest cost lever in 2026 isn't picking a cheaper model. It's caching the parts of your prompt that don't change.

Most production prompts have a long stable header (the system prompt, tool definitions, retrieved documents, few-shot examples), followed by the user's actual message. The header might be 5,000 tokens that never changes between calls. Without caching, you pay full input price for those 5,000 tokens every single call. With caching, you pay full price the first time and 10% of input price on every subsequent call within the cache window (5 minutes by default on Anthropic, longer on others).

The numbers, on Claude Opus 4.7:

• Without cache: 5,000-token header at $5/M = $0.025 per call. Across 10,000 calls: $250.
• With cache: $0.025 once, then 5,000 tokens × $0.50/M = $0.0025 per call. Across 10,000 calls: $25 + $0.025 = $25.

That's a 10x discount on the input portion of your bill. Most production AI apps could shave 50–80% off their total spend by enabling caching on their system prompt. Most don't, because they wrote the integration before caching existed and never went back.

If you're building anything that hits an LLM API more than 100 times a day with a stable header, caching is the first optimization. Not a fancier prompt. Not a smaller model. Caching.

A worked example

A code-review bot. For every PR, it gets:

• A 4,000-token system prompt (rules, style guide, examples). Stable.
• A 2,000-token diff. Changes per call.
• It outputs a 1,500-token review.

100 reviews a day, 30 days a month, on Sonnet 4.6 ($3 input, $15 output, $0.30 cached input).

Without caching:

• Input: (4,000 + 2,000) × 100 × 30 × $3/M = $54.
• Output: 1,500 × 100 × 30 × $15/M = $67.50.
• Total: $121.50/month.

With caching:

• Cached input: 4,000 × 100 × 30 × $0.30/M = $3.60.
• Fresh input: 2,000 × 100 × 30 × $3/M = $18.
• Output: $67.50 (unchanged).
• Total: $89.10/month.

A 27% cut from one configuration change. The percentage gets bigger as the cached portion grows.

Hidden costs

Three things that don't show up in the headline price:

• Tokenizer drift. As mentioned, Opus 4.7's tokenizer produces ~35% more tokens than 4.6 on the same English input. Claims of "Hindi/Devanagari is more expensive" are real: non-Latin scripts often tokenize 2–3x worse than English on the same model.
• Reasoning tokens. Reasoning-mode models (GPT-5.5 high, Claude with extended thinking) generate hidden chain-of-thought tokens before the final answer. Those tokens are billed as output. A "high-reasoning" answer can cost 10x what a normal one does.
• Tool calls. When the model calls a tool, the tool's response goes back into context as input on the next call. An agent doing 20 tool calls in a loop can rack up 50,000+ input tokens for what looked like one user request.

Always log your actual token counts in production. The bill is downstream of those numbers, and the numbers are usually higher than your back-of-envelope estimate.

Picking a model on cost

A working heuristic for 2026:

• High-volume, low-stakes work (classification, summarization, simple chat): Haiku 4.5, GPT-5-mini, or DeepSeek V4. Below $1/M output. Bills stay small.
• Daily driver, real work (coding, drafting, agentic loops): Sonnet 4.6 or GPT-5.4. Around $15/M output. Best price-per-intelligence ratio.
• Frontier reasoning, hard problems: Opus 4.7 or GPT-5.5 high. $25–30/M output. Pay it when it matters; don't pay it for everything.

The trap is using Opus for tasks Haiku could do. The other trap is using Haiku for tasks that need Opus and burning a week debugging the model's mistakes. Neither saves money.

What's next

So far we've talked about the bill. The other thing leaving your machine when you call an API isn't dollars; it's data. The next post is about what providers actually see, log, and do with the tokens you send them.