LLM API bills, and why a token costs what it costs

Someone I know shipped a tidy little summarizer feature and assumed it would cost about twenty bucks a month. The invoice came in at $4,000. Nothing was broken. The code did exactly what it was told. He just hadn't understood that an LLM API bill is one number multiplied across two prices: tokens in, tokens out. Get used to that and the bill stops surprising you.

This is post 6 of 7 in the AI Running series. The earlier posts were about the local-runtime side, the model on your own machine. This one is the cloud-API side: how token billing actually works, why output costs more than input, and how prompt caching changes the math by a factor of ten.

What a token actually is

A token is a chunk of text the model processes as one unit. Roughly:

• 1 token ≈ 4 characters of English.
• 1 token ≈ 0.75 of an English word.
• 1,000 tokens ≈ 750 words ≈ a page of text.
• 1M tokens ≈ a 750-page book.

The exact mapping is the model's tokenizer (BPE, SentencePiece, etc.). Different models slice the same text differently. Claude Opus 4.7 ships with a new tokenizer that produces up to 35% more tokens than 4.6 on the same input. So the same prompt costs 35% more even though the per-token price didn't budge. Always compare like-for-like on the same tokenizer.

For a rough feel:

• A short chat message is 10 to 50 tokens.
• A typical prompt with system instructions is 500 to 2,000 tokens.
• A document you're summarizing is usually 5,000 to 50,000 tokens.
• A full codebase loaded into context is 100,000 to 1,000,000 tokens.

Input vs output, and why output stings

Every API charges different prices for input and output. Output is always the pricier side. This catches everyone the first time.

A representative table for 2026, per million tokens:

The reason sits inside how inference works. Input tokens get processed in a single forward pass. The GPU chews through all of them at once, in parallel, in milliseconds. Output tokens come out one at a time, autoregressively. Each new output token needs a fresh forward pass over the entire context so far. A 1,000-token output runs the model 1,000 times.

That 5x or 6x ratio isn't a vendor being greedy. It's that asymmetry showing up in the price.

What this does to your bills

Two patterns trip people up.

Long context, short answers. You stuff 50,000 tokens of documentation into the prompt and ask for a one-line answer. Looks expensive because of all that input, but it's actually cheap. 50,000 input tokens at $5/M is $0.25. The 50-token answer is rounding error.

Short prompts, long answers. You hand a model a 200-token brief and ask for a 5,000-token report. Now the math flips. 200 input plus 5,000 output at $5/$25 per M is $0.001 + $0.125 = $0.126. The output runs the show. Do that 1,000 times and you're at $126.

Code generation, document drafting, agentic loops: all output-heavy. Q&A over a knowledge base is input-heavy. Always know which mode your workload sits in.

Prompt caching, the discount nobody turns on

The biggest cost lever in 2026 isn't picking a cheaper model. It's caching the parts of your prompt that never change.

Most production prompts open with a long stable header: the system prompt, tool definitions, retrieved documents, few-shot examples. Then the user's actual message comes at the end. That header might be 5,000 tokens that stay identical call after call. Without caching, you pay full input price for those 5,000 tokens every single time. With caching, you pay full price once, then 10% of input price on every later call inside the cache window (5 minutes by default on Anthropic, longer on others).

Run the numbers on Claude Opus 4.7:

• Without cache: a 5,000-token header at $5/M is $0.025 per call. Across 10,000 calls, $250.
• With cache: $0.025 the first time, then 5,000 tokens × $0.50/M = $0.0025 per call. Across 10,000 calls, $25 + $0.025 = $25.

A 10x cut on the input portion of your bill. Most production AI apps could shave 50 to 80% off their total spend just by caching the system prompt. Most don't. They wrote the integration before caching existed and never circled back.

If you're building anything that hits an LLM API more than 100 times a day with a stable header, caching is the first thing to reach for. Not a cleverer prompt. Not a smaller model. Caching.

A worked example

Say you've got a code-review bot. For every PR it gets:

• A 4,000-token system prompt (rules, style guide, examples). Stable.
• A 2,000-token diff. Changes per call.
• It outputs a 1,500-token review.

100 reviews a day, 30 days a month, on Sonnet 4.6 ($3 input, $15 output, $0.30 cached input).

Without caching:

• Input: (4,000 + 2,000) × 100 × 30 × $3/M = $54.
• Output: 1,500 × 100 × 30 × $15/M = $67.50.
• Total: $121.50/month.

With caching:

• Cached input: 4,000 × 100 × 30 × $0.30/M = $3.60.
• Fresh input: 2,000 × 100 × 30 × $3/M = $18.
• Output: $67.50 (unchanged).
• Total: $89.10/month.

A 27% cut from one config change. And the percentage climbs as the cached chunk grows.

The costs hiding off the price sheet

Three things that don't show up in the headline number.

Tokenizer drift. Like I said up top, Opus 4.7's tokenizer produces about 35% more tokens than 4.6 on the same English input. And the claim that "Hindi and Devanagari cost more" is real: non-Latin scripts often tokenize 2 to 3x worse than English on the same model.

Reasoning tokens. Reasoning-mode models (GPT-5.5 high, Claude with extended thinking) generate hidden chain-of-thought tokens before the final answer. Those get billed as output. A "high-reasoning" answer can cost 10x what a normal one does.

Tool calls. When the model calls a tool, the tool's response goes back into context as input on the next call. An agent grinding through 20 tool calls in a loop can rack up 50,000+ input tokens for what looked like one little user request.

So log your real token counts in production. The bill is downstream of those numbers, and the numbers are almost always higher than your back-of-envelope guess.

Picking a model on cost

A working heuristic for 2026:

• High-volume, low-stakes work (classification, summarization, simple chat): Haiku 4.5, GPT-5-mini, or DeepSeek V4. Below $1/M output. Bills stay small.
• Daily driver, real work (coding, drafting, agentic loops): Sonnet 4.6 or GPT-5.4. Around $15/M output. Best price-per-intelligence ratio.
• Frontier reasoning, hard problems: Opus 4.7 or GPT-5.5 high. $25 to $30/M output. Pay it when it matters. Don't pay it for everything.

One trap is using Opus for tasks Haiku could handle. The other is using Haiku for tasks that need Opus, then burning a week debugging the model's mistakes. Neither one saves you money.

What's next

So far this has all been about the bill. But dollars aren't the only thing leaving your machine when you call an API. Your data goes too. The next post is about what providers actually see, log, and do with the tokens you send them. That $4,000 invoice was the easy problem to spot.