Why your coding agent's bill grows faster than the chat

Part 1 of 2 on prompt caching.

I got curious about a number I had never actually checked: what does a long session with a coding agent really cost, and does the cost scale the way I assume it does?

My assumption was linear. Longer conversation, proportionally larger bill. That turns out to be comprehensively wrong — and wrong in a direction that gets worse the more useful the session is.

So I spent an evening building a cost model to find out how wrong. The headline gap is large. But the two things that surprised me were smaller and more specific, and both cut against advice I had been following without checking.

Prompt caching charges a reduced rate for input tokens the provider has already processed. Because agents resend the entire transcript every turn, uncached input cost grows quadratically with turn count — my model bills 13.5 million tokens for an 80-turn session whose final context is only 330,000. Caching cut that bill by 87.2%.

What prompt caching is not

Clear this up first, because the name actively misleads.

If you've cached a database query, you have a mental model: request goes out, result comes back, result gets stored, next identical request is served from the store without touching the database. Caching the output.

Prompt caching is not that. Nobody is storing the model's answers and replaying them — that would be useless, since you rarely send the identical prompt twice and you generally don't want a canned reply when you do.

Prompt caching caches the input. Specifically, it caches the processed state of a prefix of your prompt. When the provider sees a request beginning with tokens it has recently processed, it can skip re-processing that prefix and charge you a fraction of the normal rate for it.

That single distinction explains everything else, including every way to break it: the cache is a prefix match. Not a similarity match, not a semantic match. A prefix.

Why does an agent resend the whole conversation?

Language models are stateless. The model has no memory of your last message; the only thing it knows is what's in the request in front of it.

So an agent works by appending and resending:

turn 1  →  system + tools + user₁                          →  assistant₁
turn 2  →  system + tools + user₁ + assistant₁ + user₂      →  assistant₂
turn 3  →  system + tools + user₁ + assistant₁ + user₂ + …  →  assistant₃

Every turn carries the whole conversation. Some APIs hide this — pass a conversation ID and the model appears to remember — but behind the abstraction the full transcript is being re-processed on every call. Convenience, not a different mechanism.

Now count what you're billed for. The transcript grows by a fixed amount each turn, but you pay for the whole transcript, every turn. Total input billed is the sum of every transcript length so far — an arithmetic series, which grows with the square of the turn count.

The arithmetic

I modelled a fairly ordinary session: a 10,000-token system prompt with tool definitions, the user adding 1,000 tokens per turn, the assistant returning 3,000. Base input price $4.00 per million tokens, which is roughly the frontier tier.

The last row is the one to sit with. An 80-turn session — an afternoon's work, not an epic — ends with 330,000 tokens of context and bills you for 13.5 million. You paid for the transcript forty-one times over.

Note also that this is not exponential, despite how it gets described. It's quadratic: double the turns and the cost roughly quadruples. Which is quite bad enough, and worth stating correctly, because quadratic growth is predictable and therefore plannable in a way exponential growth isn't.

How much does prompt caching actually save?

Providers price cached input differently. Taking Anthropic's published structure, which is the clearest of the majors:

Re-running the same sessions, with each turn reading the previous request from cache and writing only the delta:

$54.08 becomes $6.91. Same conversation, same tokens, same model. The only difference is whether the provider is permitted to recognise a prefix it already processed.

The right-hand column is the one I'd put on a dashboard. Your effective input price on an 80-turn session is $0.51 per million against a list price of $4.00 — you are paying 13% of rate card. Conversely, if your effective rate is sitting near list price, your cache is broken and that column is how you'd know.

The wrinkle nobody mentions

Look at the first row again. A single-turn request costs 25% more with caching on.

That's the write premium. A cache write costs 1.25× base, so if you write a prefix and never read it, you've simply paid a quarter more for nothing. One-shot classification calls, single-turn completions, stateless enrichment jobs — for those, caching is a straight loss.

How many re-reads before it pays?

Break-even lands at under a third of a single re-read. One follow-up message and you're past it forever.

So the practical rule has two halves, and most advice only gives you the first: caching on for anything conversational; caching off for genuinely one-shot calls. If you run a high-volume single-turn endpoint with caching enabled by default, you're paying a 25% tax to populate a cache nobody reads.

Is the 1-hour TTL worth 2×?

The second wrinkle. Anthropic offers a longer TTL at double the write cost, and the obvious assumption is that longer is better for long sessions. Not necessarily.

I modelled 80 turns with idle gaps that let a 5-minute cache go cold, forcing a full rewrite of the transcript so far:

On an uninterrupted session the 1-hour TTL is a flat 2× markup on every write in exchange for nothing. It only starts paying at around three cold starts — which is to say, it's a bet on your own working rhythm, not on session length.

If you code in long uninterrupted blocks, stay on the default. If you're the kind of person who leaves a session open across lunch and a couple of meetings, buy the hour. Nobody can tell you which you are except your own cache-hit graph.

Does your provider cache by default?

The single highest-leverage thing here, because the answer varies and the failure is silent.

• OpenAI caches eligible input automatically — no code change.

• Anthropic and Gemini do not. You must explicitly mark cache breakpoints in the request. Miss it and you pay full freight on every turn while everything appears to work perfectly.

• Responses-style APIs that manage conversation state generally handle it for you. Raw chat-completions calls frequently don't.

• Router services add a wrinkle: a warm cache lives on the machine that built it, so if your requests get load-balanced across backends, you can miss a cache that technically exists. Pinning to one provider is worth real money.

There is no error message for "you forgot to enable caching." Your agent works. It's just quietly costing you seven times more than it should.

So what do you actually do about it?

Enabling caching is the first lever and by far the largest. But the table in section three has a second lesson that survives even a perfect cache: the multiplier is driven by turn count, not by context size. An 80-turn session bills 41× its final context. Two 40-turn sessions covering the same ground bill about 21× each — meaningfully less total, for the same work.

That reframes a few habits.

Start a new session when the topic changes. The instinct to keep one long-running thread going because "it has all the context" is expensive, and most of that context is irrelevant to the current question. A fresh session is not a loss of state; it's dropping ballast you were paying to carry on every turn.

Prefer subagents to a single growing thread. Delegating a self-contained task to a subagent with its own short context means that work gets billed against a small transcript instead of your main one. The parent thread receives a summary rather than the full working transcript — which is the same trick as compaction, applied before the cost is incurred rather than after.

Keep the system prompt lean, but not for the reason you think. A 10,000-token system prompt is not expensive in itself — it caches beautifully and after the first turn costs a tenth of rate card. It's expensive when it changes, which is part 2's subject. Optimise it for stability first and length second.

Watch the ratio, not the token count. Billed tokens divided by final context size is the single number that tells you whether your session shape is sane. Under 10× is comfortable. Past 40× you are paying for the same tokens forty times, and no amount of caching makes that a good idea — it just makes it survivable.

TL;DR

• Uncached agent cost is quadratic in turn count, because you re-send the entire transcript every turn. My 80-turn model bills 13.5M input tokens for a 330K-token final context — a 41× multiplier.

• Caching cut that from $54.08 to $6.91, an 87.2% saving, dropping the effective input price from $4.00/M to $0.51/M.

• Caching costs 25% more on a single-turn call, because a cache write is 1.25× base. Break-even is under one re-read — so enable it for conversations, disable it for one-shot endpoints.

• The 1-hour TTL is a 2× write markup that only pays after roughly three cold starts. On uninterrupted sessions it is pure loss.

• Anthropic and Gemini do not cache by default. There is no warning when you forget.

Share your thoughts in the comments — I’d love to hear how this technology is impacting your industry.

👉 Be sure to press the like button and follow me. It would be a great motivation for me.

👉 Follow me: LinkedIn | GitHub

Reproduce this

Every figure above is arithmetic on stated assumptions, not a measurement of any provider's billing.

python 3.13 · stdlib only
system prompt 10,000 tok · user +1,000/turn · assistant +3,000/turn
base input $4.00/M · write 1.25x (5-min) / 2.00x (1-hour) · read 0.10x

Swap in your own token counts and price and the shape holds; only the magnitudes move.

Next: five ways to invalidate your prompt cache — including the one-line change that cost 7.8× on an 80-turn session.

References

• Anthropic — Prompt caching — the source for the pricing multipliers used above: cache reads at 0.10x base, writes at 1.25x for the 5-minute TTL and 2.00x for the 1-hour tier.

• OpenAI — Prompt caching — worth reading alongside it, because the defaults differ: eligible prefixes here are cached automatically, with no request-side markers.

• Kwon et al., Efficient Memory Management for Large Language Model Serving with PagedAttention, SOSP 2023 — the systems work underneath all of this. Prompt caching is a billing surface on top of KV-cache reuse; PagedAttention is what made sharing those blocks across requests practical in the first place.