How to Control the Cost of an AI Agent: Budgets, Limits and Approvals

The short answer

Agent cost grows faster than the number of turns because the model is stateless: every turn re-sends the system prompt, the tool definitions and the entire history so far. Bound the run with a turn limit and a hard budget checked before every call. Then reduce what each turn carries: trim tool results, scope the context, and cache the stable prefix. Put approval gates in front of expensive or risky actions, and send easy steps to a cheaper model. Measure tokens per turn so you know which control matters for your agent.

Why agent cost is quadratic-ish

A chat model does not remember the previous call. To continue a conversation, you send it again. In an agent loop, turn one sends the system prompt, the tools and the task. Turn two sends all of that plus the model's first reply and the first tool result. Turn ten sends all of turns one to nine.

Here is an illustrative example with round numbers, not measurements. Say the fixed part (system prompt plus tool definitions) is 2,000 tokens and each turn adds 1,000 tokens of reply and tool result. Turn one sends about 3,000 input tokens. Turn ten sends about 12,000. Across ten turns, the total input is the sum of a growing series, around 75,000 tokens. Double the run to twenty turns and the total is not double. It is roughly 250,000, more than three times as much, because the later turns are each carrying the long tail of the earlier ones.

That is why a run that goes in circles is so expensive. The extra turns are not just extra calls. They are the most expensive calls in the run. It is also why a turn count alone tells you little about cost: ten turns with small tool results and ten turns with large ones are very different bills.

Where the tokens actually go

When I log a real agent, input tokens almost always dominate, and they come from four places.

Tool results. The biggest and most variable source. A search tool that returns twenty full documents, a database query that returns every column, a file read of a whole file when the agent needed one function. Each result is sent once when it arrives and then again on every later turn.

Retained history. Earlier replies, earlier tool calls and earlier results that are no longer needed but are still in the message array. This is the part that makes cost grow faster than turns.

The system prompt and tool definitions. Fixed per turn, and easy to forget. A long system prompt with many examples, plus detailed schemas for many tools, is paid on every single call. This is the part caching helps most.

Output and reasoning. Usually smaller in volume, but output tokens are commonly priced higher than input tokens, and models that reason before answering may produce many tokens you never display. Asking for long explanations the code will never read is a cost too.

To see the sizes for your own prompts and tool results, paste them into the token counter. Seeing that one tool result is larger than your whole system prompt is often the moment the fix becomes obvious.

The controls, in order of leverage

I add these roughly in this order. The first two bound the worst case. The rest lower the typical case.

1. A turn limit

The simplest backstop. Pick a maximum number of model calls per run and stop when you reach it. Set it from data: look at how many turns successful runs take, and set the limit comfortably above that. A run that needs three times the usual turns is usually lost, not working hard.

2. A token or cost budget with a hard stop

The limit that actually bounds spend. Before every model call, estimate the input for that call, add it to what the run has used, and stop if it would exceed the budget. After every call, record the real usage the provider reports. "Hard" means the check is in code and the run ends. It is not a warning in a log.

Here is the guard I use, reduced to its core:

// budget.ts - a hard budget checked before every model call.
type Usage = { inputTokens: number; outputTokens: number };

export class BudgetExceeded extends Error {}

export function createBudget(maxTokens: number) {
  let used = 0;
  return {
    /** Call before each model request with an estimate of its input size. */
    check(estimatedInput: number, maxOutput: number): void {
      if (used + estimatedInput + maxOutput > maxTokens) {
        throw new BudgetExceeded(
          `Budget of ${maxTokens} tokens would be exceeded (used ${used}).`,
        );
      }
    },
    /** Call after each response with the usage the provider reported. */
    record(usage: Usage): void {
      used += usage.inputTokens + usage.outputTokens;
    },
    get used(): number {
      return used;
    },
  };
}

Two details matter. The check includes the maximum output you allow for the call, so a call cannot overshoot the budget by producing a long reply. And the record uses the provider's reported usage, not your estimate, so errors in the estimate do not accumulate. If you prefer a money budget, multiply tokens by a per-model price map you maintain yourself. In the 200-line harness the loop catches the stop and returns a clear "budget exceeded" result instead of crashing.

3. A repeated-call guard

Models sometimes call the same tool with the same arguments again and again, usually because the result did not contain what they expected. Each repeat is a full, expensive turn. Keep a key of tool name plus serialized arguments, and when the same call repeats, stop the run or return a tool result that says plainly "you already called this, the answer was the same". This one guard removes a whole class of runaway runs.

4. Truncating or summarising tool results

Since tool results are paid on every later turn, make them small at the source. Return only the fields the agent needs. Cap result length and say in the result that it was truncated, so the model can ask for more. Paginate searches. For large documents, return a summary or the relevant section, not the whole text. This is usually the biggest single saving in the typical case, and it often improves answers too, because the model has less noise to read.

5. Scoped context

Not every past message needs to stay. Once a sub-task is done, replace its back-and-forth with a short note of the outcome. Drop old tool results the agent has already acted on. Give each tool only when the step needs it, rather than sending every tool definition every turn. Be careful: removing context the model still needs leads to repeated work, which costs more than it saves. Summarise, do not just delete.

6. Caching the stable prefix

Many providers can cache the processing of a prompt prefix that is identical across calls, and charge less for the cached part on later calls. The mechanism rewards a simple discipline: put everything that does not change at the start (system prompt, tool definitions, fixed reference material) and everything that changes at the end. Anything that varies near the start, such as a timestamp in the system prompt, breaks the cache for everything after it. Details such as how long a cache lives and whether you must mark cache points explicitly vary by provider, so check your provider's documentation. Caching lowers the cost of the fixed part. It does nothing for output tokens or for the new tail of each turn, which is why it comes after the limits, not instead of them.

7. Approval gates for expensive or risky actions

Some actions cost money directly (a purchase, a paid API call, a large batch job) or are expensive to undo (sending email, writing to production data). Mark those tools as needing approval, and have the harness pause and ask a person before running them. The gate belongs in code, not in the prompt, so it holds even when the model is confident. Approvals also bound cost indirectly: a person who sees the agent about to start its fourth large job in one run will stop it.

8. Model routing by step difficulty

Not every step needs your most capable model. Classifying input, extracting fields, picking which tool to call next from a short list, or summarising a tool result are often handled well by a smaller, cheaper model. Keep the strong model for planning and for the final answer. Route by step type in the harness, measure quality on both paths, and only keep the routing where the cheaper model holds up. Routing done blindly saves money on steps and loses it on retries.

Measure before you tune

Every one of these controls is easier to apply when you can see the numbers. For every model call, log:

• run id and turn number
• model used
• input tokens and output tokens as reported by the provider
• cached input tokens, if the provider reports them
• the size of each tool result added that turn
• the outcome of the run when it ends: success, failure, budget stop, turn limit

With that log you can answer the useful questions directly. Which turn is the most expensive in a typical run? Which tool produces the largest results? How much of each call is cached? What do failed runs cost compared with successful ones? Usually one cause dominates, and the right control is the one aimed at it.

To explore the shape before you have data, the agent run budget simulator lets you vary turns, prompt size and tool result size and see how the total grows. The LLM cost calculator turns token counts into money for a given set of prices.

How to set a budget

Do not pick a budget by feel. Work backwards from what a successful run is worth.

Here is an illustrative example with round numbers. Say a successful run saves a support agent ten minutes, and you value that at 5 units of money. Say 80% of runs succeed. Each run is then worth about 4 units on average. If you want the agent to be clearly worth it, you might cap the cost of a run at a tenth of that, 0.4 units. That is your ceiling, the number above which the agent is losing money no matter how clever it is.

Then measure. Run the agent on a representative set of tasks with the budget set at the ceiling and record the cost of each run. Set the working budget a little above the cost of a typical successful run, well under the ceiling. Runs that hit it are not failures of the budget. They are the runs worth reading, because they show you which task shape makes the agent wander.

Revisit the budget when you change the model, the tools or the prompt. Each of those changes the cost of a typical run.

My verdict

If you add only two things to an agent, add a turn limit and a hard budget checked before every call. They turn an open-ended cost into a bounded one. Then log tokens per turn, find the biggest source, which is usually tool results, and shrink it at the source. Caching, scoped context, approvals and routing all help, but they are tuning. The limits are the safety. Build the limits first, and a bad run becomes a log entry you learn from instead of a bill you explain.

FAQ

Why does an AI agent cost more than a single prompt?

Because each turn re-sends the system prompt, the tool definitions and the full conversation so far, including every earlier tool result. Input grows with each turn, so the total tokens across a run grow roughly with the square of the number of turns, not linearly.

What is the single most effective cost control for an agent?

A hard budget checked before every model call, in tokens or in money, that stops the run when it would be exceeded. A turn limit is a useful backstop, but turns vary a lot in size, so the budget is the limit that actually bounds spend.

Does prompt caching make agent cost a solved problem?

No. Caching lets a provider reuse the processing of an identical prompt prefix, which reduces cost and latency for the stable part of the prompt, such as the system prompt and tool definitions. The growing tail of the conversation still changes every turn, and output tokens are not cached, so you still need limits.

How do I pick a budget number for an agent run?

Work backwards from what a successful run is worth. Estimate the value of one success and the share of runs that succeed, then cap the cost per run well below value times success rate. Then measure real runs, set the budget a little above the cost of a typical successful run, and investigate anything that hits it.

Should an agent ask for approval before spending money?

Yes, for actions that are expensive or hard to undo: sending messages, writing data, making purchases, or starting an unusually large job. Reads can run freely. The gate should live in the harness, not in the prompt, so the model cannot talk its way past it.

How do I know where my agent's tokens are going?

Log input and output tokens for every model call, tagged with the run id and turn number, and log the size of each tool result. Within a few runs you will usually see one cause dominate, most often a large tool result that stays in the history for every later turn.

Related reading

• Your AI Agent Needs No Framework: A 200-Line TypeScript Harness - the loop where these limits live.
• Zod as a Contract: Type-Safe AI Output in Production - validating what the model returns, so a bad output costs one retry instead of a broken run.