Build a Monthly Inference Budget for an AI Agent

Before you approve an agent's model, set the per-agent spending envelope. Agentic coding workloads can consume about 1,000 times more tokens than ordinary reasoning or chat because agents reread accumulated instructions and tool results. The trade-off is between model quality and monthly spend: the bill comes from turns, tool calls, context growth, routing, and retries, not a single prompt price. The arithmetic is (expected turns × average tokens + tool-call tokens + context-growth penalty) × routed model price × retry multiplier + variance reserve, capped by a per-agent spending envelope and kill-switch thresholds.

Start with the envelope, not the model

Done means a monthly ceiling approved by product and finance, with a kill-switch threshold below it. The ceiling should cover the expected path and the bad path, not just the demo path.

Give product owners a compact view: expected monthly spend, high percentile, and the kill-switch threshold. You get a number they can defend in a budget review, not a raw token count alone.

Agent inference cost is hard to predict. Repeated runs of the same agentic task have shown token use differing by up to 30 times. A looping agent retains and processes its full prior conversation history on each subsequent turn.

The inference cost of each agentic workflow is forecast to rise several fold by 2028, even if per-token prices keep falling. That makes the monthly envelope the control point. If the envelope is too loose, the model choice becomes a preference argument. If it is too tight, the agent stops before it finishes.

Build the formula from observed runs

• Estimate expected turns per task from logged runs: median and high percentile, not a single happy-path count.
• Measure average tokens per turn. Ready when input and output tokens are separated, with system prompt, tool schema, and history counted.
• Add tool-call tokens. The output is a line item for each tool response, including file contents, API payloads, and error messages.
• Apply a context-growth penalty. A multiplier for history that grows with each turn makes later turns cost more than earlier ones.
• Price the routed model. The price per token should be for the model actually used, not the cheapest model in the catalog.
• Multiply by a retry multiplier. The factor should cover failed turns, partial successes, and re-runs after tool errors.
• Add a variance reserve. If you omit it, the budget will look clean until a slow run breaks it. It is a buffer for the spread between runs.

Keep the formula in one place. When a new tool or model changes the numbers, the budget changes with it.

Route by task, not by habit. Use a cheaper model for classification, extraction, and formatting; reserve the stronger model for planning, tool selection, and ambiguous recovery. Keep a routing table with a cost column next to each model. When the table changes, the monthly envelope changes with it.

Cap the runaway path

AICost.ai's CostWall policy engine translates budget, per-agent spending envelope, turn, token, and kill-switch settings into rules for existing gateways and model routers. AICost.ai's cost model tracks both ordinary and out-of-control agent paths, allowing for retries, growing context, and variance allowances, then measures cost by completed task, workflow, agent, or business result.

Enforce those rules at the router, not in a spreadsheet, so the spend stays inside the approved envelope.

In regulated workflows, production-agent analyses have found that tokens represent only 20% to 25% of variable run cost, with human review making up most of the remainder. That changes the budgeting question: you are not only buying tokens; you are buying a decision path that may need review, rollback, or rework.

Set the kill-switch threshold where the agent stops, not where the invoice surprises you. Review the envelope after each model or tool change, and after each routing change.