Your AI agent gives you a 200-word answer, yet a surprising number of credits disappear. It is tempting to blame the final answer or assume you picked the wrong model. Usually, neither is the main issue.
The expensive part is what happened before that answer: the agent repeatedly read a long conversation, reviewed tool definitions, called tools, absorbed large tool results, retried an uncertain step, and then did it again on the next turn.
That is the operational reality of agents. A single task is not one request and one response. It is a sequence of model calls and tool actions. Each new model call may include instructions, relevant chat history, available tools, previous tool results, and the latest user request. When that payload grows, your input usage grows with it.
The fix is not “write shorter prompts.” It is to inspect the run, find the largest repeated input, and put a sensible limit around it. This guide gives you a provider-neutral way to do that without blindly making your agent less capable.
Why a short answer can have a large bill
Think of an agent task as a working meeting rather than a text message. At every decision point, the agent may need to see a packet containing:
- Your standing instructions and rules
- The current task
- Earlier messages and intermediate reasoning
- The names, descriptions, and input rules for available tools
- Tool calls it has already made
- The results returned by those tools
- The format it must use for its next response
Then it decides what to do, perhaps calls a search, database, browser, file, code, or CRM tool, receives a result, and makes another model call to interpret it. The answer you see is simply the last visible part of that loop.
This matters because the same large material can be charged repeatedly. A 20-page document, a verbose API response, or an old conversation may be useful once. If it stays in active context for six more turns, it can become six repeated input costs.
Agent usage reporting is also becoming more detailed. Many agent environments now show request-level or step-level usage, and some show cumulative session cost. That visibility changes the practical question from “Why are agents expensive?” to “Which step in this workflow is repeating the most unnecessary material?”
Do not optimize for the lowest token count in isolation. A cheap run that fails, produces incorrect work, or creates a manual cleanup job is not a saving. Your real metric is cost per successful task.
The three repeat-token leaks to audit first
1. Oversized or stale active context
Context is everything the model can currently use to make its next decision. This commonly includes chat history, project instructions, attached material, retrieved documents, and previous tool outputs.
The leak starts when the agent carries information that is no longer relevant. For example, a coding agent may keep old error logs after the error has been fixed. A research agent may carry entire web pages after extracting the three facts it needs. A customer operations agent may receive a customer record with every historical field when it only needs name, status, last interaction, and next action.
Long-lived chats are a frequent culprit. They feel efficient because you do not need to restate background. But a session that mixes several unrelated jobs can make every new turn pay to re-read history from the old ones.
Practical control: Start a fresh task when previous work is irrelevant. Keep durable instructions short and specific. Replace completed work with a compact summary. Retrieve only the files, records, or sections required for the immediate decision.
2. Too many tools and tool results that are too large
Tools are not free just because the agent does not call them. The model may be given each tool’s name, purpose, and input schema when deciding what it can do. If you expose every integration in your stack to every job, the agent may carry a bulky tool catalogue into each turn.
Then there is the output problem. A database query that returns 500 rows, a browser extraction that returns an entire page, or a log query that returns thousands of lines can land in the working context. The agent may need only a handful of fields, but the next step can still involve re-reading the full result.
Practical control: Give the agent the smallest toolset that can complete the current job. Configure tools to return filtered, structured, short results. Ask for fields, limits, date ranges, summaries, or a single record instead of a full export.
For instance, replace “get the customer record” with “return customer name, lifecycle stage, owner, last contact date, open support count, and the latest three notes.” Replace “search logs” with “return the 20 lines around the latest error, plus timestamp and error code.”
3. Excess turns, retries, and loops
Some agent tasks need several turns. The issue is not that the agent uses tools; it is that it keeps taking turns without making enough progress. Common patterns include retrying the same failed tool call, broadening a search repeatedly, bouncing between two tools, or continuing to investigate after it has enough information to answer.
These loops are expensive because every extra turn tends to reintroduce the current context. The longer the run has already become, the more each additional turn can cost.
Practical control: Set a maximum number of turns, cap retries for the same operation, and define what should happen at the limit. Good fallback behaviour is not “keep trying.” It is “return the useful partial result, state what blocked completion, and send the task to a person or a narrower workflow.”
A turn limit is a safety rail, not a test of intelligence. Set it from your baseline data. If successful runs normally finish in four to six turns, a limit of 30 is unlikely to protect your budget.
Run a 15-minute agent cost audit
Do not begin with a large redesign. Pick one workflow that regularly consumes more credits than expected: investigating a bug, drafting a client brief, qualifying an inbound lead, producing a report, or updating records from a document.
Run a representative task once and write down the following:
- Task: What was the agent asked to complete?
- Outcome: Did it succeed, partly succeed, or fail? Would you accept the output without substantial rework?
- Total usage: Record cost, credits, or token usage shown by your agent environment.
- Input versus output: If available, record them separately. High input use is a strong clue that repeated context is the problem.
- Turns or model calls: Count how many decisions the agent made.
- Tools called: Record each tool and how often it was used.
- Largest tool result: Identify the result that was much bigger than the decision required.
- Retries: Note repeated calls, errors, or near-identical searches.
You do not need perfect observability to start. If your product only shows credits, use run logs, tool history, and the visible conversation to identify likely waste. If it offers step-level usage, use that data to locate the expensive step rather than guessing.
The goal of the audit is not to make every run short. It is to remove repeated material that does not improve the next decision.
For teams building repeatable workflows, document this audit next to the process itself. The Template and SOP guide can help turn the winning setup into a usable standard instead of a one-off adjustment.
Fix #1: Shrink the active context
Start with the content that the agent sees on every turn. Split it into three buckets:
- Stable instructions: Rules, tone, approval requirements, and output format that apply to most runs.
- Task-specific context: The files, records, conversation details, and criteria needed for this job.
- Temporary working material: Large results, old logs, intermediate drafts, and items needed only briefly.
Keep stable instructions consistent and lean. Put task-specific details in a clean task brief. Treat temporary working material as disposable: summarize it, store it outside the immediate conversation when possible, or retrieve it again only when the agent genuinely needs it.
Retrieval should be precise. “Read our knowledge base” is not a useful instruction if the actual job is to answer one billing-policy question. Give the agent the relevant policy section, or let it search for and retrieve only that section.
When a task has phases, use a handoff. Let one stage produce a concise structured result for the next stage rather than carrying every raw source forward. For example, a research stage can hand over verified findings and key constraints; the writing stage does not necessarily need every raw page or search result in active context.
If the issue is an unclear workflow rather than context alone, use the SOP-to-Automation Mapper to identify where an agent should act, where it should hand off, and where a human checkpoint prevents expensive wandering.
Fix #2: Design tools for decisions, not data dumps
Tool design is where many agent budgets quietly leak. The best tool response is not the most complete response. It is the smallest response that lets the agent make the next correct decision.
For each tool available to an agent, ask:
- Does this task actually require this tool?
- Can the description and inputs be simpler?
- Can access be limited to a relevant source, folder, account, or action?
- Can the tool return selected fields rather than a whole object?
- Can it enforce a result limit, date range, or filter?
- Can it produce a compact summary plus an ID or reference for full detail?
A reference-and-preview pattern is especially useful for large results. Return a short preview, key metadata, and a reference the agent can request if needed. This gives it a path to inspect detail without forcing every later model call to carry all detail.
Also make failure messages useful and compact. “Request failed” often triggers broad retries. “No matching record found for this email in this workspace” tells the agent what happened and what alternative it should consider. Clear tool errors reduce wasted exploration.
Fix #3: Put a budget around the run
Give every workflow an expected operating range. You may use credits, tokens, model calls, turns, tool calls, or elapsed time depending on what your system exposes. The specific unit matters less than having a threshold tied to successful work.
Start with three controls:
- Maximum turns: Stop the run before a loop becomes a large bill.
- Retry ceiling: Allow a small number of retries for a transient failure, then change approach or escalate.
- Completion rule: Tell the agent what a useful partial result looks like when it cannot finish.
For a lead qualification workflow, the completion rule might be: extract the available lead details, record missing fields, assign a confidence level, and route ambiguous cases for review. For an analysis workflow, it might be: provide the strongest supported conclusion, list the missing data, and stop. That is better than burning turns trying to manufacture certainty.
Use limits with judgment. A complex task may warrant more room than a simple classification or update. The point is to prevent an open-ended agent from treating your credit balance as an invitation to keep searching.
Fix #4: Treat caching as a measured experiment
Caching can reduce the cost of repeated, stable input. It is most relevant when a workflow repeatedly begins with the same substantial instructions, reference material, or tool setup.
But caching is not an automatic discount. Cache writes can have a different price from ordinary input, and a cache only helps when enough of the prefix is stable and reused. If changing information appears early in the request, it can reduce the reuse of everything after it.
Structure recurring tasks so stable material stays stable and task-specific details arrive separately. Then compare actual runs. Track normal input, cached input where available, cache-write usage where available, total cost, and task success. Keep the change only when it improves the cost per successful task.
This is also why a reusable prompt system helps. It keeps the durable parts of a workflow consistent while leaving clear spaces for changing details. Build one with the Reusable Prompt System Builder if your team currently rebuilds large instructions from scratch for every run.
Use a before-and-after scorecard
Change one leak at a time. If you shorten context, remove tools, tighten output limits, and switch models in the same week, you will not know which change helped or which one harmed quality.
For each version of the workflow, compare a small set of similar tasks and record:
- Number of tasks completed successfully
- Average cost or credits per successful task
- Average turns or model calls
- Average retries
- Human correction time
- The specific quality failures, if any
Keep the new configuration if it reduces total effort without lowering the outcome you need. Revert it if the apparent savings create more failures or manual cleanup. A lower monthly credit total is only useful if the workflow still does its job.
Your next move
This week, choose one costly agent workflow and inspect one representative run. Find the biggest repeated item: old conversation history, a tool catalogue, a huge tool result, or a sequence of retries. Apply one control, run comparable tasks again, and judge the result by cost per successful outcome.
That is how you make agent spending predictable. Not by hoping for shorter answers, and not by downgrading every task to a cheaper model, but by preventing the agent from repeatedly paying attention to material it no longer needs.
Once you have found a workflow worth improving, the AI Automation Opportunity Finder can help you rank it against other automation opportunities by value, effort, savings, and return.