Control per-step budgets, grounding checks, and cache hits so agent loops stay within fixed token envelopes.
Set per-iteration token ceilings
Define a hard max_tokens value for every model call inside the loop. Pass that ceiling to the provider SDK so the request itself fails fast rather than returning a partial trace that still consumes the budget.
Wrap the call in a retry block that only retries on transient network errors, never on token-limit rejections. Log the exact token count returned in the usage object so downstream monitoring can accumulate real spend without estimation.
Ground before every tool call
Require the agent to emit a short justification string and a required_input_fields list before it is allowed to invoke any external tool. Reject the plan if the justification references data the current context does not contain.
Store the justification alongside the tool request ID. When the tool response arrives, re-inject only the fields that were declared, preventing the model from re-reading the entire prior transcript on the next turn.
Cache deterministic sub-results
Hash the exact input tuple for any tool whose output is stable within a run (schema lookups, static reference data). Store the hash plus result in an in-memory map scoped to the agent instance so identical calls inside the same trace return instantly.
Expire the cache at the end of the trace or on any state mutation that could change the result. This removes the most common source of repeated token burn without introducing cross-run staleness.
Accumulate and surface cumulative usage
Maintain a running total of prompt_tokens and completion_tokens after each model response. Expose the total through a lightweight status endpoint or log line so operators can see the trajectory before the next iteration begins.
When the running total crosses a configured fraction of the overall budget, switch the agent into a read-only mode that only answers from cached context. This prevents the final steps from silently exhausting the remaining allocation.