Context Management

Long-running agents accumulate messages that exceed the model's context window. yoagent provides token tracking, overflow detection, tiered compaction, and execution limits.

Token Estimation

Fast estimation without external tokenizer dependencies:

#![allow(unused)]
fn main() {
use yoagent::context::{estimate_tokens, message_tokens, total_tokens};

estimate_tokens("Hello world");          // ~3 tokens (chars / 4)
message_tokens(&agent_message);          // estimate for a single message
total_tokens(&messages);                 // estimate for all messages
}

Context Tracking

ContextTracker combines real token counts from provider responses with estimation for new messages — more accurate than pure estimation:

#![allow(unused)]
fn main() {
use yoagent::context::ContextTracker;

let mut tracker = ContextTracker::new();

// After each assistant response, record the real usage:
tracker.record_usage(&assistant_usage, message_index);

// Get current context size (real usage + estimated trailing):
let tokens = tracker.estimate_context_tokens(agent.messages());

// After compaction, reset the tracker:
tracker.reset();
}

When no usage data is available, it falls back to chars/4 estimation.

Context Overflow Detection

When the context exceeds a model's window, providers return overflow errors. yoagent detects these automatically across all major providers.

HTTP-level detection

An HTTP error response is classified by ProviderError::classify(); an overflow becomes ProviderError::ContextOverflow, which ProviderError::is_context_overflow() reports. You only see a ProviderError when calling a StreamProvider directly — the agent loop does not recover from an overflow, it turns the error into an assistant message with StopReason::Error (calling on_error).

classify() recognizes overflow messages from Anthropic, OpenAI, Google, AWS Bedrock, xAI, Groq, OpenRouter, llama.cpp, LM Studio, MiniMax, Kimi, GitHub Copilot, and generic patterns.

Message-level detection

Whichever way the provider reported it (an HTTP error, or an in-stream error on Anthropic and OpenAI), the turn ends with an assistant message whose error_message names the overflow. Check it with Message::is_context_overflow():

#![allow(unused)]
fn main() {
let mut rx = agent.prompt("...").await;
while rx.recv().await.is_some() {}
agent.finish().await;

let overflowed = agent
    .messages()
    .last()
    .and_then(|m| m.as_llm())
    .is_some_and(|m| m.is_context_overflow());
}

Handling overflow in your application

When context_config is set (the default — see below), the loop already compacts before every turn with the configured CompactionStrategy. What it does not do is recover from an overflow the provider reports anyway; that is yours to wire:

#![allow(unused)]
fn main() {
use yoagent::context::{compact_messages, ContextConfig};

if overflowed {
    let config = ContextConfig::from_context_window(200_000);
    let mut history = agent.messages().to_vec();
    history.pop(); // drop the error message: continue_loop() cannot resume from an assistant message
    agent.replace_messages(compact_messages(history, &config));
    let mut rx = agent.continue_loop().await;
    while rx.recv().await.is_some() {}
    agent.finish().await;
}
}

For LLM-based summarization, install LlmCompaction with with_compaction_strategy(LlmCompaction::from_config(..)): it summarizes old turns in a background request and falls back to deterministic compaction whenever no summary is ready. On wasm32 it never summarizes (compaction stays deterministic). What the summaries cost is in the run's SessionStats::compaction (priced at the summarization model's own cost) and in total_cost_usd() / total_usage(); a request still in flight when a run ends is counted by the next run. Per compaction, LlmCompaction::with_event_sender reports each request on AgentEvent::ContextCompacted — largely the same spend, so do not add the two. (They can differ: events report each spliced summary, while the stats count every request when it finishes, including discarded ones.) replace_messages() and compact_messages() remain available for manual control.

ContextConfig

#![allow(unused)]
fn main() {
pub struct ContextConfig {
    pub max_context_tokens: usize,                            // Default: 100,000
    pub system_prompt_tokens: usize,                          // Default: 4,000
    pub keep_recent: usize,                                   // Default: 10
    pub keep_first: usize,                                    // Default: 2
    pub tool_output_max_lines: usize,                         // Default: 200
    pub tool_output_max_lines_overrides: HashMap<String, usize>, // Default: {"read_file": MAX}
    pub compact_target_ratio: f32,                            // Default: 0.7
    pub compact_headroom_turns: Option<usize>,                // Default: Some(30)
    pub truncate_tool_output_on_append: bool,                 // Default: true
}
}

Auto-Derivation from ModelConfig

When you set a ModelConfig but don't explicitly set a ContextConfig, the compaction budget is automatically derived from the model's context_window — reserving 80% for context and 20% for output:

#![allow(unused)]
fn main() {
// MiniMax with 1M context → compacts at 800K (no manual config needed)
let agent = Agent::from_config(ModelConfig::minimax("MiniMax-M3", "MiniMax M3"));

// Anthropic with 200K context → compacts at 160K
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"));
}

The priority chain:

1. Explicit with_context_config(...) → always wins
2. Has model_config → auto-derives from context_window (80%)
3. Neither → ContextConfig::default() (100K)

You can also derive manually:

#![allow(unused)]
fn main() {
let config = ContextConfig::from_context_window(1_000_000);
// config.max_context_tokens == 800_000
}

Tiered Compaction

compact_messages() tries each level in order, stopping as soon as messages fit the budget:

Level 1: Truncate Tool Outputs

Replaces long tool outputs with head + tail, fitting the result into tool_output_max_lines lines (exactly, for a single long block; the [... N lines truncated ...] marker is charged against that budget). The budget is the whole result's: when a result has several text blocks (an MCP server's, say), blocks no longer than an even share stay whole and the others split the rest, each cut with its own marker. A block not left whole keeps at least 5 lines (head, marker, tail), so when the even share would be under 5 lines the result goes over the budget by up to 5 lines per such block rather than being cut silently. Below a budget of 5 there is no room for a marker, and each block keeps the budget on its own. This is the cheapest level — it preserves conversation structure and typically saves 50-70% in coding sessions.

Truncation is idempotent: re-running it on an already-truncated output returns it byte for byte, so a session that sits above the budget does not re-truncate the same outputs turn after turn.

Level 2: Summarize Old Turns

Keeps the last keep_recent messages in full detail. Older assistant messages are replaced with one-line summaries like "[Summary] [Assistant used 3 tool(s)]", and their tool results are dropped. The boundary is pulled back to a turn start so an assistant message and its tool results are never split across it.

Level 3: Drop Middle Messages

Drops the smallest span of middle messages that reaches the target, keeping at least keep_first from the start and keep_recent from the end. A constant marker message stands in for what was removed; the count goes to the debug log rather than into the marker, so the text does not change from pass to pass.

Retrievable tool output

Head-tail truncation on the append path keeps a huge tool result from eating the context, but the middle is gone irrecoverably — it survives only in the event stream, which the agent cannot read.

Attach a SharedState and the full text is stashed, with the marker naming where it went:

#![allow(unused)]
fn main() {
let agent = Agent::from_config(config)
    .with_shared_state(SharedState::new());
}

[... 1847 lines truncated — full output: shared_state get "tool-out-tc_01abc-9f2a-b0" ...]

The shared_state tool is registered for the run, so the model can act on the pointer. Opt-in: with no store attached, truncation behaves exactly as before and the marker advertises no retrieval it cannot honour.

Keys are block-qualified — a result carrying several text blocks gets one key per block, suffixed by the block's position in the content vector, so text/image/text yields …-b0 and …-b2. The key combines the tool call id with a hash of the output, because Gemini synthesizes call ids as a per-response index that restarts every turn; id alone would let turn 1's frozen marker resolve to turn 5's content.

Two limits worth knowing:

• Lossy compaction drops the marker but not the stash entry. Levels 2 and 3 drop whole turns, taking the pointer with them, while the stored value lives on and keeps consuming cap quota.
• Stash entries are evictable; caller keys are not. Both backends (MemoryBackend, and the native-only FileBackend) evict oldest-first under their cap, but only tool-out-* entries — losing one degrades a marker to an ordinary "key not found" the agent can act on, and the head+tail is still in the transcript. Nothing regenerates an artifact you stored yourself, so when only caller keys remain the write reports capacity instead.

SubAgentTool::with_context_config makes the same path reachable for sub-agents; their stash is scoped, and scoped keys are excluded from the system prompt summary so a second delegation does not see the first one's keys and cold -start the prefix cache.

Prefix Cache Stability

Providers cache request prefixes — automatically on DeepSeek, explicitly via cache_control on Anthropic. A cache hit needs the new request to share a byte-identical prefix with the last one, so every rewrite of already-sent history costs full price for every token from the rewrite point onward.

Compaction is built around that:

• Levels are idempotent where they can be, so re-running on settled history changes nothing.
• compact_headroom_turns sets the compaction target from the session's observed growth rate — target = budget − turns × growth_per_turn — so the gap between compactions stays constant instead of collapsing as history accumulates. compact_target_ratio is the fallback and acts as a ceiling on retention. See Prompt Caching for the full formula and measurements.
• Markers and generated summaries carry no wall-clock timestamps or drifting counts, so the same history always compacts to the same bytes.

Bounding tool output: two layers

The largest source of cache loss is retroactive truncation — output is sent in full, cached, then rewritten by Level 1 in one sweep once the session goes over budget. Bounding it as it arrives fixes that, but a single global cap is the wrong instrument, because tools differ in how well they survive being cut:

Command output (bash, search) takes a head+tail cut well: the first error is at the top, the summary at the bottom, and the middle is repetition. tool_output_max_lines (200) is applied on append, controlled by truncate_tool_output_on_append (on by default).

File reads do not. The middle of a source file is usually the part that was asked for, so head+tail removes exactly the wrong lines. read_file bounds itself instead, by paging: it returns DEFAULT_READ_MAX_LINES (500) at a time with a header stating the true total, and the agent asks for the next range with offset/limit. That bound is lossless and directed. read_file is therefore exempt from head+tail truncation by default, via tool_output_max_lines_overrides.

Custom tools pick their own budget the same way:

#![allow(unused)]
fn main() {
let mut config = ContextConfig::from_context_window(128_000);
config.tool_output_max_lines_overrides.insert("my_paging_tool".into(), usize::MAX);
config.tool_output_max_lines_overrides.insert("noisy_tool".into(), 40);
}

To restore pre-0.15 behaviour, set truncate_tool_output_on_append: false and construct ReadFileTool { max_lines: usize::MAX, ..Default::default() } (a filesystem tool, behind the default native feature).

Measured effect

tests/context_cache_test.rs replays 300 turns on a 128K window. The tool mix (bash 41%, edit/write 36%, read 19%, search 4%) and file-size distribution come from 808 archived runs of a production agent built on yoagent.

In input-token spend that is −9.2% to −21.3% on DeepSeek and −15.2% to −22.8% on Anthropic, widening with session length. Full breakdown and the reasoning behind each default: Prompt Caching.

ExecutionLimits

Prevents runaway agents:

#![allow(unused)]
fn main() {
#[non_exhaustive]
pub struct ExecutionLimits {
    pub max_turns: usize,              // Default: 50
    pub max_total_tokens: usize,       // Default: 1,000,000
    pub max_duration: Duration,        // Default: 600s (10 min)
    pub max_consecutive_identical_tool_calls: Option<usize>,  // Default: Some(3)
}
}

When a limit is reached, the agent stops with a message like "[Agent stopped: Max turns reached (50/50)]".

All limits apply per run: each prompt / continue_loop starts again at zero turns, zero tokens and a fresh clock. A long conversation made of many prompts is never cut off by them; a single prompt that keeps working is.

What max_total_tokens counts

Every turn adds its whole prompt — cached tokens included (input + cache_read + cache_write) — plus its output. The prompt resends the conversation each turn, so a run's total is about turns × context size (quadratic in turns while the conversation is still growing), far more than the conversation itself:

Caching does not change this: cached tokens are cheaper, but they are counted. That makes the cap a ceiling on the work a run does (checked before each turn, so the last turn can go over by one turn's tokens), which is what you want for an unattended agent. A long agentic run — or a run late in a conversation, whose context is already large — stops much sooner than "1M tokens" suggests. For those, raise it — or set it to usize::MAX and let max_turns bound the run — and raise max_duration (10 minutes per run by default) too:

#![allow(unused)]
fn main() {
use std::time::Duration;
use yoagent::context::ExecutionLimits;

let agent = agent.with_execution_limits(
    ExecutionLimits::default()
        .with_max_turns(200)
        .with_max_total_tokens(usize::MAX)
        .with_max_duration(Duration::from_secs(4 * 3600)),
);
}

A SubAgentTool delegation has its own limits, counted the same way: by default 10 turns, 1M tokens and 5 minutes per delegation, set with SubAgentTool::with_execution_limits (see Sub-Agents).

Loop detection

The cheapest catastrophic failure is a model calling one tool with the same arguments forever. The three limits above all fire eventually — but only after the run has burned its entire turn, token and wall-clock budget to discover it achieved nothing.

max_consecutive_identical_tool_calls is on by default at Some(3), with two escalations that mirror the house pattern of steering before aborting:

1. First trip injects a steering message and continues. A model repeating a call is often retrying something transient, and aborting immediately would regress that legitimate case.
2. A later trip on the same signature stops the run.

Both emit AgentEvent::LoopDetected { tool_name, repetitions, aborted }, so a UI can show the intervention and an audit can tell a loop abort from a turn-limit stop.

Signatures compare serde_json::Value, not serialized text — two calls differing only in key order are the same call. Counting covers duplicates within one batch as well as across turns, because ToolExecutionStrategy defaults to Parallel and a model can emit the same call three times in a single message.

Consecutive, and that word is load-bearing: a different call resets the streak, so an alternating [a, b, a, b, …] loop is not detected. That is a deliberate trade — an agent working through a list calls one tool repeatedly and legitimately, and a detector that fired on interleaved repeats would be worse than none.

#![allow(unused)]
fn main() {
ExecutionLimits::default().with_max_consecutive_identical_tool_calls(None)  // off
}

Disabling Context Management

#![allow(unused)]
fn main() {
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .without_context_management();
}

This sets both context_config and execution_limits to None.