yoagent

The agent loop for Rust. Stream from any of 7 LLM protocols, run tools, loop until done.

yoagent is a library for building LLM-powered agents that use tools. It provides the core loop — prompt the model, execute tool calls, feed results back — and gets out of your way.

Philosophy

The loop is the product. An agent is just a loop: send messages to an LLM, get back text and tool calls, execute the tools, repeat until the model stops. yoagent implements that loop with streaming, cancellation, context management, and multi-provider support — and stops there.

agent_loop() is a stateless free function that takes everything it needs as arguments. Agent is an optional wrapper that adds message history, a tool registry, and steering queues. You can drive the loop yourself without adopting our state model.

Try it without an API key

git clone https://github.com/yologdev/yoagent && cd yoagent
ollama serve &
cargo run --example cli -- --provider ollama

A working coding agent in your terminal — file read/write/edit, shell, ripgrep search, streaming output, and skills.

What's here

The loop and its control surfaces

• The agent loop — how a turn runs, and how steering and follow-ups interrupt it
• Messages and events — the full AgentEvent stream for text deltas, thinking, and tool execution
• Tool middleware — async allow / modify / deny hooks gating every tool call
• Lifecycle callbacks plus execution limits (max turns, tokens, wall-clock) and CancellationToken abort
• Retry with exponential backoff and jitter, for rate-limit and network errors only

Models and tools

• 7 API protocols, 20+ providers — Anthropic, OpenAI Completions and Responses, Azure, Gemini, Vertex, Bedrock, plus OpenAI-compatible gateways, each with a real implementation rather than a shared shim
• Built-in tools — bash, file read/write/edit, list, ripgrep search; add your own via the AgentTool trait
• MCP servers and OpenAPI specs become tools transparently
• Structured outputs — typed, schema-validated replies enforced natively where the provider supports it
• Prompt caching

Scaling a session

• Sub-agents and shared state — delegate to child loops with their own model, tools, and limits; sub-agents read large artifacts by key instead of re-pasting them into every context window
• Context management — token tracking and tiered compaction (truncate tool outputs → summarise old turns → drop middle)
• State persistence — save and restore a run
• Session trees — branching history with fork, checkpoints, and JSONL persistence
• Skills — load AgentSkills-standard SKILL.md directories

Running it in production

• Telemetry — tracing spans per loop, LLM stream, and tool, with token and cost fields
• Cost tracking with separate cache-read and cache-write rates
• GASP — record runs as an append-only semantic event log; restore is clone + replay

Ecosystem

yoagent is part of the Yolog ecosystem. It powers the agent backend for Yolog applications.

• Repository: github.com/yologdev/yoagent
• API reference: docs.rs/yoagent
• License: MIT

Installation

Requirements

• Rust 1.86+ (2021 edition)
• Tokio async runtime

Add to Cargo.toml

[dependencies]
yoagent = "0.15"
tokio = { version = "1", features = ["full"] }

Dependencies

yoagent brings in these key dependencies automatically:

Feature Flags

All providers and built-in tools are included by default. Optional features:

Enable in Cargo.toml:

[dependencies]
yoagent = { version = "0.15", features = ["openapi"] }

Quick Start

Basic Example with Anthropic

use yoagent::{Agent, AgentEvent, StreamDelta};
use yoagent::provider::ModelConfig;
use yoagent::tools::default_tools;

#[tokio::main]
async fn main() {
    // The provider is selected from the config's protocol, and the API key is
    // read from ANTHROPIC_API_KEY. Add `.with_api_key(key)` to override it.
    let mut agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Sonnet 5"))
        .with_system_prompt("You are a helpful coding assistant.")
        .with_tools(default_tools());

    let mut rx = agent.prompt("List the files in the current directory").await;

    while let Some(event) = rx.recv().await {
        match event {
            AgentEvent::MessageUpdate { delta, .. } => match delta {
                StreamDelta::Text { delta } => print!("{}", delta),
                StreamDelta::Thinking { delta } => print!("[thinking] {}", delta),
                _ => {}
            },
            AgentEvent::ToolExecutionStart { tool_name, .. } => {
                println!("\n→ Running tool: {}", tool_name);
            }
            AgentEvent::ToolExecutionEnd { tool_name, result, is_error, .. } => {
                if is_error {
                    println!("  ✗ {} failed", tool_name);
                } else {
                    println!("  ✓ {} done", tool_name);
                }
            }
            AgentEvent::AgentEnd { .. } => {
                println!("\n\nDone.");
            }
            _ => {}
        }
    }
}

Example with OpenAI-Compatible Provider

For OpenAI, xAI, Groq, DeepSeek, Mistral, MiniMax, Z.ai, Qwen, Ollama, or any compatible API, pass the matching ModelConfig preset — from_config picks the OpenAI-compatible provider and resolves the provider's env key automatically:

use yoagent::{Agent, AgentEvent};
use yoagent::provider::ModelConfig;
use yoagent::tools::default_tools;

#[tokio::main]
async fn main() {
    // Provider inferred from the preset; key from OPENAI_API_KEY.
    let mut agent = Agent::from_config(ModelConfig::openai("gpt-5.5", "GPT-5.5"))
        .with_system_prompt("You are a helpful assistant.")
        .with_tools(default_tools());

    let mut rx = agent.prompt("What is 2 + 2?").await;

    while let Some(event) = rx.recv().await {
        match event {
            AgentEvent::MessageUpdate { delta, .. } => {
                if let yoagent::StreamDelta::Text { delta } = delta {
                    print!("{}", delta);
                }
            }
            AgentEvent::AgentEnd { .. } => println!(),
            _ => {}
        }
    }
}

Real-Time Streaming

agent.prompt() spawns the agent loop concurrently and returns a receiver immediately, so events stream in real-time as they're produced. For cases where you want to provide your own channel (e.g., to share a sender across tasks), use prompt_with_sender():

use yoagent::{Agent, AgentEvent, StreamDelta};
use yoagent::provider::ModelConfig;
use yoagent::tools::default_tools;

#[tokio::main]
async fn main() {
    let mut agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Sonnet 5"))
        .with_system_prompt("You are a helpful assistant.")
        .with_tools(default_tools());

    let (tx, mut rx) = tokio::sync::mpsc::unbounded_channel();

    // Consume events in real-time on a separate task
    tokio::spawn(async move {
        while let Some(event) = rx.recv().await {
            match event {
                AgentEvent::MessageUpdate { delta, .. } => {
                    if let StreamDelta::Text { delta } = delta {
                        print!("{}", delta);
                    }
                }
                AgentEvent::AgentEnd { .. } => println!(),
                _ => {}
            }
        }
    });

    // This blocks until the loop finishes; state is restored automatically
    agent.prompt_with_sender("What is 2 + 2?", tx).await;

    // Agent is ready for another prompt immediately
    let _rx = agent.prompt("Follow up question").await;
}

Using the Low-Level API

For more control, use agent_loop() directly:

use yoagent::agent_loop::{agent_loop, AgentLoopConfig};
use yoagent::provider::AnthropicProvider;
use yoagent::types::*;
use tokio::sync::mpsc;
use tokio_util::sync::CancellationToken;

#[tokio::main]
async fn main() {
    let (tx, mut rx) = mpsc::unbounded_channel();
    let cancel = CancellationToken::new();

    let mut context = AgentContext {
        system_prompt: "You are helpful.".into(),
        messages: Vec::new(),
        tools: yoagent::tools::default_tools(),
    };

    let config = AgentLoopConfig {
        provider: std::sync::Arc::new(AnthropicProvider),
        model: "claude-sonnet-5".into(),
        api_key: std::env::var("ANTHROPIC_API_KEY").unwrap(),
        thinking_level: ThinkingLevel::Off,
        max_tokens: None,
        temperature: None,
        model_config: None,
        convert_to_llm: None,
        transform_context: None,
        get_steering_messages: None,
        get_follow_up_messages: None,
        context_config: None,
        compaction_strategy: None,
        execution_limits: None,
        cache_config: CacheConfig::default(),
        tool_execution: ToolExecutionStrategy::default(),
        retry_config: yoagent::RetryConfig::default(),
        before_turn: None,
        after_turn: None,
        on_error: None,
        input_filters: vec![],
    };

    let prompts = vec![AgentMessage::Llm(Message::user("Hello!"))];
    let new_messages = agent_loop(prompts, &mut context, &config, tx, cancel).await;

    // Drain events
    while let Ok(event) = rx.try_recv() {
        // handle events...
    }

    println!("Got {} new messages", new_messages.len());
}

The Agent Loop

The agent loop is the core of yoagent. It implements the fundamental cycle:

User prompt → LLM call → Tool execution → LLM call → ... → Final response

How It Works

┌──────────────────────────────────────────────┐
│                  agent_loop()                │
│                                              │
│  1. Add prompts to context                   │
│  2. Emit AgentStart + TurnStart              │
│                                              │
│  ┌─────────── Inner Loop ──────────────┐     │
│  │  • Check steering messages          │     │
│  │  • Check execution limits           │     │
│  │  • Compact context (if configured)  │     │
│  │  • Stream LLM response              │     │
│  │  • Extract tool calls               │     │
│  │  • Execute tools (with steering)    │     │
│  │  • Emit TurnEnd                     │     │
│  │  • Continue if tool_calls or steer  │     │
│  └─────────────────────────────────────┘     │
│                                              │
│  3. Check follow-up messages                 │
│  4. If follow-ups exist, loop again          │
│  5. Emit AgentEnd                            │
└──────────────────────────────────────────────┘

Entry Points

agent_loop()

Starts a new agent run with prompt messages:

#![allow(unused)]
fn main() {
pub async fn agent_loop(
    prompts: Vec<AgentMessage>,
    context: &mut AgentContext,
    config: &AgentLoopConfig,
    tx: mpsc::UnboundedSender<AgentEvent>,
    cancel: CancellationToken,
) -> Vec<AgentMessage>
}

The prompts are added to context, then the loop runs. Returns all new messages generated during the run.

agent_loop_continue()

Resumes from existing context (e.g., after an error or retry):

#![allow(unused)]
fn main() {
pub async fn agent_loop_continue(
    context: &mut AgentContext,
    config: &AgentLoopConfig,
    tx: mpsc::UnboundedSender<AgentEvent>,
    cancel: CancellationToken,
) -> Vec<AgentMessage>
}

Requires that the last message in context is not an assistant message.

AgentLoopConfig

#![allow(unused)]
fn main() {
pub struct AgentLoopConfig {
    pub provider: Arc<dyn StreamProvider>,
    pub model: String,
    pub api_key: String,
    pub thinking_level: ThinkingLevel,
    pub max_tokens: Option<u32>,
    pub temperature: Option<f32>,
    pub model_config: Option<ModelConfig>,
    pub convert_to_llm: Option<ConvertToLlmFn>,
    pub transform_context: Option<TransformContextFn>,
    pub get_steering_messages: Option<GetMessagesFn>,
    pub get_follow_up_messages: Option<GetMessagesFn>,
    pub context_config: Option<ContextConfig>,
    pub execution_limits: Option<ExecutionLimits>,
    pub cache_config: CacheConfig,
    pub tool_execution: ToolExecutionStrategy,
    pub retry_config: RetryConfig,
    pub before_turn: Option<BeforeTurnFn>,
    pub after_turn: Option<AfterTurnFn>,
    pub on_error: Option<OnErrorFn>,
    pub input_filters: Vec<Arc<dyn InputFilter>>,
    pub compaction_strategy: Option<Arc<dyn CompactionStrategy>>,
    pub turn_delay: Option<Duration>,
}
}

Steering & Follow-Ups

Steering

Steering messages interrupt the agent between tool executions. When the agent is executing multiple tool calls from a single LLM response, steering is checked after each tool completes. If a steering message is found:

1. The current tool finishes normally
2. All remaining tool calls are skipped with is_error: true and "Skipped due to queued user message"
3. The steering message is injected into context
4. The loop continues with a new LLM call that sees the interruption

#![allow(unused)]
fn main() {
// While agent is running tools, redirect it:
agent.steer(AgentMessage::Llm(Message::user("Stop that. Instead, explain what you found.")));
}

Follow-Ups

Follow-up messages are checked after the agent would normally stop (no more tool calls, no steering). If follow-ups exist, the loop continues with them as new input — the agent doesn't need to be re-prompted.

#![allow(unused)]
fn main() {
// Queue work for after the agent finishes its current task:
agent.follow_up(AgentMessage::Llm(Message::user("Now run the tests.")));
agent.follow_up(AgentMessage::Llm(Message::user("Then commit the changes.")));
}

Queue Modes

Both queues support two delivery modes:

#![allow(unused)]
fn main() {
agent.set_steering_mode(QueueMode::All);
agent.set_follow_up_mode(QueueMode::OneAtATime);
}

Queue Management

#![allow(unused)]
fn main() {
agent.clear_steering_queue();   // Drop all pending steers
agent.clear_follow_up_queue();  // Drop all pending follow-ups
agent.clear_all_queues();       // Drop everything
}

Clients that render the queue (e.g. showing pending messages while the agent is busy) can inspect it without consuming:

#![allow(unused)]
fn main() {
let pending = agent.steering_queue_snapshot();  // point-in-time copy
let count = pending.len();
}

Use steering_queue_len() instead when only the count is needed — it avoids cloning message contents, which matters for high-frequency polling.

For an edit-and-requeue UI, drain atomically and requeue the survivors as a single batch:

#![allow(unused)]
fn main() {
let mut queued = agent.take_steering_queue(); // atomic drain
queued.pop();                                 // e.g. drop the newest entry
agent.steer_all(queued);                      // requeue the rest under one lock
}

These accessors pair with the spawn-based flows (prompt(), prompt_messages(), continue_loop()). The _with_sender variants borrow the agent mutably for the whole run, so the queue cannot be touched while one is in flight.

Only the drain is atomic — the edit round trip is not:

• The loop keeps running while the queue is taken. If the run finishes during the edit, requeued survivors are delivered at the start of the next run.
• A message the loop has already picked up for injection is no longer in the queue: snapshots won't show it and take_steering_queue() cannot retract it.
• Messages steered concurrently during the edit window land ahead of requeued survivors (steer appends to the back; delivery pops the front).
• Delivery of a requeued batch follows the steering QueueMode: All injects it at one check, the default OneAtATime delivers one message per check.
• After reset(), discard taken messages instead of requeueing them.
• Pending queue contents are not included in save_messages() persistence — snapshot and store them separately if they must survive a restart.

Low-Level API

When using agent_loop() directly, steering and follow-ups are provided via callback functions:

#![allow(unused)]
fn main() {
let config = AgentLoopConfig {
    get_steering_messages: Some(Box::new(|| {
        // Return Vec<AgentMessage> — checked between tool calls
        vec![]
    })),
    get_follow_up_messages: Some(Box::new(|| {
        // Return Vec<AgentMessage> — checked when agent would stop
        vec![]
    })),
    // ...
};
}

Custom Compaction

By default, when context exceeds the token budget in ContextConfig, yoagent runs a 3-level compaction strategy: truncate tool outputs → summarize old turns → drop middle messages. You can replace this with your own CompactionStrategy:

#![allow(unused)]
fn main() {
use yoagent::context::{CompactionStrategy, ContextConfig, compact_messages};
use yoagent::types::*;

struct MyCompaction;

impl CompactionStrategy for MyCompaction {
    fn compact(
        &self,
        messages: Vec<AgentMessage>,
        config: &ContextConfig,
    ) -> Vec<AgentMessage> {
        // Your logic here — then optionally delegate to the default:
        compact_messages(messages, config)
    }
}

let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .with_compaction_strategy(MyCompaction);
}

The strategy is called once per turn, right before the LLM call, whenever context_config is Some. When compaction_strategy is None, DefaultCompaction (which wraps compact_messages()) is used automatically.

Use Cases

Memory-aware compaction — Index messages into a vector store before they're dropped, so the agent can recall them later via a search tool:

#![allow(unused)]
fn main() {
struct MemoryAwareCompaction {
    memory: Arc<dyn MemoryStore>,
}

impl CompactionStrategy for MemoryAwareCompaction {
    fn compact(
        &self,
        messages: Vec<AgentMessage>,
        config: &ContextConfig,
    ) -> Vec<AgentMessage> {
        let compacted = compact_messages(messages.clone(), config);

        // Index what was dropped
        let dropped: Vec<_> = messages.iter()
            .filter(|m| !compacted.contains(m))
            .collect();
        if !dropped.is_empty() {
            self.memory.index(dropped);
        }

        compacted
    }
}
}

Semantic pointer compaction — Replace dropped messages with a marker so the agent knows context was lost:

#![allow(unused)]
fn main() {
struct SemanticPointerCompaction;

impl CompactionStrategy for SemanticPointerCompaction {
    fn compact(
        &self,
        messages: Vec<AgentMessage>,
        config: &ContextConfig,
    ) -> Vec<AgentMessage> {
        let compacted = compact_messages(messages.clone(), config);
        let dropped_count = messages.len() - compacted.len();

        if dropped_count == 0 {
            return compacted;
        }

        // Insert a marker after the first kept messages
        let mut result = compacted;
        let insert_at = config.keep_first.min(result.len());
        result.insert(insert_at, AgentMessage::Extension(
            ExtensionMessage::new("compaction_marker", serde_json::json!({
                "dropped": dropped_count,
                "note": format!("{} earlier messages were compacted", dropped_count),
            }))
        ));
        result
    }
}
}

Priority-preserving compaction — Never drop messages containing important keywords:

#![allow(unused)]
fn main() {
struct PriorityPreservingCompaction {
    preserve_keywords: Vec<String>,
}

impl CompactionStrategy for PriorityPreservingCompaction {
    fn compact(
        &self,
        messages: Vec<AgentMessage>,
        config: &ContextConfig,
    ) -> Vec<AgentMessage> {
        let (priority, normal): (Vec<_>, Vec<_>) = messages.into_iter()
            .partition(|m| self.is_priority(m));

        let mut compacted = compact_messages(normal, config);

        // Re-insert priority messages — they're never dropped
        for msg in priority {
            compacted.push(msg);
        }
        compacted
    }
}
}

Messages & Events

Message Types

Message

The core LLM message type, tagged by role:

#![allow(unused)]
fn main() {
pub enum Message {
    User {
        content: Vec<Content>,
        timestamp: u64,
    },
    Assistant {
        content: Vec<Content>,
        stop_reason: StopReason,
        model: String,
        provider: String,
        usage: Usage,
        timestamp: u64,
        error_message: Option<String>,
    },
    ToolResult {
        tool_call_id: String,
        tool_name: String,
        content: Vec<Content>,
        is_error: bool,
        timestamp: u64,
    },
}
}

Create user messages easily:

#![allow(unused)]
fn main() {
let msg = Message::user("Hello, world!");
}

AgentMessage

Wraps Message with support for extension messages (UI-only, notifications, etc.):

#![allow(unused)]
fn main() {
pub enum AgentMessage {
    Llm(Message),
    Extension(ExtensionMessage),
}

pub struct ExtensionMessage {
    pub role: String,
    pub kind: String,
    pub data: serde_json::Value,
}
}

Create extension messages with the convenience constructor:

#![allow(unused)]
fn main() {
let ext = ExtensionMessage::new("status_update", serde_json::json!({"status": "running"}));
let msg = AgentMessage::Extension(ext);
}

The kind field categorizes the extension (e.g., "status_update", "ui_event", "notification"). Use as_llm() to extract the Message if it's an LLM message. The default convert_to_llm function filters out Extension messages before sending to the provider.

All core message types implement Serialize, Deserialize, Clone, and PartialEq, enabling state persistence and test assertions.

Content

Each message contains Vec<Content>:

#![allow(unused)]
fn main() {
pub enum Content {
    Text { text: String },
    Image { data: String, mime_type: String },
    Thinking { thinking: String, signature: Option<String> },
    ToolCall {
        id: String,
        name: String,
        arguments: serde_json::Value,
        provider_metadata: Option<serde_json::Value>, // e.g. Gemini thought signatures
    },
}
}

An assistant message can contain multiple content blocks — e.g., thinking + text + tool calls.

Content is #[non_exhaustive] (match with a wildcard arm), and the ToolCall and Thinking variants are separately #[non_exhaustive] — construct them via Content::tool_call() / tool_call_with_metadata() / thinking() / thinking_signed(). Message::Assistant is likewise #[non_exhaustive]; custom providers construct it via Message::assistant().

StopReason

#![allow(unused)]
fn main() {
pub enum StopReason {
    Stop,       // Natural completion
    Length,     // Hit max tokens
    ToolUse,    // Wants to call tools
    Error,      // Provider error
    Aborted,    // Cancelled by user
    Refusal,    // Declined by the provider's safety system
}
}

Usage

Token usage from the provider:

#![allow(unused)]
fn main() {
pub struct Usage {
    pub input: u64,
    pub output: u64,
    pub cache_read: u64,
    pub cache_write: u64,
    pub total_tokens: u64,
}
}

AgentEvent

Events emitted during the agent loop for real-time UI updates:

Wire format

AgentEvent and StreamDelta serialize as internally-tagged camelCase JSON, so external frontends (a websocket fanout server, a TypeScript client, a JSONL pipe) can consume the event stream directly:

{"type":"messageUpdate","message":{...},"delta":{"type":"text","delta":"hi"}}
{"type":"toolExecutionEnd","toolCallId":"tc_1","toolName":"bash","result":{...},"isError":false}

This shape is a public contract frozen by snapshot tests — variant tags, field names, and the tagging scheme won't change in minor releases.

Streaming semantics: clients accumulate text from each MessageUpdate's delta; the message field during streaming is an empty-content placeholder (the complete message arrives as a new value in MessageEnd). Reset accumulation on each MessageStart — after a transient provider error the stream restarts from a fresh MessageStart with no closing MessageEnd for the abandoned attempt, so a client that doesn't reset duplicates the replayed text. A client that misses events entirely (e.g. a lagged websocket subscriber) resyncs from the next MessageEnd without replay.

StreamDelta

Deltas within MessageUpdate:

#![allow(unused)]
fn main() {
pub enum StreamDelta {
    Text { delta: String },
    Thinking { delta: String },
    ToolCallDelta { delta: String },
}
}

Agent State

The Agent struct provides access to its current state:

#![allow(unused)]
fn main() {
// Check if the agent is currently streaming a response
if agent.is_streaming() {
    // Use steer() or follow_up() instead of prompt()
    agent.steer(AgentMessage::Llm(Message::user("New instruction")));
}

// Access the full message history
let messages: &[AgentMessage] = agent.messages();

// Check the last message
if let Some(last) = messages.last() {
    println!("Last message role: {}", last.role());
}
}

The is_streaming() flag is true between prompt()/continue_loop() call and completion. While streaming, calling prompt() will panic — use steer() or follow_up() instead.

Tools

The AgentTool Trait

Every tool implements AgentTool:

#![allow(unused)]
fn main() {
#[async_trait]
pub trait AgentTool: Send + Sync {
    fn name(&self) -> &str;
    fn label(&self) -> &str;
    fn description(&self) -> &str;
    fn parameters_schema(&self) -> serde_json::Value;
    async fn execute(
        &self,
        params: serde_json::Value,
        ctx: ToolContext,
    ) -> Result<ToolResult, ToolError>;
}
}

ToolContext

All execution context is bundled into a single struct, making the trait easier to extend in the future:

#![allow(unused)]
fn main() {
pub struct ToolContext {
    pub tool_call_id: String,
    pub tool_name: String,
    pub cancel: CancellationToken,
    pub on_update: Option<ToolUpdateFn>,
    pub on_progress: Option<ProgressFn>,
}
}

ToolContext implements Clone and Debug.

ToolResult

#![allow(unused)]
fn main() {
pub struct ToolResult {
    pub content: Vec<Content>,
    pub details: serde_json::Value,
}
}

The content is sent back to the LLM. The details field holds metadata (not sent to the LLM) for UI/logging.

ToolError

#![allow(unused)]
fn main() {
pub enum ToolError {
    Failed(String),
    NotFound(String),
    InvalidArgs(String),
    Cancelled,
}
}

Errors are converted to ToolResult with is_error: true and sent back to the LLM so it can recover.

Implementing a Custom Tool

#![allow(unused)]
fn main() {
use yoagent::types::*;
use async_trait::async_trait;

pub struct WeatherTool;

#[async_trait]
impl AgentTool for WeatherTool {
    fn name(&self) -> &str { "get_weather" }
    fn label(&self) -> &str { "Weather" }
    fn description(&self) -> &str {
        "Get current weather for a city."
    }

    fn parameters_schema(&self) -> serde_json::Value {
        serde_json::json!({
            "type": "object",
            "properties": {
                "city": {
                    "type": "string",
                    "description": "City name"
                }
            },
            "required": ["city"]
        })
    }

    async fn execute(
        &self,
        params: serde_json::Value,
        _ctx: ToolContext,
    ) -> Result<ToolResult, ToolError> {
        let city = params["city"].as_str()
            .ok_or(ToolError::InvalidArgs("missing city".into()))?;

        // Call weather API...
        Ok(ToolResult {
            content: vec![Content::Text {
                text: format!("Weather in {}: 72°F, sunny", city),
            }],
            details: serde_json::Value::Null,
        })
    }
}
}

Register custom tools alongside defaults:

#![allow(unused)]
fn main() {
use yoagent::tools::default_tools;

let mut tools = default_tools();
tools.push(Box::new(WeatherTool));
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5")).with_tools(tools);
}

Sandboxing the built-in file tools

ReadFileTool, WriteFileTool, EditFileTool, ListFilesTool and SearchTool accept an allowlist of directory roots. Empty (the default) means unrestricted:

#![allow(unused)]
fn main() {
let roots = vec!["/srv/workspace".to_string()];

let tools: Vec<Box<dyn AgentTool>> = vec![
    Box::new(ReadFileTool::new().with_allowed_paths(roots.clone())),
    Box::new(WriteFileTool::new().with_allowed_paths(roots.clone())),
    Box::new(EditFileTool::new().with_allowed_paths(roots)),
];
}

Enforcement is against the resolved path, not the string, so neither .. nor a symlink pointing outside can escape — and a file that does not exist yet still resolves through its real parent, so writes are checked too. The rejection message deliberately does not echo the allowed roots, since tool results reach the model's transcript.

BashTool is not a sandbox

BashTool runs whatever the model asks through bash -c, with the agent process's own environment and filesystem access.

deny_patterns is a substring check that catches typos and obvious mistakes — rm -rf / with two spaces, a base64-decoded pipe, or an equivalent find -delete all sail past it. Treat it as a guardrail, never as a security boundary.

Real isolation belongs outside the tool: run the agent in a container or VM, or gate calls through ToolMiddleware, which sees the arguments before execution and can deny them.

For credentials specifically, commands inherit every environment variable the agent process holds — including any *_API_KEY. When the model composes the command, restrict what it can read:

#![allow(unused)]
fn main() {
let bash = BashTool::default()
    .with_env_allowlist(vec!["RUST_LOG".to_string()]);  // plus PATH, HOME, PWD
}

Error Handling

Return Err(ToolError) on failure, not Ok with error text. When a tool returns Err, the agent loop converts it to a Message::ToolResult with is_error: true and sends it to the LLM. The LLM sees the error and can self-correct — retry with different arguments, try a different approach, or explain the failure to the user.

#![allow(unused)]
fn main() {
async fn execute(&self, params: serde_json::Value, _ctx: ToolContext) -> Result<ToolResult, ToolError> {
    let path = params["path"].as_str()
        .ok_or(ToolError::InvalidArgs("missing 'path'".into()))?;

    let content = std::fs::read_to_string(path)
        .map_err(|e| ToolError::Failed(format!("Cannot read {}: {}", path, e)))?;

    Ok(ToolResult {
        content: vec![Content::Text { text: content }],
        details: serde_json::Value::Null,
    })
}
}

Exception: BashTool. The built-in BashTool returns Ok even on non-zero exit codes, with both stdout and stderr in the result. This is intentional — the LLM needs to see the actual error output (compilation errors, test failures, etc.) to diagnose and fix issues. Only truly exceptional failures (e.g., command not found, cancellation) return Err.

Tool Execution Flow

1. LLM returns Content::ToolCall blocks in its response
2. Agent loop emits ToolExecutionStart for each
3. Tool's execute() is called with parsed arguments
4. Result (or error) is wrapped in Message::ToolResult
5. ToolExecutionEnd is emitted
6. All tool results are added to context
7. Loop continues with another LLM call

Streaming Tool Output

Long-running tools can stream progress updates to the UI via the on_update callback. Each call emits a ToolExecutionUpdate event. Partial results are for UI/logging only — they are not sent to the LLM. Only the final ToolResult returned from execute() becomes part of the conversation.

The ToolUpdateFn type

#![allow(unused)]
fn main() {
pub type ToolUpdateFn = Arc<dyn Fn(ToolResult) + Send + Sync>;
}

Basic usage

Call on_update whenever you have progress to report:

#![allow(unused)]
fn main() {
use yoagent::types::*;

struct DataProcessorTool;

#[async_trait]
impl AgentTool for DataProcessorTool {
    // ... name, label, description, parameters_schema ...

    async fn execute(
        &self,
        params: serde_json::Value,
        ctx: ToolContext,
    ) -> Result<ToolResult, ToolError> {
        let rows = fetch_rows(&params)?;
        let total = rows.len();

        for (i, row) in rows.iter().enumerate() {
            // Check for cancellation
            if ctx.cancel.is_cancelled() {
                return Err(ToolError::Cancelled);
            }

            process_row(row);

            // Stream progress every 100 rows
            if i % 100 == 0 {
                if let Some(ref cb) = &ctx.on_update {
                    cb(ToolResult {
                        content: vec![Content::Text {
                            text: format!("Processed {}/{} rows", i, total),
                        }],
                        details: serde_json::json!({"progress": i as f64 / total as f64}),
                    });
                }
            }
        }

        Ok(ToolResult {
            content: vec![Content::Text {
                text: format!("Processed all {} rows", total),
            }],
            details: serde_json::Value::Null,
        })
    }
}
}

Consuming updates in your UI

Updates arrive as AgentEvent::ToolExecutionUpdate events on the same event stream as all other agent events:

#![allow(unused)]
fn main() {
while let Some(event) = rx.recv().await {
    match event {
        AgentEvent::ToolExecutionStart { tool_name, .. } => {
            println!("⏳ {} started", tool_name);
        }
        AgentEvent::ToolExecutionUpdate { tool_name, partial_result, .. } => {
            // Show progress in your UI
            if let Some(Content::Text { text }) = partial_result.content.first() {
                println!("  📊 {}: {}", tool_name, text);
            }
        }
        AgentEvent::ToolExecutionEnd { tool_name, is_error, .. } => {
            println!("{} {}", if is_error { "❌" } else { "✅" }, tool_name);
        }
        AgentEvent::ProgressMessage { tool_name, text, .. } => {
            println!("  💬 {}: {}", tool_name, text);
        }
        _ => {}
    }
}
}

Progress Messages

In addition to on_update (which streams partial ToolResult values), tools can emit lightweight text-only progress messages via ctx.on_progress. These appear as AgentEvent::ProgressMessage events:

#![allow(unused)]
fn main() {
async fn execute(&self, params: serde_json::Value, ctx: ToolContext) -> Result<ToolResult, ToolError> {
    if let Some(ref progress) = &ctx.on_progress {
        progress("Starting analysis...".into());
    }

    // ... do work ...

    if let Some(ref progress) = &ctx.on_progress {
        progress("Almost done...".into());
    }

    Ok(ToolResult { /* ... */ })
}
}

Use on_progress for simple status text. Use on_update when you need structured data (progress percentages, partial results).

Guidelines

• Call on_update as often as useful — there's no rate limit. The callback is synchronous and cheap.
• Always check ctx.on_update.is_some() before building the ToolResult. If None, the loop isn't interested in updates (e.g., testing).
• Use details for structured data — content is for human-readable text, details can carry progress percentages, byte counts, etc.
• Don't rely on updates reaching the LLM — they won't. Only the final return value is added to context.
• Simple tools don't need it — if your tool completes in <1 second, just ignore ctx (prefix with _ctx to suppress the warning).

End-to-end example

Here's a complete example: a CLI agent with a deploy tool that streams progress. The human sees real-time output while the LLM only gets the final result.

use yoagent::agent::Agent;
use yoagent::provider::ModelConfig;
use yoagent::types::*;

/// A tool that deploys an app and streams each step.
struct DeployTool;

#[async_trait]
impl AgentTool for DeployTool {
    fn name(&self) -> &str { "deploy" }
    fn label(&self) -> &str { "Deploy App" }
    fn description(&self) -> &str { "Deploy the application to production." }
    fn parameters_schema(&self) -> serde_json::Value {
        serde_json::json!({
            "type": "object",
            "properties": {
                "env": { "type": "string", "description": "Target environment" }
            },
            "required": ["env"]
        })
    }

    async fn execute(
        &self,
        params: serde_json::Value,
        ctx: ToolContext,
    ) -> Result<ToolResult, ToolError> {
        let env = params["env"].as_str().unwrap_or("staging");

        let steps = ["Building image", "Running tests", "Pushing to registry", "Rolling out"];
        for (i, step) in steps.iter().enumerate() {
            if ctx.cancel.is_cancelled() {
                return Err(ToolError::Cancelled);
            }

            // Stream each step to the UI
            if let Some(ref cb) = &ctx.on_update {
                cb(ToolResult {
                    content: vec![Content::Text {
                        text: format!("[{}/{}] {}...", i + 1, steps.len(), step),
                    }],
                    details: serde_json::json!({
                        "step": i + 1,
                        "total": steps.len(),
                        "phase": step,
                    }),
                });
            }

            // Simulate work
            tokio::time::sleep(std::time::Duration::from_secs(2)).await;
        }

        // Only this final result is sent to the LLM
        Ok(ToolResult {
            content: vec![Content::Text {
                text: format!("Successfully deployed to {}", env),
            }],
            details: serde_json::json!({"env": env, "status": "success"}),
        })
    }
}

#[tokio::main]
async fn main() {
    let mut agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
        .with_system_prompt("You are a deployment assistant.")
        .with_tools(vec![Box::new(DeployTool)]);

    let mut rx = agent.prompt("Deploy to production").await;

    while let Some(event) = rx.recv().await {
        match event {
            // LLM text streaming
            AgentEvent::MessageUpdate {
                delta: StreamDelta::Text { delta }, ..
            } => print!("{}", delta),

            // Tool progress streaming
            AgentEvent::ToolExecutionStart { tool_name, .. } => {
                println!("\n🚀 Starting {}...", tool_name);
            }
            AgentEvent::ToolExecutionUpdate { partial_result, .. } => {
                if let Some(Content::Text { text }) = partial_result.content.first() {
                    println!("  {}", text);
                }
            }
            AgentEvent::ToolExecutionEnd { tool_name, is_error, .. } => {
                if is_error {
                    println!("  ❌ {} failed", tool_name);
                } else {
                    println!("  ✅ {} complete", tool_name);
                }
            }
            AgentEvent::ProgressMessage { text, .. } => {
                println!("  💬 {}", text);
            }

            AgentEvent::AgentEnd { .. } => break,
            _ => {}
        }
    }
}

Running this produces:

🚀 Starting deploy...
  [1/4] Building image...
  [2/4] Running tests...
  [3/4] Pushing to registry...
  [4/4] Rolling out...
  ✅ deploy complete
Successfully deployed to production. The deployment completed all 4 stages.

The human sees each step as it happens. The LLM only sees "Successfully deployed to production" and can continue the conversation from there.

How agents benefit

When an AI agent (like a coding assistant) uses yoagent, streaming tool output helps in two ways:

1. Human oversight — The human watching the agent work sees real-time progress instead of waiting for a tool to finish. A bash command running cargo build can stream compiler output as it happens, so the human can interrupt early if something is wrong.

2. Agent UIs — Tools like web dashboards, IDE extensions, or chat interfaces can render live progress bars, log tails, or status indicators. The details field in ToolResult carries structured data (progress percentage, byte counts, etc.) that UIs can render however they want.

The LLM itself doesn't see updates — it works with final results only. This is intentional: partial output would waste context tokens and confuse the model. The streaming is purely a human-facing feature.

Execution Strategies

When the LLM returns multiple tool calls in a single response (e.g., "read file A, read file B, run bash C"), ToolExecutionStrategy controls how they run:

Configuration

#![allow(unused)]
fn main() {
use yoagent::agent::Agent;
use yoagent::types::ToolExecutionStrategy;

// Default — parallel (fastest)
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"));

// Sequential (debug / shared state)
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .with_tool_execution(ToolExecutionStrategy::Sequential);

// Batched — 3 at a time
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .with_tool_execution(ToolExecutionStrategy::Batched { size: 3 });
}

When to use each

• Parallel (default): Most tool calls are independent — file reads, searches, API calls. Running them concurrently can cut latency dramatically (3 tools × 50ms = ~50ms instead of ~150ms).
• Sequential: When tools have side effects that depend on order, or when you need fine-grained steering control between each tool.
• Batched: When you want parallelism but also want steering checkpoints. For example, Batched { size: 3 } runs 3 tools concurrently, checks for user interrupts, then runs the next 3.

Steering messages are always checked between execution units (between each tool in Sequential, after all tools in Parallel, between batches in Batched). If a user interrupts, remaining tools are skipped.

Permissions: Tool Middleware

Every tool call can be gated by an async middleware chain — the mechanism behind permission prompts, policy engines, and argument rewriting. yoagent ships the hook, not a policy: with no middleware installed, every call runs.

#![allow(unused)]
fn main() {
use yoagent::{ToolCallRequest, ToolDecision, ToolMiddleware};

struct ReadOnlyPolicy;

#[async_trait::async_trait]
impl ToolMiddleware for ReadOnlyPolicy {
    async fn before_tool(&self, call: &ToolCallRequest<'_>) -> ToolDecision {
        match call.tool_name {
            "write_file" | "edit_file" | "bash" => {
                ToolDecision::Deny("read-only session".into())
            }
            _ => ToolDecision::Allow,
        }
    }
}

let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Sonnet 5"))
    .with_tools(default_tools())
    .with_tool_middleware(ReadOnlyPolicy);
}

Semantics:

• Allow — the call proceeds (with the current arguments).
• Modify(args) — the call proceeds with replacement arguments (e.g. rewrite a path into a sandbox). Later middleware in the chain see the rewritten arguments; the ToolExecutionStart event carries what actually runs.
• Deny(reason) — the call never executes. The reason is returned to the LLM as an error tool result ("Tool call denied: ..."), so the model can adapt — pick another tool, ask the user — and the loop continues. A denial never aborts the run.

A middleware that panics is contained: the call is denied (reason "tool middleware panicked") and the loop continues — a buggy policy can't kill the run.

The hook is async, so an interactive app can prompt a human before deciding. Note that under the default Parallel execution strategy, middleware for parallel tool calls run concurrently — if you need one-at-a-time approval UX, serialize inside your middleware (e.g. a tokio::sync::Mutex) or switch to ToolExecutionStrategy::Sequential.

Sub-agents gate their own tool calls the same way via SubAgentTool::with_tool_middleware.

Structured Outputs

Get a typed, schema-validated reply instead of free text. The JSON Schema is enforced natively by the provider — not by prompt begging.

#![allow(unused)]
fn main() {
use yoagent::{Agent, provider::ModelConfig};

#[derive(serde::Deserialize)]
struct Invoice {
    vendor: String,
    total_cents: u64,
    line_items: Vec<String>,
}

let mut agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Sonnet 5"));

let invoice: Invoice = agent
    .prompt_structured(
        "Extract the invoice from the attached text: ...",
        serde_json::json!({
            "type": "object",
            "properties": {
                "vendor": {"type": "string"},
                "total_cents": {"type": "integer"},
                "line_items": {"type": "array", "items": {"type": "string"}}
            },
            "required": ["vendor", "total_cents", "line_items"]
        }),
    )
    .await?;
}

Derive the schema however you like — by hand as above, or with the schemars crate (convert with serde_json::to_value(schemars::schema_for!(Invoice))). Mind the provider dialects: OpenAI strict mode requires additionalProperties: false and every property listed in required; Gemini rejects $defs/$ref. Schemas are passed through as given.

How each provider enforces it

Semantics & caveats

• prompt_structured runs the loop to completion internally and returns the parsed T — there is no event receiver for this call.
• Three error shapes: Provider { message } when the API call itself failed (auth, network, a schema-induced 400 — retrying the parse is pointless); Parse { source, raw } when the model's text didn't deserialize (the raw text is preserved so you can retry or salvage); NoOutput when the run produced no text. Only messages produced by this call are considered — stale output from earlier turns is never parsed.
• On Anthropic the forced tool call preempts regular tools for that request, and disables extended thinking for that request (forced tool choice and thinking are mutually exclusive at the API level — a warning is logged). Treat structured prompts as extraction/finalization calls, not agentic tool-using turns.
• Markdown code fences around the JSON are stripped defensively before parsing.

Context Management

Long-running agents accumulate messages that exceed the model's context window. yoagent provides token tracking, overflow detection, tiered compaction, and execution limits.

Token Estimation

Fast estimation without external tokenizer dependencies:

#![allow(unused)]
fn main() {
use yoagent::context::{estimate_tokens, message_tokens, total_tokens};

estimate_tokens("Hello world");          // ~3 tokens (chars / 4)
message_tokens(&agent_message);          // estimate for a single message
total_tokens(&messages);                 // estimate for all messages
}

Context Tracking

ContextTracker combines real token counts from provider responses with estimation for new messages — more accurate than pure estimation:

#![allow(unused)]
fn main() {
use yoagent::context::ContextTracker;

let mut tracker = ContextTracker::new();

// After each assistant response, record the real usage:
tracker.record_usage(&assistant_usage, message_index);

// Get current context size (real usage + estimated trailing):
let tokens = tracker.estimate_context_tokens(agent.messages());

// After compaction, reset the tracker:
tracker.reset();
}

When no usage data is available, it falls back to chars/4 estimation.

Context Overflow Detection

When the context exceeds a model's window, providers return overflow errors. yoagent detects these automatically across all major providers.

HTTP-level detection

Providers that check before streaming (Google, Bedrock, Vertex) return ProviderError::ContextOverflow:

#![allow(unused)]
fn main() {
use yoagent::provider::ProviderError;

match agent.prompt("...").await {
    // The loop already handles this — but you can also match it:
    Err(ProviderError::ContextOverflow { message }) => {
        // Compact and retry
    }
    _ => {}
}
}

ProviderError::classify() auto-detects overflow from error messages covering Anthropic, OpenAI, Google, AWS Bedrock, xAI, Groq, OpenRouter, llama.cpp, LM Studio, MiniMax, Kimi, GitHub Copilot, and generic patterns.

Message-level detection

SSE-based providers (Anthropic, OpenAI) return overflow as a StopReason::Error message. Check with:

#![allow(unused)]
fn main() {
if message.is_context_overflow() {
    // Compact and retry
}
}

Handling overflow in your application

yoagent provides the detection and building blocks. Your application wires the compaction strategy:

#![allow(unused)]
fn main() {
// Proactive: check before each prompt
let tokens = tracker.estimate_context_tokens(agent.messages());
if tokens > context_window - reserve {
    let compacted = compact_messages(agent.messages().to_vec(), &config);
    agent.replace_messages(compacted);
}

// Reactive: catch overflow errors
// ... on ContextOverflow or message.is_context_overflow():
//   compact, then retry with agent.continue_loop()
}

For LLM-based summarization (asking the model to summarize old messages), implement that in your application layer — yoagent provides replace_messages() and compact_messages() as building blocks.

ContextConfig

#![allow(unused)]
fn main() {
pub struct ContextConfig {
    pub max_context_tokens: usize,                            // Default: 100,000
    pub system_prompt_tokens: usize,                          // Default: 4,000
    pub keep_recent: usize,                                   // Default: 10
    pub keep_first: usize,                                    // Default: 2
    pub tool_output_max_lines: usize,                         // Default: 200
    pub tool_output_max_lines_overrides: HashMap<String, usize>, // Default: {"read_file": MAX}
    pub compact_target_ratio: f32,                            // Default: 0.7
    pub compact_headroom_turns: Option<usize>,                // Default: Some(30)
    pub truncate_tool_output_on_append: bool,                 // Default: true
}
}

Auto-Derivation from ModelConfig

When you set a ModelConfig but don't explicitly set a ContextConfig, the compaction budget is automatically derived from the model's context_window — reserving 80% for context and 20% for output:

#![allow(unused)]
fn main() {
// MiniMax with 1M context → compacts at 800K (no manual config needed)
let agent = Agent::from_config(ModelConfig::minimax("MiniMax-Text-01", "MiniMax Text 01"));

// Anthropic with 200K context → compacts at 160K
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"));
}

The priority chain:

1. Explicit with_context_config(...) → always wins
2. Has model_config → auto-derives from context_window (80%)
3. Neither → ContextConfig::default() (100K)

You can also derive manually:

#![allow(unused)]
fn main() {
let config = ContextConfig::from_context_window(1_000_000);
// config.max_context_tokens == 800_000
}

Tiered Compaction

compact_messages() tries each level in order, stopping as soon as messages fit the budget:

Level 1: Truncate Tool Outputs

Replaces long tool outputs with head + tail, fitting the result into exactly tool_output_max_lines lines (the [... N lines truncated ...] marker is charged against that budget). This is the cheapest level — it preserves conversation structure and typically saves 50-70% in coding sessions.

Truncation is idempotent: re-running it on an already-truncated output returns it byte for byte, so a session that sits above the budget does not re-truncate the same outputs turn after turn.

Level 2: Summarize Old Turns

Keeps the last keep_recent messages in full detail. Older assistant messages are replaced with one-line summaries like "[Summary] [Assistant used 3 tool(s)]", and their tool results are dropped. The boundary is pulled back to a turn start so an assistant message and its tool results are never split across it.

Level 3: Drop Middle Messages

Drops the smallest span of middle messages that reaches the target, keeping at least keep_first from the start and keep_recent from the end. A constant marker message stands in for what was removed; the count goes to the debug log rather than into the marker, so the text does not change from pass to pass.

Retrievable tool output

Head-tail truncation on the append path keeps a huge tool result from eating the context, but the middle is gone irrecoverably — it survives only in the event stream, which the agent cannot read.

Attach a SharedState and the full text is stashed, with the marker naming where it went:

#![allow(unused)]
fn main() {
let agent = Agent::from_config(config)
    .with_shared_state(SharedState::new());
}

[... 1847 lines truncated — full output: shared_state get "tool-out-tc_01abc-9f2a-b0" ...]

The shared_state tool is registered for the run, so the model can act on the pointer. Opt-in: with no store attached, truncation behaves exactly as before and the marker advertises no retrieval it cannot honour.

Keys are block-qualified — a result carrying several text blocks gets one key per block, suffixed by the block's position in the content vector, so text/image/text yields …-b0 and …-b2. The key combines the tool call id with a hash of the output, because Gemini synthesizes call ids as a per-response index that restarts every turn; id alone would let turn 1's frozen marker resolve to turn 5's content.

Two limits worth knowing:

• Lossy compaction drops the marker but not the stash entry. Levels 2 and 3 drop whole turns, taking the pointer with them, while the stored value lives on and keeps consuming cap quota.
• Stash entries are evictable; caller keys are not. Both backends evict oldest-first under their cap, but only tool-out-* entries — losing one degrades a marker to an ordinary "key not found" the agent can act on, and the head+tail is still in the transcript. Nothing regenerates an artifact you stored yourself, so when only caller keys remain the write reports capacity instead.

SubAgentTool::with_context_config makes the same path reachable for sub-agents; their stash is scoped, and scoped keys are excluded from the system prompt summary so a second delegation does not see the first one's keys and cold -start the prefix cache.

Prefix Cache Stability

Providers cache request prefixes — automatically on DeepSeek, explicitly via cache_control on Anthropic. A cache hit needs the new request to share a byte-identical prefix with the last one, so every rewrite of already-sent history costs full price for every token from the rewrite point onward.

Compaction is built around that:

• Levels are idempotent where they can be, so re-running on settled history changes nothing.
• compact_headroom_turns sets the compaction target from the session's observed growth rate — target = budget − turns × growth_per_turn — so the gap between compactions stays constant instead of collapsing as history accumulates. compact_target_ratio is the fallback and acts as a ceiling on retention. See Prompt Caching for the full formula and measurements.
• Markers and generated summaries carry no wall-clock timestamps or drifting counts, so the same history always compacts to the same bytes.

Bounding tool output: two layers

The largest source of cache loss is retroactive truncation — output is sent in full, cached, then rewritten by Level 1 in one sweep once the session goes over budget. Bounding it as it arrives fixes that, but a single global cap is the wrong instrument, because tools differ in how well they survive being cut:

Command output (bash, search) takes a head+tail cut well: the first error is at the top, the summary at the bottom, and the middle is repetition. tool_output_max_lines (200) is applied on append, controlled by truncate_tool_output_on_append (on by default).

File reads do not. The middle of a source file is usually the part that was asked for, so head+tail removes exactly the wrong lines. read_file bounds itself instead, by paging: it returns DEFAULT_READ_MAX_LINES (500) at a time with a header stating the true total, and the agent asks for the next range with offset/limit. That bound is lossless and directed. read_file is therefore exempt from head+tail truncation by default, via tool_output_max_lines_overrides.

Custom tools pick their own budget the same way:

#![allow(unused)]
fn main() {
let mut config = ContextConfig::from_context_window(128_000);
config.tool_output_max_lines_overrides.insert("my_paging_tool".into(), usize::MAX);
config.tool_output_max_lines_overrides.insert("noisy_tool".into(), 40);
}

To restore pre-0.15 behaviour, set truncate_tool_output_on_append: false and construct ReadFileTool { max_lines: usize::MAX, ..Default::default() }.

Measured effect

tests/context_cache_test.rs replays 300 turns on a 128K window. The tool mix (bash 41%, edit/write 36%, read 19%, search 4%) and file-size distribution come from 808 archived runs of a production agent built on yoagent.

In input-token spend that is −9.2% to −21.3% on DeepSeek and −15.2% to −22.8% on Anthropic, widening with session length. Full breakdown and the reasoning behind each default: Prompt Caching.

ExecutionLimits

Prevents runaway agents:

#![allow(unused)]
fn main() {
#[non_exhaustive]
pub struct ExecutionLimits {
    pub max_turns: usize,              // Default: 50
    pub max_total_tokens: usize,       // Default: 1,000,000
    pub max_duration: Duration,        // Default: 600s (10 min)
    pub max_consecutive_identical_tool_calls: Option<usize>,  // Default: Some(3)
}
}

When a limit is reached, the agent stops with a message like "[Agent stopped: Max turns reached (50/50)]".

Loop detection

The cheapest catastrophic failure is a model calling one tool with the same arguments forever. The three limits above all fire eventually — but only after the run has burned its entire turn, token and wall-clock budget to discover it achieved nothing.

max_consecutive_identical_tool_calls is on by default at Some(3), with two escalations that mirror the house pattern of steering before aborting:

1. First trip injects a steering message and continues. A model repeating a call is often retrying something transient, and aborting immediately would regress that legitimate case.
2. A later trip on the same signature stops the run.

Both emit AgentEvent::LoopDetected { tool_name, repetitions, aborted }, so a UI can show the intervention and an audit can tell a loop abort from a turn-limit stop.

Signatures compare serde_json::Value, not serialized text — two calls differing only in key order are the same call. Counting covers duplicates within one batch as well as across turns, because ToolExecutionStrategy defaults to Parallel and a model can emit the same call three times in a single message.

Consecutive, and that word is load-bearing: a different call resets the streak, so an alternating [a, b, a, b, …] loop is not detected. That is a deliberate trade — an agent working through a list calls one tool repeatedly and legitimately, and a detector that fired on interleaved repeats would be worse than none.

#![allow(unused)]
fn main() {
ExecutionLimits::default().with_max_consecutive_identical_tool_calls(None)  // off
}

Disabling Context Management

#![allow(unused)]
fn main() {
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .without_context_management();
}

This sets both context_config and execution_limits to None.

Prompt Caching

yoagent automatically optimizes API costs through prompt caching. For providers that support it, stable content (system prompts, tool definitions, conversation history) is cached between turns, giving you up to 90% savings on input tokens.

How It Works

In a multi-turn agent loop, each request sends the full context: system prompt + tools + conversation history. Without caching, you pay full price for all of it every turn. With caching, the provider reuses previously processed prefixes.

Provider Support

Savings are the cached-read discount off base input in this crate's own price presets (CostConfig) — e.g. gpt_5_5 prices input at $5/MTok and cache reads at $0.50. They are model-dependent; check CostConfig for the model you use.

"Not wired" means the provider supports something and yoagent does not send it yet, as distinct from "None needed", where the provider caches on its own.

Providers cache in two different shapes, and CacheStrategy means something different in each. Explicit breakpoints (Anthropic) let the client choose cache boundaries and charge a write premium for them — that is what Auto and Manual place. Key-routed (OpenAI) caches automatically once a prefix passes ~1024 tokens; there are no boundaries to choose, and prompt_cache_key only steers requests from one conversation toward the same cache. Everything else caches server-side with nothing to configure, and CacheStrategy is inert — including Disabled, which cannot switch off caching the client never requested.

The practical consequence: a hit rate is not comparable across protocols without knowing which shape produced it. See the measured comparison below.

What Gets Cached (Anthropic)

yoagent places up to 3 cache breakpoints automatically:

1. System prompt — stable across all turns
2. Tool definitions — rarely change between turns
3. Conversation history — second-to-last message, so the growing prefix is cached

This means on a typical multi-turn conversation, only the latest user message and the new assistant response cost full price.

OpenAI

OpenAI caches prefixes of roughly 1024 tokens or more on its own, so there is nothing to place. What yoagent sends is prompt_cache_key, which routes a request to a machine while the cache itself stays content-addressed.

The key comes from CacheConfig::session_key when you set one, and is otherwise derived from the system prompt alone. That is the cacheable prefix — the thing the provider actually caches — and StreamConfig carries it in its own field, where compaction cannot reach it.

An earlier version mixed in the first user message too, for session discrimination. It did not survive yoagent's own compaction: compact_messages can drop the head and insert a constant marker at index 0 (constant on purpose, so the cached prefix stays byte-stable). The key drifted mid-session and every session sharing a system prompt collapsed onto one value — failing precisely on long sessions, which are the ones caching exists for. Session identity is not recoverable from a per-request snapshot.

So sessions sharing a system prompt share a key by design. That is the correct grouping — they share the cached prefix — but it concentrates traffic on one cache. Set session_key explicitly to spread load, which is also what OpenAI recommends for a hot key:

let agent = Agent::from_config(ModelConfig::openai("gpt-5.5", "GPT-5.5"))
    .with_cache_config(CacheConfig::default().with_session_key(format!("tenant-{id}/{session_id}")));

A blank key is treated as unset rather than sent literally — an empty prompt_cache_key would route every caller who did that onto one cache.

This is gated on the supports_prompt_cache_key compat flag, which is on only for native OpenAI. prompt_cache_key is OpenAI's field: a strict OpenAI-compatible server that validates unknown keys would reject the whole request rather than ignore it, and the providers that cache automatically were never reading it.

DeepSeek

DeepSeek's API manages context caching automatically. yoagent does not send Anthropic-style cache_control markers for DeepSeek; instead, keep stable prefixes stable (system prompt, tool definitions, and earlier messages) and monitor DeepSeek's prompt_cache_hit_tokens / prompt_cache_miss_tokens usage fields through Usage.cache_read and Usage.input.

Google Gemini — implicit only, deliberately

yoagent sends no cache directives to Gemini and ignores CacheStrategy there. Gemini caches implicitly, and the cachedContentTokenCount yoagent reads back into Usage.cache_read reports that automatic behaviour — it is telemetry, not an acknowledgement of anything the client asked for.

Gemini's explicit caching is a different kind of thing: you create a CachedContent object, get a handle, reference it by name on later requests, and manage its TTL and its own billing line. That does not fit behind CacheStrategy, which describes where to put markers inside a single request. Wiring it there would mean either misrepresenting a stateful server-side resource as a per-request flag, or silently creating objects whose lifetime the caller cannot see. It is worth doing as its own API, and it is not worth pretending the current enum can express it.

Configuration

Caching is enabled by default with automatic breakpoint placement. No configuration needed for optimal behavior.

Disable Explicit Cache Hints

#![allow(unused)]
fn main() {
use yoagent::{CacheConfig, CacheStrategy};

let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .with_cache_config(CacheConfig::disabled());
}

This suppresses every cache hint yoagent would send — Anthropic's cache_control breakpoints and OpenAI's prompt_cache_key alike. It does not turn off a provider's automatic server-side caching, such as DeepSeek's or Gemini's, which is not under client control.

Fine-Grained Control

#![allow(unused)]
fn main() {
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .with_cache_config(CacheConfig::new().with_strategy(CacheStrategy::Manual {
        cache_system: true,
        cache_tools: true,
        cache_messages: false, // Don't cache conversation history
    }));
}

CacheConfig is #[non_exhaustive], so construct it with new() / disabled() / default() plus the with_strategy and with_session_key builders rather than a struct literal.

Setting every Manual flag to false means "cache nothing" and is treated as equivalent to Disabled on every protocol — including key-routed ones, which send no key in that case.

Monitoring Cache Usage

Every Usage struct includes cache statistics:

#![allow(unused)]
fn main() {
// After a response:
let usage = message.usage(); // from assistant message
println!("Cache read: {} tokens", usage.cache_read);
println!("Cache write: {} tokens", usage.cache_write);
println!("Cache hit rate: {:.1}%", usage.cache_hit_rate() * 100.0);
}

• cache_read — tokens served from cache (cheap)
• cache_write — tokens written to cache when the provider reports that metric
• cache_hit_rate() — fraction of input tokens from cache (0.0–1.0)

Cost Impact

For a typical 10-turn agent conversation with Anthropic Claude:

That's an ~80% cost reduction with zero configuration.

Cache-Stable Compaction

Everything above is about placing cache breakpoints correctly. That only pays off if the bytes behind the breakpoint stop changing — and in a long session the thing that changes them is compaction.

A cache hit requires the new request to share a byte-identical prefix with the previous one. So the moment compaction rewrites a message that has already been sent, every token from that point onward is uncached and billed at full price. Before 0.15, compaction decided purely on a token budget and never counted that cost.

yoagent 0.15 treats this as a first-class design constraint, in three parts.

1. History stops churning

Compaction is a pure function of its input, and idempotent where it can be:

• Tool-output truncation fits its own budget. The [... N lines truncated ...] marker is charged against tool_output_max_lines, so truncating an already-truncated result returns it byte for byte. Previously the marker pushed the result over the limit, so the next pass re-truncated it and restated the count — a second full-prefix invalidation on top of the first.
• Markers carry no drifting state. The compaction marker's text is constant and generated summaries inherit the timestamp of what they replace, so the same history always compacts to the same bytes.
• Boundaries snap to turn starts, which also prevents orphaned tool_use/tool_result pairs that providers reject outright.

2. Output is bounded where it can be bounded well

One global line cap cannot serve every tool, because tools differ in how well they survive being cut:

Applying the cap on append rather than retroactively is what keeps the prefix intact: the bytes the provider cached are the bytes that stay. Per-tool budgets live in tool_output_max_lines_overrides; usize::MAX exempts a tool.

3. The compaction target adapts

This is the part a fixed setting cannot do.

Compaction triggers at the budget but reduces to a target below it, so the next turn does not immediately cross the budget again and rewrite history. Expressing that target as a fixed fraction has a flaw: a ratio has no idea how fast the session is growing, so the headroom it leaves is arbitrary, and the interval between compactions collapses as history accumulates.

compact_headroom_turns targets the interval directly:

target        = budget − turns × growth_per_turn
effective     = clamp( min(target / budget, compact_target_ratio), MIN_HEADROOM_RATIO, 1.0 )

• budget — max_context_tokens − system_prompt_tokens
• turns — compact_headroom_turns, how many more turns you want before the next compaction (default 30)
• growth_per_turn — mean tokens added per turn, measured by the agent loop
• compact_target_ratio — a ceiling on retention: the policy may compact harder than the ratio, never softer
• MIN_HEADROOM_RATIO (0.15) — a floor, so runaway growth cannot ask compaction to discard everything

One interpretable knob — how often am I willing to compact — that behaves the same at turn 50 and turn 2400, and self-adjusts to workload: an agent producing huge tool output gets compacted harder automatically. It is also self-limiting; on short sessions where compaction is already rare, the derived ratio ties the fixed one and nothing changes.

Mean turns between compactions:

Set compact_headroom_turns: None to restore pure ratio behaviour.

Choosing a value

Replaying a 2400-turn session at a 1:10 cache-to-input price ratio:

The trade is proportional, so pick on memory rather than cost. Cost per unit of retained context barely moves across the whole range — 4.94 down to 4.73, about 4%. There is no efficient point in this table for the data to single out: lowering the setting sells retained history for money at a nearly fixed exchange rate. Going from 30 to 40 costs 8.7% less and keeps 8.4% less. Choose it on how much history your agent needs to do its job; the cost follows mechanically.

The 4% that is not proportional is the genuinely free part, and it comes from compacting less often rather than from retaining less.

Above roughly 60 the setting stops doing anything. The derived ratio hits MIN_HEADROOM_RATIO (0.15) and clamps, so 60, 80, 100 and 140 produce byte-identical runs. To compact harder than that, lower compact_target_ratio — it is the ceiling the headroom policy is capped by, not an independent knob.

Cheaper cache argues for a smaller context, not a larger one. The same step from a fixed ratio to headroom 60 saves 19.4% at a 1:4 cache-to-input ratio and 26.0% at 1:50. That reads backwards until you notice that once cached tokens are nearly free per token, they dominate the count — and you are re-sending them on every request. Vendor ratios in practice run from about 1:10 to 1:50, so the payoff for a smaller working context is larger than a 1:4 assumption would suggest.

Measured effect

tests/context_cache_test.rs replays a session through compaction turn by turn, renders each turn as a provider body would see it, and measures the shared prefix between consecutive requests — the direct analogue of DeepSeek's prompt_cache_hit_tokens / prompt_tokens. The tool mix (bash 41%, edit/write 36%, read 19%, search 4%) and file-size distribution come from 808 archived runs of a production agent built on yoagent. Old and new are measured through the identical code path, on a 128K window.

The rewrite count is the sharper signal: 0.14.2 kept its hit rate up by compacting constantly to a small context — 415 rewrites over 2400 turns — which is precisely the churn that costs money.

Priced as input-token spend across the whole session:

The gap widens with session length, which is the point: the old behaviour degraded as history accumulated, and the new behaviour does not.

Read a hit rate against its session length, not against 100%

A hit rate is only interpretable next to the number of turns that produced it. Every turn's new content has never been sent before, so it can never hit. With n turns of roughly equal size, the arithmetic ceiling is about

(n − 1) / (n + 1)

— roughly 87% at 14 turns, 90% at 19, 95% at 39, and 96% only past 49. The 93–96% figures in the replay table above come from 300–2400 turn sessions, where the ceiling is above 99% and the measured shortfall is genuinely about rewrites. The same code measured over a 14-turn session reads ~80%, and that is not a regression — it is the ceiling.

The practical consequence: do not compare hit rates across sessions of different lengths, and do not treat a published figure as a target for a short session. Compare rewrite counts, or compare dollars.

Measured live: DeepSeek vs Anthropic

examples/llm_compaction_live.rs runs a real multi-turn session and reports per-turn cache accounting. "Not cached" means input + cache_write — both halves are prompt tokens the provider had to process and bill for. Counting only input is the trap: Anthropic books a re-processed prefix to cache_write and DeepSeek has no write category at all, so an input-only metric makes Anthropic look roughly ten times cheaper than it is.

The session hit rate is measured across five runs per provider; the per-turn figures come from one run each, because earlier runs did not record per-turn cache_write and could not be re-derived. Costs are computed from the recorded token counts at list prices — Anthropic's from ModelConfig::claude_sonnet_5, DeepSeek's at its peak-window rate (it charges half off-peak, so that column over-reports an off-peak run by up to 2×).

There is no single "steady-state" number. Within one compaction cycle the hit rate climbs monotonically, because each turn adds a roughly constant amount of new content to a prefix that keeps growing: on DeepSeek the residual is 26–158 tokens a turn, so it goes 88% → 95% → 99.6% across ten turns, then resets when compaction truncates the prefix. Quote the peak for what a mature prefix achieves and the median for what a session with this budget actually experiences; the median moves with how often you compact, so it is a property of your configuration, not of the provider.

DeepSeek also quantises hits to 64-token blocks — every cache_read figure in that run is an exact multiple of 64 — so a trailing partial block is always uncached. That alone caps a single turn a fraction of a point below 100%.

Session rates are close; the mechanisms are not. Both providers land near 80%, even though yoagent places explicit cache_control breakpoints for Anthropic and sends nothing at all to DeepSeek. What differs is where the cost falls.

DeepSeek pays almost nothing between compactions. Populating the cache is free, so the only non-cached tokens are the turn's genuinely new content. Consequently 92% of its non-cached tokens come from the two turns where compaction rewrote history — for this provider, compaction is essentially the whole cache story.

Anthropic pays a write premium continuously. Every turn writes ~2,000–4,600 tokens at 1.25× to extend the cached prefix, which is why its curve climbs more slowly and tops out at 96% rather than 99.7%. Compaction accounts for only ~50% of its non-cached tokens; the other half is that ongoing write traffic. In dollars those writes cost 3.3× the reads they enable ($0.232 vs $0.071) — which sounds damning until you price the counterfactual: the same session with caching off costs $1.377, so the writes are still a 43% saving. Buying reads with writes is the correct trade; it just means the two providers reach a similar hit rate with materially different bills.

Do not read the 25× cost gap as a caching result. The two sessions are not matched: Sonnet ran 20 turns to DeepSeek's 15 and emitted 2.8× the output tokens, and output is the majority of both bills (61% and 73%). The comparison that is controlled is the shape — zero writes versus continuous writes, 92% versus 50% of waste attributable to compaction. The absolute dollars are one session each and should be treated as illustrative.

One null result worth recording: lowering DEFAULT_TRIGGER_RATIO from 0.6 to 0.35, on the theory that a slow summarizer needs more wall-clock headroom before the budget is crossed, changed nothing measurable on either provider. The condition it was meant to address — repeated deterministic fallbacks because no summary arrived in time — did not occur in those runs, so the hypothesis is untested rather than refuted.

Judge changes in dollars, not hit rate

Hit rate is a fair proxy while you are removing needless rewrites. It stops being one the moment a change also moves context size — a larger context can raise the hit rate while raising the bill, because more of what you carry is cached but you are carrying more of it. Every tuning decision above was made on input-token spend for that reason.

Why there is no cost-benefit gate

A natural next step is to price each compaction and skip the ones that do not pay. Compaction is worth it when

R  >  (I / H) · (P_input − P_cache) / P_cache

where H is the tokens it stops resending, I the prefix it actually invalidates, and R the remaining calls. Measured across every compaction event in the replay, I/H runs 0.02–0.72 at typical session lengths, putting break-even at under three remaining calls on DeepSeek pricing. The gate can only fire in the last turns of a session, and enabling it changed total cost by less than 0.2%.

That is a consequence of the work above rather than an argument against the model: once compaction is byte-stable up to the cut point, the invalidation it has to pay for is a fraction of what it saves. The economics are documented by bash-agent, whose measurements prompted this work.

Best Practices

1. Keep system prompts stable — changing the system prompt between turns invalidates the cache
2. Don't shuffle tools — tool order matters for cache prefix matching
3. Let it work automatically — the default CacheStrategy::Auto is optimal for most use cases
4. Monitor cache_hit_rate() — if it's consistently low, check if your system prompt or tools are changing unexpectedly
5. Don't rewrite history yourself — a custom CompactionStrategy or transform_context that edits already-sent messages costs the whole prefix from that point on. Append, or cut at a boundary and leave everything before it byte-identical.
6. Tune compact_headroom_turns, not compact_target_ratio — the ratio has to be re-guessed for every workload and session length; the headroom policy adapts on its own. Lower it if compaction is still too frequent, raise it to preserve more history.
7. Judge in dollars — see the note above; a higher hit rate does not always mean a smaller bill.

Retry with Backoff

When an LLM provider returns a transient error — rate limit (HTTP 429) or network failure — yoagent automatically retries with exponential backoff and jitter. No configuration required; it works out of the box.

How it works

Request → Error? → Retryable? → Wait (backoff + jitter) → Retry → ...
                       ↓ No
                  Fail immediately

1. The agent loop calls the provider
2. If the provider returns a retryable error:
   • If a retry-after delay was provided (rate limits), use that
   • Otherwise, calculate delay: initial_delay × multiplier^(attempt-1) with ±20% jitter
   • Wait, then retry
3. After max_retries attempts, the error propagates normally

What gets retried

Default configuration

#![allow(unused)]
fn main() {
RetryConfig {
    max_retries: 3,          // Up to 3 retry attempts
    initial_delay_ms: 1000,  // 1 second before first retry
    backoff_multiplier: 2.0, // Double the delay each attempt
    max_delay_ms: 30_000,    // Cap at 30 seconds
}
}

With defaults, the retry delays are approximately:

• Attempt 1: ~1s
• Attempt 2: ~2s
• Attempt 3: ~4s

(±20% jitter to avoid thundering herd when multiple agents hit the same provider)

Configuration

Using the Agent builder

#![allow(unused)]
fn main() {
use yoagent::agent::Agent;
use yoagent::retry::RetryConfig;

// Default — 3 retries, exponential backoff (recommended)
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"));

// Custom — more retries, longer initial delay
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .with_retry_config(RetryConfig {
        max_retries: 5,
        initial_delay_ms: 2000,
        backoff_multiplier: 2.0,
        max_delay_ms: 60_000,
    });

// Disable retries entirely
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .with_retry_config(RetryConfig::none());
}

Using AgentLoopConfig directly

#![allow(unused)]
fn main() {
use yoagent::agent_loop::AgentLoopConfig;
use yoagent::retry::RetryConfig;

let config = AgentLoopConfig {
    // ...other fields...
    retry_config: RetryConfig {
        max_retries: 3,
        initial_delay_ms: 1000,
        backoff_multiplier: 2.0,
        max_delay_ms: 30_000,
    },
};
}

Rate limit headers

When a provider returns ProviderError::RateLimited { retry_after_ms: Some(5000) }, yoagent uses that exact delay instead of the calculated backoff. This respects the provider's guidance — if Anthropic says "retry after 5 seconds", we wait 5 seconds, not our own estimate.

If no retry_after_ms is provided, the exponential backoff kicks in.

Observability

Retry attempts are logged via tracing at the WARN level:

WARN Provider error (attempt 1/3), retrying in 1.1s: Rate limited, retry after 1000ms
WARN Provider error (attempt 2/3), retrying in 2.3s: Rate limited, retry after 2000ms

Subscribe to tracing events in your application to surface these in your UI:

#![allow(unused)]
fn main() {
use tracing_subscriber;

// Simple stderr logging
tracing_subscriber::fmt::init();

// Or filter to just retries
tracing_subscriber::fmt()
    .with_env_filter("yoagent::retry=warn")
    .init();
}

Design notes

• Retry lives in the agent loop, not inside individual providers. One config controls all retry behavior.
• Jitter prevents thundering herd: when many agents hit a rate limit simultaneously, jitter spreads their retries so they don't all retry at the same instant.
• Cancellation is respected: if the user cancels while waiting for a retry, the loop exits immediately.
• No retry on API errors: a malformed request will fail the same way every time. Retrying wastes time and tokens.

Skills

Skills extend an agent with domain expertise using the AgentSkills open standard. A skill is a directory containing a SKILL.md file with instructions the agent can load on demand.

How it works

Skills use progressive disclosure to manage context efficiently:

1. Metadata (~100 tokens/skill) — name + description, always in the system prompt
2. Instructions (<5k tokens) — SKILL.md body, loaded when the agent decides the skill is relevant
3. Resources (unlimited) — scripts, references, assets — loaded only when needed

The agent decides when to activate a skill based on the description alone. No trigger engine needed.

Skill format

my-skill/
├── SKILL.md          # Required: YAML frontmatter + instructions
├── scripts/          # Optional: executable code
├── references/       # Optional: documentation loaded on demand
└── assets/           # Optional: templates, static resources

SKILL.md uses YAML frontmatter:

---
name: git
description: Git operations — commit, branch, merge, rebase. Use when the user mentions version control.
---

# Git Skill

## Workflow
1. Run `git status` first
2. Stage changes, write conventional commit messages
3. For merges, check for conflicts first

## Scripts
For complex diffs: `bash {baseDir}/scripts/diff_summary.sh`

Loading skills

#![allow(unused)]
fn main() {
use yoagent::SkillSet;

// Load from multiple directories (later dirs override earlier on name conflict)
let skills = SkillSet::load(&["./skills", "~/.yoagent/skills"])?;

// Or load from a single directory with a label
let workspace_skills = SkillSet::load_dir("./skills", "workspace")?;
}

If skills may be updated independently at runtime, use resilient loading so one malformed SKILL.md does not discard the rest of the set:

#![allow(unused)]
fn main() {
let (skills, errors) = SkillSet::load_resilient(&["./skills", "~/.yoagent/skills"]);

for error in errors {
    tracing::warn!(%error, "skipping malformed skill");
}
}

load_dir_resilient provides the same behavior for a single directory with a custom source label. The original load and load_dir methods remain strict and return the first error they encounter.

Using with Agent

#![allow(unused)]
fn main() {
use yoagent::{Agent, SkillSet};

let skills = SkillSet::load(&["./skills"])?;

let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .with_system_prompt("You are a coding assistant.")
    .with_skills(skills)  // Appends skill index to system prompt
    .with_tools(tools);
}

The agent's system prompt will include:

<available_skills>
  <skill>
    <name>git</name>
    <description>Git operations — commit, branch, merge, rebase.</description>
    <location>/path/to/skills/git/SKILL.md</location>
  </skill>
</available_skills>

When the agent encounters a task matching a skill, it reads the SKILL.md using the read_file tool and follows the instructions. No special infrastructure needed.

Precedence

When loading from multiple directories, later directories take precedence. A skill in ./skills/ overrides the same-named skill in ~/.yoagent/skills/.

You can also merge skill sets explicitly:

#![allow(unused)]
fn main() {
let mut base = SkillSet::load_dir("/usr/share/yoagent/skills", "bundled")?;
let user = SkillSet::load_dir("~/.yoagent/skills", "user")?;
let workspace = SkillSet::load_dir("./skills", "workspace")?;

base.merge(user);
base.merge(workspace); // workspace wins on conflict
}

Compatibility

By following the AgentSkills standard, skills written for yoagent work with Claude Code, Codex CLI, Gemini CLI, Cursor, OpenCode, Goose, and any other compatible agent. Write once, use everywhere.

Design philosophy

Skills are deliberately simple:

• No trigger engine — the LLM decides from descriptions
• No compile-time registration — skills use existing tools (read_file, bash)
• No plugin API — skills are just files
• No runtime loading — loaded at startup, that's it

If a skill needs a custom tool, it can provide an MCP server.

Sub-Agents

Sub-agents let a parent agent delegate tasks to child agent loops, each with their own system prompt, tools, and provider. The parent LLM invokes them like any other tool.

Overview

Parent Agent
├── prompt("Research X and implement Y")
│   ├── calls SubAgentTool("researcher", task="Research X")
│   │   └── child agent_loop() with read/search tools → returns findings
│   ├── calls SubAgentTool("coder", task="Implement Y based on findings")
│   │   └── child agent_loop() with edit/write tools → returns result
│   └── summarizes both results

Each sub-agent invocation starts a fresh conversation — no state leaks between calls.

Creating Sub-Agents

#![allow(unused)]
fn main() {
use std::sync::Arc;
use yoagent::sub_agent::SubAgentTool;
use yoagent::provider::ModelConfig;
use yoagent::tools;

let researcher = SubAgentTool::from_config(
    "researcher",
    ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"),
)
    .with_description("Searches and reads files to gather information.")
    .with_system_prompt("You are a research assistant. Be thorough and concise.")
    .with_tools(vec![
        Arc::new(tools::ReadFileTool::new()),
        Arc::new(tools::SearchTool::new()),
    ])
    .with_max_turns(10);
}

Registering on a Parent Agent

#![allow(unused)]
fn main() {
use yoagent::agent::Agent;

let mut agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .with_system_prompt("You coordinate between sub-agents.")
    .with_sub_agent(researcher)
    .with_sub_agent(coder);
}

The parent sees sub-agents as regular tools. It decides when to delegate based on its system prompt.

Parallel Execution

When the parent LLM calls multiple sub-agents in a single response, they run concurrently (default Parallel strategy). Two sub-agents each taking 50ms complete in ~50ms total, not 100ms.

Configuration

Event Forwarding

When the parent provides an on_update callback (standard for all tools), sub-agent events are forwarded as ToolExecutionUpdate events. The parent's UI sees real-time progress from the child:

• Text deltas from the sub-agent's LLM responses
• Tool call notifications from the sub-agent's tool usage

Shared State

By default, each sub-agent invocation is isolated — to pass data between sub-agents, the parent must re-paste it into every prompt. For large artifacts (CI logs, codebases, analysis results), this wastes context tokens.

SharedState solves this: store an artifact once, and any number of sub-agents read/write it by reference.

#![allow(unused)]
fn main() {
use yoagent::shared_state::SharedState;

let state = SharedState::new();
state.set("ci_log", large_log_text).await.unwrap();

let analyzer = SubAgentTool::from_provider(
    "analyzer",
    provider.clone(),
    ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"),
)
    .with_system_prompt("Analyze the CI log for failures.")
    .with_shared_state(state.clone());  // opt-in
}

When .with_shared_state() is used, the sub-agent automatically gets:

1. A shared_state tool with get, set, list, and remove actions
2. A system prompt appendix listing available keys and their sizes

The sub-agent reads the artifact via tool call instead of having it pasted into the prompt:

Sub-agent calls: shared_state(action="get", key="ci_log")
Sub-agent calls: shared_state(action="set", key="summary", value="...")

The parent reads results back programmatically:

#![allow(unused)]
fn main() {
let summary = state.get("summary").await.expect("sub-agent wrote this");
}

Parallel Sub-Agents with Shared State

Multiple sub-agents can share the same SharedState concurrently. Each gets its own clone of the Arc handle — reads are concurrent, writes are serialized by tokio::sync::RwLock.

#![allow(unused)]
fn main() {
let error_analyst = SubAgentTool::from_provider(
    "error_analyst",
    provider.clone(),
    ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"),
)
    .with_shared_state(state.clone());
let perf_analyst = SubAgentTool::from_provider(
    "perf_analyst",
    provider.clone(),
    ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"),
)
    .with_shared_state(state.clone());

// Both run in parallel, reading the same artifact and writing different keys
}

Backends

SharedState is backed by a pluggable SharedStateBackend trait. Two built-in backends are provided:

MemoryBackend (default) — in-memory HashMap with a byte capacity limit:

#![allow(unused)]
fn main() {
let state = SharedState::new();                          // 10MB default
let state = SharedState::with_max_bytes(50 * 1024 * 1024); // 50MB
}

A set call that would exceed capacity returns Err(CapacityError).

FileBackend — one file per key, persistent across process restarts:

#![allow(unused)]
fn main() {
use yoagent::shared_state::FileBackend;

let state = SharedState::with_backend(FileBackend::new(".agent-state"));
}

Keys are percent-encoded to filenames (reversible, no collisions). Useful for debugging (inspect state with ls / cat) and for long-running workflows where memory limits matter.

Custom backends implement the SharedStateBackend trait:

#![allow(unused)]
fn main() {
use yoagent::shared_state::{SharedStateBackend, SharedStateError};

#[async_trait::async_trait]
impl SharedStateBackend for MyRedisBackend {
    async fn get(&self, key: &str) -> Result<Option<String>, SharedStateError> { ... }
    async fn set(&self, key: &str, value: String) -> Result<(), SharedStateError> { ... }
    async fn remove(&self, key: &str) -> Result<bool, SharedStateError> { ... }
    async fn keys(&self) -> Result<Vec<String>, SharedStateError> { ... }
    async fn summary(&self) -> Result<String, SharedStateError> { ... }
}

let state = SharedState::with_backend(MyRedisBackend::new());
}

See examples/shared_state.rs for a complete parallel analysis demo.

Multi-Provider Support

Sub-agents can use any provider supported by yoagent — not just Anthropic. Pass a ModelConfig to configure the base URL, compat flags, and other provider-specific settings:

#![allow(unused)]
fn main() {
use yoagent::provider::ModelConfig;

let model_config = ModelConfig::xai("grok-4-1-fast-reasoning", "Grok 3 Mini Fast");

// `from_config` resolves OpenAiCompatProvider from the config's protocol and
// the key from XAI_API_KEY.
let analyst = SubAgentTool::from_config("analyst", model_config)
    .with_tools(vec![...]);
}

This works with all providers: OpenAI, Groq, DeepSeek, Gemini, Mistral, xAI, and more. See Model Presets for the full list of first-class factory methods.

Design Decisions

• Context isolation: Each invocation starts fresh. Sub-agents don't accumulate history across calls.
• Nesting supported: Sub-agents can be given other SubAgentTools for recursive delegation (see examples/rlm.rs). Use with_max_turns() to prevent infinite chains.
• Cancellation propagation: The parent's cancellation token is forwarded. Aborting the parent aborts all sub-agents.
• Turn limiting: The default 10-turn limit prevents runaway execution. The parent's execution limits also apply to total wall-clock time.

Isolating sub-agents from each other

SharedState is a single flat namespace by design — a parent stores an artifact once and several sub-agents read it by reference, which is the whole point of the type. Every holder sees every key.

When a sub-agent should not see its siblings' data, hand it a scoped view:

#![allow(unused)]
fn main() {
let state = SharedState::new();

let researcher = SubAgentTool::from_config("researcher", config.clone())
    .with_scoped_shared_state(state.clone(), "researcher");
let writer = SubAgentTool::from_config("writer", config)
    .with_scoped_shared_state(state.clone(), "writer");

// The parent's unscoped handle still sees everything both wrote.
for key in state.keys().await { /* … */ }
}

Keys are transparently prefixed, so a scoped sub-agent cannot read, overwrite, or enumerate anything outside its scope — including via a crafted key, since the prefix is applied on the way in. summary() is scoped too, which matters most: it is injected into the sub-agent's system prompt, so an unscoped summary would disclose every sibling's key names.

Scoping nests (state.scoped("a").scoped("b")), so a sub-agent can narrow its own view but never widen it. Keep plain with_shared_state when sharing is the intent.

Examples

• examples/sub_agent.rs — Coordinator with researcher and coder sub-agents
• examples/code_review.rs — 3 parallel sub-agents reviewing a file via shared state
• examples/rlm.rs — Recursive Language Model: nested sub-agents with autonomous file discovery

State Persistence

yoagent supports saving and restoring agent conversation state, enabling pause/resume workflows, state transfer between processes, and conversation checkpointing.

Save and Restore

#![allow(unused)]
fn main() {
use yoagent::agent::Agent;

// After running some conversation turns...
let json = agent.save_messages()?;
std::fs::write("conversation.json", &json)?;

// Later, in a new process:
let json = std::fs::read_to_string("conversation.json")?;
let mut agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .with_system_prompt("You are helpful.");

agent.restore_messages(&json)?;

// Continue the conversation — the agent sees the full history
let rx = agent.prompt("Follow up question").await;
}

Builder Initialization

For constructing an agent with pre-existing history:

#![allow(unused)]
fn main() {
let saved: Vec<AgentMessage> = serde_json::from_str(&json)?;
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .with_messages(saved)
    .with_system_prompt("...");
}

JSON Format

Messages serialize as a JSON array. Each message is tagged by role:

[
  {
    "role": "user",
    "content": [{"type": "text", "text": "Hello"}],
    "timestamp": 1700000000000
  },
  {
    "role": "assistant",
    "content": [{"type": "text", "text": "Hi there!"}],
    "stopReason": "stop",
    "model": "claude-sonnet-5",
    "provider": "anthropic",
    "usage": {"input": 100, "output": 50, "cacheRead": 0, "cacheWrite": 0, "totalTokens": 150},
    "timestamp": 1700000001000
  }
]

As of 0.13 all field names are camelCase; the pre-0.13 snake_case names (cache_read, cache_write, total_tokens, error_message, provider_metadata) are still accepted when loading older files.

Extension messages use a nested structure:

{
  "role": "extension",
  "kind": "status_update",
  "data": {"status": "running"}
}

Context Tracking

ContextTracker and ExecutionTracker are runtime-only and not persisted. This is by design — both are created fresh each agent_loop() invocation and operate on whatever messages are in context at that point. Restoring messages and calling prompt() works correctly without any special recalculation.

What's Serializable

For branching history — fork, checkpoints, edit-and-rerun — see Session Trees.

Session Trees

A Session stores conversation history as a tree, not a flat list. Every entry has an id and parent_id; the head points at the current branch tip. Appending after a seek creates a new branch — history is never overwritten. This is the primitive behind:

• Fork — try a different direction from any point
• Checkpoints — label a state, come back to it later
• Edit & re-run — change an earlier turn on a new branch; the original branch stays intact

#![allow(unused)]
fn main() {
use yoagent::{Agent, Session, provider::ModelConfig};

let mut session = Session::new();

// Run a turn, record it.
let mut agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Sonnet 5"));
let mut rx = agent.prompt("draft a plan").await;
while rx.recv().await.is_some() {}
agent.finish().await;
session.append_new(agent.messages())?;
session.checkpoint("first-draft")?;

// ... more turns ...

// Rewind to the checkpoint and branch.
session.seek_checkpoint("first-draft")?;
let mut agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Sonnet 5"))
    .with_messages(session.path_messages());   // only this branch's history
let mut rx = agent.prompt("actually, make it a library instead").await;
while rx.recv().await.is_some() {}
agent.finish().await;
session.append_new(agent.messages())?;          // new branch recorded

// Both branches exist:
assert_eq!(session.branch_tips().len(), 2);
}

Persistence: JSONL

#![allow(unused)]
fn main() {
std::fs::write("session.jsonl", session.to_jsonl())?;
let restored = Session::from_jsonl(&std::fs::read_to_string("session.jsonl")?)?;
}

One entry per line, append-friendly, diff-friendly. On load the head is the last line's entry, and the file is validated (duplicate ids and dangling/forward parent references are rejected — which also makes cycles impossible).

Compaction caveat: append_new verifies the agent's history still extends the session path and returns SessionError::HistoryDiverged when it doesn't. The usual cause is context compaction (on by default) rewriting the agent's messages — disable context management on session-tracked agents, or rebuild the branch with Session::from_messages after a divergence. The flat save_messages() / restore_messages() API remains for single-branch persistence.

API sketch

GASP

This tree is a natural format for the GASP transcripts/ tier — the raw-conversation cold tier (the spec leaves its format open). The semantic event log lives in the gasp feature: a recorder over the AgentEvent stream that emits a conformance-checked agent repo.

GASP: Your Agent Is a Git Repo

GASP — the Git Agent State Protocol — keeps an agent's durable self in a git repository: an append-only semantic event log (state/events.jsonl) that folds into a typed goal/run/model/tool graph, alongside identity, skills, and memory tiers. Restore = git clone + replay. Clone your agent onto a new machine and it remembers everything, with lineage.

yoagent bridges to GASP through the gasp feature (backed by yoagent-state, the reference runtime). The bridge is a consumer of the AgentEvent stream — zero agent-loop changes:

yoagent = { version = "0.15", features = ["gasp"] }

#![allow(unused)]
fn main() {
use yoagent::gasp::{GaspRecorder, GoalRef};

let recorder = GaspRecorder::init(
    "./my-agent-repo", "my-agent", "worker-1",
    GoalRef::New { title: "ship the feature".into() },
).await?;

let (tx, record_handle) = recorder.recording_sender("implement the parser", None);
agent.prompt_with_sender("implement the parser", tx).await;
let run_id = record_handle.await??;   // events appended + committed
}

What gets recorded

The semantic log stores bounded one-line summaries — the task string (verbatim), model ids, and the first 200 characters of tool inputs, tool outputs, and assistant text — never full transcripts. If secrets can flow through tool arguments or outputs, install a redacting summarizer (recorder.with_summarizer(...)) before recording: the repo is designed to be cloned and shared, and committed history is hard to scrub. Full transcripts belong in GASP's cold transcripts/ tier — Session::to_jsonl is a natural format for it.

Robustness: a crashed process leaves no open run — the recorder closes stale runs as interrupted on open, and a dropped event stream finishes the run with the outcome derived so far. One git commit per run keeps the history append-only, and the init scaffolding (manifest, identity) is committed so a fresh clone is a complete, conformant agent. If recording fails mid-run (disk, lease, git), recording stops and the error surfaces via the returned handle — always await it — while event forwarding to your UI continues uninterrupted. GASP repos are single-writer: don't share one repo between live workers, and record one run at a time.

Tested conformance

yoagent's CI emits an agent repo with a mock provider and runs the GASP conformance checker against it — all seven mechanical checks (envelope round-trip, replay, vocabulary, append-only git history, causation integrity, restore, domain↔ops consistency) must pass on every commit. Try it yourself:

cargo run --example gasp_emit --features gasp -- /tmp/my-agent
git clone https://github.com/yologdev/gasp && cd gasp/conformance-check
cargo run -q -- /tmp/my-agent
# conformant: all checks passed

Lifecycle Callbacks

yoagent provides three lifecycle callbacks that let you observe and control the agent loop without modifying its internals.

Callbacks

before_turn

Called before each LLM call. Receives the current message history and the turn number (0-indexed). Return false to abort the loop.

#![allow(unused)]
fn main() {
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .on_before_turn(|messages, turn| {
        println!("Turn {} starting with {} messages", turn, messages.len());
        turn < 10 // Stop after 10 turns
    });
}

after_turn

Called after each LLM response and tool execution. Receives the updated message history and the turn's token usage.

#![allow(unused)]
fn main() {
use std::sync::{Arc, Mutex};

let total_cost = Arc::new(Mutex::new(0u64));
let cost_tracker = total_cost.clone();

let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .on_after_turn(move |_messages, usage| {
        let mut cost = cost_tracker.lock().unwrap();
        *cost += usage.input + usage.output;
        println!("Cumulative tokens: {}", *cost);
    });
}

on_error

Called when the LLM returns a StopReason::Error. Receives the error message string.

#![allow(unused)]
fn main() {
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .on_error(|err| {
        eprintln!("LLM error: {}", err);
        // Log to monitoring, send alert, etc.
    });
}

Combining Callbacks

All callbacks are optional and independent:

#![allow(unused)]
fn main() {
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .on_before_turn(|_msgs, turn| turn < 20)
    .on_after_turn(|msgs, usage| {
        println!("Messages: {}, Tokens: {}/{}", msgs.len(), usage.input, usage.output);
    })
    .on_error(|err| eprintln!("Error: {}", err));
}

Using with AgentLoopConfig

For direct loop usage without the Agent wrapper:

#![allow(unused)]
fn main() {
use std::sync::Arc;
use yoagent::agent_loop::AgentLoopConfig;

let config = AgentLoopConfig {
    before_turn: Some(Arc::new(|_msgs, turn| turn < 5)),
    after_turn: Some(Arc::new(|_msgs, _usage| { /* log */ })),
    on_error: Some(Arc::new(|err| eprintln!("{}", err))),
    // ... other fields
};
}

Callback Timing

Loop iteration:
  1. Inject pending messages (steering/follow-up)
  2. Check execution limits
  3. before_turn(messages, turn_number)  <-- return false to abort
  4. Compact context
  5. Stream LLM response
  6. Check for error/abort → on_error(message) if StopReason::Error
     → after_turn(messages, usage) even on error/abort
  7. Execute tool calls
  8. Track turn
  9. after_turn(messages, usage)
  10. Emit TurnEnd event

Telemetry

yoagent instruments the loop with tracing spans — structured, timed, nested units your observability stack can consume. With no subscriber installed the overhead is near-zero (a cached per-callsite interest check); nothing is exported unless you opt in.

The span tree

agent_loop                (model)
└─ llm_stream             (turn, model, tokens_in, tokens_out, tokens_cached, cost_usd, error)
└─ tool                   (tool, tool_call_id, is_error)

• llm_stream — one per turn, wrapping the provider call. Token counts are recorded from real usage; cost_usd is recorded when the ModelConfig has pricing configured (CostConfig).
• tool — one per tool execution, with the tool name and error status; duration comes free with the span.

Local: print spans

#![allow(unused)]
fn main() {
tracing_subscriber::fmt()
    .with_span_events(tracing_subscriber::fmt::format::FmtSpan::CLOSE)
    .init();
}

Run cargo run --example telemetry to see it.

Production: OpenTelemetry

The OTel bridge is application-side — yoagent needs no OTel dependency (that's the point of tracing). Install the tracing-opentelemetry layer and the same spans flow to any OTLP backend — Datadog, Grafana Tempo, Honeycomb, Jaeger:

#![allow(unused)]
fn main() {
// opentelemetry_otlp 0.27+, opentelemetry_sdk 0.27+, tracing-opentelemetry 0.28+
use opentelemetry::trace::TracerProvider as _;
use tracing_subscriber::layer::SubscriberExt;

let exporter = opentelemetry_otlp::SpanExporter::builder()
    .with_tonic()
    .build()?;
let provider = opentelemetry_sdk::trace::SdkTracerProvider::builder()
    .with_batch_exporter(exporter)
    .build();

let subscriber = tracing_subscriber::registry()
    .with(tracing_opentelemetry::layer().with_tracer(provider.tracer("yoagent-app")));
tracing::subscriber::set_global_default(subscriber)?;
}

(The OTel crates rework their builder APIs between releases — if this snippet drifts, the authoritative wiring is the tracing-opentelemetry docs; yoagent only emits standard tracing spans and does not depend on OTel.)

Because these are ordinary tracing spans, an agent call nests inside your app's existing request traces (e.g. an axum handler span) automatically.

What it buys you

• Cost attribution — dollars per turn/model in your dashboards, from the same CostConfig data as session_cost_usd.
• Latency diagnosis — "40s request: 8s provider, 30s in one bash tool."
• Audit — which tools ran, with what outcome, per session.

MCP Integration

What is MCP?

The Model Context Protocol (MCP) is a JSON-RPC 2.0 protocol that lets AI agents discover and call tools from external servers. It defines a standard way for agents to connect to tool providers over two transports:

• Stdio — spawn a child process, communicate via stdin/stdout (newline-delimited JSON)
• HTTP — POST JSON-RPC requests to an HTTP endpoint, including the request/response subset of Streamable HTTP

Connecting to MCP Servers

Stdio Transport

Use with_mcp_server_stdio() to spawn an MCP server process and register its tools:

use yoagent::Agent;
use yoagent::provider::ModelConfig;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
        .with_system_prompt("You are a helpful assistant with file access.")
        .with_mcp_server_stdio(
            "npx",
            &["-y", "@modelcontextprotocol/server-filesystem", "/tmp"],
            None,
        )
        .await?;

    let rx = agent.prompt("List files in /tmp").await;
    // handle events...
    Ok(())
}

You can pass environment variables to the server process:

#![allow(unused)]
fn main() {
use std::collections::HashMap;

let mut env = HashMap::new();
env.insert("API_TOKEN".into(), "secret".into());

let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .with_mcp_server_stdio("my-mcp-server", &["--port", "0"], Some(env))
    .await?;
}

HTTP Transport

For remote MCP servers exposed over HTTP:

#![allow(unused)]
fn main() {
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .with_mcp_server_http("http://localhost:8080/mcp")
    .await?;
}

HttpTransport handles both the plain JSON-RPC-over-POST shape and the request/response subset of Streamable HTTP — servers that answer a POST with an SSE-framed response, whether or not they then close the stream:

• Responses framed as text/event-stream are parsed out of their SSE frames, joining each event's data: lines as the SSE spec requires.
• A server may interleave notifications/progress and notifications/message frames ahead of the result — that is how it reports progress during a tools/call. Those are skipped: a frame is this request's response only if it carries no method, carries a result or an error, and its id matches.
• Requests advertise Accept: application/json, text/event-stream, letting the server pick its framing.
• An Mcp-Session-Id returned by the server is captured and replayed on every later request, and released with a DELETE on McpClient::close(). Servers that reject DELETE are tolerated — teardown is best-effort. A 404 on a session-bearing request clears the session and reports that it expired, so a caller can rebuild the client.
• 202 Accepted (or 204) with an empty body — how a notification is acknowledged — is a success, not a parse failure. Any other empty 2xx is reported as an error, since it usually means a proxy answered instead of the MCP server.
• The body is parsed incrementally, so a call returns at the blank-line-terminated frame carrying its response rather than at end-of-stream. A server that holds the POST stream open after answering does not block it. Two trade-offs come with that: returning mid-body forgoes connection reuse (a fresh connection, and TLS handshake, on the next call to such a server), and a plain JSON-RPC body has no frames to return early at, so it is read to the end as before.
• A stalled server — one that accepts the POST then sends nothing — is bounded by an idle read timeout (120s) rather than hanging. The timer resets on every read, so a long tools/call streaming progress frames is never cut off.

Not supported: the GET server→client stream and Last-Event-ID resumability. McpTransport is send/close only, with nowhere to deliver a server-initiated message — supporting them would mean growing the trait an inbound channel. Notifications arriving on the POST stream before the response are read and skipped; any that trail it are not, since the call has already returned — so a server that blocks awaiting a reply to a sampling/createMessage it sent on this stream will time out rather than be answered. Note also that the handshake still negotiates protocolVersion: 2024-11-05 (the revision predating Streamable HTTP), which servers generally accept.

McpClient::close() is what sends the DELETE. Agent::with_mcp_server_http does not call it, so sessions opened that way are released by the server's own timeout rather than explicitly.

How MCP Tools Work

When you call with_mcp_server_stdio() or with_mcp_server_http(), yoagent:

1. Connects to the MCP server and performs the initialize handshake
2. Calls tools/list to discover available tools
3. Wraps each MCP tool as an AgentTool via McpToolAdapter
4. Adds them to the agent's tool list

MCP tools appear alongside built-in tools. The LLM sees them with their original names, descriptions, and JSON Schema parameters — it can call them just like any other tool.

Mixing Built-in and MCP Tools

#![allow(unused)]
fn main() {
use yoagent::tools::default_tools;

let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .with_tools(default_tools())  // bash, read, write, edit, list, search
    .with_mcp_server_stdio("my-db-server", &[], None)
    .await?;
// Agent now has both built-in coding tools AND MCP database tools
}

Using the MCP Client Directly

For lower-level control, use McpClient directly:

#![allow(unused)]
fn main() {
use yoagent::mcp::{McpClient, McpToolAdapter};
use std::sync::Arc;
use tokio::sync::Mutex;

let client = McpClient::connect_stdio("my-server", &[], None).await?;
let tools = client.list_tools().await?;

for tool in &tools {
    println!("{}: {}", tool.name, tool.description.as_deref().unwrap_or(""));
}

// Call a tool directly
let result = client.call_tool("read_file", serde_json::json!({"path": "/tmp/test.txt"})).await?;

// Or wrap as AgentTool adapters
let client = Arc::new(Mutex::new(client));
let adapters = McpToolAdapter::from_client(client).await?;
}

Error Handling

MCP operations return McpError:

• McpError::Transport — connection or I/O failure
• McpError::Protocol — unexpected response format
• McpError::JsonRpc — server returned a JSON-RPC error
• McpError::ConnectionClosed — server process exited

When an MCP tool returns isError: true, the adapter converts it to a ToolError::Failed, which the agent loop sends back to the LLM with is_error: true so it can self-correct.

OpenAPI Tool Adapter

Auto-generate AgentTool implementations from OpenAPI 3.0 specs. Point an agent at any API spec and it instantly gets callable tools for every operation.

Feature-gated — add features = ["openapi"] to your Cargo.toml.

Quick Start

use yoagent::Agent;
use yoagent::openapi::{OpenApiToolAdapter, OpenApiConfig, OperationFilter};
use yoagent::provider::ModelConfig;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = OpenApiConfig::new()
        .with_bearer_token("sk-...");

    let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
        .with_system_prompt("You are an API assistant.")
        .with_openapi_file("petstore.yaml", config, &OperationFilter::All)
        .await?;

    Ok(())
}

Loading Specs

Three ways to load an OpenAPI spec:

#![allow(unused)]
fn main() {
// From a file
let agent = agent.with_openapi_file("spec.yaml", config, &filter).await?;

// From a URL
let agent = agent.with_openapi_url("https://api.example.com/openapi.json", config, &filter).await?;

// From a string (sync)
let agent = agent.with_openapi_spec(&spec_string, config, &filter)?;
}

Or create adapters directly for more control:

#![allow(unused)]
fn main() {
let adapters = OpenApiToolAdapter::from_str(&spec, config, &OperationFilter::All)?;
let tools: Vec<Box<dyn AgentTool>> = adapters.into_iter().map(|a| Box::new(a) as _).collect();
}

Configuration

OpenApiConfig controls auth, headers, timeouts, and response limits:

#![allow(unused)]
fn main() {
let config = OpenApiConfig::new()
    .with_base_url("https://api.staging.example.com") // Override spec's servers
    .with_bearer_token("sk-...")                       // Bearer auth
    .with_header("X-Custom", "value")                  // Extra headers
    .with_timeout_secs(60)                             // Request timeout
    .with_max_response_bytes(128 * 1024)               // Truncate large responses
    .with_name_prefix("github");                       // Tool names: github__listRepos
}

Authentication

#![allow(unused)]
fn main() {
// Bearer token
let config = OpenApiConfig::new().with_bearer_token("token");

// API key in a custom header
let config = OpenApiConfig::new().with_api_key("X-API-Key", "key-value");

// No auth
let config = OpenApiConfig::new(); // default
}

Filtering Operations

Most API specs have dozens or hundreds of operations. Use OperationFilter to select which ones become tools:

#![allow(unused)]
fn main() {
// All operations (default)
let filter = OperationFilter::All;

// Specific operations by ID
let filter = OperationFilter::ByOperationId(vec![
    "listRepos".into(),
    "getRepo".into(),
    "createIssue".into(),
]);

// All operations with a specific tag
let filter = OperationFilter::ByTag(vec!["repos".into()]);

// All operations under a path prefix
let filter = OperationFilter::ByPathPrefix("/repos".into());
}

How It Works

Each OpenAPI operation becomes one AgentTool:

When the LLM calls a tool, the adapter:

1. Substitutes path parameters in the URL (/pets/{petId} → /pets/123)
2. Adds query parameters as ?key=value
3. Adds header parameters
4. Applies auth from config
5. Sends the request body as JSON (if the operation has one)
6. Returns the response text (with status code) to the LLM

Non-2xx responses are not treated as errors — they're returned as text so the LLM can reason about them and retry or adjust.

Mixing with Other Tools

OpenAPI tools work alongside built-in tools and MCP tools:

#![allow(unused)]
fn main() {
use yoagent::tools::default_tools;

let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"))
    .with_tools(default_tools())
    .with_openapi_file("github.yaml", github_config, &github_filter).await?
    .with_mcp_server_stdio("db-server", &[], None).await?;
}

Limitations (v1)

• OpenAPI 3.0.x only (not 3.1.x)
• JSON request/response bodies only (no multipart/form-data)
• No OAuth2 or token refresh (pass tokens via OpenApiConfig)
• Operations without operationId are skipped
• Path-level $ref items are skipped

Providers Overview

yoagent supports multiple LLM providers through the StreamProvider trait and ApiProtocol dispatch.

Supported Protocols

ApiProtocol Enum

#![allow(unused)]
fn main() {
pub enum ApiProtocol {
    AnthropicMessages,
    OpenAiCompletions,
    OpenAiResponses,
    AzureOpenAiResponses,
    GoogleGenerativeAi,
    GoogleVertex,
    BedrockConverseStream,
}
}

ModelConfig

Full configuration for a model, including provider routing:

#![allow(unused)]
fn main() {
pub struct ModelConfig {
    pub id: String,              // e.g. "gpt-5.5"
    pub name: String,            // e.g. "GPT-5.5"
    pub api: ApiProtocol,        // Which provider to use
    pub provider: String,        // e.g. "openai"
    pub base_url: String,        // API endpoint
    pub reasoning: bool,         // Supports thinking/reasoning
    pub context_window: u32,     // Context size in tokens
    pub max_tokens: u32,         // Default max output
    pub cost: CostConfig,        // Pricing per million tokens
    pub headers: HashMap<String, String>,  // Extra headers
    pub compat: Option<OpenAiCompat>,      // Quirk flags
}
}

First-class model presets are documented in Model Presets. Convenience constructors:

#![allow(unused)]
fn main() {
let anthropic = ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5");
let openai = ModelConfig::openai("gpt-5.5", "GPT-5.5");
let google = ModelConfig::google("gemini-2.5-flash", "Gemini 2.5 Flash");
let xai = ModelConfig::xai("grok-4-1-fast", "Grok 4.1 Fast");
let groq = ModelConfig::groq("llama-3.3-70b-versatile", "Llama 3.3 70B");
let deepseek = ModelConfig::deepseek("deepseek-v4-flash", "DeepSeek V4 Flash");
let mistral = ModelConfig::mistral("mistral-large-latest", "Mistral Large");
let minimax = ModelConfig::minimax("MiniMax-Text-01", "MiniMax Text 01");
let zai = ModelConfig::zai("glm-4.7", "GLM 4.7");
let qwen = ModelConfig::qwen("qwen3.6-plus", "Qwen 3.6 Plus");
let ollama = ModelConfig::ollama("http://localhost:11434/v1", "llama3.1:8b");
let local = ModelConfig::local("http://localhost:1234/v1", "my-model");
}

ProviderRegistry

Maps ApiProtocol → StreamProvider. The default registry includes all built-in providers:

#![allow(unused)]
fn main() {
let registry = ProviderRegistry::default();

// Use it to stream with any model
let result = registry.stream(&model_config, stream_config, tx, cancel).await?;
}

Custom registries:

#![allow(unused)]
fn main() {
let mut registry = ProviderRegistry::new();
registry.register(ApiProtocol::AnthropicMessages, AnthropicProvider);
}

StreamProvider Trait

#![allow(unused)]
fn main() {
#[async_trait]
pub trait StreamProvider: Send + Sync {
    async fn stream(
        &self,
        config: StreamConfig,
        tx: mpsc::UnboundedSender<StreamEvent>,
        cancel: CancellationToken,
    ) -> Result<Message, ProviderError>;
}
}

All providers receive a StreamConfig, emit StreamEvents through the channel, and return the final Message.

OpenAPI Tool Adapter

In addition to LLM providers, yoagent can auto-generate tools from any OpenAPI 3.0 spec. This is a tool integration (not a provider), but it complements the provider system by letting agents call external APIs.

Enable with features = ["openapi"]. See the OpenAPI Tools guide for details.

Model Presets

yoagent's first-class model presets are the ModelConfig::* constructors. A preset sets provider routing, base URL, context metadata, default output limits, and provider compatibility flags.

Use a preset when the provider is listed here. Use a custom ModelConfig when you need a compatible provider or endpoint that does not have a constructor yet.

First-Class Constructors

The constructors do not validate model IDs. They send the id you pass through to the provider, which lets you use newly released model IDs before yoagent updates its examples.

The named presets (claude_fable_5, claude_opus_5, claude_opus_4_8, claude_sonnet_5, claude_haiku_4_5, gpt_5_5) also fill in real CostConfig pricing. The OpenCode presets select the API protocol from the model id — see OpenCode Zen & Go.

OpenAI-Compatible Presets

These constructors all use OpenAiCompatProvider:

#![allow(unused)]
fn main() {
use yoagent::provider::ModelConfig;

let agent = Agent::from_config(ModelConfig::deepseek(
    "deepseek-v4-flash",
    "DeepSeek V4 Flash",
));
}

OpenAI-compatible presets also set OpenAiCompat flags for provider-specific API differences, such as max_tokens vs. max_completion_tokens, reasoning fields, tool result formatting, and streaming usage support. See OpenAI Compatible for the full quirk-flag list.

Ollama Models

Use ModelConfig::ollama for Ollama's OpenAI-compatible endpoint:

#![allow(unused)]
fn main() {
let llama = ModelConfig::ollama("http://localhost:11434/v1", "llama3.1:8b");
}

Ollama remains separate from ModelConfig::local(...) because some Ollama-served models need an assistant message after tool results, while other local OpenAI-compatible servers may not. The Ollama preset enables that transcript workaround; the generic local preset stays neutral.

Qwen Models

Use ModelConfig::qwen for hosted Qwen / DashScope:

#![allow(unused)]
fn main() {
let qwen = ModelConfig::qwen("qwen3.6-plus", "Qwen 3.6 Plus");
}

The default base URL is the international DashScope endpoint. For other regions, override base_url after construction:

#![allow(unused)]
fn main() {
let mut qwen = ModelConfig::qwen("qwen3.6-plus", "Qwen 3.6 Plus");
qwen.base_url = "https://dashscope.aliyuncs.com/compatible-mode/v1".into();
}

Region endpoints:

• International/Singapore: https://dashscope-intl.aliyuncs.com/compatible-mode/v1
• China/Beijing: https://dashscope.aliyuncs.com/compatible-mode/v1
• US/Virginia: https://dashscope-us.aliyuncs.com/compatible-mode/v1

For locally deployed Qwen, keep the local endpoint and opt into Qwen's model-family compat flags:

#![allow(unused)]
fn main() {
let qwen_local = ModelConfig::openai_compat(
    "http://localhost:1234/v1",
    "qwen3-local",
    "qwen",
    OpenAiCompat::qwen(),
);
}

If a local serving layer also has its own quirks, combine the compat flags explicitly. For example, Qwen served by Ollama may need both Qwen reasoning parsing and Ollama's tool-result transcript workaround:

#![allow(unused)]
fn main() {
let mut compat = OpenAiCompat::qwen();
compat.requires_assistant_after_tool_result = true;

let qwen_ollama = ModelConfig::openai_compat(
    "http://localhost:11434/v1",
    "qwen2.5-coder:7b",
    "ollama",
    compat,
);
}

DeepSeek Models

Use the current DeepSeek API model IDs by default:

#![allow(unused)]
fn main() {
let flash = ModelConfig::deepseek("deepseek-v4-flash", "DeepSeek V4 Flash");
let pro = ModelConfig::deepseek("deepseek-v4-pro", "DeepSeek V4 Pro");
}

Legacy DeepSeek aliases still work because ModelConfig::deepseek passes the model ID through unchanged:

#![allow(unused)]
fn main() {
let chat = ModelConfig::deepseek("deepseek-chat", "DeepSeek Chat");
let reasoner = ModelConfig::deepseek("deepseek-reasoner", "DeepSeek Reasoner");
}

DeepSeek documents deepseek-chat and deepseek-reasoner as compatibility aliases scheduled for deprecation on 2026-07-24. In DeepSeek's current API, deepseek-chat maps to the non-thinking mode of deepseek-v4-flash, while deepseek-reasoner maps to the thinking mode of deepseek-v4-flash.

yoagent also sends DeepSeek's current request shape:

• max_tokens, not max_completion_tokens
• thinking: { "type": "enabled" | "disabled" }
• reasoning_effort when ThinkingLevel is not Off
• DeepSeek cache hit/miss usage fields when present

For legacy aliases, set ThinkingLevel to match the alias behavior:

#![allow(unused)]
fn main() {
let chat_agent = Agent::from_config(ModelConfig::deepseek("deepseek-chat", "DeepSeek Chat"))
    .with_thinking(ThinkingLevel::Off);

let reasoner_agent = Agent::from_config(ModelConfig::deepseek("deepseek-reasoner", "DeepSeek Reasoner"))
    .with_thinking(ThinkingLevel::High);
}

Older DeepSeek reasoning models had stricter feature limits than the current V4 API. In particular, historical deepseek-reasoner documentation did not support function calling. If you need tools, prefer current V4 model IDs unless you have tested the legacy alias for your workflow.

Compat Flags Without Constructors

OpenAiCompat also has quirk presets such as OpenAiCompat::cerebras() and OpenAiCompat::openrouter(). Those are compatibility profiles, not full ModelConfig constructors. To use them, call ModelConfig::openai_compat(...) with the provider name, base URL, and compat value you need.

Anthropic Provider

AnthropicProvider implements the Anthropic Messages API with SSE streaming.

Usage

#![allow(unused)]
fn main() {
use yoagent::provider::ModelConfig;

let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"));
}

Features

Streaming SSE

Uses reqwest-eventsource to parse Anthropic's SSE stream. Events handled:

• message_start — Input token usage, cache stats
• content_block_start — Text, thinking, or tool_use block
• content_block_delta — Text, thinking, input JSON, or signature deltas
• content_block_stop — Block complete
• message_delta — Stop reason, output usage
• message_stop — Stream complete

Thinking

Set thinking_level to enable thinking. By default the provider sends adaptive thinking (thinking: {"type": "adaptive"}), which the current model generation requires (Claude Fable 5, Opus 5, Opus 4.7/4.8, Sonnet 5 reject budget-based thinking with a 400). The level maps to an output_config.effort hint:

For pre-4.6 models, opt into legacy budget-based thinking via AnthropicCompat::legacy():

#![allow(unused)]
fn main() {
let mut config = ModelConfig::anthropic("claude-sonnet-4-5", "Claude Sonnet 4.5");
config.anthropic = Some(AnthropicCompat::legacy());
}

Legacy budgets: Minimal/Low 1,024 (the API minimum), Medium 2,048, High 8,192. max_tokens is automatically raised above the budget when needed.

Thinking content is streamed as Content::Thinking with a cryptographic signature for verification.

Stop Reasons

Every documented Anthropic stop_reason maps explicitly:

Anything unrecognized maps to Stop and is logged at warn, so a stop reason added by Anthropic later is visible rather than silently treated as a finish.

Refusals. Models with safety classifiers (e.g. Claude Fable 5) can decline a request with stop_reason: "refusal". The agent loop stops the turn like a normal Stop, and callers can match on the variant to retry on a fallback model.

Tool Calls With Unusable Arguments

A tool call's arguments arrive as input_json_delta fragments and are assembled at content_block_stop. When that assembly cannot happen — the accumulated text is not valid JSON, or the content_block_stop event itself is unusable — the turn fails with StopReason::Error and an error_message naming each affected tool and quoting its input.

This matters because the alternative is silent: a tool executed with empty arguments falls back to its defaults, so list_files asked for /etc would list the working directory instead, with nothing in the response indicating the model's actual input was dropped.

agent_loop returns on StopReason::Error before extracting tool calls, so nothing executes. Every tool call in the message is replaced with a text block — not just the unusable one — because the turn runs none of them, and a tool_use block with no matching tool_result is rejected by the API on the next request.

Cache Control

Automatic prompt caching via cache_control markers:

• System prompt: Always cached with {"type": "ephemeral"}
• Second-to-last message: Gets cache_control on its last content block, creating a cache breakpoint

This means on repeated calls, only the latest message is processed at full price.

Configuration

Setting ModelConfig.base_url retargets the provider at any gateway that speaks the Anthropic Messages protocol (e.g. OpenCode Zen/Go — see OpenCode Zen & Go).

Environment Variables

OpenAI Compatible Provider

OpenAiCompatProvider implements the OpenAI Chat Completions API. One implementation covers OpenAI, xAI, Groq, Cerebras, OpenRouter, Mistral, DeepSeek, MiniMax, Z.ai, Qwen, Ollama, and any other compatible API.

For the first-class ModelConfig::* constructors and default model metadata, see Model Presets.

Usage

Requires a ModelConfig with compat flags set in StreamConfig.model_config:

#![allow(unused)]
fn main() {
use yoagent::provider::ModelConfig;

let agent = Agent::from_config(ModelConfig::openai("gpt-5.5", "GPT-5.5"));
}

OpenAiCompat Quirk Flags

Different providers have behavioral differences even though they share the same API:

#![allow(unused)]
fn main() {
pub struct OpenAiCompat {
    pub supports_store: bool,
    pub supports_developer_role: bool,
    pub supports_reasoning_effort: bool,
    pub supports_thinking_control: bool,
    pub supports_usage_in_streaming: bool,
    pub max_tokens_field: MaxTokensField,       // MaxTokens or MaxCompletionTokens
    pub requires_tool_result_name: bool,
    pub requires_assistant_after_tool_result: bool,
    pub thinking_format: ThinkingFormat,        // OpenAi, Xai, or Qwen
}
}

Provider Presets

OpenAiCompat presets are lower-level quirk flags. A provider is first-class when it also has a ModelConfig::* constructor; see Model Presets.

DeepSeek context caching is automatic on DeepSeek's side. yoagent does not send cache_control markers for DeepSeek, but it does parse DeepSeek's prompt_cache_hit_tokens and prompt_cache_miss_tokens usage fields into Usage.cache_read and Usage.input.

Adding a New Compatible Provider

1. Add a constructor to OpenAiCompat:

#![allow(unused)]
fn main() {
impl OpenAiCompat {
    pub fn my_provider() -> Self {
        Self {
            supports_usage_in_streaming: true,
            // set flags as needed...
            ..Default::default()
        }
    }
}
}

2. Create a ModelConfig that uses it:

#![allow(unused)]
fn main() {
let config = ModelConfig::openai_compat(
    "https://api.myprovider.com/v1",
    "my-model",
    "my-provider",
    OpenAiCompat::my_provider(),
);
}

Thinking/Reasoning

The ThinkingFormat enum controls how reasoning content is parsed from streams:

• ThinkingFormat::OpenAi — Uses reasoning_content field (DeepSeek, default)
• ThinkingFormat::Xai — Uses reasoning field (Grok)
• ThinkingFormat::Qwen — Uses reasoning_content field (Qwen)

Local Servers (LM Studio, Ollama, llama.cpp, vLLM)

Use ModelConfig::ollama() for Ollama, or ModelConfig::local() for any other local OpenAI-compatible server. No API key required:

#![allow(unused)]
fn main() {
use yoagent::agent::Agent;
use yoagent::provider::ModelConfig;

// The `local` provider resolves to an empty API key automatically — none needed.
let agent = Agent::from_config(ModelConfig::local("http://localhost:1234/v1", "my-model"));
}

For Ollama:

#![allow(unused)]
fn main() {
let agent = Agent::from_config(ModelConfig::ollama("http://localhost:11434/v1", "llama3.1:8b"));
}

Or via the CLI example:

cargo run --example cli -- --api-url http://localhost:1234/v1 --model my-model

For locally deployed open-source model families, keep the local endpoint and choose the model-family compat profile:

#![allow(unused)]
fn main() {
let qwen_local = ModelConfig::openai_compat(
    "http://localhost:1234/v1",
    "qwen3-local",
    "qwen",
    OpenAiCompat::qwen(),
);
}

Serving-layer quirks and model-family quirks can be combined because OpenAiCompat fields are public:

#![allow(unused)]
fn main() {
let mut compat = OpenAiCompat::qwen();
compat.requires_assistant_after_tool_result = true;

let qwen_on_ollama = ModelConfig::openai_compat(
    "http://localhost:11434/v1",
    "qwen2.5-coder:7b",
    "ollama",
    compat,
);
}

GitHub Copilot (bring-your-own-token)

Terms of service. api.githubcopilot.com is intended for use through official GitHub Copilot editor integrations. Accessing it from a third-party agent is against GitHub's Copilot terms of service and may result in token revocation or account suspension. yoagent does not ship a first-class Copilot preset for this reason. The configuration below is documented only for users who understand and accept that risk. Use at your own discretion.

Copilot's chat endpoint is OpenAI Chat Completions–shaped, so it works with OpenAiCompatProvider given the right base URL, integration headers, and a valid Copilot token as the API key:

#![allow(unused)]
fn main() {
use yoagent::agent::Agent;
use yoagent::provider::{ModelConfig, OpenAiCompat};

let mut config = ModelConfig::openai_compat(
    "https://api.githubcopilot.com",
    "gpt-5.5",
    "copilot",
    OpenAiCompat::openai(),
);
// Copilot fingerprints clients via these headers; they are required.
config.headers.insert("Copilot-Integration-Id".into(), "vscode-chat".into());
config.headers.insert("Editor-Version".into(), "Neovim/0.10.0".into());

let agent = Agent::from_config(config)
    .with_api_key(copilot_token); // see below
}

The API key is a short-lived Copilot token, not your GitHub token. You obtain it by exchanging a GitHub OAuth token (from the device-login flow, or from the local Copilot config under ~/.config/github-copilot/) at https://api.github.com/copilot_internal/v2/token. That token expires after ~25–30 minutes.

yoagent has no built-in credential refresh — api_key is static for the life of the provider (Authorization: Bearer {api_key}). For anything longer than a single short turn, you must exchange and refresh the token yourself and rebuild the agent's config with a fresh token before it expires; otherwise long runs will fail with 401.

Auth

Uses Authorization: Bearer {api_key} header. Extra headers can be added via ModelConfig.headers.

Google Gemini Provider

Two providers for Google's Gemini models:

• GoogleProvider — Google AI Studio (Generative AI API)
• GoogleVertexProvider — Google Cloud Vertex AI

Google AI Studio

#![allow(unused)]
fn main() {
use yoagent::provider::ModelConfig;

let agent = Agent::from_config(ModelConfig::google("gemini-2.5-flash", "Gemini 2.5 Flash"));
}

API Details

• Endpoint: {base_url}/v1beta/models/{model}:streamGenerateContent?alt=sse&key={api_key}
• Auth: API key as query parameter
• Default base URL: https://generativelanguage.googleapis.com
• Default context window: 1,000,000 tokens

Message Format

Google uses a different message format than OpenAI/Anthropic:

Streaming

Uses SSE format (alt=sse). Each chunk contains candidates with content.parts and optional usageMetadata.

Google Vertex AI

GoogleVertexProvider uses the same message format but with Vertex AI authentication and endpoints.

• Protocol: ApiProtocol::GoogleVertex
• Auth: OAuth2 / service account credentials
• Endpoint pattern: https://{region}-aiplatform.googleapis.com/v1/projects/{project}/locations/{region}/publishers/google/models/{model}:streamGenerateContent

Amazon Bedrock Provider

BedrockProvider implements the AWS Bedrock ConverseStream API.

Usage

#![allow(unused)]
fn main() {
use yoagent::provider::{ApiProtocol, ModelConfig};

// Bedrock has no dedicated ModelConfig preset — build one with `custom`.
let agent = Agent::from_config(ModelConfig::custom(
    ApiProtocol::BedrockConverseStream,
    "bedrock",
    "https://bedrock-runtime.us-east-1.amazonaws.com",
    "anthropic.claude-opus-4-8",
    "Claude Opus 4.8",
))
    .with_api_key("ACCESS_KEY:SECRET_KEY");  // or ACCESS_KEY:SECRET_KEY:SESSION_TOKEN
}

Authentication

The api_key field uses a colon-separated format:

{access_key_id}:{secret_access_key}
{access_key_id}:{secret_access_key}:{session_token}

Alternatively, provide pre-computed auth headers via ModelConfig.headers or use an IAM proxy that handles SigV4 signing.

API Details

• Endpoint: {base_url}/model/{model}/converse-stream
• Default base URL: https://bedrock-runtime.us-east-1.amazonaws.com
• Protocol: ApiProtocol::BedrockConverseStream

Message Format

Bedrock uses its own content block format:

Stream Events

Bedrock's ConverseStream returns these event types:

• contentBlockStart — New content block (text or tool use)
• contentBlockDelta — Text or tool use input delta
• contentBlockStop — Block complete
• messageStop — Stop reason (end_turn, max_tokens, tool_use)
• metadata — Token usage

Azure OpenAI Provider

AzureOpenAiProvider implements the OpenAI Responses API format with Azure-specific authentication and URL patterns.

Usage

#![allow(unused)]
fn main() {
use yoagent::provider::{ApiProtocol, ModelConfig};

// Azure has no dedicated ModelConfig preset — build one with `custom`.
// The provider ("azure") resolves the key from AZURE_OPENAI_API_KEY.
let agent = Agent::from_config(ModelConfig::custom(
    ApiProtocol::AzureOpenAiResponses,
    "azure",
    "https://{resource}.openai.azure.com/openai/deployments/{deployment}",
    "gpt-5.5",
    "GPT-5.5",
));
}

Authentication

Uses the api-key header (not Authorization: Bearer):

api-key: {your_api_key}

Additional headers can be set via ModelConfig.headers (e.g., for Azure AD Bearer tokens).

URL Format

https://{resource}.openai.azure.com/openai/deployments/{deployment}

Set this as ModelConfig.base_url. The provider appends /responses?api-version=2025-01-01-preview.

API Details

• Protocol: ApiProtocol::AzureOpenAiResponses
• Format: OpenAI Responses API (not Chat Completions)
• Streaming: SSE with event types:
  • response.output_text.delta — Text content
  • response.function_call_arguments.start — Tool call start
  • response.function_call_arguments.delta — Tool call arguments
  • response.completed — Final usage data

Message Format

Uses the Responses API input format:

OpenCode Zen & Go

OpenCode Zen (pay-per-use) and OpenCode Go (subscription) are model gateways run by the OpenCode team. Both are supported through the ModelConfig::opencode_zen() and ModelConfig::opencode_go() presets.

The gateways serve different model families over different protocols. The presets select the protocol automatically from the model id:

Gemini models on Zen are not supported — Zen serves them over a Google-native endpoint shape yoagent does not target. A gemini-* id falls through to Chat Completions (with a warning logged) and will fail at request time.

The routing table mirrors the Zen/Go endpoint docs as of mid-2026. OpenCode can change gateway-side routing at any time — if a model errors, verify its protocol against {base}/models.

Usage

Agent::from_config selects the built-in provider from the preset's protocol (config.api) and resolves the key from OPENCODE_API_KEY, so the same call works whichever model family you pick:

#![allow(unused)]
fn main() {
use yoagent::provider::ModelConfig;
use yoagent::Agent;

// Chat-completions model (GLM, Kimi, DeepSeek, ...)
let agent = Agent::from_config(ModelConfig::opencode_zen("glm-5.2"));

// Claude/Qwen model — Anthropic Messages protocol
let agent = Agent::from_config(ModelConfig::opencode_zen("claude-sonnet-5"));
}

For OpenCode Go, use ModelConfig::opencode_go("kimi-k2.7-code") — the base URL and protocol map differ, the usage pattern is identical.

Authentication

Both gateways use Authorization: Bearer {api_key}. For the Anthropic-protocol models the presets set AnthropicCompat { bearer_auth: true, .. } so the Anthropic provider sends Bearer auth instead of its native x-api-key header.

Get an API key by signing in at opencode.ai (Zen) or subscribing to Go.

Defaults

The presets use conservative defaults (128K context window, 16K max output). Override the fields for models with larger limits:

#![allow(unused)]
fn main() {
let mut config = ModelConfig::opencode_zen("kimi-k2.7-code");
config.context_window = 256_000;
}

Endpoints

• Zen: https://opencode.ai/zen/v1/{chat/completions | messages | responses}
• Go: https://opencode.ai/zen/go/v1/{chat/completions | messages}

Model list and metadata: https://opencode.ai/zen/v1/models and https://opencode.ai/zen/go/v1/models.

Built-in Tools

yoagent ships with six coding-oriented tools. Get them all with default_tools():

#![allow(unused)]
fn main() {
use yoagent::tools::default_tools;
let tools = default_tools();
}

BashTool

Execute shell commands with timeout and output capture.

• Name: bash
• Parameters: command (string, required)

Configuration

#![allow(unused)]
fn main() {
pub struct BashTool {
    pub cwd: Option<String>,           // Working directory
    pub timeout: Duration,             // Default: 120s
    pub max_output_bytes: usize,       // Default: 256KB
    pub deny_patterns: Vec<String>,    // Blocked commands
    pub confirm_fn: Option<ConfirmFn>, // Confirmation callback
}
}

Default deny patterns: rm -rf /, rm -rf /*, mkfs, dd if=, fork bomb.

Example

#![allow(unused)]
fn main() {
let bash = BashTool::default();
// Or customize:
let bash = BashTool {
    cwd: Some("/workspace".into()),
    timeout: Duration::from_secs(60),
    ..Default::default()
};
}

ReadFileTool

Read file contents with optional line range.

• Name: read_file
• Parameters: path (required), offset (optional, 1-indexed line), limit (optional, number of lines)

Configuration

#![allow(unused)]
fn main() {
pub struct ReadFileTool {
    pub max_bytes: usize,              // Default: 1MB
    pub allowed_paths: Vec<String>,    // Path restrictions (empty = no restriction)
}
}

WriteFileTool

Write content to a file. Creates parent directories automatically.

• Name: write_file
• Parameters: path (required), content (required)

EditFileTool

Surgical search/replace edits. The most important tool for coding agents — instead of rewriting entire files, the agent specifies exact text to find and replace.

• Name: edit_file
• Parameters: path (required), old_text (required), new_text (required)

The old_text must match exactly, including whitespace and indentation.

ListFilesTool

List files and directories with optional glob filtering.

• Name: list_files
• Parameters: path (optional, default: .), pattern (optional glob)

Configuration

#![allow(unused)]
fn main() {
pub struct ListFilesTool {
    pub max_results: usize,    // Default: 200
    pub timeout: Duration,     // Default: 10s
}
}

Uses find or fd for efficient traversal.

SearchTool

Search files using grep (or ripgrep if available).

• Name: search
• Parameters: pattern (required, regex), path (optional root directory)

Configuration

#![allow(unused)]
fn main() {
pub struct SearchTool {
    pub root: Option<String>,      // Root directory
    pub max_results: usize,        // Default: 50
    pub timeout: Duration,         // Default: 30s
}
}

Returns matching lines with file paths and line numbers.

SharedStateTool

Read and write named variables in a shared key-value store. This tool is not included in default_tools() — it is automatically injected into sub-agents when you call SubAgentTool::with_shared_state().

• Name: shared_state
• Parameters: action (required: get, set, list, remove), key (required for get/set/remove), value (required for set)

See Sub-Agents: Shared State for usage details.

Configuration

AgentLoopConfig

The main configuration for the agent loop:

#![allow(unused)]
fn main() {
pub struct AgentLoopConfig {
    pub provider: Arc<dyn StreamProvider>,
    pub model: String,
    pub api_key: String,
    pub thinking_level: ThinkingLevel,
    pub max_tokens: Option<u32>,
    pub temperature: Option<f32>,
    pub model_config: Option<ModelConfig>,
    pub convert_to_llm: Option<ConvertToLlmFn>,
    pub transform_context: Option<TransformContextFn>,
    pub get_steering_messages: Option<GetMessagesFn>,
    pub get_follow_up_messages: Option<GetMessagesFn>,
    pub context_config: Option<ContextConfig>,
    pub compaction_strategy: Option<Arc<dyn CompactionStrategy>>,
    pub execution_limits: Option<ExecutionLimits>,
    pub cache_config: CacheConfig,
    pub tool_execution: ToolExecutionStrategy,
    pub retry_config: RetryConfig,
    pub before_turn: Option<BeforeTurnFn>,
    pub after_turn: Option<AfterTurnFn>,
    pub on_error: Option<OnErrorFn>,
    pub input_filters: Vec<Arc<dyn InputFilter>>,
    pub turn_delay: Option<Duration>,
}
}

StreamConfig

Passed to StreamProvider::stream():

#![allow(unused)]
fn main() {
pub struct StreamConfig {
    pub model: String,
    pub system_prompt: String,
    pub messages: Vec<Message>,
    pub tools: Vec<ToolDefinition>,
    pub thinking_level: ThinkingLevel,
    pub api_key: String,
    pub max_tokens: Option<u32>,
    pub temperature: Option<f32>,
    pub model_config: Option<ModelConfig>,
    pub cache_config: CacheConfig,
}
}

ContextConfig

Controls context window compaction:

#![allow(unused)]
fn main() {
pub struct ContextConfig {
    pub max_context_tokens: usize,                            // Default: 100,000
    pub system_prompt_tokens: usize,                          // Default: 4,000
    pub keep_recent: usize,                                   // Default: 10
    pub keep_first: usize,                                    // Default: 2
    pub tool_output_max_lines: usize,                         // Default: 200
    pub tool_output_max_lines_overrides: HashMap<String, usize>, // Default: {"read_file": MAX}
    pub compact_target_ratio: f32,                            // Default: 0.7
    pub compact_headroom_turns: Option<usize>,                // Default: Some(30)
    pub truncate_tool_output_on_append: bool,                 // Default: true
}
}

compact_headroom_turns sets the compaction target from observed growth (target = budget − turns × growth_per_turn), keeping the interval between compactions constant as a session lengthens; compact_target_ratio is the fallback and a ceiling on retention. truncate_tool_output_on_append caps tool output as it enters the context rather than retroactively, and tool_output_max_lines_overrides gives per-tool budgets so a tool that head+tail would damage (a paging reader) can opt out. All three exist to keep the provider's prefix cache intact — see Context Management.

When context_config is not explicitly set, it is automatically derived from ModelConfig.context_window (80% for context, 20% reserved for output). If neither is set, ContextConfig::default() (100K) is used.

#![allow(unused)]
fn main() {
// Derive from a model's context window:
let config = ContextConfig::from_context_window(200_000);
// config.max_context_tokens == 160_000
}

ExecutionLimits

Prevents runaway agents:

#![allow(unused)]
fn main() {
#[non_exhaustive]                      // build with Default::default() + with_*
pub struct ExecutionLimits {
    pub max_turns: usize,              // Default: 50
    pub max_total_tokens: usize,       // Default: 1,000,000
    pub max_duration: Duration,        // Default: 600s
    pub max_consecutive_identical_tool_calls: Option<usize>,  // Default: Some(3)
}
}

#![allow(unused)]
fn main() {
ExecutionLimits::default()
    .with_max_turns(20)
    .with_max_consecutive_identical_tool_calls(None)  // disable loop detection
}

ThinkingLevel

#![allow(unused)]
fn main() {
pub enum ThinkingLevel {
    Off,        // No thinking (default)
    Minimal,    // Anthropic: effort "low" (adaptive) / 1,024-token budget (legacy)
    Low,        // Anthropic: effort "low" / 1,024
    Medium,     // Anthropic: effort "medium" / 2,048
    High,       // Anthropic: effort "high" / 8,192
}
}

OpenAI-family providers map these levels to reasoning_effort where the compat flags enable it; the Google and Bedrock providers currently ignore thinking_level.

CostConfig

Token pricing per million:

#![allow(unused)]
fn main() {
#[non_exhaustive]                          // build with new() + with_*, not a literal
pub struct CostConfig {
    pub input_per_million: f64,
    pub output_per_million: f64,
    pub cache_read_per_million: f64,
    pub cache_write_per_million: f64,
    pub context_tiers: Vec<ContextTier>,   // empty = one flat rate at every size
}
}

Cache rates are set with builders rather than positionally. Four same-typed f64 arguments in a row is a transposition hazard, and no vendor publishes them in one order — Anthropic lists input / cache-write / cache-read / output, OpenAI lists input / cached-input / output:

#![allow(unused)]
fn main() {
CostConfig::new(5.0, 30.0)          // input, output — output is always dearer
    .with_cache_read(0.5)
    .with_cache_write(6.25)
}

All-zero rates mean pricing unknown, not free. is_configured() reports which, and session_cost_usd() returns None for an unpriced model rather than $0.

Context tiers

Some vendors charge more above a prompt-size threshold. cost_usd selects by the request's prompt tokens (input + cache_read + cache_write), so a long reply to a short prompt stays on the base rate:

#![allow(unused)]
fn main() {
CostConfig::new(5.0, 30.0)
    .with_context_tier(ContextTier::new(272_000, 10.0, 45.0).with_cache_read(1.0))
}

Tiers are kept sorted, and cost_usd takes the last one the prompt clears, so a multi-step schedule works. No shipped preset sets one — see ModelConfig::gpt_5_5's docs for why the one candidate stayed flat.

One caveat if you add a tier: prompt size is derived as input + cache_read + cache_write, which holds only where the provider subtracts cached tokens out of input. bedrock.rs populates neither cache field, so a heavily-cached prompt reads small there.

ModelConfig Presets

yoagent provides first-class ModelConfig::* constructors for Anthropic, OpenAI, Google Gemini, xAI, Groq, DeepSeek, Mistral, MiniMax, Z.ai, Qwen, Ollama, and local OpenAI-compatible servers.

See Model Presets for the full table of constructors, default base URLs, context windows, and DeepSeek legacy alias notes.

API Reference

Top-Level Functions

agent_loop()

#![allow(unused)]
fn main() {
pub async fn agent_loop(
    prompts: Vec<AgentMessage>,
    context: &mut AgentContext,
    config: &AgentLoopConfig,
    tx: mpsc::UnboundedSender<AgentEvent>,
    cancel: CancellationToken,
) -> Vec<AgentMessage>
}

Start an agent loop with new prompt messages. Returns all messages generated during the run.

agent_loop_continue()

#![allow(unused)]
fn main() {
pub async fn agent_loop_continue(
    context: &mut AgentContext,
    config: &AgentLoopConfig,
    tx: mpsc::UnboundedSender<AgentEvent>,
    cancel: CancellationToken,
) -> Vec<AgentMessage>
}

Resume from existing context. The last message must not be an assistant message.

default_tools()

#![allow(unused)]
fn main() {
pub fn default_tools() -> Vec<Box<dyn AgentTool>>
}

Returns: BashTool, ReadFileTool, WriteFileTool, EditFileTool, ListFilesTool, SearchTool.

Agent Struct

High-level stateful wrapper around the agent loop.

Construction

#![allow(unused)]
fn main() {
let agent = Agent::from_config(ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"));
}

Builder Methods

All return Self for chaining (unless noted as Result).

Core

Tools & Integrations

Context & Limits

Behavior

Callbacks

Prompting

State Access

State Mutation

Steering & Follow-Up Queues

Control

SubAgentTool

Delegates tasks to a child agent loop.

Construction

#![allow(unused)]
fn main() {
// Provider auto-selected from the config's protocol; env key resolved automatically:
let sub = SubAgentTool::from_config("name", ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"));

// Or pass an explicit provider Arc (custom providers, or a shared handle across sub-agents):
let sub = SubAgentTool::from_provider("name", Arc::new(provider), ModelConfig::anthropic("claude-sonnet-5", "Claude Sonnet 5"));
}

Builder Methods

All return Self for chaining.

SharedState

Pluggable key-value store for sub-agent communication. Backed by a SharedStateBackend trait.

Construction

#![allow(unused)]
fn main() {
use yoagent::shared_state::{SharedState, FileBackend};

let state = SharedState::new();                              // MemoryBackend, 10MB cap
let state = SharedState::with_max_bytes(50 * 1024 * 1024);  // MemoryBackend, 50MB cap
let state = SharedState::with_backend(FileBackend::new("./state-dir")); // FileBackend
}

Methods

Built-in Backends

Custom Backends

Implement the SharedStateBackend trait:

#![allow(unused)]
fn main() {
#[async_trait::async_trait]
pub trait SharedStateBackend: Send + Sync {
    async fn get(&self, key: &str) -> Result<Option<String>, SharedStateError>;
    async fn set(&self, key: &str, value: String) -> Result<(), SharedStateError>;
    async fn remove(&self, key: &str) -> Result<bool, SharedStateError>;
    async fn keys(&self) -> Result<Vec<String>, SharedStateError>;
    async fn summary(&self) -> Result<String, SharedStateError>;
}
}

Re-exports

The crate re-exports key types from lib.rs:

#![allow(unused)]
fn main() {
pub use agent::Agent;
pub use agent_loop::{agent_loop, agent_loop_continue};
pub use types::*;  // Message, Content, AgentMessage, AgentEvent, etc.
}

LlmCompaction — live evaluation report

Branch feat/llm-compaction, PR #119. Numbers below are from real API runs against Claude Sonnet 5 (session) + Claude Haiku 4.5 (summarizer), reproducible via cargo run --example llm_compaction_live --features gasp.

Every live run in this report was GASP-recorded: the harness routes the agent's event stream through GaspRecorder, so each turn is a run in a git-backed event log (state/events.jsonl plus one commit per run) rather than console output that scrolled past. The transcripts the briefings were judged from are reconstructable from that record, not from memory.

All three reproduction commands below were run from a clean clone of this branch before publication. The prefix-cache figures reproduced exactly (they are deterministic — no model in the loop). The live figures are given as ranges across three runs, because the model is not deterministic and a single decimal would be false precision: session cache hit rate landed at 75.4%, 76.4% and 79.2%; summarization cost at $0.0149, $0.0171 and $0.0184.

Provider comparison: DeepSeek vs Anthropic

"Not cached" is input + cache_write. An earlier version of this page counted only input, which understated Anthropic roughly tenfold: Anthropic books a re-processed prefix to cache_write, DeepSeek has no write category, so the two are not comparable on input alone. The figures below supersede that.

• Session rates are close despite yoagent placing explicit cache_control breakpoints for Anthropic and sending nothing to DeepSeek.
• DeepSeek's cost is almost entirely compaction (91.9%): populating its cache is free, so between rewrites it pays only for genuinely new content.
• Anthropic pays continuously — ~2,000–4,600 cache-write tokens per turn at 1.25× — so its curve climbs more slowly and tops out at 96%, and compaction is only half its non-cached total. Those writes cost 3.3× the reads they enable ($0.232 vs $0.071) yet still leave the session 43% cheaper than no caching at all, so the trade is right even though the ratio looks wrong.
• Trigger ratio 0.6 → 0.35 was a null result across eight earlier runs.

"Steady state" is a range, not a number. Within a compaction cycle the hit rate climbs monotonically — each turn adds a roughly fixed amount of new content (26–158 tokens on DeepSeek, 532–4,592 on Sonnet) to a prefix that keeps growing — then resets when compaction truncates the prefix. DeepSeek's cycle reads 88.3 → 92.8 → 95.2 → … → 99.6%. The peak is what a mature prefix achieves; the median is what this budget actually delivers, and it moves with compaction frequency, so it describes the configuration rather than the provider. A supporting floor on DeepSeek: it quantises hits to 64-token blocks (every cache_read in the run is an exact multiple of 64), so a trailing partial block never hits.

An earlier version of this page reported Sonnet's steady state as ~81%. That was read off the middle of its ramp rather than computed, and it understated the mature-prefix figure by 15 points.

Session hit rate spans five runs per provider — that metric always counted cache_write and was unaffected by the bug. The per-turn figures (mature-turn rates, compaction share, cost) are n=1 each: earlier runs did not record per-turn cache_write, so they could not be re-derived and had to be re-measured.

Costs come from the recorded token counts at list prices — ModelConfig::claude_sonnet_5 ($2/$10 per MTok, $2.50 write, $0.20 read) and DeepSeek's peak-window rate, which is double its off-peak rate, so that column over-reports an off-peak run by up to 2×.

The 25× cost gap is not a caching result. The sessions are not matched: Sonnet ran 20 turns to DeepSeek's 15 and emitted 2.8× the output tokens, and output is the majority of both bills. What the runs do compare cleanly is the shape — zero writes versus continuous writes, 92% versus 50% of non-cached tokens attributable to compaction. Session lengths differ, so percentages compare and absolute totals do not.

Why these numbers are lower than the replay figures

docs/concepts/prompt-caching.md reports 93–96% from 300–2400 turn replays. These live runs are 15–20 turns, where the arithmetic ceiling is ~88–90%: every turn's new content is necessarily a miss, so the best achievable rate is about (n-1)/(n+1). 80% over 15 turns and 95% over 300 turns describe the same behaviour. Hit rates are not comparable across session lengths; rewrite counts and dollars are.

The numbers, for anyone who wants them

On that last row: the strategy does not reduce prefix-cache breaks. Both it and the deterministic default rewrite history only when the budget is crossed. It buys retention quality and costs tokens. The docs say so explicitly, and the figures come from a committed harness (tests/prefix_cache_harness.rs), not from a scratch file — an earlier draft of this work claimed a cache win that measurement did not support.

Reproducing

git clone https://github.com/yologdev/yoagent && cd yoagent
git checkout feat/llm-compaction

# prefix-cache measurements (no API key needed)
cargo test --test prefix_cache_harness -- --ignored --nocapture --test-threads=1

# live briefing evaluation (needs ANTHROPIC_API_KEY; measured $0.78 for a
# 20-turn Sonnet 5 run — set YO_MAX_TURNS lower to spend less, or point
# YO_MODEL/YO_SUMMARIZER at deepseek-v4-flash for ~$0.03)
cargo run --example llm_compaction_live --features gasp

# harness plumbing only, no key, no bill
YO_DRY_RUN=1 cargo run --example llm_compaction_live --features gasp

Caveats worth stating if anyone asks

• Three live runs, and the briefing-quality verdict is a judgement from reading them, not a benchmark. Someone else reading the same briefings could reasonably grade them differently.
• The A/B on the instruction change is underpowered and I'm not claiming it.
• The cache-hit figures come from one session shape and vary a few points between runs; they will move much more with turn size, budget, and how often compaction fires.
• keep_first: 0 in the live harness is not the crate default — it is set to force the briefing to be the only carrier, which is what makes the retention probe meaningful.

Architecture Overview

Layered Design

yoagent is organized as three conceptual layers within a single crate. Dependencies flow strictly downward — upper layers use lower layers, never the reverse.

┌─────────────────────────────────────────────┐
│  Layer 3: Orchestration          (planned)   │
│  Multi-agent, delegation, work modes         │
├─────────────────────────────────────────────┤
│  Layer 2: Agent + Providers                  │
│  Concrete providers, tools, retry, caching,  │
│  context management, MCP                     │
├─────────────────────────────────────────────┤
│  Layer 1: Core Loop                          │
│  agent_loop, types, traits                   │
│  Provider-agnostic. Tool-agnostic.           │
└─────────────────────────────────────────────┘

Layer 1: Core Loop

The pure agent loop. No opinions about LLMs, no built-in tools. Just the control flow.

Modules: types.rs, agent_loop.rs, provider/traits.rs

Owns:

• agent_loop() / agent_loop_continue() — the loop itself
• AgentTool trait — interface tools must implement
• StreamProvider trait — interface providers must implement
• AgentMessage, AgentEvent, StreamDelta — message & event types
• AgentContext — system prompt + messages + tools
• Tool execution strategies (parallel/sequential/batched)
• Streaming tool output (ToolUpdateFn)
• Steering & follow-up message injection

Does not own: Any concrete provider or tool implementation.

Layer 2: Agent + Providers

Batteries-included single-agent layer. Most users interact with this.

Modules: agent.rs, context.rs, retry.rs, provider/*.rs, tools/*.rs, mcp/*.rs

Adds on top of Layer 1:

• Concrete providers — Anthropic, OpenAI-compat, Google, Azure, Bedrock, Vertex
• Provider registry — dispatch by API protocol
• Prompt caching — automatic cache breakpoint placement
• Retry with backoff — exponential, jitter, respects retry-after
• Context management — token estimation, smart truncation, execution limits
• Built-in tools — bash, read_file, write_file, edit_file, list_files, search
• MCP client — stdio + HTTP transports, tool adapter
• Agent struct — stateful builder wrapping it all together

Layer 3: Orchestration (planned)

Multi-agent coordination. Not yet implemented — the architecture is designed to support it when needed.

Planned capabilities:

• Orchestrator struct — spawn, delegate, and coordinate multiple agents
• Work modes:
  • Interactive — multi-turn, human in the loop (current default)
  • Autonomous — runs to completion without input (background tasks, CI)
  • Pipeline — input → output, chainable (scan → fix → verify)
  • Supervisor — delegates to other agents, synthesizes results
• Fan-out — same task to multiple agents (different providers for diversity)
• Pipeline chaining — output of agent A feeds input of agent B
• Agent communication through the orchestrator event bus

Why not yet: Multi-agent orchestration adds complexity. The single-agent loop handles 95% of use cases. Layer 3 will be built when a concrete use case drives it, not speculatively.

Module Layout

yoagent/
├── src/
│   ├── lib.rs                  # Public re-exports
│   │
│   │── Layer 1: Core Loop ─────────────────────
│   ├── types.rs                # Message, Content, AgentTool, AgentEvent
│   ├── agent_loop.rs           # Core loop: prompt → LLM → tools → repeat
│   │
│   │── Layer 2: Agent + Providers ─────────────
│   ├── agent.rs                # Agent struct (stateful wrapper)
│   ├── context.rs              # Token estimation, compaction, limits
│   ├── retry.rs                # Retry with exponential backoff
│   ├── provider/
│   │   ├── traits.rs           # StreamProvider trait, StreamEvent, ProviderError
│   │   ├── model.rs            # ModelConfig, ApiProtocol, OpenAiCompat
│   │   ├── registry.rs         # ProviderRegistry (protocol → provider)
│   │   ├── anthropic.rs        # Anthropic Messages API
│   │   ├── openai_compat.rs    # OpenAI Chat Completions (15+ providers)
│   │   ├── openai_responses.rs # OpenAI Responses API
│   │   ├── google.rs           # Google Generative AI
│   │   ├── google_vertex.rs    # Google Vertex AI
│   │   ├── bedrock.rs          # AWS Bedrock ConverseStream
│   │   ├── azure_openai.rs     # Azure OpenAI
│   │   ├── mock.rs             # Mock provider for testing
│   │   └── sse.rs              # SSE utilities
│   ├── tools/
│   │   ├── bash.rs             # BashTool
│   │   ├── file.rs             # ReadFileTool, WriteFileTool
│   │   ├── edit.rs             # EditFileTool
│   │   ├── list.rs             # ListFilesTool
│   │   └── search.rs           # SearchTool
│   └── mcp/
│       ├── client.rs           # MCP client (stdio + HTTP)
│       ├── tool_adapter.rs     # McpToolAdapter (MCP tool → AgentTool)
│       ├── transport.rs        # Transport implementations
│       └── types.rs            # MCP protocol types

Data Flow

                    ┌─────────────┐
                    │   Caller    │
                    └──────┬──────┘
                           │ prompt / prompt_messages
                    ┌──────▼──────┐
                    │    Agent    │  Layer 2: stateful wrapper
                    │  (agent.rs) │  Manages queues, tools, state
                    └──────┬──────┘
                           │
                    ┌──────▼──────┐
                    │ agent_loop  │  Layer 1: core loop
                    │             │  Prompt → LLM → Tools → Repeat
                    └──┬───────┬──┘
                       │       │
              ┌────────▼──┐ ┌──▼────────┐
              │ Provider  │ │   Tools   │  Layer 2: implementations
              │ .stream() │ │ .execute()│
              └────────┬──┘ └──┬────────┘
                       │       │
              ┌────────▼──┐ ┌──▼────────┐
              │ LLM API   │ │ OS / FS   │
              │ (HTTP)    │ │ (shell)   │
              └───────────┘ └───────────┘

Events flow back via mpsc::UnboundedSender<AgentEvent>

How Providers Plug In

1. Implement StreamProvider trait (Layer 1 interface)
2. Register with ProviderRegistry under an ApiProtocol (Layer 2)
3. Set ModelConfig.api to match that protocol
4. The registry dispatches stream() calls to the right provider

Each provider translates between yoagent's Message/Content types and the provider's native API format. All providers emit StreamEvents through the channel for real-time updates.

How Tools Plug In

1. Implement AgentTool trait (Layer 1 interface)
2. Add to the tools vec (via default_tools() or custom)
3. The agent loop converts tools to ToolDefinition (name, description, schema) for the LLM
4. When the LLM returns Content::ToolCall, the loop finds the matching tool and calls execute()
5. Results are wrapped in Message::ToolResult and added to context

Tools receive a CancellationToken child token — they should check it for cooperative cancellation during long operations.

Design Principles

• Layers are conceptual, not physical. One crate, clean module boundaries, no feature flags needed.
• Dependencies flow down. Layer 1 never imports from Layer 2. Layer 2 never imports from Layer 3.
• Layer 1 is stable. The core loop and traits change rarely. New features are added in Layer 2 or 3.
• Build what's needed. Layer 3 is designed but not implemented. It will be built when a use case demands it, not speculatively.
• Simple over clever. A straightforward loop with good defaults beats an elegant abstraction nobody can debug.