Skip to content

Models

A model performs the inference a harness needs to carry out an agent’s work. A model may interpret context, produce structured output, or propose a tool call. The harness supplies the context and controls the actual invocation of tools. A model provider serves the selected model through an authorized account.

Select for the actual task. A document extraction step may need vision input and structured output. A writer can use a text model with a smaller input. A spoken interaction can use a realtime audio model or a combination of speech and text services. Tool calling, input size, output validation, and cost are part of selection.

Step Required input Required output
Summarize accepted calendar data Structured data represented in text A summary validated against named fields
Inspect a shared tab Masked image and the visible control list A browser action that passes validation
Give a spoken answer Audio or a transcript Audio, with tool calls checked by the same service

Illustrative decision logic for a connector-backed reasoning step.

if model cannot accept the requested input: hold for setup
if model cannot return the required output: hold for setup
otherwise run within the step budget and validate the returned fields

Model output is variable. The harness can check output structure, limits, and tool permissions. A specialist can still need to verify a claim against its source. Capture the selected model and actual usage for an attempt. ### Evaluate a replacement model

Test a replacement model with the same inputs, tools, and evals before using it for the same playbook. Check both output quality and failure behavior. For example, when a source is missing, a replacement summarizer should request the evidence instead of inventing a citation.

See model providers, evals, and Coach’s vision model.