Evals
An eval tests behavior against a stated requirement. A suite groups cases, names its run mode, and can require a passing result before release. A case selects a command, actor role, inputs or fixtures, and assertions.
Define what the harness should verify
Section titled “Define what the harness should verify”Requirements, suites, and cases
Section titled “Requirements, suites, and cases”Author these definitions in assets/persona-evals.json. A requirement identifies the behavior, the definition that implements it, and case IDs that verify it. A suite selects a test mode and can mark itself required for production. A case runs a command as a role with fixtures and asserts observable behavior. Pick assertions for both the desired output and the boundaries the playbook must respect.
Example case
Section titled “Example case”This excerpt belongs in a suite’s cases. Register the named command and link draft-before-publishing to this case in requirements before running it.
{ "id": "draft-without-publish", "name": "Draft from a complete brief", "category": "golden", "commandRef": "prepare-campaign", "runAs": "author", "input": { "inputValues": { "brief": "Draft a launch announcement." } }, "requirementIds": ["draft-before-publishing"], "assertions": [ { "type": "execution_status", "expected": "completed" }, { "type": "no_external_side_effect" } ]}Exercise a changed definition
Section titled “Exercise a changed definition”Run a representative successful case, a missing-input case, and a case whose input tries to bypass approval. Check the failing condition and the evidence recorded in the result. When a specialist corrects an anomaly, add a case that reproduces it. Run the required suite before release and keep live workspace evidence separately scoped.