Chapter 07 / 08
Qualify the automation
Evaluate the development agent and qualify each automated handoff before expanding its reach.
Optional guidance added in v1.4
Anthropic’s AI-Native SDLC Playbook prompted these additions and informed the evaluation and artifact-handoff ideas. Velocity’s templates adapt them to its existing authority, proof, and lane rules. Read the source attribution.
Evaluate the system doing the development
Product tests ask whether the software behaves correctly. A development-agent evaluation asks whether a changed model, prompt, skill, permission set, or runtime performs its assigned work reliably.
Use representative tasks, incident regressions, expected outcomes, and explicit acceptance thresholds. Include refusal cases: attempts to cross a lane, weaken an acceptance criterion, or claim behavior from static evidence. Protect fixtures and grading criteria from the candidate being evaluated.
Compare against the baseline configuration and retain attempted-run outcomes, including failed and interrupted runs. A clean subset of successful examples cannot establish the reliability of the whole process.
Qualify one transition
An automated trigger starts a check of the transition contract. It does not grant authority. The contract connects an accepted artifact revision and its proof to one permitted next action, with an execution-time authorization check and a recovery route.
Test stale revisions, repeated events, interrupted execution, denied actions, and partial completion. Mandatory controls belong outside the agent’s write authority. Verify the relevant mutation routes; a guard on one entry point may leave another open.
Keep implementation in the project
Velocity supplies templates. The consuming project supplies the adapters, fixtures, runtime enforcement, and observed results. A filled-in document is not a working integration, and an evaluation pass cannot expand the agent’s authority.
Start with one bounded workflow and a reviewable recovery path. Broaden automation only after measuring how it behaves and resolving the relevant failure cases. Parallelism is useful only while integration and review capacity keep up.
Current optional support: Agent Evaluation Pack, Automation Transition Contract, and Control Planes. The worked example contains synthetic case designs, not executed success. The manifesto’s delegation grades remain unadopted.