Chapter 08 / 08
Measure and learn
Use delivery results and failure evidence to decide which changes deserve to become repeatable practice.
Measure the whole delivery outcome
Faster code generation can still leave more waiting, review, or rework. The optional measured-pilot template added in v1.4 helps a project compare its existing process with one bounded automation change.
Declare the workflow, baseline, observation window, exclusions, and decision thresholds before interpreting the results. Record open or censored work so unfinished items do not disappear from the comparison. Separate measured outcomes from estimates and unknowns.
| Measure | Question it answers |
|---|---|
| Delivery elapsed time | How long from accepted work to acceptance at the named lane? |
| Active human effort and waiting | Did the process save attention or simply move a queue? |
| Rework and escaped defects | Did speed carry an additional quality cost? |
| Resource use | What did the workflow consume, including retries? |
| Authority and recovery failures | Did the workflow stay within its contract and recover as expected? |
Route observations back into Capture
Define monitoring thresholds, debounce and deduplication behavior, and a destination for evidence. A threshold breach can produce an issue for diagnosis and triage. Any automated response must fit the action already authorized by its runbook or transition contract.
Keep observed symptoms separate from diagnoses. A monitoring alert does not establish a root cause, and an agent’s explanation does not demonstrate recovery. Collect the proof appropriate to the action and the claim.
Promote a lesson through its owner
Some findings belong in a project’s commands, fixtures, or overlay. Others justify a reusable Velocity proposal. Name that disposition so a lesson is neither lost in conversation nor silently converted into a universal rule.
Maintain separate authority for delivery changes and automation changes. When the process doing the evaluation is itself modified, preserve the baseline and the evidence needed to judge that modification.
The current evidence limit
Velocity publishes no universal speedup or benchmark result. The measured-pilot template and synthetic example are starting tools. A claim of improvement needs observed results from a defined project workflow.
Optional v1.4 support: Measured Automation Pilot and Control Planes. The September additions were informed by the credited playbook review.