Orchestrated AI change cadence (L4)
Model updates, prompt changes, and workflow changes ship on a cadence with release notes, owners, and a rollback, the same as software. Silent changes are the leading cause of broken trust in AI workflows.
The steps
- 01
Version prompts and configs
Tool: Manual
Prompts, retrieval configs, and agent instructions live in version control with a change history. If a prompt lives in a text box only, you cannot debug a regression. Owner: RevOps or platform team. DoD: production prompts are versioned.
- 02
Ship release notes people read
Tool: Slack
What changed, why, expected effect, who to tell if it breaks. Three sentences. Users discovering changes by surprise stop reporting problems. Pitfall: technical changelogs written for the builders only. DoD: release note per production change.
- 03
Keep a tested rollback
Tool: Manual
Every change has a previous known good version and a rollback that someone has actually executed in a drill. Owner: platform owner. DoD: one rollback drill completed per quarter.
- 04
Watch a regression suite
Tool: observability
Maintain a fixed set of test inputs with expected behaviour and run them on every change. Without it, quality changes are only discovered by users. Owner: platform owner. DoD: regression suite runs automatically pre release.
Tools in this playbook
- Manual
- Slack
- observability
Next playbooks
Unfamiliar terms are defined in the AI and Revenue Dictionary. Related frameworks live in the framework library.