Standard AI vendor evaluation (L4)
One evaluation standard for every AI vendor: data handling, meter and pricing mechanics, retrieval quality on your own content, exit path, and roadmap dependency. Orchestrated so procurement, security, and the business team run it in parallel rather than in sequence.
The steps
- 01
Test on your own data
Tool: Manual
Demos use vendor data and prove nothing. Require a bounded pilot against your content and your edge cases with a scoring rubric written before the pilot starts. Owner: business owner plus RevOps. DoD: rubric signed before the pilot begins.
- 02
Interrogate the meter
Tool: Manual
Get the unit of consumption, the rate, rollover rules, overage behaviour, and what happens at renewal in writing. Consumption pricing surprises are the most common AI budget failure. Pitfall: budgeting from a pilot month. DoD: meter mechanics documented from vendor written answers.
- 03
Run security and legal in parallel
Tool: Slack
Data residency, retention, training use, subprocessors, and incident terms start on day one of the pilot, not after the business decides. Owner: security plus legal. DoD: parallel review is standard in the workflow.
- 04
Write the exit path before signing
Tool: Manual
Data export format, prompt and configuration portability, and notice terms. Without an exit path you are buying a dependency, not a tool. Owner: procurement. DoD: exit path section completed in every evaluation.
Tools in this playbook
- Manual
- Slack
Next playbooks
Unfamiliar terms are defined in the AI and Revenue Dictionary. Related frameworks live in the framework library.