Customer health scoring with AI signals (L4)
Combine product usage, support sentiment, and exec-engagement signals into a single churn-risk score with explicit experiments.
The steps
- 01
Define and segment churn events
A 'health score' is meaningless if you haven't defined what 'unhealthy' looks like. Work with your Data/Finance team to create three distinct binary definitions of churn: Logo Churn (contract termination), Downgrade (moving to a lower tier), and Contraction (seat reductions or usage-based revenue drops). This is critical because the signals for a user leaving entirely are often different from a CFO cutting costs. • Open your CRM (Salesforce/HubSpot) or Snowflake/BigQuery environment. • Create three boolean fields or columns: `is_logo_churned`, `is_downgraded`, and `is_contracted`. • Set a historical lookback period of 12-24 months. For each account, flag if they met these criteria. • Owner: CS Ops + Data Analyst. • Time: 4-6 hours. • Pitfall: Mixing 'voluntary' churn (they left) with 'involuntary' (credit card failed). Only model voluntary churn for this playbook. • Definition of Done: A validated dataset where every historical account is tagged with at least one churn type.
- 02
Build a multi-dimensional signal inventory
Inventory all digital footprints that correlate with customer health. You need three categories of signals: Product (Usage), Support (Sentiment), and Relationship (Exec presence). • Product signals: Use tools like Mixpanel, Pendo, or Amplitude to export 'Usage Decay' (e.g., % drop in active users over 30 days) and 'Key Feature Adoption'. • Support signals: Connect your Zendesk or Intercom to an LLM or sentiment tool (like Gong or MonkeyLearn). Extract a 'Sentiment Score' (0-1) for the last 5 tickets. • Relationship signals: Track 'Exec Login Frequency',did someone with a 'VP' or 'C-Suite' title log in during the last 90 days? • SQL prompt example: `SELECT account_id, AVG(sentiment_score) as avg_mood, COUNT(DISTINCT exec_logins) as exec_pulse FROM activities GROUP BY 1;` • Owner: CS Ops. • Time: 8-10 hours. • Pitfall: Using 'total logins' which is a vanity metric; focus on 'active days' or 'core action' frequency instead. • Definition of Done: A table (The 'Signal Inventory') mapping account IDs to these specific metrics.
- 03
Train and backtest the AI model
You must prove the signals actually predict churn before asking CSMs to act. Use a 'Lookback' methodology. Take your data from 6 months ago and see if the signals at that time would have predicted the churn that actually happened. • Tooling: Use a tool like Akkio, Pecan, or a Python notebook (Random Forest or Logistic Regression). • Process: Upload your Signal Inventory and your Churn Definition table. Use 70% of data for training and 30% for testing. • Target Metric: Aim for an AUC (Area Under Curve) of >0.75. This means the model is significantly better than a random guess. • If AUC is <0.70, go back to Step 2 and add 'Invoice Dispute' or 'NPS' data. • Owner: Data Science or RevOps. • Time: 15-20 hours. • Pitfall: Overfitting. If your model is 99% accurate, it’s probably just identifying people who already cancelled their subscription. • Definition of Done: A backtest report showing the model’s precision and recall at various score thresholds (e.g., 'At a score of 80, we catch 70% of churners').
- 04
Deploy the score to the CSM workflow
The raw AI score needs to be translated into a human-readable Health Score (0-100) and pushed into the CSM's daily workspace (Salesforce, Gainsight, or Totango). • Mapping: Convert the AI probability (0.12) into a score (12/100). • Categorization: Red (0-40), Yellow (41-70), Green (71-100). • Automation: Create a workflow that refreshes this score every Monday morning at 8:00 AM. • Setup an 'AI Reason' field next to the score. Use an LLM prompt like: 'Given these signals (Usage -20%, Ticket Sentiment 'Angry'), summarize the primary risk factor in 10 words.' • Owner: CRM Admin / CS Ops. • Time: 5-8 hours. • Pitfall: Not explaining the score. If a CSM sees 'Score: 42' with no explanation, they will ignore it. • Definition of Done: A live dashboard in the CRM showing 'At-Risk Accounts' sorted by the new AI Health Score.
- 05
Execute mandatory save-play sequences
A score without a play is just noise. Define a 'Save Play' for the Top-20 at-risk accounts that is mandatory for the CSM to execute within 5 business days of a score dropping into the 'Red' zone. • Trigger: A 'Red' score (below 40) creates a Task in the CRM: 'High Churn Risk: Execute Save Playbook'. • The Save Playbook: 1. Executive Outreach (email template provided), 2. Usage Audit (find why features aren't used), 3. Value Realization Call (re-confirming ROI). • Example email: 'Hi [Name], I noticed a change in how your team is interacting with [Product]. I want to ensure we're still aligned with your Q3 goals...' • Reporting: Track 'Save Rate',the percentage of accounts that moved from Red back to Yellow/Green within 60 days. • Owner: VP of Customer Success. • Time: 3 hours for setup; ongoing for execution. • Pitfall: Treating every Red score as a crisis; some accounts are naturally 'zombies' that have already decided to leave. Focus on those with high ARR first. • Definition of Done: A 'Save Play' activity history record attached to at least 80% of identified Red-zone accounts.
Next playbooks
Unfamiliar terms are defined in the AI and Revenue Dictionary. Related frameworks live in the framework library.