Appendix B from The AI Contact Center Handbook by Sho Shimoda. Available on Amazon.
Six diagnostic instruments
These are the diagnostic tools referenced throughout the book. They exist because CX leaders in the middle of a transformation asked for them: something they can print out, walk into a room with, and use to structure a hard conversation. Nothing here is a substitute for judgment. But structure helps.
| Section | Tool | Use it when |
|---|---|---|
| B.1 | Voice AI Readiness Diagnostic (20 questions, 100 points) | Deciding whether you are ready for a pilot, a production launch, or an enterprise-wide rollout. |
| B.2 | Voice AI Vendor Security Questionnaire (25 questions) | Sending to every vendor on a shortlist. |
| B.3 | Voice AI DPIA Template (5 sections) | Meeting the GDPR Article 35 obligation before any voice AI deployment. |
| B.4 | CX Survey Templates (5 designs) | Measuring the interactions your AI or your agents just handled. |
| B.5 | Executive Readiness Conversation (2-hour agenda) | Before any large AI-in-CX investment is committed. |
| B.6 | Weekly Operating Rhythm (4 cadences) | Once the AI is running in production. |
B.1 The Voice AI Readiness Diagnostic
Twenty questions, scored 0–5 each (0 = "not at all," 5 = "fully in place"). Total possible: 100.
| Score | Interpretation |
|---|---|
| Below 40 | Not yet ready to deploy voice AI at meaningful scale. |
| 40–65 | Ready to start with a limited pilot. |
| 66–85 | Ready for production deployment. |
| Above 85 | Ready for enterprise-wide rollout. |
Data and infrastructure (25 points)
- Do you have a unified customer profile accessible in real time to the AI, spanning at least CRM, order/billing, and interaction history?
- Is your call recording infrastructure capable of delivering transcripts to the AI within seconds of the call ending (or streaming during the call)?
- Do you have documented APIs to your core backend systems (CRM, order management, billing, knowledge base)?
- Is your identity system (SSO, agent authentication) capable of extending to service accounts that the AI can use to act on behalf of customers?
- Do you have a data warehouse or analytics platform that can ingest AI interaction logs at production scale?
Operational readiness (25 points)
- Does the operations team have a defined process for handling escalations from AI to human agents, including warm-transfer of full conversation context?
- Do you have real-time monitoring in place that will show you when the AI's containment rate, CSAT, or latency deviates from baseline?
- Do you have an on-call rotation defined for AI-specific incidents (misroutes, hallucinations, safety flags)?
- Have you documented the specific business processes the AI is authorized to execute, and the ones it is not?
- Do you have a rollback plan for individual AI capabilities that can be executed in under an hour?
Governance and compliance (25 points)
- Have you completed a Data Protection Impact Assessment (DPIA) for the AI deployment?
- Do you have a documented model registry recording which AI model version is in production, what data it was trained on, and who approved its deployment?
- Do you have an incident response plan for AI-specific failure modes (data leak, harmful output, hallucination causing customer harm)?
- Are your call recording, transcription, and AI processing environments in scope for the relevant regulatory frameworks (PCI DSS, HIPAA, GDPR, etc.) that apply to your business?
- Have you obtained appropriate customer notice and consent for AI-mediated interactions in the jurisdictions where you operate?
People and culture (25 points)
- Have you communicated to the frontline workforce what the AI will and will not do, and how it affects their jobs?
- Is there a designated internal owner of the AI product — someone with authority to make trade-offs between accuracy, cost, and containment?
- Have you identified the internal candidates who could transition into new AI-adjacent roles (AI Agent Builder, Conversational Designer, Guardian Agent Operator)?
- Does your executive leadership have realistic expectations for the value delivery timeline (typically 12–24 months for meaningful business impact)?
- Do you have a cross-functional governance forum (operations, IT, security, legal, HR) that meets at least monthly on the AI program?
B.2 The Voice AI Vendor Security Questionnaire
Send this to every voice AI vendor you evaluate. If the vendor cannot answer any of these questions clearly, that is itself the answer.
Data handling
- Where is customer data (audio, transcripts, derived metadata) physically stored, in what jurisdictions, and can we require it be kept in specific regions?
- Is customer data used to train the vendor's base models, either the customer's own data or aggregated across customers? If yes, what opt-out is available?
- What is the data retention policy for customer audio, transcripts, and interaction logs? Can it be customized?
- On contract termination, what is the process, timeline, and cost for returning or destroying all customer data?
- Do you support customer-managed encryption keys (BYOK)?
Model and inference security
- What language models does the AI use, and can we specify or restrict which models are used in our instance?
- What defenses do you have against prompt injection attacks? Can you demonstrate them?
- How do you detect and mitigate hallucinations in production, before they reach the customer?
- Can you provide model version control — a record of which model version handled each interaction, and the ability to rollback to a prior version?
- What is your process and timeline for patching models when a new safety issue is discovered?
Access control
- Do you support single sign-on integration with major identity providers (Okta, Azure AD, Ping)?
- What role-based access controls are available for administrative functions?
- How is service-to-service authentication handled between the AI and our backend systems?
- Do you provide audit logs of all administrative actions, and how long are they retained?
- What is your process for revoking access when an employee (yours or ours) leaves?
Compliance certifications and audits
- Which certifications do you hold: SOC 2 Type II, ISO 27001, HIPAA (BAA available), PCI DSS (as a service provider), FedRAMP, others?
- When was your most recent third-party penetration test, and can we see a summary?
- Do you support DPIA documentation and cooperation with our own DPIA process?
- What is your process for notifying us of a security incident, and what is your maximum notification window?
- Do you have documented policies for AI-specific compliance requirements (EU AI Act obligations, U.S. state AI laws, etc.)?
Operational resilience
- What is your service level agreement for uptime, and what credits apply when you miss?
- How is disaster recovery structured? What is your published Recovery Time Objective (RTO) and Recovery Point Objective (RPO)?
- How do you handle carrier or telecommunications outages that affect your platform?
- What is your process for capacity planning, and how do you handle traffic spikes that exceed forecast?
- Do you provide a public status page with historical uptime data?
B.3 Voice AI DPIA Template
A Data Protection Impact Assessment is required under GDPR and increasingly under other frameworks whenever a new processing activity is likely to result in high risk to individuals. Voice AI in a contact center almost always triggers this obligation. The template below is a structural starting point; adapt to your legal counsel's guidance.
| Section | What to document |
|---|---|
| 1. Description of the processing | Purpose of the AI system; categories of personal data processed; categories of data subjects; recipients (internal, service providers, sub-processors, regulators); retention periods; international transfers. |
| 2. Necessity and proportionality | Legal basis for processing (with justification); assessment of whether AI is the minimum necessary means; alternatives considered; data minimization measures; accuracy safeguards. |
| 3. Risk assessment | For each identified risk: nature; likelihood (low/med/high); severity (low/med/high); residual risk after mitigations. Common risks below. |
| 4. Mitigations | Technical (input filtering, Guardian Agent, encryption); organizational (access controls, audit review, training); contractual (vendor obligations, sub-processor limits, indemnification); data subject rights facilitation. |
| 5. Consultation and sign-off | DPO review; internal team consultation; if required, supervisory authority; executive sign-off (typically Chief Privacy Officer); review cadence (annual plus material change). |
Common risks to assess (Section 3)
- Model hallucination causing incorrect information delivery
- Prompt injection causing unauthorized action
- Voice deepfake defeating biometric authentication
- Discriminatory routing or treatment based on protected characteristics
- Cross-customer data leakage through the model
- Excessive data retention beyond regulatory requirements
- Inadequate transparency to the data subject
B.4 CX Survey Templates
Five ready-to-use survey designs. Deploy them at the interaction points where they actually measure what you think they measure — which is almost never right after the call ended.
| Template | When to send | What it measures |
|---|---|---|
| 1. Post-interaction CSAT | Within one hour of interaction close | Satisfaction, resolution, effort (CES) |
| 2. AI agent experience | Within one hour of interaction close | Understanding, resolution, willingness to repeat |
| 3. Relationship NPS | Once per year or after tenure milestone | Loyalty + "why" |
| 4. Post-resolution follow-up | 48–72 hours after interaction | True first-contact resolution |
| 5. Agent experience | Quarterly, anonymous | Workload, tools, job security, AI impact |
Template 1 — Post-interaction CSAT
- "Thinking about your recent contact with [Company], how satisfied were you with the experience?" (1–5)
- "Was your issue resolved?" (Yes / Partially / No / N/A)
- "How easy was it to get your issue resolved?" (1–7) — this is your CES
- Optional open text: "Is there anything you'd like to tell us about this experience?"
Keep it to four questions. Every additional question drops response rate by roughly 10–15%.
Template 2 — AI agent experience
- "You were helped by [Company]'s automated assistant. How well did it understand what you needed?" (1–5)
- "Did the automated assistant resolve your issue?" (Yes / Partially / No — please describe / Was transferred to a person)
- "Would you prefer to use the automated assistant again for similar issues?" (Yes / No / Depends on the issue)
- Optional open text: "What could the automated assistant have done better?"
The critical question is number three. If most customers say "yes" or "depends," your AI has customer permission to grow. If most say "no," you have a design problem, not a technology problem.
Template 3 — Relationship NPS
- "On a scale of 0 to 10, how likely are you to recommend [Company] to a friend or colleague?"
- Detractors (0–6): "What is the most important reason for your score?"
- Passives (7–8): "What one thing could we do to earn a higher score from you?"
- Promoters (9–10): "What do we do particularly well that you'd want us to keep doing?"
The follow-up questions matter more than the score itself. NPS without the "why" is a vanity metric.
Template 4 — Post-resolution follow-up
- "Two days ago you contacted us about [topic]. Is your issue still resolved?" (Yes / No — please tell us what happened)
- "Have you had to contact us again about the same issue?" (No / Yes — please describe)
- Optional: "Is there anything else we can help with related to this?"
This survey measures true first-contact resolution — the version that catches the "resolved-at-the-time-but-came-back-the-next-day" pattern that in-call surveys miss.
Template 5 — Agent experience
- "How manageable was your workload in the past quarter?" (1–5)
- "How well do the tools you use support you in helping customers?" (1–5)
- "How confident do you feel about your job security over the next 12 months?" (1–5)
- "Do you feel the AI tools deployed on your team have made your job better, worse, or unchanged?" (Better / Unchanged / Worse — please describe)
- Open text: "What one change would most improve your daily work?"
The third question is the one most companies avoid asking. Ask it. If you do not ask it, your agents will still have the anxiety — they will simply not tell you about it until they resign.
B.5 The Executive Readiness Conversation
Not a checklist. A structured agenda for a two-hour conversation between the CX leader, the CIO, the CFO, and the CEO before any large AI-in-CX investment is committed. Use it once, or use it quarterly.
| Segment | Duration | Content |
|---|---|---|
| Opening | 15 min | CX leader presents current-state baseline: cost per contact, CSAT, FCR, turnover. No forward-looking slides. |
| Problem statement | 20 min | Which baseline metrics is leadership most dissatisfied with? What outcome in dollar terms would count as success in two years? Frequently exposes divergent implicit theories — align on primary goal. |
| Capability gap | 25 min | Given the target outcome, what capabilities need to be built or acquired? Skills, roles, systems, data. Reference B.1 explicitly. |
| Vendor & platform | 20 min | Which vendor category best matches the strategy? What existing commitments constrain the choice? Appetite for multi-vendor complexity vs. single-vendor simplicity? |
| People plan | 20 min | What happens to the workforce? Commitment to reskilling? Honest headcount projection two years out? Who owns the transition? |
| Budget & timeline | 15 min | Multi-year investment envelope. Go/no-go criterion at each phase. Who has authority to slow the program if metrics regress? |
| Commitment | 5 min | Each executive states what they will personally own in the next 90 days. Written down. Circulated within 24 hours. |
The 90-day commitments matter more than the two-hour discussion. Programs that leave the room with a plan but without individually named executive owners for the first three months' work almost invariably drift.
B.6 The Weekly Operating Rhythm
Once the AI is running in production, four operational cadences matter. Not one of them is optional.
| Cadence | Duration | Attendees | Content |
|---|---|---|---|
| Daily | 30 min | AI operations team | Previous day's containment rate, escalation categories, customer-flagged issues, Guardian Agent safety-flag queue. Anything anomalous gets ticketed. |
| Weekly | 60 min | Head of CX, head of AI product, head of contact center ops, rotating Legal/Compliance rep | Week-over-week trends. Intents that improved or regressed. Model versions shipped this week and measured impact. Vendor incidents. |
| Monthly | 2 hr | Extended leadership including finance | Full CX and AI metrics vs. target. Cost per contact, CSAT, FCR, containment, agent satisfaction, and the specific ROI metric from B.5. |
| Quarterly | 2 hr | Executive team | Abbreviated re-run of B.5: what has changed in baseline, capability gap, and industry landscape. Recommitment on next 90 days. |
How to use this appendix
Pick the diagnostic that matches the phase you are in. Pre-decision: B.5. Pre-launch: B.1 and B.2. Regulatory: B.3. Post-launch: B.4 and B.6. None of these are one-shot — the readiness diagnostic in particular is worth re-running every six months as the operation matures and the gaps shift.