The industry's measurement framework for AI in customer experience has become a liability. Deflection rates and average handle time—metrics designed for traditional agent productivity—are masking whether AI deployments actually drive business value. Brad Birnbaum's argument at Kustomer cuts to a fundamental misalignment: CX teams are optimizing for activity metrics when they should be optimizing for outcomes that matter to revenue, retention, or growth. A retailer protecting margin through exchange-over-refund decisions operates under entirely different success criteria than a subscription business fighting churn or a B2B SaaS company hunting upsells. The shift from "AI handled this interaction" to "AI moved this metric that moves the business" requires teams to reverse-engineer their measurement strategy—defining the North Star outcome first, then building AI governance and observability around it. This reframing has immediate implications for teams already deep into Zendesk, Salesforce, or Freshdesk deployments: your current dashboards may be reporting success while your AI is optimizing for the wrong thing.
The practical challenge lies in execution velocity and governance. Birnbaum advocates for speed—pick your most important outcome, get something live quickly, build from there—but this sits in tension with the governance question that CommBank's AU$140MN investment in human support and the Apple customer service failures have exposed. AI observability and continuous iteration are non-negotiable, yet they require infrastructure most CX platforms are still building. The human-in-the-loop model Kustomer describes—where agents shift from creators to reviewers—assumes your team has the capacity and training to review at scale. For support leaders managing lean teams, this raises a harder question: does outcome-driven AI measurement actually require you to invest more in human oversight, not less? The vendors winning this transition will be those who make observability and outcome tracking native to their platform, not bolt-on features.
The question of how to measure AI in customer experience is becoming harder to sidestep. As more CX teams build out their AI deployments, the pressure to show real ROI is sharpening, and a growing number of leaders are realizing that the metrics they’ve relied on simply aren’t telling th