Enterprise AI organizations are shipping agents to production with evaluation frameworks they don't actually trust. Across 157 enterprises surveyed, half have already deployed an agent that passed internal evaluations only to fail when customers encountered it in the wild—yet only one in twenty fully trusts their automated evaluation systems. This represents a fundamental misalignment between what organizations measure in controlled environments and what actually performs under real-world conditions. The problem isn't that evaluation coverage is insufficient; it's that the evaluations themselves lack predictive validity for production behaviour. For CX teams already running Agentforce or similar platforms, this raises an uncomfortable question: if half of enterprises are discovering evaluation failures post-deployment, what confidence can you place in your own pre-launch testing, particularly when those tests are built on the same automated frameworks the industry collectively distrusts?
This evaluation gap sits within a broader pattern of enterprise AI deployment outpacing operational maturity. Organizations are simultaneously granting agents greater autonomy—moving beyond simple retrieval and into multi-step decision-making—whilst simultaneously reducing their confidence in the gatekeeping mechanisms meant to prevent failures. The infrastructure problem compounds the evaluation problem: as noted in parallel research, context retrieval is being built faster than it can be trusted, yet teams are shipping anyway. For support leaders and CX consultants, the implication is stark: the vendors and platforms you're evaluating are likely operating under the same misalignment. Your internal evaluations may be systematically optimistic about production performance, which means either your testing methodology needs fundamental redesign, or you need to accept that early-stage agent deployments will generate customer-facing failures as a cost of learning what your evaluations actually measure.
The strategic question for CX organizations is whether to treat this as a temporary calibration problem or a structural one. If evaluation frameworks are fundamentally misaligned with production reality across the industry, then incremental improvements to testing won't solve it—you need different evaluation approaches entirely, ones that explicitly test for failure modes your current systems can't predict. This becomes especially critical as agent autonomy expands beyond simple customer queries into actions that affect customer records, billing, or service commitments. The teams shipping agents today are effectively running live experiments on customer satisfaction, which works only if you're prepared to measure and respond to failures faster than your evaluation frameworks can predict them.
Across 101 enterprises, the infrastructure that feeds AI agents their business context is being built faster than it can be trusted. Retrieval-augmented generation is already the default context source, and provider-native retrieval has quietly overtaken the dedicated vector databases that define th
Across 157 enterprises, organizations are granting AI agents more autonomy while trusting the evaluations meant to gate that autonomy less. Half have already shipped an agent that passed their internal evaluations and then failed a customer in production; only one in twenty fully trusts automated ev
Across 101 enterprises, agent orchestration is consolidating onto model-provider platforms — Anthropic’s Claude leads by a wide margin — chosen for the gravity of the underlying model and judged on reliable multi-step execution. But the ambition runs well ahead of the reality: most deployed “agents”