Half of enterprises have deployed an AI agent or LLM feature that passed their internal evaluations and then caused a customer-facing failure. A quarter have had it happen more than once.
That figure comes from VentureBeat’s June 2026 VB Pulse survey of 157 qualified respondents at organizations with 100 or more employees. A July wave of the same instrument, with 108 respondents, put it at 49% and 24% – statistically indistinguishable. The sample is self-selected rather than a probability sample and skews toward the mid-market, so read it as directional. But two independent waves landing on the same number is harder to wave away than a single survey.
The uncomfortable part is what it says about the release gate. If roughly half of surveyed organizations have watched their evaluation suite approve of something that later failed in front of customers, then a passing score is not evidence of production reliability. It is evidence that the agent handled the cases you thought to test.
Most writing about AI agent implementation right now is about pilots that never ship. This is the opposite problem, and in some ways the worse one: the pilot shipped.
Answer first: evaluations test a distribution you constructed. Production is a distribution you do not control, and the gap between them is where failures live.
When VentureBeat asked what limits evaluation today, the single most-cited answer was that evaluations align poorly with real-world outcomes – named by 29% of respondents. Not coverage. Not tooling. Alignment with reality.
Four specific gaps account for most of what gets through.
Evaluation sets are written by people who understand the system. Real users do not. They paste half a document, ask two questions in one sentence, refer to something from three messages ago, use internal shorthand nobody wrote down. An agent that answers your test questions at 94% may be facing a materially different input distribution on day one.
In evaluation, the connected systems are usually a snapshot, a sandbox, or a mock. In production they are live and inconsistent: a record was updated mid-conversation, an API returned a 200 with an empty payload, a document exists in two versions and the agent found the stale one. The model behaved correctly. The answer was still wrong.
Most evaluations run as a single privileged identity. Production has roles. The question “what is our policy on X” has a different correct answer depending on who is asking, and an agent that ignores that distinction will pass every accuracy test and still disclose something it should not have. Worth noting alongside this: the same VentureBeat research found 69% of companies let at least some of their agents share credentials, with multiple agents running under one API key or service account.
Single-turn evaluation of a multi-step agent is the most common structural mistake in AI agent implementation. A 95% per-step success rate across six steps is roughly 74% end to end. The evaluation reports 95%. The user experiences 74%.
Here is the finding that should concern the CTO more than the failure rate itself.
Two-thirds of enterprises (66%) either already allow an agent to push a change to production on automated evaluation results alone, with no human review, or are actively engineering toward that within twelve months. Only 5% say they fully trust the automated evaluations that would make that call.
Read those two numbers together. Organizations are removing the human check while openly saying they do not trust the machine check replacing it. VentureBeat’s own cross-tab found that enterprises which had been burned by an evaluation escape were among the most aggressive in moving toward autonomy – not the least.
There is a benign reading available: the companies running the most agents encounter the most incidents and also have the most deployment infrastructure. The survey cannot settle which explanation dominates. What it does settle is that a customer-visible failure is not, in practice, stopping the march toward autonomy.
The recommendation VentureBeat draws from its own data is direct: before removing human review from any workflow, test your evaluations against production outcomes rather than internal benchmarks. That means keeping a held-out set drawn from real traffic, re-scoring it on the same suite, and measuring how often the suite and reality disagree. If you have never calculated that disagreement rate, you do not know what your release gate is worth.
Accuracy scoring after the fact tells you how often the agent was wrong last month. Source-grounding tells you whether this answer is supported right now. An agent that verifies its output against the retrieved document before returning it converts a class of silent failure into a visible refusal – which is a far better outcome in front of a customer than a confident wrong answer.
Run the same evaluation set as three different roles. If the answers are identical, your agent is not enforcing permissions, regardless of what the access control diagram says.
When an agent fails in front of a customer, the first question is always “why did it do that?” If your logs contain the final response but not the retrieval, the tool calls, and the intermediate reasoning, you cannot answer it – and you cannot fix it. Decision-level logging is what turns an incident into a test case.
Every customer-facing failure should end as a new evaluation case. Organizations that do this narrow the gap between eval and reality over time. Organizations that do not keep discovering the same class of failure with different nouns.
We built Nabla Agent around the assumption that the release gate will sometimes be wrong. Answers are checked against source documents before they are returned, permissions are enforced per role rather than per deployment, and every decision the agent makes is logged and auditable.
That does not eliminate the evaluation gap. Nothing does. It changes what happens when the gap opens: the failure is visible, attributable, and reproducible, rather than something a customer discovers on your behalf.
It is the distance between the autonomy an organization grants its agents and the trust it places in the evaluations meant to govern that autonomy. VentureBeat’s June 2026 research found 66% of enterprises permit or are building toward production deployment without human review, while only 5% fully trust the automated evaluations informing those decisions.
In VentureBeat’s June 2026 survey of 157 enterprise respondents, 50% had deployed an agent or LLM feature in the prior year that passed internal evaluations and then caused a customer-facing failure; 25% had seen it more than once. A July wave found 49% and 24%.
No. The most-cited limitation was that evaluations align poorly with real-world outcomes, not that models are inaccurate. The failures cluster around inputs, system state, permissions and multi-step compounding – not raw model capability.
For any workflow where a wrong answer reaches a customer, yes, until you can show your evaluation suite agrees with production outcomes on held-out real traffic. The decision should rest on that measured agreement, not on a benchmark score.
Measure disagreement, not accuracy. Take real production interactions, have humans score them, re-score them with your suite, and track how often the two disagree and in which direction.
Input your search keywords and press Enter.
