The AI Testing Gap: 57% in Production, 15% With a Purpose-Built Approach
A joint study by QualiZeal and HFS Research, based on a survey of 101 enterprise quality and technology leaders in the United States, has put numbers to a gap most practitioners recognise but few can size. Fifty-seven per cent of enterprises run generative AI in some form of production and 36 per cent have agentic systems live, while only 15 per cent have a purpose-built approach to AI testing. Fifty-two per cent test probabilistic AI systems using frameworks designed for deterministic software, and 37 per cent of enterprises running AI in production have no named function accountable for its quality. QualiZeal sells AI assurance software and services, which is worth holding alongside every finding in this article.
Why the 52 per cent matters most
Of all the figures in the study, the one about testing frameworks is the most consequential and the least discussed, because it describes a category error rather than a budget shortfall.
A deterministic test asserts an expected output. Given this input, the function returns this value; if it returns something else, the test fails. That model has served software engineering for decades because deterministic software genuinely behaves that way.
A generative model does not. The same prompt can produce different outputs on consecutive calls, and both may be correct. A test that asserts equality against a stored expected response will fail constantly and teach a team nothing, so teams either loosen the assertion until it passes everything or stop running it. Neither outcome tells anyone whether the system works.
Testing a probabilistic system requires a different discipline: asserting properties rather than values, evaluating outputs against criteria, sampling across a distribution and measuring rates rather than outcomes. That is a genuinely different body of practice, and the study’s finding is that roughly half the market has not made the transition. The consequence is not that these enterprises test badly. It is that many of them cannot say what their AI testing establishes, which is a different and more uncomfortable position.
What trust engineering would consist of
If trust engineering is the proposed answer, it is worth saying what the practice would contain. Four capabilities appear in the study’s own findings by their absence.
The first is an evaluation harness: a repeatable way to run a model against a defined set of inputs and score the outputs against criteria rather than against stored answers. The second is a measurement programme for the properties that matter beyond accuracy, which means defining fairness for the specific application and documenting that definition. The third is a record that survives scrutiny, because a regulator asking how a system was validated wants documentation rather than a description. The fourth is a named owner, which costs nothing to appoint and appears to be missing in 37 per cent of cases.
Only the first of those requires new tooling. The other three require decisions, and the study suggests the decisions are the harder part. That is why AI testing is not primarily a procurement problem: an enterprise can buy an evaluation platform and still have nobody accountable for what it reports, which is the situation 37 per cent of respondents describe.
The accountability gap
The second finding is organisational rather than technical, and it may be the harder one to fix. Thirty-seven per cent of enterprises running AI in production have no named function accountable for AI quality.
Accountability is what converts a practice into a control. An organisation with no named owner for AI quality has no one who signs off before a model reaches production, no one to ask when a system behaves unexpectedly, and no one whose job depends on the answer. That is a defensible position while AI is experimental and an indefensible one once it is processing customer requests.
Kalyan Konda, co-founder and chief executive of QualiZeal, frames it as a boardroom question rather than a testing one, and the framing holds. A risk that belongs to nobody in particular belongs to the board by default, and most boards do not yet have a mechanism for it.
Accuracy is proven; trust is not
The study’s most quotable line describes a 53-point gap. Eighty-eight per cent of enterprises measure model accuracy. Forty-seven per cent measure bias and fairness, 37 per cent measure explainability, and 35 per cent can provide regulatory compliance evidence.
The pattern is consistent and it makes sense. Accuracy is the easiest of the four to measure, because it requires a labelled dataset and a metric, both of which a data science team already has. Bias and fairness require defining what fairness means for a specific application, which is a policy question before it is a technical one. Explainability requires an account of why a model produced an output, which for large models remains an active research area rather than a solved tool. Regulatory evidence requires all three plus documentation that survives an audit.
Hansa Iyengar of HFS Research puts the consequence directly: accuracy gets AI into production, trust is what lets it scale. That is the right order of operations, and it explains why the gap persists. Accuracy is sufficient to launch and insufficient to expand.
The mandate without the budget
The fourth finding describes a mismatch between expectation and investment. Eighty-two per cent of enterprises expect the quality engineering role to grow significantly, and 24 per cent report proportional budgets for it. That is a 58-point gap, the widest in the study.
The reframing QualiZeal and HFS propose is that quality engineering becomes trust engineering: the discipline moves from assuring software to assuring intelligence. Madhu Murty of QualiZeal argues that AI and automation elevate rather than replace quality engineers, which is the more plausible reading of a function whose remit is expanding faster than its headcount.
Whether the title change sticks is a separate question from whether the work changes. The work clearly changes: testing a system that reasons probabilistically, auditing its outputs for fairness and documenting the result for a regulator are not the same tasks as regression-testing an application, and the people who can do them are scarce.
What the sample can and cannot support
The study surveyed 101 enterprise quality and technology leaders in the United States, and that number deserves stating plainly because it bounds what the findings establish.
One hundred and one respondents is a reasonable sample for directional findings and too small for fine distinctions. The study does not report a margin of error, a breakdown by company size or sector, or the sampling method, so a reader cannot tell whether the 37 per cent without accountable ownership skews toward smaller organisations or clusters in a particular industry. The figures describe a US enterprise population and should not be read as global.
None of that undermines the central finding, which is about a structural mismatch that practitioners will recognise. It does mean the numbers are best read as an indication of scale rather than as precise measurements, and the direction of travel matters more than the decimal.
Why the timing matters
Regulatory pressure is the force that will close the gap, and it is arriving unevenly. The 35 per cent figure for regulatory compliance evidence is the lowest in the study, and it is the one most likely to move quickly, because the obligation to demonstrate rather than assert is becoming explicit across jurisdictions.
An enterprise that cannot produce evidence today has a window in which to build the capability before an examiner asks for it, and the size of that window varies by sector and jurisdiction. That window is what makes the study timely rather than merely interesting, and it is also why the commercial interest behind it deserves stating rather than dismissing: the category is new, the obligation is arriving, and the firms selling into it have an interest in the gap being understood.
What would strengthen the next edition
Three additions would make a follow-up study considerably more useful.
A sector and size breakdown would let a reader act on it. The AI assurance problem looks different in a bank than in a software company, and different again in an organisation with fifty engineers rather than five thousand.
A time series would be the second. These are single-point figures, and the question a reader actually has is whether the gap is widening or closing. Repeating the survey annually would answer it, and the trend would be more valuable than any individual number.
And a definition of purpose-built AI testing would help. Fifteen per cent is a striking figure and a reader cannot assess it without knowing what the study counted. If the bar is a dedicated evaluation harness with bias testing, the number is unsurprising. If it is any structured evaluation process, the number is alarming.

What to watch
Whether a second edition establishes a trend, because a single survey describes a moment and a series describes a market. Whether named accountability becomes standard practice, since that is the change an enterprise can make without buying anything. And, whether bias and explainability measurement closes on accuracy as tooling matures, which would be the clearest sign that trust engineering has become engineering rather than aspiration. And whether the vendors selling AI assurance can demonstrate their own products against the criteria their research sets, because that is the test this category will eventually face.

