A simple experiment with an uncomfortable result
Last year, a small group of researchers ran an experiment that should make every engineering leader pause. They gave sixteen experienced open-source developers 246 real tasks drawn from their own codebases. For half the tasks, the developers could use AI coding tools. For the other half, they couldn't. The assignment was randomized, ensuring nobody could choose the easier work.
Before they began, the developers predicted AI would make them about 24% faster. That expectation aligns with much of the conversation around AI-assisted software development. It didn't. With AI tools, they actually took 19% longer to complete the same work.
More surprising was what happened next. Even after experiencing the slower performance firsthand, the developers still believed AI had made them about 20% faster.
The stopwatch told a different story
They were 19% slower, yet convinced they were 20% faster. That's roughly a 40-point gap between perception and reality. None of them could detect it from the inside. Objective measurement revealed what was actually happening. But why? The researchers don't claim to have a definitive answer, but several explanations are plausible. And they extend well beyond software engineering.
AI removes friction, not necessarily time. Watching an assistant generate a plausible answer instantly feels like progress, even if the next twenty minutes are spent checking, correcting, and re-prompting it. The ease of getting started can be mistaken for moving faster.
Expectations also shape perception. When everyone says a tool improves productivity by 20 or 30 percent, we're primed to believe we're experiencing those gains. We remember how smooth the work felt, not what the clock says. And AI often makes work more enjoyable. Instead of staring at a blank page, you're reacting to suggestions. That experience feels more productive, even when the overall task takes longer.
None of this makes the developers careless. It makes them human.
The bigger illusion inside every engineering organization
If experienced engineers can't reliably judge whether they're working faster, what does that mean for the rest of us?
Most organizations don't measure delivery performance directly. They measure how people describe it, through stand-ups, sprint reviews, and steering meetings. At every layer, perception replaces measurement. Scale one developer's blind spot across hundreds of engineers and multiple management layers, and leadership can end up managing a very different reality from the one the data would reveal.
That's not a people problem. It's an instrumentation problem.
Why intuition stops working at scale
Pilots don't fly by feel, and engineering organizations shouldn't either. Aircraft carry instruments because human perception becomes unreliable at high altitudes and speeds. The instruments replace instinct with evidence.
Software delivery now faces a similar challenge. AI-generated code, autonomous testing, and increasingly rapid release cycles have outpaced our ability to assess delivery health solely through observation. That's why continuous quality and delivery observability has become essential, not as another report produced on Friday afternoon, but as the instrument panel for the entire software delivery operation.
This is the thinking behind Insights360, part of ZenseAI.QI, Zensar's Quality Intelligence platform. Instead of relying on self-reported confidence, Insights360 continuously captures objective delivery signals, including defect trends and test coverage, as well as performance metrics and risk indicators. Leaders can make decisions based on what the software delivery system is doing, not what people believe it's doing. It's also why we've come to think of quality as intelligence rather than a checkpoint.
A checkpoint asks a single question: Did we pass? Quality intelligence asks a more useful one: What do we actually know right now, and how confident are we? That distinction matters most when everything appears to be going well, because that's exactly when perception is least likely to be challenged.
The next competitive advantage is proof
Every organization today is pursuing the same ambition: adopt AI faster, ship faster, and claim higher productivity. Far fewer can demonstrate that those claims stand up to scrutiny because far fewer are measuring continuously enough to know. That creates an opportunity.
Organizations that win won't necessarily be the ones making the boldest productivity claims. They'll be the ones who can prove them with evidence, not intuition; measurement, not perception. In the AI era, speed alone won't be enough for a competitive advantage. Being able to prove your speed will.
