AI adoption is everywhere, but AI impact is proving much harder to find.
Nearly nine in ten organisations now use AI regularly in at least one business function, according to McKinsey’s 2026 research. Around 80% say it has improved individual productivity, yet only 37% report a positive impact on operating profits. Getting people to use AI and getting a business to benefit from it are two very different things.
Plenty of organisations can point to thousands of licences, dozens of pilots, and impressive-looking dashboards. Employees are prompting copilots, teams are experimenting with agents and leaders can tell the board that the organisation is embracing AI.
But have customers had a better experience? Have costs come down? Are engineers releasing software faster? Has risk fallen or revenue increased? If those outcomes aren’t changing, high adoption levels don’t necessarily mean meaningful value.
When experimentation becomes theatre
Organisations need space to test new technology and discover what works. The problem starts when experimentation continues without a clear route to value.
Research published by the UK Government’s Department for Science, Innovation and Technology in 2026 found that just over half (54%) of organisations already using AI felt ready to scale it, and only 13% described themselves as completely ready. The findings underline how different it is to experiment with AI and to embed it confidently across a business.
We’ve also seen what can happen when AI moves from experiment to real-world use without sufficient operational safeguards. Air Canada was held liable by the British Columbia Civil Resolution Tribunal after its chatbot gave a customer incorrect information about its bereavement policy. More recently, enterprise software company Retool revealed that an experimental AI out-of-office responder unexpectedly began consuming around $10,000 a day in processing costs.
These are very different examples, but both illustrate the same point: putting technology into a process doesn’t automatically improve it. It needs to operate reliably, have appropriate controls, and produce an outcome worth the cost.
Start with the friction, not the shiny thing
One of the biggest mistakes I see is starting with the product. A new model launches, a vendor demonstrates an agent, or somebody sees a competitor talking about AI, and suddenly the organisation is searching for somewhere to use it.
A better approach is to start with the business problem and work backwards. In practice, that means following a few simple steps:
Step 1: Find the friction. Identify a specific problem that is costing the organisation time, money, quality, or customer satisfaction. Maybe customers are consistently getting stuck at a certain point, employees are losing hours to repetitive work, decisions are taking too long, or manual handoffs are creating errors.
Step 2: Establish the baseline. Work out how the process performs today. How long does it take? What does it cost? Where do errors occur? What is the customer or employee experience?
Step 3: Define the outcome. Decide what the end result needs to look like for the experiment to count as a success. A customer service team might want to reduce resolution times without damaging quality. An engineering organisation could need to increase release velocity while reducing defects. A security team might want to detect threats faster.
Step 4: Test AI against that outcome. Run the experiment and compare the result with the baseline, including the cost, risk, and operational complexity required to achieve it.
Step 5: Scale, redesign or stop. If the evidence shows a meaningful improvement that the business can sustain, look at how it can work across the wider workflow. If it doesn’t, change the approach or stop the experiment.
Measure the outcome, not the activity
There is no universal AI metric because there is no universal AI problem. For customer experience, we might look at resolution quality, conversion, or loyalty. In engineering, it could be release velocity or defect reduction. For security, detection speed and response time might matter more.
The challenge is choosing metrics that capture value rather than activity. Counting activations, prompts, or chatbot interactions tells you whether something is being used, but not whether that use is worthwhile. Imagine a customer spends 20 minutes with an AI chatbot. An engagement dashboard might make that look like success, while the customer might have spent those 20 minutes repeatedly explaining the same problem before giving up and calling a human.
If a process previously took four hours and now takes one, measure that. If software releases happen faster with fewer defects or customers get accurate answers sooner, measure that. AI needs the same commercial discipline as any other significant technology investment.
Turn individual productivity into organisational impact
Many organisations have started by using AI to help individuals perform existing tasks faster. There is real value in using it to summarise information, analyse documents or help with code, but making one person faster doesn’t automatically make the organisation faster.
Imagine AI helps an employee complete a task in ten minutes rather than an hour, but the output then sits in somebody else’s inbox for two days waiting for approval. Individual productivity has jumped, yet the customer experiences almost no difference. That’s why the next phase of AI needs to focus on workflows rather than simply adding more tools.
Follow the whole process: where work starts, how data enters the system, where decisions happen, which steps require a person and which can be automated. This is where data foundations, operating models, and governance become critical.
It’s also where pilots often hit the wall. Something that works beautifully with clean sample data and a handful of enthusiastic users can behave very differently when it meets fragmented systems, inconsistent data, security requirements and thousands of employees.
With AI, failures don’t always look like failures, at least not at first. A retrieval layer built over inconsistent data can return confident, well-formatted answers that happen to be wrong, and nobody notices until someone checks. A system that performs well on clean sample data can struggle with the messiness and edge cases it encounters in production. And models can change over time without anyone inside the organisation touching them, so a system that passed its tests in March may behave differently in June.
None of this is a reason to avoid AI. It’s a reason to treat it like production software from the beginning, with monitoring, versioning, rollbacks and a way to tell whether the answers are still good.
Moving from demo to production requires engineering, operational thinking and organisational change.
From AI possibility to AI performance
There is no shortage of excitement around AI, and much of it is justified. The challenge now is translating that excitement into performance.
That means moving beyond adoption as the measure of success. The organisations that get the most from AI will be those that stay focused on the problems worth solving, measure what changes and are prepared to stop when the value isn’t there.
AI doesn’t need to be everywhere. It needs to make a difference where it is used.
Divya Sampath
Divya Sampath, Chief Delivery Officer at Beyond, has previously held senior delivery and programme roles at Google, Stripe, Accenture and Vodafone, giving her first-hand experience of turning major technology investments into operational change.
