AI pilots stall when nobody owns the result, so start with one department

Two weeks ago, economists at the National Bureau of Economic Research published a survey of almost 6,000 executives in the US, the UK, Germany and Australia. Nearly 90 percent of their firms said AI had made no difference to employment or productivity over the past three years.

That is a striking number, given how much money has gone in. I don’t read it as proof that AI doesn’t work. I read it as proof that most companies are running AI the wrong way: as a company-wide experiment that belongs to nobody.

AI is everywhere in these companies and changing very little

According to The Register’s write-up of the study, 69 percent of the businesses surveyed already use some form of AI. Yet 89 percent saw no change in productivity, measured as sales per employee.

The executives themselves are light users. Fortune reported that about two-thirds of them use AI, for an average of 1.5 hours a week, and a quarter don’t use it at work at all. Looking forward, they expect AI to lift productivity by 1.4 percent over the next three years. That is a modest forecast from the people signing the checks.

Deloitte’s survey of more than 3,000 executives, released in January, points the same way. Only 25 percent of respondents had moved 40 percent or more of their AI pilots into production. Most pilots are still pilots.

By the numbers

89%

of surveyed firms saw no change in productivity from AI over three years

Source: theregister.com

So the tools are in the building. People have logins. The work itself hasn’t changed.

Why company-wide pilots stall

I have built companies for twenty years, and I have watched every new technology arrive the same way. Leadership buys licenses for everyone. A committee forms. Someone runs a lunch-and-learn. Six months later, a few people use the tool to clean up emails, and nobody can say what it did for the business.

The AI version of this pattern has three specific problems.

Nobody owns the outcome. When the pilot belongs to “the company,” it belongs to no manager in particular. The head of finance didn’t ask for it. The head of support has a queue to clear today. IT owns the licenses but not the work. A pilot without an owner never gets the hard decisions it needs, such as which steps to stop doing by hand.

There is no before number. A company-wide rollout rarely starts by measuring how long anything takes. So when someone asks whether it worked, the only evidence is a survey asking people if they feel more productive. That is how you end up with executives reporting no measurable change.

The work is never redesigned. Giving someone a chatbot and leaving their job exactly as it was produces a slightly faster version of the old job. The gains show up when a specific task moves from a person to an agent and the person’s day is rebuilt around reviewing that output. That redesign only happens inside a specific team, with a specific manager, on a specific process.

When executives themselves use the tools for an hour and a half a week, it tells me AI is still being treated as an assistant for individuals. It hasn’t become a way of running a department.

One department gives you an owner, a baseline and a finish line

The alternative is boring and it works. Pick one department. Pick one workflow inside it. Make one manager responsible for the result, and give them a date.

Say you run a 60-person distribution company. Instead of rolling out an AI assistant to all 60 people, you go to accounts payable. Three people there spend their days matching invoices to purchase orders, chasing missing paperwork and keying entries into the accounting system. You time that work for a week before anything changes. Then you put an agent on the matching and the chasing, with a person approving every entry before it posts. Thirty days later, you time it again.

Before and after

Company-wide pilot

Every employee gets a license
A committee owns it
Nobody timed the work first
Success is a survey

One-department start

One team changes one workflow
One manager owns the result
The task is timed before launch
Success is hours and errors

The second approach is smaller, and it is the one that produces a number you can defend.

That one department gives you things a company-wide pilot never will.

A manager who wants it to work, because the result shows up in their own team’s week.
A before number and an after number, measured on the same task.
A clear line between what the agent does and what a person checks.
A story you can tell the next department, with their peer’s numbers in it.

The second department moves faster than the first, because it has seen a colleague’s results instead of a vendor’s demo. The third moves faster again. That is how adoption actually spreads inside a company. Mandates from the top don’t do it.

Two objections, and why I don’t buy them

The first is that starting with one team is too slow and the company will fall behind. I think the opposite is true. The NBER numbers describe companies that went wide and got nothing measurable. A single department that saves real hours in a month is ahead of a company that has had licenses for a year and can’t say what they did.

The second is that the first department might pick the wrong workflow. It might. That is fine. A wrong pick in one team costs you a few weeks and teaches you how your own processes work. A wrong pick across the whole company costs you a year and teaches you nothing, because there was never a baseline to compare against.

There is also a cost argument that rarely gets made. A license for every employee is a fixed monthly bill whether anyone changes how they work or not. A single department running one agent on one workflow costs a fraction of that, and you only expand the spend after you have seen the hours come back in that first team.

When I help a company decide where AI should do its work, the first question I ask is who owns the number. If nobody can answer, the project isn’t ready, however good the tool is.

What to do this week

List the three departments with the most repetitive, rules-based work. Accounts payable, customer support and sales operations are common answers.
Pick one. Choose the one whose manager is most willing to own the result, even over the one with the biggest theoretical savings.
With that manager, choose one workflow and write down every step in it.
Time that workflow for five working days before you touch any tool. Record how many items came in and how long each took.
Set a 30-day finish line and agree, in writing, what number would count as success.

Do not buy anything new until step four is done. You will learn more from one week of timing the work than from another round of demos.