Skip to content
6 min read AI

Unmeasured Productivity: AI Investments are Flying Blind

Boards are committing huge spend to AI with very little understanding of how it will interact with the people using it

Unmeasured Productivity: AI Investments are Flying Blind
Photo by İsmail Enes Ayhan / Unsplash

Your People Are Already Using AI

Microsoft research shows 52% of knowledge workers hide their use or AI from their managers. Data is flowing to external systems without governance. Contracts are being drafted, code is being written and emails are being sent through tools your IT department may not even know exist.

Researchers call this the "secret cyborg" problem, and if you have not told your teams how to use AI, they are using it in a way that suits them.

In April 2023, Samsung learned what that looks like when three separate employees at the semiconductor division pasted confidential information into ChatGPT within a month. One uploaded source code, another shared code for identifying defective equipment, and a third fed a meeting transcript into the tool to generate minutes.

Samsung banned ChatGPT, spent two years building their own internal AI, then cautiously allowed it back with strict controls.

It Just Got Much More Serious

I had three calls last week from companies suddenly trying to buy GPU capacity. Large organisations have realised they cannot keep running sensitive work through US AI providers and need to build their own capability, which means they need data centres, network infrastructure and chip capacity they do not have.

Anthropic's release of Claude Cowork had just dropped share prices in a number of white-collar listed companies, especially in legal and data processing, because the tool quite clearly demonstrates how it can do the work of junior people faster and cheaper.

The penny has dropped.

But the queue to buy capacity, networks and compute is getting longer by the day.

The logic driving the infrastructure panic is the following:

If you look at the usage curve for ChatGPT alone, it launched on 30 November 2022, hit one million users in five days, and reached 100 million within two months, making it the fastest-growing consumer application in history.

The bubble - if there is one - still has some way to go before popping...

AI's Poor Economics

Strip away the hype and an AI system is a machine that converts enormous amounts of electricity into pattern recognition.

A single ChatGPT query uses roughly ten times the electricity of a Google search, and OpenAI handles over 2.5 billion queries per day. The hyperscalers are spending over fifty billion dollars annually on new data centres. Both Microsoft and Google are missing their climate targets specifically because of AI energy demands.

Do not forget, this is a market war over metals, manufacturing and fossil-fuel powered machines that get very hot.

China controls 60% of rare earth mining and 90% of processing. Around 90% of advanced chips are manufactured on a single island, Taiwan.

At some point there will be a supply crunch - and perhaps a political one too. Elon Musk believes given the pace of growth, data centre capacity will dry up this year, forcing companies to look at other options - such as data centres in space.

OpenAI's leaked internal documents show the company expects to lose $9 billion in 2025 and projects operating losses of $74 billion by 2028. It spends $1.69 for every dollar of revenue it generates. Anthropic is burning through capital at a slightly slower rate, projecting to break even by 2028.

One venture capital investor told The Economist this is "the WeWork story on steroids."

The dot-com bust still produced Amazon, Google and eBay. The companies building AI may or may not survive, but the capability itself is not going away. That is exactly why everyone is scrambling to own infrastructure rather than rent it from providers who might not exist in five years.

Your strategy needs to be resilient to either outcome.

When It Works, the Gains Are Real

In September 2023, Harvard Business School and Boston Consulting Group ran an experiment with 758 BCG consultants. Half got access to GPT-4, half did not. The consultants using AI completed 12.2% more tasks, worked 25.1% faster and produced results rated 40% higher quality.

But for a task designed to fall outside AI's capabilities, consultants using AI performed 19 percentage points worse than those without it.

They trusted outputs that looked plausible but were subtly wrong. It made things worse.

In May 2023, a New York attorney submitted a legal brief citing six precedents that ChatGPT had invented entirely, complete with fabricated quotes and fictional citations. The judge fined him $5,000 and required letters of apology to every judge falsely named. Dozens of similar sanctions have followed across the United States.

AI will lie to you with complete confidence and you will pay the price if you do not verify.

The Productivity Gap Nobody Is Measuring

The BCG experiment is the most cited proof that AI delivers. But it was 758 consultants performing defined tasks over a controlled period with researchers tracking every output.

That is not how your organisation works.

In practice, your people are using AI intermittently, on tasks that blend judgment with execution, inside workflows that were designed for humans working with other humans.

One person drafts a proposal with Claude, edits it with their own judgment, sends it to a colleague who rewrites half of it without AI, and the final version is reviewed by a senior leader who cannot tell which parts were machine-generated and which were not.

That is the reality of work in 2025, and almost nobody is measuring it.

We have decades of mature frameworks for measuring human productivity. But for the blended reality most companies are actually living in, where the same team is part human, part AI-assisted and part fully automated, often within the same person's working day, there is almost no measurement infrastructure at all.

Nobody has a reliable way to answer the most basic question: is AI actually making your organisation more productive, or is it making individuals faster while creating new friction, new errors and new coordination costs that cancel out the gains?

The honest answer, for most organisations, is that they do not know.

They are making capital allocation decisions worth tens of millions on the basis of results achieved under laboratory conditions by other people's employees.

This is the resilience question that matters more than any technology choice.

A company that has invested heavily in AI but cannot measure its hybrid workforce's actual output is flying blind in exactly the way that boards are supposed to prevent.

The competitive advantage will not go to whoever adopts AI fastest. It will go to whoever understands first how their people and their technology actually perform together, and builds the measurement capability to prove it.


Selling to the C-Suite

On Thursday 26th February at 10am, I'm running a free webinar on selling to the C-suite. It's the course I have given to a number of Fortune tech companies to help them shift the narrative to get better engagement.

Believe it or not, these companies face the same challenges as everyone else: confidence, clarity and charisma in dealing with CEOs. If your team is underperforming, this is a good use of their time - and it's free.

Most sales approaches are built for procurement and volume. They lead with product, stack up features and follow a process designed for people whose job is to evaluate vendors. But when you're selling to a CEO, CFO or board member, none of that lands.

You have to be the expert problem solver and the trusted advisor.

How do you become that?

Click the link and register.

Selling to the C-Suite: Listen. Create. Engage. · Luma
Most sales teams talk at senior buyers. They lead with product, pile on features and wonder why the CEO stopped replying after the first meeting. The ones who…