The Compute Delusion

For forty years software engineering research has been telling us the same thing: the bottleneck isn't typing code. It's understanding reality. AI is finally proving it.

Every technological bubble develops a strange relationship with reality.

The railway bubble believed every town needed a railway.

The dot-com bubble believed every business needed a website.

The crypto bubble believed every database needed a token.

The AI bubble believes every human decision is simply a larger version of autocomplete.

That last assumption matters more than most people realize.

Because underneath the trillion-dollar investments, giant data centers, benchmark wars, and increasingly confident predictions that software engineers will disappear by Christmas sits a simple belief:

If we keep adding enough compute, intelligence will eventually emerge in a form capable of replacing human judgment.

The problem is that software engineering may be the first major industry demonstrating the opposite.

Not because models aren't improving.

Because the bottleneck was never where people thought it was.

And strangely enough, software engineering researchers have been warning us about this for decades.

In 1986, Fred Brooks published No Silver Bullet, one of the most influential essays ever written about software development. Brooks argued that most of the difficulty in software comes not from writing code but from understanding the problem itself. He distinguished between accidental complexity and essential complexity.

Accidental complexity is syntax.

Boilerplate.

Framework configuration.

Tooling.

The mechanical work.

Essential complexity is everything else.

Understanding reality.

Understanding customers.

Understanding constraints.

Understanding contradictions.

Nearly forty years later, AI has become the greatest accidental-complexity reduction tool ever invented.

The problem is that software engineering was never mostly accidental complexity.

The Strange Problem With "Better"

One of the more interesting observations emerging from experienced engineers is not that models are getting worse.

It's that they're becoming harder to distinguish.

This sounds absurd.

Benchmarks continue improving.

Context windows continue expanding.

Reasoning scores continue climbing.

Yet many experienced engineers report something strange.

The latest generation feels incrementally better but not fundamentally different.

The improvement exists.

The value doesn't.

Those are not the same thing.

This distinction is largely absent from AI discussions because the industry has become obsessed with measuring intelligence while ignoring workflow.

A Formula 1 car is objectively faster than a Toyota Corolla.

That doesn't mean your commute becomes ten times shorter.

The bottleneck was traffic.

Not horsepower.

Software engineering increasingly resembles traffic.

The Hidden Assumption Behind Full Automation

The popular AI narrative quietly assumes that programming is primarily code generation.

This is why so much discussion revolves around models writing more code.

More files.

More repositories.

More pull requests.

More output.

But experienced engineers know something uncomfortable.

Writing code is usually not the difficult part.

Understanding what should be written is.

The actual work is ambiguity reduction.

Understanding requirements.

Resolving contradictions.

Discovering hidden constraints.

Identifying edge cases.

Negotiating tradeoffs.

Managing risk.

Maintaining mental models.

Coordinating humans.

The code is often the receipt.

Not the meal.

Yet most AI discourse treats software development as though the receipt were the business.

Information Has a Compression Limit

There is a deeper issue hiding beneath the hype.

Information theory.

Large software systems contain enormous amounts of information.

Not documentation.

Not code.

Information.

Every business rule.

Every exception.

Every integration.

Every dependency.

Every operational assumption.

Every political compromise.

Every historical accident.

Every bug nobody dares touch.

This information cannot be infinitely compressed.

At some point, a prompt stops being a specification and becomes a wish.

The fantasy behind one-shot software generation is that a few paragraphs can somehow contain enough information to reconstruct thousands of decisions accurately.

But information lost during compression does not magically reappear.

The model fills the gaps.

And the gaps are precisely where billion-dollar mistakes live.

The more complex the system becomes, the more the problem shifts from generation to interpretation.

The issue is no longer whether the model can generate an answer.

The issue is whether anyone can trust the answer.

The Hallucination Problem Is Actually a Trust Problem

The industry talks constantly about hallucinations.

This is the wrong frame.

The real problem is verification.

Recent research on AI coding assistants has uncovered something fascinating.

Developers spend less time writing code.

But they spend more time reviewing, directing, validating, correcting, and supervising AI output.

The researchers describe the emergence of a new form of work:

Supervisory Engineering.

In other words, AI removed typing.

It did not remove responsibility.

Consider two scenarios.

In the first, an AI generates a perfect 50,000-line system.

In the second, an AI generates a flawed 50,000-line system.

To a human reviewer, these situations are nearly identical.

Both require inspection.

Both require understanding.

Both require validation.

Both require accountability.

The verification burden remains.

Which creates an uncomfortable asymmetry.

Generating software scales.

Reviewing software does not.

As models produce increasingly large outputs, the cost of validation grows with them.

Eventually the review process becomes more expensive than decomposition.

The engineer stops asking for bigger solutions.

Not because the AI lacks capability.

Because the human lacks trust.

This is the hidden economic force behind diminishing returns.

Not model quality.

Verification cost.

Verification Debt

For twenty years we've talked about technical debt.

Technical debt is what happens when code is written faster than it can be maintained.

AI is creating a new phenomenon.

Verification debt.

Verification debt occurs when code is generated faster than humans can confidently understand it.

The result is an illusion of progress.

Pull requests multiply.

Repositories grow.

Velocity metrics improve.

Meanwhile nobody actually knows whether the system is becoming better or merely larger.

Organizations begin accumulating uncertainty faster than they accumulate knowledge.

This looks like productivity until something breaks.

Then everyone discovers the missing work item.

Understanding.

The Airline Pilot Effect

A useful analogy is aviation.

Modern aircraft can fly themselves remarkably well.

They can maintain altitude.

Navigate routes.

Adjust power.

Handle many routine operations.

Yet airlines still employ pilots.

Not because automation failed.

Because rare events dominate risk.

The moments that matter are precisely the moments automation struggles with.

Software engineering works similarly.

The easy work is increasingly automated.

The difficult work remains stubbornly human.

Not because humans are smarter.

Because humans own responsibility.

Responsibility is the part nobody seems to include in benchmark charts.

Why Compute Is Producing Smaller Returns

This is where diminishing returns begin to appear.

Early AI gains were spectacular because the models learned enormous amounts of publicly available knowledge.

Documentation.

Libraries.

Patterns.

Frameworks.

Tutorials.

Stack Overflow.

The low-hanging fruit was everywhere.

Adding more compute unlocked massive capabilities.

But eventually you encounter a different problem.

Not knowledge.

Context.

And context is expensive.

Not computationally.

Organizationally.

The AI industry increasingly resembles a mining operation that has extracted the richest ore and is now processing mountains of rock to recover tiny additional amounts of value.

The ore is still there.

The economics are changing.

The frontier is no longer intelligence.

The frontier is organizational complexity.

The Status Game Nobody Wants To Admit

The replacement narrative persists partly because it serves important psychological functions.

It signals sophistication.

It signals futurism.

It signals membership in the winning side of history.

Predicting augmentation sounds boring.

Predicting replacement sounds revolutionary.

One attracts venture funding.

The other sounds like process improvement.

Unfortunately reality has a habit of preferring boring explanations.

The history of technology is largely the story of tools making humans more productive rather than eliminating them.

Spreadsheets didn't eliminate finance.

CAD didn't eliminate architects.

Excel didn't eliminate accountants.

The internet didn't eliminate journalists.

Each transformed the profession.

Each changed who survived.

Each altered economics.

None eliminated the need for judgment.

The Local Model Future

Perhaps the most telling signal is emerging outside the frontier labs.

Local models.

Every year, smaller models achieve capabilities that previously required massive infrastructure.

This creates an awkward question.

If next year's laptop can perform ninety percent of today's premium coding tasks, what exactly are trillion-dollar compute investments buying?

The answer may be uncomfortable.

Marginal improvements.

Not transformative ones.

The cloud giants may increasingly find themselves selling increasingly expensive solutions to increasingly narrow problems.

The future may look less like centralized superintelligence and more like ubiquitous competent intelligence.

Less nuclear power plant.

More electrical outlet.

The Real Bottleneck

The industry's biggest misconception is that software engineering is constrained by code production.

It isn't.

Decades of software delivery research have repeatedly reached the same conclusion.

High-performing organizations do not win because individual developers type faster.

They win because they coordinate better.

Requirements flow faster.

Feedback loops close faster.

Decisions happen faster.

Trust moves faster.

Most organizations are constrained by:

• Poor requirements
• Conflicting incentives
• Slow approvals
• Risk avoidance
• Coordination failures
• Political behavior
• Documentation debt
• Unclear ownership

AI generates code faster than organizations generate clarity.

And clarity remains stubbornly human.

This creates a paradox.

The more capable AI becomes, the more obvious the non-technical bottlenecks become.

Organizations expecting AI to eliminate engineers may discover that engineers were never the slowest part of the system.

They were simply standing closest to the bottleneck.

The Realization

The most important thing happening in AI may not be the rapid improvement everyone talks about.

It may be the slowing improvement nobody wants to discuss.

Not because progress has stopped.

Because we've reached the point where intelligence is no longer the scarce resource.

Trust is.

Understanding is.

Verification is.

Responsibility is.

The AI industry keeps trying to solve organizational problems with more compute.

This is like trying to solve traffic congestion by building faster cars.

The cars are no longer the bottleneck.

The roads are.

The evidence increasingly suggests that AI is not hitting diminishing returns in intelligence.

Benchmarks continue improving.

Models continue improving.

Hardware continues improving.

What is hitting diminishing returns is the ability of human organizations to absorb those improvements.

The frontier is no longer model capability.

The frontier is organizational capability.

A calculator can outperform every mathematician who has ever lived at arithmetic.

Yet mathematics still exists.

Not because arithmetic mattered.

Because arithmetic was never the point.

Software engineering may be discovering the same thing.

Code generation was the arithmetic.

The industry mistook it for intelligence.

And now that machines are becoming excellent at it, we're discovering where the real work was all along.