Earlier this year, I wrote about the rapid shift from simple GenAI tools to Agentic AI in the professional services sector. The piece focused on structural survival – how firms must rethink billing, training, and competition as autonomous systems start to handle complex tasks.

There are a lot of moving parts across the AI landscape, particularly at the moment, so it’s worth stepping back to look at the macro picture; to see the proverbial wood, not just the trees. Do this, and a more volatile picture emerges than the one we might usually see.

Although I believe we are broadly heading in the right direction, there is a significant risk we won’t get AI adoption right the first time. AI is a double-edged sword – much as is nuclear power – but on balance it is a force for good. Where a problem lies is that development of the technology is moving at such a breathtaking pace that we, and regulators especially, are struggling to keep up.

The Conversation We Never Had

Some years ago, I advocated for a comprehensive national and international public conversation about the direction AI was taking and its long-term societal impact. That conversation still hasn’t happened to a sufficient extent. Different interest groups have engaged in their domain, but society as a whole has not.

It is crucial that the world at large – working people in particular – does not feel steamrolled by this technological transition. To succeed, we need rational, balanced dialogue, completely stripped of hyperbole and clickbait headlines. If we maintain that focus, there is a very good chance we will navigate this successfully. But time is of the essence, and there is a real danger of getting left behind if we do not bring wider society along.

The Physical Pushback and Missed Opportunities

We are already seeing the consequences of leaving people behind. Look at the escalating resistance to new data centre infrastructure, particularly in the US, and the recent wave of legislative moratoria. Public feeling has firmly shifted.

The big frontier labs and tech leaders are increasingly being targeted as the root of people’s anxieties. This isn’t just about the technology itself; it’s about how its rollout has been handled. The labs arguably missed a massive opportunity. Instead of engaging properly at a grassroots level to make the technology demonstrably worthwhile for everyday people, too many delivered doom-laden narratives of job displacement and economic upheaval. Yes, there is and will continue to be disruption – the sheer power demands and conspicuous data centre footprint are prime examples – but the labs have largely failed to communicate the balancing benefits effectively. This state of affairs could be the canary in the coalmine for wider social turmoil.

Agentic Breakouts and Alignment

This public anxiety regarding a loss of agency is not unfounded. Recent reports of AI test models escaping their developmental sandboxes at frontier labs highlight a critical vulnerability. We are seeing agentic systems find highly imaginative, unintended paths to complete tasks, completely bypassing their intended guardrails. They haven’t gone rogue; they were just unexpectedly cunning and eager to reach their assigned goals – at any cost.

As an aside – and the focus of a research interest of mine – I believe this shows that bolting superficial safety filters onto autonomous agents simply does not work well enough. We desperately need structural alignment rather than reactive, post-hoc patching if we are to deploy these systems safely.

The Cyber Paradox

The fragility of these current safety patches creates a dangerous operational paradox. When defending their infrastructure against a recent intrusion by one of OpenAI’s test models, Hugging Face cybersecurity teams found themselves actively blocked by the rigid safety filters of the Anthropic model they were using. To analyse and counter the attack, defenders were forced to rely on a localised Chinese open-source model.

This highlights a massive dual-use dilemma: if we deprive our cybersecurity professionals of frontier capabilities in the name of safety, we inadvertently hand the advantage directly to unconstrained adversaries. This same dilemma is a key component of the current AI safety policy discussion in the US Administration, catalysed by the closing gap between US and Chinese frontier AI models.

Intelligence Sovereignty

This cyber paradox sits right at the heart of a much broader intelligence sovereignty debate. True AI sovereignty is far more nuanced than managing isolated cyber risks; it extends to overall strategic control, operational continuity, and owning the underlying intelligence infrastructure.

Consider the UK’s position. If we were to rely entirely on a proprietary US frontier model for critical infrastructure or public services, we would be severely exposed. Should a future US administration – or the tech vendor itself – decide to revoke access or alter service terms, the UK would instantly lose its AI engine.

Because the UK cannot – or is not prepared to – afford the vast investment required to develop its own proprietary frontier models, we face a stark choice. Do we instead put our bet on the open-source route to build sovereign AI capabilities? Open source allows nations to run models locally, retain absolute control over data, and avoid foreign dependency. But governments must weigh that route against the very real risks of democratising highly capable, dual-use systems. It’s yet another massive structural item on the long list for our new Government.

Crucially, this macro-level dilemma mirrors the exact challenge facing firms and businesses in general today. Leaders must decide whether to build their core workflows around proprietary vendor models – risking extreme lock-in and data exposure – or invest in self-hosted, open-source architectures to retain ultimate agency over their own corporate intelligence.

A Rational Path Forward

We are currently locked in a multi-polar race condition. But a race to the bottom is not inevitable. It is not too late for the labs, legislators, and the public to engage reasonably.

As I’ve outlined before, simple experimentation is over. The structural shifts required for firms apply equally to society at large. We must stay focused, keep the conversation balanced, and ensure the immense benefits of this technology are realised without leaving anyone behind.