The old adage 'trust but verify' is rapidly losing its meaning in the age of advanced artificial intelligence, according to technology analysts and legal experts. As AI models become more complex and are connected to external services, the ability to independently check their reasoning and outputs is diminishing — leaving UK businesses and consumers exposed to hidden errors, bias, and security vulnerabilities.
Speaking at a cybersecurity conference this week, researchers highlighted that modern AI systems — particularly large language models and AI agents that interact with third-party APIs — produce results that are often impossible to trace back to a clear chain of logic. 'When a model gives you an answer, you cannot simply look under the hood and see why it said what it said,' explained one independent AI safety researcher. 'This creates a fundamental problem for regulated industries like finance, healthcare, and legal services.'
The regulatory landscape in the UK is still catching up. The Information Commissioner’s Office (ICO) has previously issued guidance requiring organisations to explain automated decisions that significantly affect individuals. Meanwhile, the European Union’s AI Act, which came into force this year, imposes strict transparency obligations on high-risk AI systems. UK companies that trade with the EU or process EU citizens’ data must comply — but many lack the tools to audit their own AI deployments effectively.
For UK businesses, the implications are stark. A bank that uses an AI model to approve loans, or an insurer that relies on an algorithm to set premiums, could face legal challenges if it cannot demonstrate that the system is fair and accurate. 'If you cannot verify what your AI is doing, you are essentially flying blind,' said a partner at a London-based technology law firm. 'That is a recipe for regulatory fines, reputational damage, and consumer harm.'
Consumers, too, are at risk. From AI-powered recruitment tools to automated benefit assessments, opaque algorithms increasingly shape everyday life. Without robust verification mechanisms, individuals may have no way to challenge incorrect or biased decisions. 'Trust but verify only works if verification is actually possible,' the researcher added. 'Right now, for many AI systems, it is not.'
Industry groups are calling for the UK government to accelerate work on a national AI assurance framework, similar to proposals floated in the 2023 AI White Paper. Without clear standards and independent auditing requirements, experts warn that the gap between AI capability and accountability will continue to widen — leaving both businesses and the public vulnerable to the consequences of machines that cannot explain themselves.