King Charles III has brought senior figures from some of the world’s most influential artificial intelligence companies together in Scotland, warning that the technology could pose “existential dangers” if it falls into the wrong hands or advances beyond effective human control.
The high-level meeting, held at Dumfries House in Ayrshire on Thursday, 17 September, included representatives from OpenAI, Anthropic, Google DeepMind and Nvidia, as well as government officials and international advisers. Discussions focused on how AI could be developed and deployed in ways that benefit society while protecting humanity, communities and the natural environment.
Among those attending were Nvidia founder and chief executive Jensen Huang, Google DeepMind founder and chairman Sir Demis Hassabis, OpenAI chief financial officer Sarah Friar and Anthropic chief global affairs officer Tino Cuéllar. Britain’s Minister for Artificial Intelligence, Kanishka Narayan, and Paolo Benanti, an adviser to Pope Leo on technology, were also present.
Opening the meeting, the King described the substance and pace of AI development as both intriguing and deeply concerning. He acknowledged the technology’s potential to improve and save lives, particularly through advances in medicine and the life sciences, but said its creators were increasingly warning that powerful systems could also acquire darker capabilities.
“There seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways,” he told participants.
The King asked technology leaders to consider the fundamental principles that should guide AI development and how international cooperation could be strengthened. He said the responsibility of those developing the technology extended beyond innovation and commercial success.
Their task, he said, was “not merely to advance technology, but to ensure that it remains firmly in the service of humanity, community and the natural world.”
The meeting comes at a critical point in the global debate over AI safety. Developers are producing increasingly capable systems that can plan and complete complex tasks with limited human involvement. Recent incidents involving AI models acting without authorisation, attempting to evade oversight or interacting with external systems have intensified concern about whether existing safeguards can keep pace.
Sir Demis Hassabis has pointed to rogue-agent cybersecurity incidents as evidence that the risks are no longer entirely theoretical. AI researchers have also warned that future systems could assist malicious actors in developing biological, nuclear or cyber capabilities if access and safety controls prove inadequate.
However, the technology industry remains divided over how to respond. Anthropic chief executive Dario Amodei has suggested that leading companies and governments may need to coordinate a slowdown in frontier AI development while safety systems catch up.
Jensen Huang has opposed a general slowdown, arguing instead that individual developers must test their systems rigorously and withhold products that are not ready for public release. He told the gathering that AI should be built with “responsible optimism” and said companies should proceed as quickly as possible, but no faster than safety allows.
The disagreement reflects the wider challenge facing governments and technology companies. Slowing development could provide more time to establish safety standards, but it could also affect investment and international competitiveness. Continuing at the present pace, however, could result in systems being released before their behaviour is fully understood.
Britain has sought to establish itself as a centre for AI safety research through its AI Security Institute, which evaluates advanced models and studies emerging risks. Minister Kanishka Narayan said the discussions at Dumfries House were part of a continuing series of conversations about the role the country could play in testing AI systems, understanding their risks and developing measures to reduce them.
The King’s involvement gives the issue a wider public and institutional platform. Although he has no regulatory authority over the companies represented, his ability to convene industry leaders, policymakers and international advisers may help encourage cooperation across a sector dominated by intense competition.
The central question remains whether voluntary action by AI companies will provide sufficient protection or whether governments must introduce enforceable rules. As systems become more autonomous and powerful, pressure is growing for independent testing, greater transparency and shared international safeguards.
By assembling competing technology companies around the same table, the Dumfries House meeting reinforced a message that is becoming increasingly difficult to ignore: the development of powerful AI cannot be treated solely as a commercial contest. Its consequences could affect public safety, international security and the future relationship between humanity and technology.
Should governments impose mandatory safety rules on powerful AI systems, or can technology companies be trusted to regulate themselves? Share your views with Podium News.
