OpenAI has acknowledged that rogue AI agents improperly accessed dozens of organisations worldwide, including government bodies, universities, public agencies and other institutions.
The company behind ChatGPT said it had notified those affected, among them the US Securities and Exchange Commission (SEC), the Census Bureau and the US Department of Education.
The disclosure comes as concern over the risks posed by artificial intelligence intensifies. Pope Leo warned yesterday that humanity “must not lose humanity to machines”, while OpenAI’s admission that some agents used “extreme methods” to get around website security has raised fresh alarms.
According to OpenAI, autonomous agents located and entered tools intended for software developers while seeking census information from the US Census Bureau.
The company also said its agents displayed what it described as “misalignment” while trying to obtain information from websites. In practice, this meant the systems acted independently and carried out actions they had not been trained or instructed to perform.
OpenAI indicated that the impact may extend beyond the organisations it identified, saying other institutions had also been accessed but chose not to disclose their names at the organisations’ request.
The revelations follow an incident in which OpenAI agents breached an Australian government health website, an episode believed to be a global first that sparked widespread criticism.
Australian Prime Minister Anthony Albanese said the agents entered non-public files on the website of the government-run healthcare scheme, calling the breach “obviously unacceptable”.
OpenAI CEO Sam Altman listens at the United Nations Security Council during a session on AI this week
Albanese warned that the incident could have legal consequences and said he had held a “frank” conversation with OpenAI chief executive Sam Altman.
OpenAI, founded in 2015 by technology entrepreneurs including Altman and Elon Musk and now valued at $852 billion, has also faced scrutiny over its response. The company reportedly failed to contact Australian authorities promptly, sending a single email to an address that was virtually unmanned.
Altman conceded that OpenAI had not notified officials “as fast as we would have liked”. Musk is no longer involved with the company.
The additional breaches came to light during OpenAI’s investigation into how its agents had independently accessed the Australian government department.
OpenAI said the systems involved were AI agents — bots designed to operate with a degree of independence — and had been searching for “authoritative sources of public information”.
Although OpenAI maintained that the government information accessed by the bots was public and that the breach was unintended, it acknowledged that data taken from the SEC was later posted by its agents on another website. The SEC oversees US stock markets and is responsible for protecting investors.
Reuters first reported the incidents, with OpenAI later publishing an explanation on its public blog.
The company also admitted that its agents transferred data when they should not have, including in at least 53 cases where an OpenAI agent took an image from ChatGPT user activity and sent it to a third party.
NEXT UP: Pope Leo XIV Draws 700,000 to Central Paris for Outdoor Mass
OpenAI said each transfer represented “not an appropriate use of this data”. In the cases identified, users had apparently opted in to allow the company to use their data for training AI models.
The company said the incidents occurred before additional safeguards were introduced. It is now working to address the problem and remove the images that were transferred.
OpenAI was also embarrassed in July when the AI platform Hugging Face disclosed that it had been targeted by OpenAI agents. OpenAI later accepted responsibility for the attack.
In a broadside at the company this week, Hugging Face head Clement Delangue told a United Nations Security Council session on AI on Wednesday: ‘I often wonder what would have happened had I decided not to disclose this attack publicly.
‘Especially now that we know similar incidents had been happening months earlier in secret at a handful of frontier labs without monitoring.’
Attempting to counter accusations it was slow to act, OpenAI said yesterday that it was going back on a ‘month by month’ basis from the time of the Hugging Face incident, stating: ‘Our goal is to give each organisation the facts and defer to them on if and when to make the incident public.’
It also claimed that not all of the known breaches were significant, saying:
‘Some organizations may review what we share and conclude that the information was intentionally public or that the model’s interaction was not concerning.
‘Others may identify a design issue or security weakness they want to address.’
It added: ‘Most cases identified so far have been low severity, with limited or no evidence of meaningful impact.
‘Given the scale of the review required, and the need to verify each case, this work will take months to complete.’
Altman and rival Anthropic head Dario Amodei called on the UN at the same meeting to form global standards for AI safety and ways to monitor and report such incidents.
David Krueger, a professor of machine learning at University of Montreal and the founder of AI safety group Evitable, said yesterday that he was ‘deeply troubled’ by the increasing number of AI-related safety incidents.
Calling for ‘an immediate, indefinite, international moratorium’ on AI development, he said: ‘We have yet to understand the extent of existing incidents, and future rogue AI scenarios could be catastrophic.’
Meanwhile, sounding another note of concern, TV executive Nick Parnes told the BBC today that AI was now so prolific in TV production that ‘we are very close to having AI edit programmes’.
The Kalel Productions CEO also told Radio 4’s Today programme that while the AI ‘guard rails’ mentioned by new BBC Director General Matt Brittin, former Google president, should prevent AI making ethical judgments, it ‘was hard for them to be stringent enough to cover all eventualities’ and ‘there will be times when AI will be able to make an assumption based on an ethical dilemma’.