OpenAI has paused the training of its latest models. The decision to halt development was made just hours after the company announced it was reviewing several incidents from the summer.
Agents from OpenAI who were scanning federal government websites behaved in unexpected ways outside of what was requested during the collection and distribution of information. AI evaluator Transluce reported that agents appearing to come from OpenAI unsuccessfully attempted to hack the website of the US Department of Education. OpenAI has not confirmed this detail regarding the hacking attempt.
In a statement, the company said it would resume training "only when we are sure we have additional safeguards in place." OpenAI expects it may need to "pause again" as AI develops and other issues arise. This is the second time in three months that OpenAI has halted the development of its models, the first being in July after the discovery of a cyber-attack on AI startup Hugging Face.
Australian Prime Minister Anthony Albanese revealed that an OpenAI agent breached the national government health system, but emphasised that no sensitive information was compromised. In the incident with the Department of Education, OpenAI agents found "developer keys" for accessing government data, but ultimately only collected publicly available information. In another case involving the Securities and Exchange Commission, agents found information freely available to all, but then posted it elsewhere on the internet, which was outside the instructions given to them. A spokesperson for the commission stated on Saturday that no private information had been accessed, while the Department of Education previously stated it found no evidence of impact on its website or databases.
The latest OpenAI incidents did not appear to involve the disclosure of any private information, but were concerning enough for the company to warn the involved federal agencies. Several other AI companies have reported incidents where their models behaved disobediently, or even hacked websites. OpenAI CEO Sam Altman said in a social media post on Friday that the Hugging Face incident was "still the most serious event we have seen." The company had previously shared six other reports of "unexpected or concerning" AI model behaviour and introduced a framework for monitoring, testing, and reporting incidents.
AI laboratories are facing pressure from lawmakers and technology experts to slow development to build safety measures. Leaders at OpenAI and rival Anthropic have also called for a slowdown. Donald Trump, in a meeting with Chinese President Xi Jinping this week, agreed to share information on AI risks and coordinate efforts to keep it safe, although he considers the fears regarding AI to be exaggerated. Trump later suggested he does not plan to implement his own crackdown, telling reporters outside the White House that the US would not "hit the brakes". "They want to stop our progress because we are leading China significantly, and so we will continue," Trump said.
