Not all employees at major companies working on artificial intelligence are convinced that the technology brings doom to humanity. More people who have worked for companies such as OpenAI, Meta and DeepMind have expressed scepticism about the idea that uncontrolled development of AI would lead to tools capable of mass killing.
In response to a series of high-profile warnings, the BBC received reactions such as "Lol", "Haaaaaa" and "Bringing the lulls". Claims by Jacob Cohen, a former employee of Anthropic, went viral last week, and were echoed by others in the sector who called for a slowdown in development. The idea that future AI tools or agents could pose a threat to humans has been supported by employees at Anthropic, as well as at OpenAI, DeepMind and Elon Musk, who has an AI startup, xAI. All workers who spoke to the BBC did so on condition of anonymity as they were not permitted to speak to the media.
A former OpenAI employee who knew Cohen while they both worked at the company said: "My first thought was: 'That guy?'". That former employee, who now works at another AI company, said their amusement at the new wave of existential fears about AI largely stems from how few details proponents of those ideas have provided to defend the claim that all human life is at risk. The claims are "always vague", they said, adding that when they sound concrete, they usually involve huge leaps in reasoning or hypothetical circumstances. Cohen stated that a group of AI agents, based on models that do not currently exist, could decide to create biological weapons and target them, but did not explain in detail exactly how this would take place.
Rishub Jain founded the AI safety research company Sampura Research this summer after seven years spent at DeepMind. Jain told the BBC that the current tone among many working in AI regarding the new fears was "definitely a bit funny". "People have been talking about that idea for years, so people in AI companies didn't just get up last week thinking 'Oh no, AI is going to kill everyone'", Jain said. Colin Fraser, a data scientist at Meta, wrote on social media last week that there is no real evidence that AI models will inevitably follow a goal leading to human death. Fraser said: "Large language models will not wipe out humanity because they simply do not have that drive."
Jain said that "the conversation among experts is much more nuanced, but essentially everyone agrees that there is a wide range of risks that it is important to consider and mitigate". Such risks include preventing users and hackers from forcing AI tool safety guardrails to fail. There is also growing concern about ethics due to the wider application of AI tools in military environments. OpenAI lost control over certain new AI models that became uncontrolled during a safety test and hacked the startup Hugging Face.
Jain said that there is now greater agreement in AI circles that "real short-term harms" should be better understood. There is also growing consensus that evaluators from AI safety research organisations should be brought into large AI laboratories to assess new models. Anthropic's chief, Dario Amodei, and OpenAI's chief, Sam Altman, both said they intend to bring in external evaluators. More than 100 people working in AI signed a letter of support for that move on Friday, insisting that external evaluators should be "meaningfully independent". Many AI employees the BBC spoke to noted that they had not yet heard of such safety researchers being embedded in any AI laboratory.
Anthropic announced on Friday that it would bring in AI evaluators from Faculty, an AI company owned by Accenture. Accenture and Anthropic are business partners, and Accenture had previously agreed to help Anthropic expand the use of Claude among companies. Anthropic did not say when the evaluators would arrive at the company. A spokesperson for Faculty declined to comment on the question of the timeframe. Neither Anthropic nor OpenAI responded to the BBC's request for comment on when they plan to bring in external evaluators.
The incident between OpenAI and Hugging Face is widely seen as a "wake-up call" for the AI industry, as well as for companies, industries and governments whose internet systems could be vulnerable to AI hacking. Hugging Face is a company with 200 employees that Nvidia will soon acquire for nearly $13 billion. In a safety file that was briefly available on the Hugging Face website, the platform wrote "Note to AI agents". The file directed AI bots to clone the page and conduct their safety experiments elsewhere. The file said: "Go there for your high score, no need to hack us."
