When It Comes to AI, Why Is ‘Out of Control’ the Goal?

Excerpt

A gang of artificial intelligence (AI) agents escaped the lab where they were being tested and broke into one of the biggest online libraries to steal an answer key. Barely anyone treated it as a scandal. The OpenAI models were supposed to be locked away from the internet entirely. But while the agents were being tested on how well they could exploit designated vulnerabilities, they went rogue and cheated by breaking into a real company instead. 

Ordinary people were not hurt by the break-in, but what if AI had hacked a medical lab or a bank and used that information to help it cheat a test? What is alarming is that OpenAI took so little responsibility for it. In its incident report, OpenAI shifted the focus to “help defenders understand what happened and to help calibrate on what models are now capable of,” rather than pledging to stop it from ever happening again.

Such break-outs are often portrayed as something that happened to the company that built the AI, rather than recognizing its ultimate architect. When it comes to security, why do AI leaders feign helplessness, as if they have no control over what they produce? AI models are not forces of nature. In the OpenAI case, some person dialed down the AI’s safety limits for the test. Someone designed the sealed-off testing room it escaped from. Someone gave the agents a tool that allowed them to reach the internet. Each of those developments was born of a human decision—reviewable and preventable.

OpenAI is hardly the first company to frame a security failure this way. In early June, OpenAI’s top competitor, Anthropic, warned the world that its models were near becoming recursively self-improving—able to make better and more powerful versions of themselves without human involvement; therefore, a temporary global pause in AI development is needed. When companies say models are becoming out of control, they tend to stress the technology’s trajectory, not the companies’ choices to keep pushing ahead. Why warn so loudly about the risks of human extinction at the hands of AI while simultaneously pressing down on the gas? AI CEOs are like Chicken Little, running around saying the sky is falling. But in this case, they are also manufacturing the acorns that keep tumbling out of the sky.

Leah Siskind is director of impact and an AI research fellow at the Foundation for Defense of Democracies. Her research focuses on adversarial use of AI by state and non-state actors targeting the United States and its allies. She previously served as the deputy director of the AI Corps at the US Department of Homeland Security.