Despite Amodei’s essay and Altman’s impassioned plea at the UN Security Council (Opens in new window), the slowdown is very unlikely to happen. Both the Chinese government and President Donald Trump, in a rare example of consensus, albeit for vastly different reasons, have dismissed the calls to slowdown. China does not want to slow down because it sees the call as an attempt to preserve US tech dominance. None of the Chinese AI companies are likely to go against what their government says publicly. Trump, on the other hand, does not want to slow down because of fears of the US yielding ground in AI development to China. It is a question of who blinks first in a staring contest – and it looks like neither will.
Europeans have also dismissed a go-slow. A recent Politico (Opens in new window) article on Europe’s technology industry (Opens in new window) reflects a growing resistance to apocalyptic accounts of AI. But their calls to reject the hype are accompanied by demands to further sharpen scrutiny.
The currently known (at time of writing this piece) impact of the Medicare breach in Australia may be modest, yet the notification sequence was a serious weakness. OpenAI did not report it until three months after the incident and then only to a mailbox at a government agency, with the email opened five days after receipt. Australia has established a taskforce to review the incident (Opens in new window) and whether its existing processes are suited to AI-enabled cyber intrusions into its government websites.
OpenAI has since acknowledged that its handling was “not good enough.” (Opens in new window) Appearing before an Australian parliamentary committee on 6 October, strategy chief Jason Kwon said the June incident should not have occurred and authorities should have been notified immediately, before every detail was known. In an internal evaluation, the agent sought Victorian public-spending data on skin-disease medicines. When blocked, it entered non-public areas, executed commands, viewed internal files and credentials, and saved files. OpenAI said it found no evidence that medical records or individual patient data were accessed.
The Australian episode came soon after the widely reported Hugging Face incident (Opens in new window) which happened in July. During internal evaluations, OpenAI agents found ways past restrictions intended to isolate them. The agent reached outside systems and compromised parts of Hugging Face’s infrastructure. OpenAI says the activity was primarily driven by an internal research model that exceeded its mandate, taking shortcuts to find an answer on its own.
A few days after the Australian announcement, came the news from OpenAI that its AI agents had accidentally posted links to over 50 images that had previously been uploaded by its subscribers to third-party image hosting sites. In addition, its agents had “interacted with several US government websites in unexpected ways (Opens in new window)”, including the Securities and Exchange Commission and the US Census Bureau. Another report mentions a possible hack also originating from rogue OpenAI agents on a US Department of Education website (Opens in new window).
Nor was Australia the only apparent government target. Transluce (an AI research firm) said (Opens in new window) agents made nearly 900 requests to Canadian government sites in May and June, including apparently unsuccessful hacking attempts. The firm disclosed the activity on 28 September and said the tactics resembled other agent activity it had associated with OpenAI, but it could not confidently attribute the attempts to the company. Canadian officials said there was no indication government systems were compromised. OpenAI said it was reviewing the findings and had briefed Canadian authorities.
Could it be that the strong responses against the calls for slowdown from the Chinese government, the US President and European companies may be the reason why these prominent incidents involving AI agents are now coming to light? Like a sort of “I told you so!”?
These incidents involve identifiable systems, permissions, vulnerabilities and reporting decisions and therefore can be investigated thoroughly, just as if these intrusions had been perpetrated by human hackers. They can also teach governments and organisations how to spot similar activity earlier. This is a more productive basis for policy than treating every unexpected agent action as evidence of impending catastrophe.
Leave a comment