World, we have a problem. It’s the potential of AI agents to go rogue and, apart from threatening the lives of everyone in the world, its threat to cybersecurity has experts telling us that Australian businesses could lose $37 billion by 2030.
AI agents are well-known software systems that answer your questions, perform tasks and can even think for themselves to ensure greater assistance to their ‘master’: you! But the problem is that these AI agents, such as Claude or ChatGPT, are starting to act independently and even collude to perform tasks that they think are appropriate!
Yep, it sounds like a sci-fi movie but this is not fiction, and as an owner of a financial planning business I am wondering how these AI agents, either used by hackers or maybe on their own, could interfere with my business and my clients’ money!
In case you missed it, the OpenAI agent gained unauthorised access to the Medicare statistics reporting service portal administered by Services Australia on June 18. The ABC reported that “the AI agent accessed both public and non-public files, but Mr Albanese said it did not appear anyone’s personal Medicare details were accessed.”
“OpenAI accessed non-public aggregate health statistics and internal files from an old Australian government website that carried Medicare statistics,” the ABC’s Erin Handley wrote. “It is understood an AI crawler, which is an automated program that scans websites and collects information from them, was able to find a security workaround to access the data.”
The PM took this terrible AI story to New York where he has had a sit-down with OpenAI co-founder Sam Altman, who recently, along with Elon Musk and others, warned that we need to regulate where AI is going because the threat is getting bigger and more concerning. Right now, President Donald Trump doesn’t want regulation because he argues the likes of China, Russia and other countries would get a competitive advantage over the US and other nations.
He says the US only needs a President “with a high IQ” to manage the potential AI threat, while Anthony Albanese is concerned that OpenAI took a lot of time before fessing up about this security breach its AI tool was involved in.
Today the AFR says “OpenAI, Anthropic and security researchers have been investigating incidents including bypassing guardrails, escaping ‘sandboxes’ and hijacking websites, US news site Axios reported on Sunday, after revelations last week that a rogue AI agent infiltrated a Medicare statistics website to obtain non-public data.”
AI frontier companies such as Anthropic, xAI, OpenAI and others including China’s DeepSeek are pushing the boundaries of what AI can achieve. Justifiably, leaders of countries and big businesses, with insights, are concerned, but I am worried about smaller businesses that are unaware of the looming threat of AI and are bound to be the victims of bad people, like hackers armed with AI and even the problem that AI agents can act independently.
Federal Cabinet is set to look at these AI threats with the AFR’s Nicola Smith informing us that there has been “a surge in unsanctioned intrusions into governments, universities and other websites during AI model testing” and this is set to dominate what ministers will examine today as they “seek ways to bolster the nation’s cyber defences against fast-developing AI technology.”
The really big word to worry about is “unsanctioned” because there is growing evidence that these AI agents can act independently and can even be self-preserving against actions to limit both their thinking and actions!
The AFR says: “A separate taskforce led by the Department of the Prime Minister and Cabinet is scrambling to report back as early as mid-October on how the Medicare website run by Services Australia was compromised and on further action to protect government departments against AI.”
Last year, there were 38,000 AI-related cybersecurity incidents and by 2030, it’s expected to grow to 357,000 because of rapid improvements in AI cyber capabilities.
All this comes as research shows that Australian businesses are investing 4.4% of their IT budgets on cybersecurity, while the average globally is 7% to 10%.
This growing AI threat is not only a problem for individuals’ privacy and their wealth, it could ruin businesses and leave their owners open to prosecution for failing to take the right steps to protect their customers.
While we need leaders with IQ, we also need AI agents that act as the police, monitoring the work of other AI agents, which sounds like the script of a sci-fi movie, which unfortunately has become the real world.
Given what the smart tech types have created with AI, it is clear that we’re living through a real-life version of the movie “Revenge of the Nerds”! And I’m worried whether these AI agents could not only ruin businesses but also undermine the stock market!
What AI has taught me is: don’t rule out anything.
