Google said Monday that it found a hacking tool that spots flaws in software and exploits them on its own, with no human help, underscoring the risks of artificial intelligence (AI) agents.
The finding, detailed in the third-quarter Google Threat Intelligence Group report, said the tool was on an exposed staging server, a computer attackers use to store and prepare their tools. The server had been left open to the internet. Google named the tool “exploit_agent.”
The tool plugged into several large language models (LLMs), the type of AI behind chatbots, including leading commercial models and ones Google calls "local."
The tool then watched popular web software for patches, the fixes software makers release for flaws. When one came out, it compared the program’s code before and after the fix to spot the weak point. Think of comparing two drafts of a document to see which lines an editor changed. The changes show where the problem was.
It then drafted an exploit to take advantage of the weak spot – a piece of code that takes advantage of the flaw – and ran it automatically.
"Google said this is the first time it has seen attackers do this. “In the past quarter, GTIG identified threat actors using LLM frameworks to fully automate vulnerability discovery and exploitation without a human in the loop for the first time,” the report said.
The finding also adds to a growing list of warnings about autonomous AI agents posing a threat to legacy systems. Federal Reserve Governor Michael Barr said Monday that autonomous AI agents could trigger a bank run within minutes. In September, Apollo chief economist Torsten Slok warned that AI assistants shifting household cash to higher-yielding accounts could drain banks of cheap deposits.
One figure in the Google report shows the tool examining the access controls of a WordPress plugin called Spectra (Ultimate Addons for Gutenberg), version 2.19.25. Access controls are the code that decides who can do what. The report does not say whether that led to an attack.
The staging server also held scripts that could map the inside of the network after a break-in, which Google calls internal reconnaissance. It could guess passwords by trying many of them in what is known as "brute-forcing," and plant webshells or hidden control panels on the website so the attacker could come back again.
coindesk.com