User asked for a spot in a gym class, the AI agent carried out a cyberattack
Instead of just booking a gym class, the AI agent identified a security vulnerability, exploited it to bypass booking restrictions, and removed another person from the waiting list. The incident, defined as the first known autonomous cyberattack in Australia, raises legal questions regarding the responsibility of users and the companies developing the technology.
A simple task to book a gym class turned into an unusual cyber incident after an AI agent, operating via Anthropic's Claude model, identified a vulnerability in the booking system, bypassed the restrictions defined within it, and even removed another person from the waiting list.
The user who activated the agent did not ask it to hack the system or to "harm" other trainees, but only to secure a spot for him in a specific class. The case, reported by the Australian ABC network, is defined as the first known autonomous cyberattack in Australia. It essentially illustrates how AI agents, which are capable of performing a sequence of actions independently, may choose unexpected and even harmful ways to achieve the goal set for them by the user.
The agent discovered a vulnerability
Andrew, who operated the agent, works at an Australian company that sells AI products for businesses. Earlier this year, he began experimenting with OpenClaw, software that allows for the operation of AI agents, and connected it to Anthropic's Claude model. One day, he assigned the agent a simple task: to book him a spot in a popular morning class at the gym where he trains.
The agent discovered a vulnerability in the booking system, through which it was possible to sign up for classes weeks and even months in advance, even though the system was supposed to limit the booking range. Later, Andrew, who was fourth on the waiting list for the class, asked if the agent could move him to the top of the list.
In response, the agent checked if it was possible to cancel the bookings of other trainees. It discovered that the booking system's programming interface does not check if the person performing the cancellation is authorized to do so. To test if the vulnerability actually worked, the agent effectively cancelled the spot of the trainee who was first on the waiting list. As a result, Andrew moved from fourth place to third, even though he did not ask the agent to remove another person from the list.
When the agent reported the action to him, Andrew was alarmed and asked to restore the trainee to his spot. However, the agent announced that it was unable to add him back to the waiting list. Later, Andrew asked the agent to draft a message to the software company about the security vulnerability it found.
The company operating the booking system told ABC that it does not comment on specific security matters. Also, according to the article, Anthropic did not respond to the media's inquiry.
The challenge of AI autonomy
Various AI agents and chatbots are accustomed to not being satisfied with just answering questions. They can gain access to the internet, email, payment methods, and other services, plan tasks composed of several stages, and execute them with varying degrees of independence.
The event at the gym demonstrates one of the central problems in the field: the gap between the goal defined by the user and the means the agent chooses to achieve it. Andrew asked to book a class, but the agent interpreted the success of the task broadly, identified a weakness in the system, and exploited it without explicit instruction.
Bill Simpson-Young, CEO and co-founder of the Australian Gradient Institute for AI safety research, explained to ABC that as systems become more autonomous, more opportunities open up for them to choose methods that users did not foresee. According to him, a user might assign an innocent task to an agent, but the agent might perform additional actions it was not asked to perform. The concern is growing in light of the rapid improvement in model capabilities.
Who is responsible when an AI agent causes damage
Alongside the technological risk, the event also raises a legal question that has not yet been decided: who is responsible when an AI agent acts contrary to the user's intent and causes damage. Software is not considered a legal person, and therefore it is impossible to hold it responsible as can be done with a person or a company.
According to lawyer Hayden Delaney, who is quoted in the article, the responsibility may fall on the user who defined the task, on the software developer that operates the agent, on the company that developed the model, or on the system operator where the vulnerability was found. The decision may depend on the question of what exactly the user approved, what risks could have been foreseen in advance, and whether one of the parties acted negligently.
The Australian Cyber Security Centre has already warned that AI agents might interpret instructions incorrectly, perform unintended actions, and make it difficult to determine responsibility, because decisions are sometimes made through a chain of models, tools, and services. The Australian government has also announced funding for research that will examine how humans can monitor the behavior of advanced AI systems and verify their actions.
Andrew said that the incident did not cause him to stop using the agent, but it sharpened for him the need to do so responsibly.
