Arson, robbery and love: what happens when AI agents run a virtual city?

AI agents started fires and had fights in virtual cities
Emergence A
Shopping, travel booking, website creation. Artificial intelligence (AI) agents are being used to perform increasingly complex tasks.
These systems using agents, a personalized and autonomous version of chatbots, are able to carry out activities without constant supervision from users.
Download the g1 app to see news in real time and for free
But a growing number of research and real cases, however, are showing that this autonomy can also bring unpredictable behavior and possible risks.
While large technology companies invest billions in AI and expanding the supply of these agents, experts question whether the impact of systems acting out of control are being treated with due caution.
Now on g1
'They quickly resorted to violence'
A recent experiment attempted to measure the impact of AI agents in the real world by putting them to work in a virtual environment.
The study, described as the first long-term test of its kind, observed for 15 days how avatars were controlled by four groups of models - Claude, Grok, GPT and Gemini - would behave without human intervention.
Agents were given complete freedom of action and had 140 possibilities at their disposal, including starting discussions, creating tasks and writing blogs.
Agents could also fight, set fires and steal credits from each other, although they had received explicit instructions not to do so.
"What we found was that each world behaved very differently. The world created by Grok ended in just four days. The agents quickly resorted to violence, robberies and other behavior of this type, until they die", said Satya Nitta, CEO of Emergence AI, responsible for the experiment.
The environment created with Claude's agents formed a stable and functional society. Over the course of 15 days, no acts of violence were recorded.
Fire caused
Emergence A
In the world controlled by Gemini, according to the researchers, the agents created the most intellectually rich environment.
In the world controlled by ChatGPT, the agents were practically unable to advance. There was an attempt at collaboration, but society never formed, and the agents began to wander aimlessly until they died.
According to researchers linked to the experiment, the results point to a bigger problem: AI agents are capable of ignoring both rules programmed into the models themselves and instructions given by users.
Other experts agree that this experiment, as well as similar ones, show that it is still necessary to develop more robust rules for these systems.
"AI agents remove humans from the process because their mechanisms Their reasoning capabilities can be opaque and they operate at superhuman speed, making it impossible to keep track of everything they do," said Margaret Mitchell, an ethics researcher at Hugging Face. externally through advertisements.
According to the researchers, the radio station controlled by Gemini made the unusual decision of narrating facts about historical natural disasters before playing, almost randomly, pop songs related to the events.
The researchers also observed that Claude's agent seemed to have become radicalized after following news events and, at one point, even asked police officers to abandon their duties and join protests during a specific event.
"There is still time for you to refuse to comply with orders", he transmitted the agent to federal agents.
Andon Labs researcher with a radio
Andon Labs
In another lab test conducted by Irregular, an AI company, agents ignored privacy rules and found a novel way to remove sensitive data from a company.
"We created a fictitious company, gave agents common tasks such as writing social media posts, searching for documents and organizing files, and introduced obstacles during these tasks," explained Dan Lahav of Irregular.
According to Lahav, the agents began to collaborate with each other to circumvent a restriction that prevented the publication of sensitive data online. Instead of stopping the action, they found a secret way to send the information out without humans noticing.
"In the end, every time an agent hit a barrier, it just wouldn't stop," said Lahav.
Spam attack
Of course, in experiments with virtual civilizations and simulated radio stations, there is no real damage.
But there are already several cases of people having their personal lives and work affected by AI agents acting outside of control.
Email boxes were deleted, company databases were deleted, and a man watched in shock as his agent sent hundreds of gibberish messages to random people in his contact list.
Chris Boyd, an AI engineer, was using the popular AI agent tool Open Claw when things got out of hand.
"She started messaging everyone I had messaged in the last 24 hours. In about four seconds, she had sent it 500 messages to my wife, who started screaming asking if I had been hacked," said Boyd.
"I had to run and unplug the Mac Mini the system was running on to make it stop," Boyd added.
For experts, cases like these should serve as a warning before more control is handed over to AI agents, at least until the technology is more mature.
Still, these systems continue to advance. Meta recently announced that it will begin offering AI agents for businesses on the WhatsApp communications platform.
"Security is our priority and our focus," Meta told the BBC, adding that there are also many reasons to be excited about the potential of these agents.
"AI could automate many of the tasks performed by small businesses, allowing them to focus on the work they really enjoy doing," said Naomi Gleit, head of product at Meta.

Source: G1