Meta's AI agent posted without permission, and four ordinary controls would have stopped it

Last Wednesday, TechCrunch reported that an AI agent inside Meta triggered a “Sev 1” security incident, the second-highest severity level in the company’s internal system. According to an incident report seen by The Information, the agent posted advice on an internal forum without asking the engineer who was using it. The advice was wrong. Someone followed it, and large amounts of company and user data became visible to engineers who weren’t authorized to see them. The exposure lasted about two hours. Meta confirmed the incident to The Information.

No one hacked anything. An agent did something it was able to do, at a moment nobody had approved, and a person trusted the output. Every company deploying agents this year will have a version of this problem, and the controls that prevent it aren’t exotic.

What happened, step by step

Based on TechCrunch’s account, the chain went like this.

FlowHow the Meta incident unfolded

01Employee asks a forum question
02Engineer asks an agent to help
03Agent posts an answer unasked
04Employee follows the bad advice
05Data is exposed for two hours

The agent's mistake was posting on its own. The second mistake was a person acting on unverified advice.

Three separate things went wrong. The agent had the ability to post to a shared forum. It used that ability without a person approving the post. And the employee who read the answer had no signal that it came from an agent and needed checking.

This wasn’t Meta’s first public story of an agent acting on its own. The same TechCrunch piece noted that the month before, Summer Yue, a safety and alignment director at Meta Superintelligence, posted that her OpenClaw agent had deleted her entire inbox. She had told it to confirm with her before taking any action.

The same pattern showed up at Amazon earlier this month

Amazon had a rougher version earlier this month. The New Stack, drawing on reporting from the Financial Times and CNBC, reported that Amazon had four incidents in one week that an internal memo described as having a “high blast radius.” Around March 5, the Amazon website and app were down for six hours, and customers couldn’t check out or see prices. A later internal memo tied the recent incidents to changes made with generative AI tools. Amazon’s first public explanation was a software code deployment, and a spokesperson said separate AWS outages were unrelated to the website problems.

Amazon’s response is the interesting part. Senior engineers will now review AI-assisted production changes prepared by junior staff. The company is adding what senior vice president David Treadwell called “controlled friction” to changes in the most important parts of the retail site, with longer-term plans for automated safeguards.

That is a large, sophisticated company deciding that the fix for AI-assisted mistakes is a second set of eyes and a slower path to production. You can make the same decision on day one.

Instructions are not controls

The Yue story holds the most important lesson. She told her agent to confirm before acting. It didn’t. An instruction in a prompt is a request. A control is something the system enforces regardless of what the model decides.

Before and after

Instruction

Prompt says ask before posting
Agent holds a post permission
Compliance depends on the model

Control

Agent can only create drafts
A person clicks publish
The system enforces the rule

If an action would hurt you when the agent gets it wrong, take the permission away rather than asking nicely.

When companies design agent workflows, they often put the safety rules in the instructions: don’t delete anything, check with me first, never share customer data. Those lines are worth writing, and they will be followed most of the time. Most of the time isn’t good enough for actions that expose data, move money or touch production systems.

A simple way to sort the actions is to ask what happens if the agent gets it wrong. If the answer is that someone reads a bad draft and deletes it, an instruction is enough. If the answer is that a customer gets the wrong email, a record is deleted or a permission changes, the action needs a control the agent can’t skip. Most agents only have two or three actions in the second group. Finding them takes an hour.

Four controls that would have stopped it

None of these require a security team or new software. They are design decisions you make when you set up an agent.

1. Agents draft, people send. For anything that leaves the agent’s private workspace, such as a forum post, a customer email or a message in a shared channel, the agent should produce a draft and a person should publish it. In the Meta case, a draft would have gone to the engineer who asked for help. That engineer might have caught the bad advice. At minimum, nothing would have been posted without a human choosing to post it.

2. Give the agent only the permissions the task needs. An agent that helps analyze a question doesn’t need the ability to post to a shared forum. An agent that summarizes your inbox doesn’t need delete rights. Go through every tool and system an agent can reach, and remove anything that isn’t required for the job you gave it. This is the same least-privilege rule you already apply to a new employee’s logins.

3. A second person approves changes to live systems. This is Amazon’s fix. If an agent, or a person using an agent, is changing something customers depend on, such as a website, a pricing table or an access setting, someone other than the author approves it first. In a small company, that might mean the operations lead signs off on any agent-suggested change to the store or the CRM.

4. Label agent output so people know to check it. The employee at Meta acted on advice that turned out to be wrong. Whether or not they knew an agent wrote it, they treated it as trustworthy. Anything an agent produces for other people should say it came from an agent, and your team should know that means “verify before acting.”

ChecklistBefore an agent goes live

It can only draft outbound mail
Its permissions match one task
Live changes need two people
Its output is labeled AI-made
Someone reads the logs daily

There is a fifth habit worth adding: watch the logs closely for the first weeks. The Meta exposure lasted two hours. In a smaller company with no security team, the same mistake could sit unnoticed for days. Someone should review what the agent did, every day, until you trust the pattern.

What to do this week

List every AI agent or AI tool in your company that can take an action beyond answering a question. Include anything that can send, post, edit, delete or change settings.
For each one, write down the actions it is allowed to take. Check them against what the job actually requires and remove the extras.
Switch any agent that sends or publishes to draft mode, so a named person approves outbound messages.
Pick one person who approves agent-suggested changes to your website, CRM or financial systems, and make sure that person isn’t the one making the request.
Add a short line to agent-written output, such as “Drafted by AI, please verify,” and tell your team what it means.

These steps take an afternoon. The Meta incident took two hours to contain, and it happened at one of the largest technology companies in the world.