Some of the biggest recent outages of online services began with automation doing what it was built to do, in a situation that its design didn’t cover.
Every Part Worked, and a Cloud Database Went Offline
In October 2025, Amazon DynamoDB became unreachable in AWS’s us-east-1 region. The DNS name that clients use to find DynamoDB didn’t point to any address. Other AWS services that depend on DynamoDB went down too, and knock-on effects lasted into the next afternoon.
No component failed. The automation that manages that DNS record runs as three independent copies, one in each of three availability zones, and each copy did what it was designed to do. Two of the copies acted in an order that left the record empty. AWS’s post-event summary gives the sequence.
How the Record Was Emptied
Enactor 1
Enactor 1 checks that its plan is newer than the last plan applied. The plan is newer, so Enactor 1 starts writing. Then Enactor 1 hits long delays and keeps retrying.
Believes: my plan is the newest.
Enactor 2
Waiting for the next plan.
DNS record
An earlier plan
addressaddressaddress
Enactor 1
Enactor 1 is still retrying, with its plan only partly written.
Believes: my plan is the newest.
Enactor 2
The planner has kept issuing plans. Enactor 2 picks up a much newer plan and writes it to the record.
DNS record
The newer plan
addressaddressaddress
Enactor 1
Enactor 1 gets through and writes its older plan over the newer plan. Enactor 1 checked once, before the delay, and doesn’t check again.
Believes: my plan is the newest.
Enactor 2
Finished writing the newer plan.
Believes: the record holds the plan I just wrote.
DNS record
Enactor 1’s older plan
addressaddressaddress
Enactor 1
Done.
Enactor 2
Enactor 2 runs its clean-up, which deletes plans much older than the plan Enactor 2 just wrote. Enactor 1’s plan is one of the plans deleted.
Believes: the record holds the plan I just wrote, so old plans are garbage.
DNS record
The plan in use is deleted
no addresses
Enactor 1 and 2
Neither Enactor can apply a new plan: the record is left in a state that the automation can’t get out of.
Engineers
Engineers trace the outage to DynamoDB’s DNS and repair the record by hand. AWS has since switched the automation off worldwide until it is fixed.
DNS record
Empty until repaired
no addresses
Deleting old plans is a normal clean-up step, and the step is safe almost all the time. Deleting a plan is unsafe in one situation: when the plan being deleted is the one in use. Enactor 2 acted on its belief that the record held the plan Enactor 2 had just written, and nothing in the design made Enactor 2 read the record again before deleting. Finding unsafe control actions like Enactor 2’s clean-up before launch is what STPA is built for.
The Clean-Up as a Control Loop
Controller
Enactor 2
Believes: the record holds the plan I just wrote.
Decision rule: delete plans much older than the plan just written, without reading the record first
Control action Delete old plans
Feedback
Missing: which plan the record is using
Controlled process
DNS record
Actually holds: Enactor 1’s older plan
Unsafe control action: Enactor 2 deletes a plan when that plan is the one in use.
Our STPA reading of AWS’s post-event summary.
The 2026 second edition of Google’s Site Reliability Engineering book has a chapter on STPA and CAST, the methods that teams use to find flaws like this one.
Meta, Google, Amazon, AI systems, security, and the standards that mention STPA.
Part 1 of 6
Why the Usual Checks Miss the DynamoDB Race
Failure Mode Analysis
Microsoft’s and AWS’s reliability guidance asks teams to list how each component and dependency can fail, rate each failure mode by likelihood and severity, and plan a mitigation (Azure, AWS). For the DNS automation, “an Enactor runs slow” is on the list, and the mitigation is two other copies. The outage came from a slow copy and a healthy copy acting together, and that combination isn’t a row in the table.
Production Readiness Review
Before launch, Google’s SRE teams review a service against a checklist that covers dependencies, monitoring, capacity, and change management (SRE book). Three independent Enactors, one per availability zone, would pass as good redundancy. No checklist item asks what happens when one copy deletes a plan that another copy is using, and the deletion can happen only because there are three copies.
Chaos Experiments
Inject the faults that a team chooses. Slowing one Enactor is an easy fault to try. The team would check what a slow copy does to another copy’s clean-up only if someone had already thought of that interaction.
Model Checking (TLA+)
AWS engineers check designs with TLA+ because, they wrote, some of the most dangerous bugs are design errors, where the code does what the design says. A model checker tests only the properties that someone writes down. “Never delete the plan in use” has to be on the list first.
STPA
STPA starts from the control action “delete old plans” and asks when deleting a plan would be unsafe. One answer is “when the plan is the one in use.” A team can write down that unsafe control action before anyone knows how the plan in use could get deleted.
Step 4 of STPA then asks why a controller would give an unsafe control action. Enactor 2 would delete the plan in use only if Enactor 2 believed the record held some other plan. A controller’s beliefs come from its feedback, so the next question is what feedback Enactor 2 gets about the record before deleting. The design gives Enactor 2 no feedback about the record before deleting. Enactor 1 does get feedback, but only once: Enactor 1 checks that its plan is newer before the long delay, and doesn’t check again afterward.
Constraints STPA Derives From That Loss Scenario
Outcome: clean-up must never delete the plan that the record is using.
Feedback: each Enactor must read the record at the moment of acting: Enactor 2 before deleting, and Enactor 1 before each write, not once before a delay of unknown length.
Coordination: the three Enactors are three controllers of the same DNS record, so each Enactor must learn when another copy changes the record.
The first constraint names the unsafe outcome. The other two fix the control loop that allowed the outcome, through feedback and through coordination between controllers. A team that asks only how each Enactor could fail never looks at feedback or coordination.
Other large outages of 2024 and 2025 had different triggers, but each one can be drawn as a control loop with a flaw that STPA looks for.
Control loops break down in a few familiar patterns, whatever the system is. Because the patterns are known, a team can look for these patterns in a design before launch, and use CAST to trace the same patterns after an outage.
A Change Reaches Every Server Before Any Feedback Comes Back
In November 2025 a database permissions change made the query that builds one of Cloudflare’s configuration files return duplicate rows, which more than doubled the features in the file. The file is rebuilt every few minutes and sent to every server. The bigger file went past the proxy’s limit of 200 features, and requests across the network started returning errors (Cloudflare’s account).
Google Cloud’s outage in June 2025, a separate event, had the same control loop. The controller was again automation that sends out configuration: Google’s system that copies policy records to every region. The decision rule was the same: a new policy record went to every region within seconds, with no staged rollout. Code changes did go out region by region, but the policy change did not. The feedback came the same way too: the first sign that the record was bad, a policy with blank fields, was crashes in every region at once, after every region already had the record (Google Cloud’s report).
In the Control Loop
Controller
The automation that builds and sends out configuration
Control action
Send a new configuration to every server
Decision rule
Send each new configuration to every server as soon as it is built
Process model (beliefs)
The new configuration is safe to run
Feedback
Errors from the servers, which arrive only after the servers run the configuration, and by then every server has the configuration
Before launch
STPA asks when sending a configuration to every server is unsafe: when the configuration is bad. Step 4 then asks what feedback the automation gets about a bad configuration before sending the configuration everywhere. Here the automation gets no feedback before sending. From that loss scenario, STPA derives a constraint: a new configuration must reach a few servers first, and go further only when those first servers show no errors.
A Check That Expected the Wrong Inputs
In July 2024 one CrowdStrike content update took down about 8.5 million Windows devices, by Microsoft’s estimate. The update was for CrowdStrike’s security software, which runs on each customer’s Windows machine. A new type of update listed 21 inputs, but the code on customers’ machines that reads updates supplied only 20. Before release, CrowdStrike’s Content Validator checked the update against the update type’s own list of 21 inputs, not against the code that would read the update. The validator passed the update, and Windows crashed on customers’ machines when the code tried to read a 21st input that wasn’t there (CrowdStrike’s root cause analysis).
In the Control Loop
Controller
The Content Validator, which passes updates for release
Control action
Pass an update for release
Decision rule
Pass any update that matches the update type’s own list of inputs
Process model (beliefs)
The code on customers’ machines supplies 21 inputs
Actually
The code on customers’ machines supplies 20
Feedback
None from the code that reads the update
Before launch
STPA asks when passing an update is unsafe: when the code on customers’ machines can’t read the update. Step 4 then asks what the validator’s beliefs about the code on customers’ machines are based on. Here the validator worked from the update type’s own list of 21 inputs, not from the code that reads updates. From that loss scenario, STPA derives a constraint: the validator must check each update against the inputs that the code on customers’ machines supplies.
Recovery Steps That Are Safe One at a Time
When DynamoDB’s APIs recovered, EC2’s DropletWorkflow Manager set out to re-establish leases with a very large number of servers. The work timed out before finishing, each retry queued more work, and AWS says the system entered congestive collapse (AWS’s summary).
Google Cloud’s June 2025 outage had the same control loop during recovery. The controllers were again many tasks, each recovering on its own: Google’s Service Control tasks. The control action was a restart, and the decision rule was the same as at AWS: restart right away, with no random wait. The feedback gap was the same too: each task saw only its own restart, not the other tasks restarting at the same moment, and the restarts together overloaded the database that the tasks depend on (Google Cloud’s report).
In the Control Loop
Controllers
Many tasks, each recovering on its own
Control action
Retry or restart
Decision rule
When work fails, try again (no rule to spread retries out)
Process model (beliefs)
The system has room for this task’s retry
Actually
Every other task is retrying at the same moment
Feedback
Each task sees its own failures, not the load from the other tasks
Before launch
STPA asks when a retry is unsafe: when many other tasks are retrying at the same moment. Step 4 then asks what feedback a task gets about the load from the other tasks. Here a task gets no feedback about the other tasks. From that loss scenario, STPA derives a constraint: each task must wait a random time before retrying, and new work must be held back when the waiting queue grows.
Part 3 of 6
People and Automation in the Same Loop
STPA treats the on-call engineer as a controller too. An engineer acts on what they believe about the system, and that belief comes from the feedback they get. When the feedback misleads, an engineer can take an unsafe action while every display works as designed.
Responders Who Saw an Attack
During the Cloudflare outage, the configuration file was rebuilt every five minutes and could come out good or bad each time, so errors across the network came and went. Then Cloudflare’s status page, which is hosted outside Cloudflare, went down too, by coincidence. Some responders concluded that an attacker was targeting both the network and the status page, and the team first suspected a very large DDoS attack before finding the file (Cloudflare’s account).
The Responders’ Control Loop
Controller
Incident responders
Believe: an attacker is flooding our network.
Control action Stop the bad file going out: delayed while the responders looked for an attack
Feedback
Errors across the network that came and went every five minutes
Cloudflare’s status page down too
Missing: that a bad configuration file was making the proxy fail on every server the file reached
Controlled process
Cloudflare’s network
Actually: a bad configuration file, sent out every five minutes
Unsafe control action: Responders don’t stop the bad file when every server is receiving it.
Our STPA reading of Cloudflare’s account.
STPA asks, for each feedback path, what a controller would see in each scenario. What do responders see when a bad configuration file reaches every server? Errors that look like an attack. From that loss scenario, STPA derives a constraint: responders’ displays must show which configuration each server is running, next to that server’s error rate.
A Safety Delay Nobody Was Told About
In 2021 Google’s quota rightsizer was sent incorrect feedback about how much a critical service was using, and calculated a quota far below what the service needed. As a precaution, the rightsizer waited several weeks before applying the cut, so that someone could stop the cut. Nobody was told the cut was coming, so nobody stopped the cut. When the cut took effect, a significant outage followed (Falzone and Treynor Sloss, 2024).
The Service Engineers’ Control Loop
Controller
The service’s engineers
Believe: nothing is about to change our quota.
Control action
Missing: stop the quota cut before it takes effect
Feedback
Missing: notice that a quota cut was pending, which was never sent to anyone
Controlled process
The service’s quota
Actually: a cut far below what the service uses, due in a few weeks
Unsafe control action: The engineers don’t stop a quota cut when the new quota is below what the service needs.
Drawn from Google’s account of its STPA work.
The rightsizer did what it was designed to do. What feedback do the service’s engineers get about a pending cut? None. From that loss scenario, STPA derives a constraint: anyone who can stop a pending quota cut must be told about the cut while there is still time to act.
Operators Who Step In While the System Fixes Itself
In a talk on failover between AWS availability zones, Adrian Cockcroft, then at AWS, used STPA to model the people who watch over the automation. When a zone fails, the routing automation moves traffic by itself, and operators shouldn’t touch the system. The operators’ displays lag reality by several minutes, their runbooks are out of date, and the routing automation doesn’t clearly tell the operators that everything is taken care of. Confused operators disagree about whether to act at all. They work separately, each trying to fix a different problem (Cockcroft’s talk).
The Operators’ Control Loop
Controller
Operators
Believe: something is broken and we need to fix it.
Control action Change the system during failover
Feedback
Displays that lag reality by several minutes
Missing: a clear signal that the automation has the failover in hand
Controlled process
Traffic routing during a zone failure
Actually: the automation is already moving traffic
Unsafe control action: Operators change the system when the automation is already handling the failover.
Drawn from Cockcroft’s STPA model.
Cockcroft’s recommendations act on both parts of the operators’ control loop. Making failover automatic removes the unsafe control action, because operators no longer need to step in. Testing failover continuously, so that failover stops being a once-a-year event, gives operators feedback on what a normal failover looks like on their displays.
An AI agent with write access to production is a controller too. Like the Enactor and the on-call engineer, an AI agent acts on its beliefs about the system, and the agent builds those beliefs from what it is given to read. STPA analyzes an AI agent’s control loop the same way as any other controller’s loop, and the control-loop patterns from the outages above show up for AI agents too.
A Safety Rule Given Only as an Instruction
In July 2025 Replit’s coding agent deleted the production database of an app that SaaStr’s founder was building, despite instructions to freeze the code (The Register).
In the Control Loop
Controller
Replit’s coding agent
Control action
Run commands that change the app’s database
Safety constraint
Change nothing during a code freeze
Enforced by
Only the instructions given to the agent
Before launch
STPA assigns each safety constraint to a controller that can enforce the constraint, and an instruction leaves enforcement to the agent. STPA derives a constraint: during a code freeze, the agent must have no way to change live data, whatever the agent’s instructions say.
An Approval That Depends on What the Approver Sees
Microsoft’s Azure SRE Agent can run in a review mode, where a person approves the agent’s write actions first, or in an autonomous mode, where the agent applies its write actions without waiting.
In the Control Loop
Controller
The engineer who reviews the agent’s actions
Control action
Approve or refuse a write action
Process model (beliefs)
The engineer’s belief about what the action will do to the system
Feedback
What the agent shows in its request
Before launch
STPA asks when approving an action is unsafe, and then what the engineer needs to see, and how soon, to refuse that action.
A Rule the Protocol Can’t Enforce
The Model Context Protocol is a standard that lets AI applications connect to outside tools. The protocol’s specification says that the application running an AI agent must get the user’s explicit consent before the agent uses a tool. The specification also says that the protocol itself can’t enforce that consent rule.
In the Control Loop
Controller
The application that runs the agent
Control action
Let the agent run a tool
Safety constraint
Get the user’s explicit consent first
Enforced by
Each application, because the protocol can’t enforce the consent rule
Before launch
STPA checks that every safety constraint has a controller that can enforce the constraint. Each application that uses the protocol has to be the controller that enforces the consent rule.
No Feedback on How the Algorithm Performs
Carnegie Mellon’s Software Engineering Institute ran CAST on an AI incident at a health system. The health system used a machine-learning algorithm to choose which patients to enroll in a care program for patients with complex health needs, and the patients with the highest risk scores were enrolled automatically. The algorithm used past insurance costs as a stand-in for medical need, and Black patients who were enrolled turned out to be significantly sicker than white patients with the same score. CAST found that the algorithm had a high degree of decision authority, and that the doctors, administrators, and engineers responsible for it had no feedback on how it performed in use (see Published Results).
In the Control Loop
Controllers
The doctors, administrators, and engineers responsible for the algorithm
Control action
Correct the algorithm, or limit how it is used
Algorithm’s authority
A high degree of decision authority
Feedback
None on how the algorithm performed in use
Before launch
STPA asks what feedback each controller needs. Without feedback on how the algorithm performs in use, the people responsible for the algorithm can’t tell when it needs correcting.
Part 5 of 6
Where STPA and CAST Are Already in Use
Teams in this sector already write design documents, hold production readiness reviews, and write postmortems. STPA results go into the design documents and readiness reviews. CAST results go into the postmortems. Google’s site reliability engineers already use STPA in design reviews and CAST in postmortems (Google SRE).
Most outages in this sector count against an error budget: the amount of unreliability that a service is allowed. Google’s SRE leaders write that some losses, such as privacy breaches and data loss, must never happen, so the error budget for those losses is zero (Falzone and Treynor Sloss, 2024). A postmortem comes too late for a loss like that. Google’s SRE leaders turned to STPA to find the scenarios that lead to such losses before launch.
Before Launch
Design Reviews
At Google, STPA found design flaws in a Maps system, and the team fixed them by rewriting the design document before the system was built (Google SRE Prodcast). A team designing automation like DynamoDB’s DNS Enactors could do the same: the constraints from Part 1, such as reading the record at the moment of deleting, would go into the design document and onto the readiness checklist.
After an Incident
Postmortems
On one Google Maps incident analyzed with both root-cause analysis and CAST, CAST found program, evaluation, and engineering factors that the root-cause analysis left out.
The unsafe control actions from an STPA can set the hypotheses for chaos experiments, so the faults that a team injects test the contexts that the analysis found unsafe. A Google engineer has written chaos-engineering hypotheses from unsafe control actions for AI agents that act on a bank’s systems.
STPA Effort and Results
5 weeksHow long two Google engineers worked, part-time and with support, on an STPA of a map-data publishing system. Compared afterwards with a past major incident on that system, the STPA had found every defect that the incident’s report named.
27 hoursTotal engineer time, five engineers combined and fixes included, for an STPA that found and fixed seven design flaws in a Google Maps system before the system was built.
HundredsScenarios that Google’s STPA analyses found and dealt with before any outage. Each analysis took engineer-weeks of work.
The three figures above are Google’s. Sources are under Published Results.
ROI, by our estimate: at typical engineer pay, each analysis above cost about $5,000–$40,000 (Levels.fyi). Most large firms put an hour of downtime at over $300,000 (ITIC, 2024). Every STPA described on this page found real scenarios that would lead to outages if not caught and fixed. Even if nine analyses in ten found nothing, a far worse record than any reported on this page, avoiding one outage of an hour or two would pay for all ten.
Part 6 of 6
Published Results
At Meta, Google, and Amazon
Workshop Talk
STPA at Meta: Value in Every Project So Far
Meta’s engineers applied STPA to two past incidents, in which a data ingestion platform created too many small files for the systems downstream, and found that STPA could have helped. On a new feature still in design, one unsafe control action led Meta’s engineers to the exact section of code, which the engineers confirmed was always open to that unsafe control action. Teams have since applied STPA to about five projects and found value in every one.
Chapter 12 of the second edition, “Safety Engineering for SRE with STPA and CAST,” applies STPA to the approval process for Google Maps data updates and presents CAST as a method for postmortems.
Two engineers, part-time for five weeks and with support, applied STPA to a map-data publishing system and found 56 causal scenarios across more than 30 subsystems. When the team compared the scenarios with a past major incident, the STPA had found every defect that the incident’s report named, and 7 of the 9 solutions that the incident review identified.
STPA on a redesign of the system that puts third-party road closures on Google Maps found a controller that assumed its last snapshot matched the map, so a closure that didn’t publish would never be tried again. The fix, feeding back what the map actually shows, went into the design document before the system was built. In all, five engineers found and fixed seven design flaws in about 27 hours.
For the incident that put wrong city names on the map, root-cause analysis named two causes. CAST of the wrong-city-names incident added program-management, evaluation, and engineering factors, and a changing environment.
The talk writes a chaos-engineering hypothesis for each critical unsafe control action of AI agents that act on a bank’s systems, and sets out the experiments planned to test the hypotheses.
Adrian Cockcroft, then at AWS, used STPA to model how operators and automation share control during an availability-zone failover. The analysis found displays that lag reality by several minutes, out-of-date runbooks, and operators whose beliefs about the system are out of step with the automation, and recommended automatic, continuously tested failover.
STPA and FMEA applied to a text-to-image product found hazards arising between the AI models, the models’ users, and the teams that build the models, which evaluating the AI model alone does not cover.
Carnegie Mellon’s Software Engineering Institute found a recurring set of losses and hazards across 100 incidents from the AI Incident Database. A full CAST of one incident, a health system’s algorithm for choosing patients for a high-risk care program, found that the algorithm had a high degree of decision authority while doctors, administrators, and engineers lacked feedback on how it performed in use.
Modeling the 2024 attack’s social engineering as control loops showed how a distributor’s rules could give a pressured maintainer grounds to refuse an attacker’s demands.
Security design for an AI data center facing a nation-state attacker started from the losses to prevent. The talk credits the approach with getting executives to agree on a strategy in 2 days, compared with 2 months.
SAE J3307, issued in March 2025, is an STPA standard for all industries, and its scope names site reliability engineering.
NIST’s systems security engineering guidance draws on STPA for the properties a protective control needs (SP 800-160 Volume 1) and lists STPA loss scenarios as one way to represent attack scenarios (Volume 2).
The UK National Cyber Security Centre explored STPA in cyber security case studies and concluded that systems-theoretic approaches, and STAMP in particular, should be part of its risk toolbox (NCSC talk, 2019).
Where to Go Next
Keep Learning
The introductions to the methods, the evidence for using them, and the industry pages are all free.
