## Overview Charles Perrow's Normal Accidents framework holds that in tightly coupled, complex systems, catastrophic failures aren't bugs — they're structural features. Barrat applies this to AI: as intelligent systems grow more complex and tightly coupled with infrastructure, "normal accidents" become inevitable and increasingly catastrophic. ## Core Framework Perrow identifies two properties that guarantee catastrophic failure: 1. **Tight coupling** — components have immediate, serious impact on each other with no buffer 2. **Interactive complexity** — failures propagate through hidden paths that designers cannot anticipate When both are present, accidents are "not only unexpected, but are incomprehensible for some critical period of time." The system defeats its own safety devices through paths hidden in its complexity. **Empirical evidence**: Three Mile Island (1979) — 4 simple failures cascaded into near-meltdown. Knight Capital (2012) — badly programmed algorithm lost $440M in 30 minutes. ## Cross-Domain Applications **Software Engineering**: Distributed microservices are tightly coupled through shared state and dependency chains. Outages cascade through "hidden paths" — exactly Perrow's pattern. **AI Systems**: Self-improving AI combines both properties at maximum intensity — tight coupling (modifies its own substrate) and interactive complexity (emergent behavior from self-modification). Normal accidents in AI aren't recoverable because the system evolves faster than human understanding. ## Critical Analysis Perrow's framework explains *why* accidents happen but offers limited guidance on *prevention* — loose coupling and reduced complexity are the prescriptions, but AI development trends toward more of both. ## Related Concepts - [[Convergent Instrumental Goals]] — the drives that make AI systems increasingly complex and tightly coupled - [[Intelligence Explosion as Inscrutability Cascade]] — inscrutability is the mechanism through which normal accidents become incomprehensible ## Source - [[Four Basic Drives]] — *Our Final Invention* Ch.6 (James Barrat), pp. 77-92 - Charles Perrow, *Normal Accidents: Living with High-Risk Technologies* (1984)