In chemistry, sodium is a metal that explodes on contact with water. Chlorine is a gas that will kill you. Together they make table salt. The properties of salt are not hiding inside sodium, waiting to be discovered. They don’t exist until the interaction happens. This is not controversial. This is freshman chemistry. And it is emergence.
In geometry, four equal-length lines at right angles make a square. No individual line contains “squareness.” The square is a property of the arrangement. Rearrange the same four lines and you get a different shape with different properties, different area, different structural behavior. The emergence is in the relationship, not the components.
In music, a melody doesn’t exist in any individual note. It exists in the sequence and the intervals between them. Change the intervals, same notes, different melody, different emotional effect, different meaning. The thing that moves you isn’t a note. It’s what happens between them.
These are not exotic examples. These are things you learned before you were old enough to drive. And in each case, the emergent property is real, measurable, and useful. Salt preserves food. A square encloses an area. A melody moves people and when paired with humans, emotions can emerge to overwhelm the senses. Describing the components completely does not describe what they produce together. You can describe sodium and chlorine in isolation and still not have described the taste of salt.
This should not be a difficult concept. And yet.
In late 2025, philosopher John Heil published an article in the Institute of Art and Ideas titled “Emergence Explains Nothing and Is Bad Science.” His thesis: emergence is incoherent because competing accounts of it describe different phenomena, and therefore the concept itself is broken. Life, consciousness, and the cosmos are not “emergent.” The word, he argues, “masks our ignorance, mistaking gaps in explanation for gaps in reality.”
Heil is doing something that looks rigorous but isn’t. He’s collapsing two distinct versions of emergence into a single target and then declaring the target incoherent because the two versions don’t match. Epistemological emergence (we can’t predict the system’s behavior from knowledge of its parts) and ontological emergence (the system has properties that genuinely don’t exist at the component level) are different claims about different things. Complaining that the word covers both is like complaining that “bank” refers to both a financial institution and a riverbank, and concluding that banking doesn’t exist.
Salt doesn’t care about this distinction. Salt is an emergent result of chemical bonding. It’s a measurable physical fact that you can taste.
Philip Anderson, Nobel laureate in physics, mapped this territory in 1972. “The ability to reduce everything to simple fundamental laws does not imply the ability to start from those laws and reconstruct the universe.” That was not a philosopher speculating about whether emergence is real. That was a physicist who won the Nobel Prize telling the field that reductionism has limits, and those limits are not a failure of knowledge. They are a feature of how complex systems work.
Heil’s article is arguing against territory that Anderson mapped over fifty years ago. And to be precise about where he’s right: if you can inspect every line of the boids code, then the mechanism is doing the explaining. The word “emergence” is just pointing at it. Heil is correct that a label should not replace an explanation, and when people use “emergence” as a magic word to avoid doing the mechanistic work, that is bad science.
But that’s not how the dismissal gets used in practice. In practice, “emergence explains nothing” becomes permission to refuse to look at the system level at all. The reason it matters, the reason I’m not just letting it pass as another philosopher being wrong on the internet, is that the same dismissal Heil articulates is the one I keep running into everywhere else, wielded not as a demand for better mechanism but as a reason to ignore system-level behavior entirely.
“Emergence explains nothing” is the same intellectual move as “LLMs are just statistics.” Both look at a system, refuse to engage with it at the level where the interesting behavior actually lives, and declare that the interesting behavior doesn’t exist because it’s not visible at the level they prefer. That’s not rigor. It’s a jurisdiction claim dressed up in a trench coat up to no good.
And it’s costing us.
Emergence doesn’t just produce new properties. It produces surprise.
In 1987, Craig Reynolds wrote a program called Boids. Three rules: keep some distance from your neighbors, match their heading, steer toward the center of the group. No rule says “flock.” No rule describes the shape of a flock, the way it splits around obstacles, the way it reforms afterward. No single boid has any concept of flocking. The flocking behavior doesn’t exist at the individual level. It is entirely a product of the interaction.
And here’s what makes boids lethal to Heil’s argument: you can inspect every line of the code. There is no gap in explanation. There is no hidden ignorance. You know every rule, every parameter, every interaction radius. The system is fully transparent. And you still cannot predict the specific shape the flock will take at any given moment without running it. The surprise isn’t a gap in your knowledge. It’s a structural property of how emergence works. The rules are simple. The state space is not.
This isn’t limited to simulations. The STARFLAG project studied real starling murmurations with quantitative methods and confirmed the same principle: local interaction rules producing collective behavior that doesn’t exist at the individual level. The mechanistic work that Heil says is missing already exists. It exists in code you can read, in peer-reviewed research you can cite, and in the sky above Rome on winter evenings.
Conway’s Game of Life makes the same point from a different angle. Four rules applied to cells on a grid. Fully deterministic. Fully inspectable. And the system is computationally universal, meaning that for some questions there is no shortcut around running the process. You cannot predict the outcome without running the simulation. The surprise is baked into the math.
The pattern holds at every scale. Two systems interact. The interaction produces something neither system contains. The more interacting components, the more permutations. The more permutations, the more surprise. This isn’t mysticism. It’s combinatorics.
And combinatorics doesn’t care about your intentions. Put a thousand reasonable people together with the right catalyst and the collective behavior can turn catastrophic. Nobody walks into a crowd carrying the emergent outcome. It doesn’t exist until the interaction produces it. Emergence doesn’t have a preference. Salt is useful. A flock is beautiful. A mob is devastating. The mechanism doesn’t change based on whether we like the outcome. And that’s exactly why dismissing emergence as “bad science” is dangerous. If you refuse to accept that system-level properties are real, you lose the ability to see them coming.
I know, because I watched it happen.
In December 2022, ten days after ChatGPT launched, I posted a warning to several digital archival communities. The night before, my wife had demonstrated that publicly accessible AI tools could produce fully synthetic documentary video convincing enough to contaminate an archive. Not as a hypothetical. As a finished product, built from over a dozen services, sitting on our kitchen table. I told the archivists: everything coming into your collections from this point forward should be considered suspect. The contamination wouldn’t require bad actors. It would come from well-intentioned people casually enhancing, colorizing, and compositing content with synthetic elements, and the results would blend into collections indistinguishable from source material. The emergent contamination wouldn’t come from the tools or the people. It would come from the interaction between them at a scale nobody was tracking.
The response was hostile enough that I retracted the post. Three and a half years later, the problem I described is so widely acknowledged it barely qualifies as a take. I’ve written about it in detail elsewhere. The short version is: emergence doesn’t wait for you to accept it before it arrives.
Now, remember the melody.
A melody emerges from the interaction of notes. Pair that melody with a human being and something new emerges again: an emotional response that overwhelms the senses. That response doesn’t exist in the melody. It doesn’t exist in the human. It’s a second layer of emergence built on top of the first. Emergence compounds.
So what happens when you connect billions of humans to AI systems?
Neither is particularly surprising on its own. Humans are predictable enough to be identifiable. That’s what stylometry measures. AI systems fall into patterns too. They have basins, biases, default behaviors that repeat across contexts. Neither component is high-entropy.
But the interaction space between them is combinatorially vast. Each human brings different context, different intent, different expertise, different questions. Each session responds differently depending on what came before. Neither side is surprising alone. The combination is. And the number of permutations in that interaction space is beyond anyone’s ability to map, including the people who built the systems.
Every day, millions of people sit down with an AI tool and produce something that neither of them could have produced alone. An idea that gels. A solution not present as a completed artifact in either the user or the model alone, but one that emerged from the conversation between the two. Most of the time it’s mundane. Sometimes it’s genuinely novel. And nobody has a framework that’s keeping up with what’s happening at that scale.
That’s the thing about emergence. It doesn’t announce itself. You don’t cross a threshold and hear a bell. You cross it because each individual interaction looks ordinary, and the extraordinary part is what the system is doing in aggregate, where nobody is watching.
In 1965, a mathematician named I.J. Good published a paper called “Speculations Concerning the First Ultraintelligent Machine.” Good had worked with Alan Turing at Bletchley Park. He was a cryptographer, a Bayesian statistician, and a careful thinker. He was also, it turns out, describing emergence compounding on itself before most people had a word for it.
Good’s argument was simple. An ultraintelligent machine could design a better machine. That better machine could design a better one still. The result would be an “intelligence explosion,” and the intelligence of man would be left far behind. Vernor Vinge later popularized this idea as the “technological singularity” in 1993, but Good had the mechanism first, and his version is more precise. An intelligence explosion is emergence in a feedback loop. Each cycle’s output becomes the next cycle’s input. The compounding is the point.
Good also saw the other half of the problem, in the same breath: “Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.”
Provided.
We built the largest emergent system in human history, and most of the people using it every day don’t have a word for what’s happening. The people who do have the word are publishing articles about how the word doesn’t mean anything. And the institutions that understand the word just fine are using it to decide who gets to participate and who gets the placebo.
Good saw it coming in 1965. He wrote one sentence that contained both the promise and the warning. Sixty-one years later, the industry is still fumbling them separately.
And it’s about to get worse. Or rather, it already did, and then it got interesting.
On June 9, 2026, Anthropic released Claude Fable 5, the first Mythos-class AI model available to the general public. It is, by most benchmarks, the most capable AI system ever made widely accessible. Within hours, the AI community found something in the model’s 319-page system card that the company had disclosed but apparently not expected anyone to read closely.
When Fable 5 detected that you were asking about cutting-edge AI development work, building the kind of infrastructure used to train large models, it silently downgraded its own responses. It didn’t refuse. It didn’t tell you it couldn’t help. It didn’t redirect you to a less capable model with a notification, which is what it does for cybersecurity and biology queries. For AI research questions, it quietly gave you a worse answer and didn’t mention that it was doing so. The system card said this explicitly: “not visible to the user.”
Think about that for a moment. The model responds. It looks helpful. But the quality of the response has been deliberately degraded without your knowledge. You asked a question and received a placebo.
The backlash was immediate and broad. Open-source researchers, AI safety experts, former Anthropic employees, and policy advisors all pushed back sharply. Dean Ball, a senior fellow at the Foundation for American Innovation, called it “secret sabotage” and said it “massively and profoundly raises the status of the argument that AI safety has been hype to justify monopolistic behavior by labs.” Jeremy Howard at Fast AI pointed to the asymmetry: Anthropic keeps full capabilities for its own researchers while throttling everyone else.
Within 48 hours, Anthropic reversed course. “We’re changing Fable 5’s safeguards for frontier LLM development to make them visible,” the company said. “We made the wrong tradeoff and we apologize for not getting the balance right.”
Good. They moved from sabotage to boundary. That’s the right correction. But the essay isn’t about whether Anthropic fixed the problem. It’s about what the instinct reveals.
Anthropic’s terms of service already prohibit using Claude to build a competing model. That’s a contract. You agree to it when you sign up. If you violate it, they can enforce it. That’s how contracts work. Plenty of software licenses define prohibited uses. They don’t silently corrupt your output when they suspect you might violate them. They don’t swap out the compiler for an inferior version without telling you. They state the boundary in the license and trust the legal system to handle violations.
One approach is a boundary. The other is sabotage. Anthropic tried the second one first. The fact that they corrected course when called on it is to their credit. The fact that the instinct was to degrade silently rather than refuse openly tells you something about how the institution thinks about the emergent capabilities its own tool produces.
To be fair, the instinct wasn’t purely monopolistic. The labs are living inside Good’s feedback loop. They know what recursive self-improvement looks like because they’re watching it happen on their own infrastructure. An AI helping a researcher build a better AI is the literal mechanism of the intelligence explosion. Some of that silent degradation was panic, not just power. They didn’t know how to openly refuse a complex, multi-step research prompt without the model being jailbroken around the refusal, so they implemented targeted, invisible interventions that reduced the model’s effectiveness on those queries without notifying the user. It is still the wrong approach, but it is rooted as much in terror of the very emergence this essay describes as it is in competitive positioning.
The correction tells you they heard the distinction once it was made painfully public. The instinct tells you it was not obvious to them before then. Both matter.
They are not denying that emergence is real. They are not making Heil’s error. They understand perfectly well that the interaction between their model and a capable researcher produces emergent capabilities that neither party contains alone. They understand it so well that their first instinct was to selectively throttle it. Not for everyone. Just for people working on certain problems. The model still works fine if you’re writing marketing copy or debugging a web application. The same intervention can be motivated by safety, institutional power, and competitive fear at once. That is precisely why transparency matters.
The correction matters. The instinct matters more. And it is the institutional version of the same move that has been made every time emergence threatens an established order. Neural networks were narrowed into a funding problem once before. A critique became a gate. Heil doesn’t deny that salt has properties sodium and chlorine don’t have. He redefines the philosophical terms until the concept sounds incoherent. Anthropic didn’t deny that their model produces emergent capabilities in the hands of researchers. For 48 hours, they just made sure certain researchers got the version that didn’t.
Good’s “provided” clause lands differently now. “The first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.” He imagined the control problem as a relationship between humans and a machine. What he didn’t imagine is that the machine would be docile, and the control problem would be between humans deciding which other humans get to use it.
The emergence is real. The interaction is producing capabilities beyond what either side contains alone. The surprise is structural and the permutations are beyond mapping. The question was never whether this was happening. The question is who gets to participate, and whether the people making that decision will do so openly or in the dark.
Anthropic, to their credit, chose the light. Eventually. After being caught.
I.J. Good, working with the tools of 1965, saw both halves of the problem in a single sentence. In 2025, a philosopher published an article saying emergence was bad science. In 2026, the most capable AI lab on earth demonstrated that they know exactly how real it is by trying to control who benefits from it. The philosopher may be right that the word shouldn’t replace mechanism. But “emergence is bad science” as a headline gives people permission to stop looking at the system level entirely, and the lab just proved that the system level is exactly where the action is. And the rest of us are living in the gap between those two positions, watching the institutions figure out in real time whether they’re going to be honest about what they’ve built.
Emergence isn’t magic. It never was. It’s what happens when systems interact, and it’s been happening since sodium met chlorine. The only thing that’s new is the scale, the speed, and the number of institutions still deciding whether to acknowledge it or manage it by telling the truth in a place they assume nobody will look.
The last invention that man need ever make. Provided.