The whole is more (and less) than the sum of its parts
A reflection on specialization, the big picture, and everything lost when we take things apart.
I was eight years old when I took apart the household alarm clock to figure out how it worked. The result: forty-two pieces on the table, each perfectly understandable, and none of them telling the time. My father, understandably, was not amused.
Repeated over several centuries on a planetary scale, this scene is pretty much the history of modern thought. Then came AI, bearing good news and bad. The good news: it produces the pieces for you, polished, in four seconds. The bad news: still no one is telling the time. And in the meantime, you have forgotten how to put it back together.
The contract we signed without reading
A few centuries ago, we struck a deal with ourselves: divide every problem into small enough pieces that we can understand them. Descartes turned it into a method, Adam Smith into a pin factory, Frederick Taylor into a stopwatch. And it worked spectacularly well. Medicine, microchips, airplanes, vaccines: almost everything that makes our lives comfortable comes from someone who spent twenty years on a tiny fraction of something.
Yet contracts always have fine print, and this one roughly read: you gain competence, and in return no one is responsible for the whole anymore. We didn't read it, because the early decades of payoffs were too great. Today the bill arrives, and with AI it arrives with interest.
Where problems live
Systems thinking was born in the twentieth century among biologists, engineers, and mathematicians who rarely spoke to one another (von Bertalanffy, Wiener, Forrester, and later Donella Meadows), yet they all argued the same thing: serious problems do not live inside the parts; they live between the parts. In the interfaces, in handoffs, where department A ends, department B begins, and in between lies no-man's-land.
When a hospital malfunctions, it is rarely because cardiologists do not understand hearts: it is because cardiology, radiology, and administration each optimize their own metric, and the patient is the only one who has to cross it all. When a city chokes in traffic, every individual piece (a parking lot, a road, a shopping mall) makes sense and often works fine. The disaster emerges from the sum, without anyone having decided it.
We all know the feeling: 'No one is at fault, yet it doesn’t work.' That, right there, is a system.
Russell Ackoff loved a thought experiment: take the best engine in the world, the best transmission, the best brakes, each from a different car, and assemble them. You don't get the best car; you get a very expensive pile of junk, because the parts were never designed to talk to each other. Improving each part separately can destroy the whole.
And the world is full of real-life versions of this game. In 1958 China, war was waged on sparrows, accused of eating seeds; but they also ate insects, locusts thrived, and the agricultural collapse contributed to a terrible famine (alongside many other decisions, because complex systems never have a single cause, which is precisely the point). At Wells Fargo, sales targets so aggressive they were divorced from reality drove employees to open millions of unauthorized accounts; the scandal erupted in 2016 and concluded in 2020 with a $3 billion settlement. Incentives are a way of speaking to a system, and systems reply literally and poorly.
None of these are stories of stupidity. They are stories of local, brilliant intelligence applied in the wrong place.
The bill, 2025 edition
Now look at AI through these eyes: it is the finest specialist humanity has ever built. It knows everything about every single piece, and it has never seen the clock.
In 2025, the MIT Media Lab released a preprint with a telling title: Your Brain on ChatGPT: Accumulation of Cognitive Debt. Fifty-four participants wearing EEG caps, writing essays in three ways: with ChatGPT, with Google, and unassisted. Those using ChatGPT exhibited the weakest brain connectivity and struggled to recall what they had written only minutes before.
'Cognitive debt': borrowing future thinking capacity in exchange for present convenience.
Before anyone panics, some hygiene: it is a preprint, it has not undergone peer review, the sample is small (only eighteen participants completed the final session), and no one can honestly claim that AI 'damages the brain.' Anyone writing that in a headline is in a rush, or has something to sell.
Yet inside the study lies a detail far more interesting than the alarm. Participants who began on their own and only then used AI regained robust brain activity. And in other studies, students who asked for ready-made answers fared worse once the tool was removed, while those who asked for hints performed on par with those who never used it. Same tool, opposite effects. The problem is the architecture of the relationship between the machine and the human using it.
In other words, once again, a system. Cognitive debt shares all of its features: a stock that accumulates, a delay (you feel nothing until you must manage without the tool), and a self-reinforcing loop, where less effort leads to less competence, which leads to greater dependence. It works like an adjustable-rate mortgage: everything is fine until you have to refinance.
Aviation has known this for decades. It is called automation complacency: pilots who rely too heavily on the autopilot lose manual dexterity and vigilance. The response was not to abolish automation, which would have been foolish: it was to design the human-machine system, with explicit rules on when to fly manually and when not to. No one blamed the autopilot. Nor the pilot.
Three ideas that seem technical but are philosophy
The important things lie in between. No single water molecule is wet, no single neuron is conscious: the most interesting properties of the world emerge from relationships, not building blocks. And no generated paragraph is an idea: the idea lies in how the paragraphs hold together, and that is precisely the work we are unlearning how to do.
Looking for a culprit is laziness. Ordinary thought runs in straight lines: A causes B, therefore A is to blame. In systems, B feeds back onto A, C alters both, and the structure produces its behavior regardless of who occupies it. Put great people in a bad system and the system wins. It isn't the fault of AI, and it isn't yours either: it is that everything—from deadlines to metrics to urgency—pressures you to accept the first answer. Instead of hunting for the villain, you examine the design.
The observer is part of the scene. The boundaries of a system are not inscribed in nature: they are drawn by whoever is looking. Where does 'the task' end and 'the person who ought to learn something by doing it' begin? Every map is a choice, and every choice leaves something out. 'The map is not the territory,' Korzybski noted; and George Box, more wryly: all models are wrong, but some are useful.
These intuitions carry an ancient ring: we find them in Heraclitus, in Taoism, in all traditions that view the world as a web of relationships rather than a storehouse of objects. Perhaps the West, having chosen to take the clock apart, had to wait three centuries to notice what it had stopped looking at.
Leverage points, and where we look for them
Donella Meadows, in a 1999 essay that became legendary, wrote that complex systems contain leverage points where a small shift can yield immense change, and that we almost always search for them in the wrong place. The weakest levers are numbers: budgets, quotas, tax rates. Stronger are information flows (who sees what) and the rules of the game. Higher still stand the goals the system pursues, and at the summit lies the paradigm—the story a society tells itself.
Apply this to our situation. Measuring 'hours saved' or 'output per person' is the weakest lever: the number. Deciding what to never delegate is a rule. Asking whether we want people who produce more documents or people who understand more is a goal. And the paradigm? The one claiming that thinking is a cost to eliminate, rather than the reason we exist.
A company that rewards only speed is paying its employees to accumulate debt, and then acts surprised when no one is left capable of discerning whether the machine's answer is wrong.
The final irony
Systems thinking can easily become yet another insular specialty: a jargon (loops, stocks, leverage points) that divides initiates from outsiders, a way of chanting 'complexity' to avoid getting one's hands dirty with facts. There are consultants who draw gorgeous diagrams with arrows everywhere and have never repaired a thing.
And now AI generates those diagrams in four seconds. A causal loop diagram that no one actually thought through is the purest form of cognitive debt: the map without the journey, the system without ever having lived in it.
This is why the rule is straightforward and slightly unpleasant, because it demands effort: zoom in, zoom out. First, you think, with your own mind, even clumsily. Then you summon the machine. Then you rewrite in your own words, getting your hands dirty in the details. Finally, you ask where that piece fits into all the rest. The first and last steps are the ones no one can do for you, and they happen to be the very steps the system tempts you to skip.
The big picture without depth is idle chatter; depth without the big picture is blind efficiency. Systems thinking, at its core, is the only discipline that cannot be delegated, because the whole, by definition, resides in no single piece: not even the generated ones.
You can have all the pieces you want. But telling the time, for now, is still up to you.
Further reading
Donella Meadows, Thinking in Systems and Leverage Points: Places to Intervene in a System; Peter Senge, The Fifth Discipline; Russell Ackoff, Re-creating the Corporation; Ludwig von Bertalanffy, General System Theory; Nataliya Kosmyna et al., Your Brain on ChatGPT (preprint, 2025).
More writing in Debug dei Desideri on Substack.