Nobody Knows How the Whole System Works (And That’s Been True for Decades)

There’s a classic tech interview question: “What happens when you type a URL into your browser?” You can go deep—HTTP, DNS, TCP, IP, ARP, radio modulation, transistor physics. But here’s the uncomfortable truth: nobody actually understands all the layers.

A fascinating conversation is happening on LinkedIn about AI, system complexity, and the limits of human understanding. And the conclusion might surprise you: the “AI making things too complex to understand” crisis has actually been our reality for thirty years.

The Core Insight

Simon Wardley is worried: AI lets us build things without understanding how they work.

Adam Jacob counters: AI is changing development, yes, but the benefits exceed the risks.

Bruce Perens goes deeper: This has already happened. Modern CPUs and operating systems are so complex that most developers already work with fundamentally incorrect mental models.

And MIT engineering professor Louis Bucciarelli, writing back in 1994, nailed it: “Does anyone know how their telephone works?”

He lists what “knowing” might mean: the physics of diaphragms and magnetic fields, routing algorithms, echo suppression, satellite transmission, corporate financing, regulatory frameworks. Nobody understands all of it. Nobody ever has.

Why This Matters

The technical interview equivalent is revealing. Consider everything that happens when you hit Enter on that URL:

• The interrupt that fires in your OS kernel
• Which 802.11ax modulation scheme your WiFi is using
• The difference between QAM and QPSK
• The ARM processor’s relaxed memory model
• JVM garbage collection
• How field effect transistors implement digital logic

Can you explain all of these? Neither can anyone else.

Brendan Gregg (legendary performance engineer) used to interview by pushing candidates to the limits of their knowledge. His goal wasn’t to find someone who knew everything—that person doesn’t exist. He wanted to see if they’d admit “I don’t know” or bluff.

The implication: our industry has always been built on partial understanding. AI makes it worse, but it’s a difference of degree, not kind.

Key Takeaways

• “Magic” as an epithet exists for a reason. Frameworks that hide complexity (hello, Rails) make building easier but understanding harder. This trade-off predates AI.

• Mental models are already wrong. If you think you understand how your computer works, you’re likely incorrect in fundamental ways. You’ve just built useful-enough abstractions.

• Complexity is inherent to modern systems. Telephony, the internet, modern operating systems—these are too complex for any individual to fully comprehend. This was true in 1994.

• AI accelerates an existing trend. Yes, AI lets us build more while understanding less. But we crossed the “no one understands the whole system” threshold decades ago.

• Bluffing vs. admitting limits matters. The healthiest response to complexity is acknowledging what you don’t know, not pretending expertise.

Looking Ahead

Here’s the paradox: all four perspectives are correct.

Wardley is right that building without understanding is dangerous.

Jacob is right that AI’s benefits outweigh its risks.

Perens is right that developers already work with broken mental models.

Bucciarelli is right that complex systems exceed individual comprehension.

The takeaway isn’t that we should stop worrying about AI-induced complexity. It’s that we should recognize this as an escalation of an existing condition, not a new category of problem. We’ve been building on foundations we don’t fully understand for a long time.

The question isn’t “how do we ensure everyone understands the whole system?” That ship sailed. The question is: how do we build reliable systems from partial understanding, and how do we maintain enough experts in each layer to debug when things break?

AI makes this harder. But it’s the same game we’ve been playing since the first person built something on top of someone else’s abstraction.

Based on analysis of “Nobody knows how the whole system works” from surfingcomplexity.blog