AI Can Write Production-Ready Code, but No One Dares Let It Run a Distributed Cluster Alone
Code's solution space aligns with training data distribution, so AI sprints through it; operations must handle the real world beyond training data—this divide determines where AI hits its next bottleneck.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
Observability Pays for Reliability and Predictability
Companies spend 5% to 20% of their infrastructure budget on observability, but what they really want to buy is reliability and predictable spending. In the past, higher spending correlated clearly with higher reliability, but after migrating to microservices, data collection volume at each layer exploded—spending growth now outpaces business output growth, and marginal reliability gains have flattened. This is the real reason people think observability is too expensive.
— JadAI Didn't Flatten Engineer Gaps. It Widened Them.
Jad observed that truly great engineers turn AI around to train themselves—learning how to dialogue with it, how to decompose tasks, when to spawn sub-agents. The result isn't that mediocre engineers get better; great engineers get better, and the gap between 'competent' and 'excellent' on the team goes straight to ten times wider. He attributes this to complexity: the more complexity a human mind can handle, the finer-grained the task decomposition they can hand to AI—and that's precisely what AI can't do yet.
— JadOne Engineer, Ten Days, Six Months of Former Work
Jad recently completed a project solo with AI help: auto-promotion for a data lake—shredding each field in logs into independently queryable columns, involving figuring out how the metadata system submits and how multiple writers coordinate. A decade ago, this complexity would have taken one engineer six months; this time it took about ten days—a ten to twenty-fold efficiency gain, and it's real production code running live, not a demo.
— JadLead with Anomalies, Not Backward Root-Cause Inference
Most AI SRE systems on the market take a lagging metric and then work backward to infer root cause in a complex system—Jad thinks that path is hard to travel. Grepper does the opposite: first scan massive logs for novel patterns or early warning signals appearing for the first time in an environment (like connection exhaustion or memory limits), then feed that signal to AI reasoning. Real cases already exist in internal and customer environments where ignoring such signals led to actual production incidents.
— JadQuery Optimization's Near. Full Cluster Takeover Isn't.
Asked how soon AI can replace humans at query optimization, Jad answers 'very soon.' Asked whether AI can operate a distributed cluster independently, the way a systems expert would, and absorb its failures, his answer is 'not yet.' Both agree the complexity order is different: code's solution space matches training data distribution, but ops must handle real-world states beyond training data, with long tails and edge cases.
— JadSecurity Problems Are One Order Harder Than Operations
Martin points out that operations faces systems stumbling into unlucky edge states by chance; security adds an active adversary hunting for flaws the system can't defend. Jad agrees this makes security harder than ordinary ops. Even though Grepper applies similar logic to security signals (anomalies in statistical features), it still requires AI to generate candidate causal hypotheses and verify them one by one—not deliver answers outright.
— MartinSelf-Driving Taught Us: Real-World Economics Is Slow
Martin reminds us that when Sebastian Thrun won the DARPA challenge at Stanford, everyone thought autonomous driving was imminent—then came twenty years and $170 billion in spending, and the unit economics still can't match Uber. He uses this to show that whenever the solution space equals the real-world physical distribution, AI reaching economic breakeven takes far longer and costs far more than problems where the solution space equals training-data distribution—like code.
— MartinCustomers Buy Savings, Not AI Stories
When Grepper pitched 'AI finds and fixes problems before they crash your system,' the customer interrupted: 'Wait—you said you can cut our Datadog bill by 90%? That I'll buy right now; never mind the AI part.' Jad's take: the value of code-focused AI is easy to price—engineer efficiency is already a big cost line; ops-focused AI is harder for customers to price directly, and even if it's more useful, it's much harder to sell.
— JadIn their own words · checked verbatim
AI can now write remarkable amounts of production code. But writing software and running it are two very different problems.
if you're a good engineer, you're just a better good engineer. It's not like there's some like bad engineers all of a sudden became good engineers.
Jad13:16
The question I think is maybe about complexity. And like, to what extent can you take the things that you've described that are operational and like that happen in production and convert them into something that an AI can reason about.
Jad26:37
the dirty secret for the longest time of running like a distributed cluster was just human beings managing it.
Martin32:45
there's like this, there's like the AI, the world that the AI lives in, which is all digital and like whatever it is, the training data, the in-distribution world, and there's the actual world. And like, those things are not necessarily the same.
Martin52:12
it took 20 years and $170 billion of investment to get to a worse unit economics than Uber, right?
Martin54:14
Roll back. You said you can save 90% of our data dog bills? Yeah. That's what I want to buy. But the AI thing, you know, like, no, no, no, no. That's what we want to buy right now.
Jad57:16
Figures
| Share of infrastructure spending on observability | 5% to 20% | 7:07 |
| Data/telemetry cost reduction from Grepper | 90% | 9:12 |
| Data lake auto-promotion project timeline (AI-assisted, one engineer) | About 10 days (previously about 6 months) | 23:34 |
| Concurrent AI agent windows running simultaneously | 3–4 | 21:34 |
Glossary
- Semantics vs. syntax
- Syntax is how code is written; semantics is how the machine actually executes it.
- AI SRE
- Using AI to replace manual troubleshooting and incident response in production systems.
- Auto-promotion (shredding)
- Converting individual fields from logs into independently queryable columns for improved query efficiency.
- Chaos engineering
- Deliberately injecting failures to test system resilience in operations.
- WSL
- Windows Subsystem for Linux—a compatibility layer for running Linux environments directly on Windows.
How to listen
Engineers focused on AI engineering efficiency and ops automation boundaries, technical VCs, and infrastructure founders trying to predict where AI's next ceiling is.
Skip the opening small talk about graduate school and smoking motorcycles—it doesn't affect the technical discussion that follows.