In-House Silicon Isn't About Saving Money, It's About Escaping Nvidia's Generational Lock-In
The real motive behind hyperscaler in-house chips isn't cost savings but control: avoiding being held hostage to Nvidia's generational cadence, while using scale to compress networking and compute costs. The Broadcom tax is cheaper than the Nvidia tax, but only the biggest players can afford to play.
The argument · tap a timestamp to hear it
The first motive for in-house chips is control, not cost
The first reason hyperscalers build their own chips isn't to save money, it's control. They don't want to hand something business-critical entirely to Nvidia. In-house silicon lets them control their own vertical integration, run their own optimizations, decide the direction of those optimizations, and avoid long-term purchasing commitments to a single supplier. Nvidia's chips change substantially each generation, and buyers must commit to large orders far in advance, which constitutes strategic risk. In-house silicon is fundamentally an active grip on the supply chain and product roadmap.
Meta tore down a $1 billion data center under construction
Chip generations change so much that Meta once tore down and rebuilt a data center that was already under construction. About $1 billion had already been invested, but the chip change forced a complete redesign of the data center architecture. This is an extreme case: on a general-purpose chip roadmap, the buyer can only follow passively. If you control the silicon design yourself, you know the chip specs in advance and can avoid this kind of tear-it-all-down waste. This example turns "uncontrollable generations" into the core argument for in-house chips.
Google keeps TPUs for itself; Amazon rents its chips out
Google is the most typical case of in-house silicon: TPU has gone through many generations, a large amount of infrastructure runs on its own silicon, and yet it still buys a lot of Nvidia. The difference between Google and Amazon is this: Google's TPUs are mainly for internal workloads, with the whole system designed around internal needs; Amazon turns its in-house chips into externally rentable instances, a different business model. Google also has top-tier large models like Gemini, and TPUs carry those too. There are already rumors that OpenAI will use TPUs in the future. So even when both build in-house, the profit model and the customer served are completely different.
In-house CPUs are almost all Arm, not x86
Beyond AI accelerators, hyperscalers have also started building their own CPUs. The logic is the same as with accelerators: specialize a general-purpose CPU for their own workloads. Almost all in-house CPUs are Arm architecture. Arm's data center opportunity suddenly exploded, driven precisely by hyperscalers. Unlike the traditional x86 general-purpose chip model, these players want full control from instruction set to silicon. Physical design companies like Broadcom and Marvell take on the concrete implementation, while Arm provides compute subsystems (CSS), forming a new chain different from the Nvidia supply chain.
However expensive the Broadcom tax is, it's cheaper than the Nvidia tax
Can in-house chips save money? The key is who does it. Hyperscalers don't deal directly with foundries; design companies like Broadcom interface with TSMC. Because Broadcom holds multiple customers, it can push prices down, so even paying the "Broadcom tax" is cheaper than the "Nvidia tax." This is a counterintuitive conclusion: Nvidia's margins on AI chips are extremely high, and going in-house through a design company can squeeze out that premium. But it also means hyperscalers haven't fully disintermediated — they've just swapped in a cheaper middleman.
Amazon's in-house effort started with networking, not AI
Amazon's in-house effort started with networking, not AI. Early on, every server had to be fitted with a network chip and then connected to a top-of-rack switch, but network utilization wasn't high. Amazon first implemented Nitro on FPGAs, letting multiple CPUs share one network interface; later it decided to replace FPGAs with ASICs, and so acquired the chip company Annapurna. The root of this decision was scale: with a large number of microservice instances, network activity rates aren't enough to justify one network chip per CPU, and compressing network connections saves enormous cost. Nitro later evolved into an elastic fabric architecture and spawned chips like Graviton and Trainium.
Without a million servers, don't even think about in-house chips
Why not build your own chips? Because the scale is too large. For example: if you have 1.2 million servers and compress network connections 4:1, you only need 300,000 network adapters, saving 900,000 adapters plus their switches and infrastructure. The money saved is enough to fund building a CPU or AI chip in-house. This explains why only the biggest players can do this — scale determines the break-even point for in-house silicon. Ordinary companies don't have enough deployment volume to amortize R&D costs, so they can only buy general-purpose chips.
10,000 servers serving 5 million people: here's the math
Humans struggle to picture hyperscale numbers. Here's a conversion: serving 5 million people requires 10,000 servers; the US has about 300 million to 330 million people, so it would need 60 warehouses of that scale, i.e. 600,000 servers; and that's just the US. Global population is about 8 billion-plus, and even if per-capita demand in other countries is lower, the total is still enormous. It's precisely this order of magnitude that lets every specialized optimization of in-house chips turn into real returns across a large-scale deployment, and it forms the economic basis for hyperscalers going in-house.
In their own words · checked verbatim
they don't want to be beholden entirely to Nvidia for something that is so, so important to their business, right?
Facebook um demolished a data center they were in the process of building because the chips had changed so much gen on gen that they had to redesign the architecture of their deployment.
A general chip has to be general for everyone.
It turns out Broadcom is cheaper as weird as it is to say.
having your own in-house silicon design team can be really beneficial if you want to optimize every cent.
So, if you turn around and say, "Well, I've got 1.2 million servers deployed." Like yeah, I can't picture that number.
if you can identify patterns, you can increase watch time on your platform.
Figures
| AI accelerator market share (including TPU) | about 85:15 | 11:09 |
| Annapurna acquisition timing | around 2015-2016 | 15:12 |
| Example deployment scale | 1.2 million servers | 18:13 |
| Network compression ratio | 4:1 | 18:13 |
| Number of warehouses corresponding to US population | 60 | 27:17 |
Glossary
- TPU / Tensor Processing Unit
- A specialized chip Google designed to accelerate neural network training/inference.
- Nitro / (AWS network virtualization hardware)
- AWS's in-house networking and I/O offload hardware, initially FPGA-based and later moved to ASIC, letting multiple CPUs share a network interface.
- RISC-V / open-source instruction set architecture
- An instruction set any company can use for free to design processors, seen as a potential alternative to Arm.
- Arm CSS / Arm Compute Subsystem
- A pre-integrated compute subsystem Arm provides, containing CPU cores and interconnect, shortening the custom chip design cycle.
How to listen
Founders and investors watching the compute supply chain, cloud infrastructure engineers, and chip industry people who want to understand Nvidia's competitive landscape.
The intro and the Arteris ad starting at 14:10 can be skipped; the rest is high density.