AI Chip Shipments Will Jump an Order of Magnitude: From a Million to 15 Million
Broadcom's president reveals an AI chip roadmap: from a million units to 15 million, with the 'beast' being the largest package of 8 stackable compute dies plus 16 HBM modules, and four of the five frontier labs are collaborating.
The argument · tap a timestamp to hear it
Custom chip makers still outsource the final step
Google, Meta, Amazon, and some frontier model companies can have their own chip design teams, define their own architecture and interconnect, but ultimately they still have to 'get the thing built': dealing with foundries and development kits, managing the timing and latency of connections between transistors. Broadcom is the largest of these fee-based back-end design service providers. Clients come to it not just for design capability but also for a batch of ready-made IP including high-speed I/O, high-speed compute, memory controllers, and more.
— Ian CutressThe most tight-lipped chip company suddenly starts talking
Broadcom is a mystery to the outside world: it has almost no PR department and rarely discloses proactively. Occasionally it releases a Tomahawk networking chip or chats with Patrick Kennedy on ServeTheHome. Over the past year, president Charlie has rarely stood on stage at launch events to talk about what Broadcom is doing and why it has expanded its customer base from traditional networking chips to custom AI chips. The chart showing three tiers of chips is the clue.
— Ian CutressToday's AI servers are just the smallest of the three tiers
Of the three tiers Charlie showed, the leftmost tier is already shipping in hundreds of thousands of systems, about the size of this Cisco chip on the desk: a central compute die surrounded by four to six HBM modules, packaged using something like TSMC's CoWoS. Broadcom says that over the past year, it has shipped more than one million units of this type of chip for customers. This tier is the reality of today's AI servers.
— Ian CutressThe larger the package, the higher the shipment volume
The middle tier is noticeably larger: two to four compute/IO dies surrounded by eight to twelve HBM modules, and AMD's MI455X also fits this design form. Charlie's expected shipment volumes are the key: the middle tier is expected to ship 5 million units, and the rightmost tier 15 million. Placed next to 'one million units in the past year,' this is not linear growth but an order-of-magnitude jump in shipments for the next generation of custom AI chips.
— Ian CutressThe rightmost chip may stack compute dies on compute dies
The rightmost beast: the red parts are eight compute dies arranged in two columns of four, and they are stacked. Charlie lifts the red to reveal purple underneath, which could be SRAM, or SRAM could be on top; if it is compute-on-compute, it effectively equals sixteen compute dies. Each compute die is paired with two HBM modules, so one package contains sixteen HBM modules, with two light-blue IO dies at the top and bottom edges, and openings left at the package edges for optical modules.
— Ian CutressFrontier model companies are almost all making their own chips
Charlie said that of the five most advanced frontier model labs, four are working with Broadcom on custom compute chips. Companies like OpenAI and Anthropic define the designs they want, Broadcom does the back-end services and interfaces with TSMC, and then partners like Supermicro and WiWynn are brought in for data center scale deployment. This also explains why Broadcom is expanding capacity: it not only serves traditional networking customers but is also taking orders from large model companies.
— Ian CutressThe bottleneck to scaling may not be packaging but HBM
The real bottleneck may not be packaging but whether there is enough high-bandwidth memory to go around: could a new form between DDR/LPDDR and HBM emerge to fill the gap? Also, Broadcom said on stage that it has already deployed a 2nm chip with a partner, slightly smaller than the beast, and sooner in time. This reminds us that in the coming years, advanced packaging and HBM capacity will be swept up along with custom chip orders.
— Ian CutressThe first 2nm custom chip to land may come from Japan
The host pinned down the 'coming soon, smaller' 2nm chip Broadcom mentioned: he asked George from Chips and Cheese, and the two judged it to be Fujitsu's Monaka—a 2nm compute die paired with a 5nm SRAM die, with a central IO die in the middle. This chip has previously appeared at several conferences as a mockup or test silicon.
— Ian CutressIn their own words · checked verbatim
You can have a chip design team. You can decide what your architecture is. You can decide what your interconnect is. But then you still have to go build the thing. You still have to work with the foundry, work with the development kits in order to put those ideas into silicon.
Ian Cutress1:06
But the point is Cisco doesn't have a chip design team. They had to go out and find a company to go build their silicon for them.
Ian Cutress2:07
Broadcom are a little bit of an enigma from the outside because they don't really have a press department. They don't really tell anybody anything much about what they're doing.
Ian Cutress3:09
Charlie stated that in the past year they have shipped just over a million of these for their customers.
Ian Cutress4:09
Charlie said that the one in the middle, they expect to ship 5 million of those chips. The one on the right, they expect to ship 15 million.
Ian Cutress5:09
What you're seeing there is eight compute dies. That's the red. That's split into uh two columns and four rows. But these are stacked compute dies.
Ian Cutress6:11
What Charlie said on stage is that they're working with four of the five leading frontier model lab companies. So thinking your open AIs and your anthropics to go build compute chips for them.
Ian Cutress7:11
Figures
| Expected shipment volume for the middle tier of the roadmap | 5 million units | 5:09 |
| Expected shipment volume for the rightmost tier (beast) of the roadmap | 15 million units | 5:09 |
| Number of HBM modules in the beast | 16 | 6:11 |
| Number of frontier model labs collaborating | 4 out of 5 | 7:11 |
Glossary
- back-end design services
- The physical design phase after chip front-end design and before foundry production, including place and route, timing closure, etc.
- HBM
- High-bandwidth memory: DRAM stacked close to the compute chip via advanced packaging, with bandwidth far exceeding ordinary DDR/LPDDR.
- CoWoS
- TSMC's 2.5D packaging process that places multiple compute dies and HBM side by side on a silicon interposer.
- scaleup fabric
- A proprietary interconnect that connects multiple accelerator chips into one logical large machine, distinct from ordinary data center networking.
How to listen
Investors and chip engineers focused on ASIC custom chips, advanced packaging, and the HBM supply chain, as well as founders who want to see clearly the custom chip strategies of OpenAI/Anthropic.
Those familiar with back-end design services can skip 0:01–2:07; the key numbers come after 5:09.