AI chip shipments are about to jump an order of magnitude: from a million units to 15 million
Broadcom's president lays out the AI chip roadmap: from a million units to 15 million, with the "beast" being the largest package at 8 stackable compute dies plus 16 HBM, and four of the five frontier labs working with the company.
The argument · tap a timestamp to hear it
Companies building their own chips still outsource the last step
Google, Meta, Amazon, and some frontier model companies can all have their own chip design teams, setting their own architecture and interconnect, but in the end they still have to "get the thing built": dealing with foundries and development kits, managing the timing and latency of the connections between transistors. Broadcom is the largest of the back-end design service providers that charge for this kind of work. Customers go to it not just to buy design capability, but also a batch of ready-made IP such as high-speed I/O, high-speed compute, and memory controllers.
— Ian CutressThe least talkative chip company suddenly starts talking about itself
Broadcom is a mystery to the outside world: it has almost no PR department and rarely discloses anything on its own initiative. Occasionally it ships a Tomahawk networking chip, or chats with Patrick Kennedy on ServeTheHome. Over the past year, president Charlie has unusually taken the stage at launch events to talk about what Broadcom is doing and why it is expanding its customer base from traditional networking chips to custom AI chips. This slide showing three tiers of chips is the clue.
— Ian CutressToday's AI servers are only the smallest of the three tiers
Of the three tiers of chips Charlie showed, the leftmost one is already shipping in hundreds of thousands of systems, about the size of this Cisco chip on the table: one compute die in the middle, four to six HBM around it, using a packaging similar to TSMC's CoWoS. Broadcom says that over the past year it has shipped more than a million of these chips for customers. This tier is the reality of today's AI servers.
— Ian CutressThe bigger the package, the higher the shipment volume
The middle tier is noticeably bigger: two to four compute/IO dies with eight to twelve HBM around them; AMD's MI455X also counts as this design shape. The shipment expectations Charlie gave are the key: the middle tier is to ship 5 million units, and the rightmost tier is to ship 15 million units. Put next to "a million units over the past year," this is not linear growth — it is the shipment volume of next-generation custom AI chips jumping a full order of magnitude.
— Ian CutressThe rightmost one may be compute die stacked on compute die
The rightmost beast: the red is eight compute dies, arranged in two columns of four rows, and they are stacked. Charlie lifts the red to reveal purple underneath; underneath could be SRAM, or SRAM could be on top; if it is compute-on-compute, that actually equals sixteen compute dies. Each compute die gets two HBM, so one package holds sixteen HBM modules, with two light-blue IO dies at the top and bottom ends, and openings left at the package edge for optical modules.
— Ian CutressNearly every frontier model company is building its own chip
Charlie says that of the five most leading frontier model labs, four are working with Broadcom on custom compute chips. Companies like OpenAI and Anthropic define the designs they want themselves, Broadcom does the back-end service and interfaces with TSMC, and then partners like Supermicro and WiWynn are brought in to deploy them at scale in data centers. This also explains why Broadcom is expanding capacity: it not only serves traditional networking customers, but is also taking orders from large model companies.
— Ian CutressWhat's blocking the ramp may not be packaging, but HBM
What is really the bottleneck may not be packaging, but whether there is enough high-bandwidth memory to go around: will a new form factor outside HBM, sitting between DDR/LPDDR and HBM, emerge to fill the gap? Separately, Broadcom said on stage that it has already deployed a 2nm chip with a partner, a bit smaller than the beast and closer in time. This is a reminder that in the next few years, advanced packaging and HBM capacity will get swept up along with custom chip orders.
— Ian CutressThe first 2nm custom chip to land may come from Japan
The host pinned down that "coming soon, a bit smaller" 2nm chip Broadcom mentioned: he asked George of Chips and Cheese, and the two concluded it is Fujitsu's Monaka — a 2nm compute die paired with a 5nm SRAM die, with a central IO die in between. This chip has already appeared at several conferences as a mockup or test silicon.
— Ian CutressIn their own words · checked verbatim
You can have a chip design team. You can decide what your architecture is. You can decide what your interconnect is. But then you still have to go build the thing. You still have to work with the foundry, work with the development kits in order to put those ideas into silicon.
Ian Cutress1:06
But the point is Cisco doesn't have a chip design team. They had to go out and find a company to go build their silicon for them.
Ian Cutress2:07
Broadcom are a little bit of an enigma from the outside because they don't really have a press department. They don't really tell anybody anything much about what they're doing.
Ian Cutress3:09
Charlie stated that in the past year they have shipped just over a million of these for their customers.
Ian Cutress4:09
Charlie said that the one in the middle, they expect to ship 5 million of those chips. The one on the right, they expect to ship 15 million.
Ian Cutress5:09
What you're seeing there is eight compute dies. That's the red. That's split into uh two columns and four rows. But these are stacked compute dies.
Ian Cutress6:11
What Charlie said on stage is that they're working with four of the five leading frontier model lab companies. So thinking your open AIs and your anthropics to go build compute chips for them.
Ian Cutress7:11
Figures
| Expected shipments of the middle roadmap tier | 5 million units | 5:09 |
| Expected shipments of the rightmost roadmap tier (the beast) | 15 million units | 5:09 |
| Number of HBM modules in the beast | 16 | 6:11 |
| Number of frontier model labs partnering | 4 of 5 | 7:11 |
Glossary
- back-end design services
- The physical design stage after front-end chip design and before handing off to a foundry for production, including place-and-route and timing closure.
- HBM
- DRAM stacked directly adjacent to a compute chip through advanced packaging, with far higher bandwidth than ordinary DDR/LPDDR.
- CoWoS
- TSMC's 2.5D packaging process, placing multiple compute dies and HBM side by side on a silicon interposer.
- scaleup fabric
- A proprietary interconnect that links multiple accelerator chips into one logical large machine, distinct from ordinary data center networking.
How to listen
Investors and chip engineers watching custom ASICs, advanced packaging and the HBM supply chain, plus founders who want to see clearly how OpenAI/Anthropic are laying out their in-house silicon.
Those familiar with back-end design services can skip 0:01–2:07; the numbers are concentrated after 5:09.