The war over foundation models is over; vertical copilots are where the veterans get their shot
Foundation models are an emergence built by young people on Microsoft's compute, but the vertical copilots that actually make money belong to startup veterans — two-thirds of the vertical opportunities have already been killed off by ChatGPT itself, and what's left is a combined war of data, social and business model.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
ChatGPT trains each version on the last, so the gains are not linear
Wang Huainan points out that ChatGPT's leap from 3.5 to 4.0 was not a version iteration: it used the full intelligence of the previous version to train the next one, so each jump is non-linear. He quotes Lu Qi: there are only two moments in life when information explodes beyond your ability to absorb it — one was the arrival of the browser in '98, the other is now. That non-linear rate of progress turns what looked like an opportunity two months ago into something taken away from you: you wanted to build smart Excel reports, ChatGPT did it itself; you wanted to do medical records, a Microsoft-backed company is already on it.
— Wang HuainanThe foundation belongs to the young; the industry copilot belongs to the veterans
Wang Huainan offers a polarised judgment: the foundation large language model is an emergence that young people, under Sam's leadership, pulled off with Microsoft's compute and Bing's data — it belongs to new thinking, to innovation with no regard for the rules. But the industry-copilot opportunities growing on top of it will most likely belong to startup veterans. The work is complex; it demands the ability to connect things, to judge, and to assemble a team, and a fresh-out-of-the-gate technical young person can't handle it. He observes that the first wave of people activated by this were Wang Huiwen, Wang Xiaochuan, Zhou Hongyi and himself — all veterans over 35, in an internet industry that used to screen out 35-year-olds.
— Wang HuainanTwo-thirds of vertical industries will be killed off by ChatGPT itself
Wang Huainan judges that not every vertical industry copilot has a shot — more than half, perhaps two-thirds, of industries will be killed off by ChatGPT itself. The survivors are complex, requiring industry knowledge, data and cognition, and belong only to people who have dug deep in that industry for a long time. He gives the example of a lawyer AI: a tool can always be replaced — you find 100 lawyers and can train a lawyer AI, someone else finds 200 lawyers and can train lawyer AI 2. Only a community is irreplaceable, because it has person-to-person exchange. So going from tool to community to a business model — all three layers have to be done in one breath.
— Wang HuainanAmazon ships a large model plus a small model at the end of May, pre-empting the future
Wang Huainan reveals something not yet public: at the end of May Amazon will ship two models, one large language model and one small language model. He finds Amazon interesting — it has pre-judged the future of this: I'll fight for a ticket on the general large-model ship, but at the same time do a small language model and seize the possibility of entering vertical copilots first. Amazon runs dozens of teams in parallel; most people are working on the large language model, but very strong people are working on the small one, and there are Chinese people on the team. He judges this day will certainly arrive within a month, and what Amazon is fighting for is certainly tied to e-commerce and transactions.
— Wang HuainanThe war over foundation models is over, and Google should concede
Wang Huainan states plainly: the war over foundation models is over. Last time they talked he said to watch a while longer; today he thinks it's already lost, and Google should concede — though it won't concede right away. He argues emergence can only emerge once: because it leads the world, the best search engine is still only Google, and the large model is fiercer than the search-engine era, a single dominant player, because it gets all the world's knowledge first. There is only one God; there cannot be several. The only opportunity is in verticals: if there are a hundred vertical opportunities, 60 are already off the table, and you have to find the remaining 30%.
— Wang HuainanThe vertical moat isn't algorithms or compute — it's data plus social plus business model
Wang Huainan argues the moat in verticals is not the algorithm, not even compute — compute is a moat, like nuclear war, but the more critical thing is data. In the coming year, companies with data and capability will feverishly build their own independent vertical copilots, and those with data but no capability will be feverishly acquired. But data isn't entirely a moat either: social on top of data is the moat, and third is the business model. He takes Bilibili as an example: it has data and it has social, but it hasn't made money — the people producing content haven't made money on Bilibili, and if that problem isn't solved, the good content and data producers will flee.
— Wang HuainanThe first shot is search, the second e-commerce, the third entertainment
Wang Huainan judges the large-model war's first shot is search, the second e-commerce, the third entertainment. E-commerce is a search scenario, but the pain is that no one browses anymore, so an intelligent copilot is needed to restore browsing — for instance, a woman asking on her first day at work what cosmetics to use: not a copilot for after you know what to buy, but someone who goes shopping with you when you don't know what to buy. On entertainment: put ChatGPT and Midjourney together and everyone becomes the maker of their own entertainment content; the cost of creating excellent content races toward zero, entertainment will get bloodily wiped, and everyone becomes a film director.
— Wang HuainanThe internet used to screen out 35-year-olds; this time it's all over-35s finding their youth again
Wang Huainan observes something strange: the internet used to be the domain of the young, screening out 35-year-olds, but this time everyone interested in this is over 35 — from Zhou Hongyi to Wang Xiaochuan, Wang Huiwen, and himself. He thinks young people's error rate in judgment may be very high, while veterans have the experience, the feel, and the ability to handle complex situations — exactly what this needs. He suggests listing China's top 100 platforms by traffic today and seeing whether an attack would work: if Xiaohongshu doesn't cut its own throat, for instance, it will have its throat cut by someone else in these three waves.
— Wang HuainanIn their own words · checked verbatim
The foundation large language model — at least in the US — was an emergence pulled off by a bunch of crazy people with other algorithms, using Microsoft's compute and Bing's data. It belongs to young people, to new thinking, to innovation with no regard for the rules.
底层的大语言模型 起码在美国是被一帮疯狂的 有其他算法的人借助了微软的算力 和必应的数据走出来的这种涌现 他属于年轻人 属于新思维 属于毫不顾忌的创新
Wang Huainan7:25
More than half of industries, even two-thirds of industries, will be killed off by Chat itself.
一半以上的行业 甚至三分之二的行业 都会被chat自己干掉
Wang Huainan11:08
A tool can always be replaced; only a community is irreplaceable, because it has person-to-person exchange.
工具是永远可被替代的 只有社区才能不可替代 因为他有人的交流
Wang Huainan21:18
I think the moat is not in the algorithm, not even in compute — compute is a kind of moat, because it's a war of how many chips others have versus how many chips you have, very much like nuclear war — but I think the third aspect is data. The moat is in data.
我觉得壁垒在于 不在于算法 甚至不在于算力 算力是一种壁垒 因为别的人多少芯片 你有多少芯片的战争 这跟核战争特别像 但是我觉得在于第三个方面是数据 壁垒在于数据
Wang Huainan27:30
This is too strange: the internet used to be the domain of young people, right? And it screened out 35-year-olds. This time you find that everyone interested in this is over 35.
这事情太怪异了 就是互联网曾经就是年轻人的天下 对吧 而且他是排除35岁的 这次你发现所有对这个事情感兴趣的 人全是35岁以上的了
Wang Huainan37:50
The more I think about it, the less I believe a person is an independent, autonomous thing. More and more I feel he is someone else operating a machine. We are so imperfect.
我越想越不觉得人是一个独立自主的一个东西 我越来越觉得他是别人操纵了一台机器 我们如此之不完美
Wang Huainan47:40
Figures
| Number of models Amazon plans to release | 2 (one large language model and one small language model) | 13:10 |
| Share of YC startup projects killed off by ChatGPT | 60% to 70% | 5:05 |
| Share of vertical industries killed off by ChatGPT | More than half, even two-thirds | 11:08 |
| How much earlier emergence came than Sam Altman expected | 5 to 8 years | 7:55 |
| Layoff ratio at Twitter after Musk's acquisition | 70% to 90% | 29:23 |
| Share of Chinese text in ChatGPT4's training data | Less than 1% | 41:30 |
Glossary
- vertical copilot
- An intelligent assistant aimed at a specific industry or scenario, such as a lawyer AI or a shopping copilot.
- LLM
- Large Language Model, a language model trained on massive amounts of data.
- transformer
- A neural network architecture based on the attention mechanism, the foundation of large language models.
How to listen
Founders and investors watching foundation-model startups and investment opportunities, especially those hunting for a vertical entry point and sizing up competitive moats.
43:30 to 49:32, the discussion of Hinton's departure and the Berkshire Hathaway annual meeting — only weakly connected to the main thread.