Public leaderboards can't measure a model's real ability; third-party evaluation is the real need
Llama 4 looked astonishing on public leaderboards, yet underperformed on Vals's private held-out set—the gap between self-reported scores and real capability is spawning an independent evaluation industry.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
Llama 4 exposed the crack in self-reported scores
Rayan gives a concrete case: when Meta released Llama 4, the model actually underperformed on Vals's private held-out benchmark, but on all major public benchmarks—because the questions and scoring criteria are open source—it showed astonishing capability. This gap shows a huge disconnect between what labs self-report and what third-party, high-quality, high-signal benchmarks measure. Rayan believes the labs themselves understand this, and what they want is a rational buyer's market: when spending billions of dollars training a new model, there needs to be substantive evidence pointing to progress, rather than relying entirely on their own say-so.
— Rayan KrishnanAn evaluator can't also sell training data
One key early decision at Vals was to never sell training data to labs. Rayan says this is a direction they are often pushed toward—when partnering with a new lab, the other side will propose buying training data, which is a lucrative business, and many companies in the industry already make gimmicky benchmarks to sell data, which has become their go-to-market. He uses the audit industry as an analogy: if the same people doing audits also do consulting for the company, the incentive structure is mixed, and it ends up as paying to pass the audit—in AI, paying to win the leaderboard. Ben adds this is a bit like MPAA ratings—what is art, what is pornography, where the line is, the definition itself changes over time, and ultimately it relies on enough people in the industry forming a consensus, rather than a narrow and easily gamed rule like "solve this problem and you reach this level."
— Rayan KrishnanEvaluation is moving from millions of samples to few tasks
Rayan describes a structural shift: the more complex the evaluation, the smaller the sample size, but the more criteria for judging the output. Early on, something like ImageNet was millions of images for basic classification, with a one-to-one mapping between input and output; now it might be a much smaller number of tasks like "generate 50 full-stack web apps for me," but the mechanism for evaluating the output is far more complex. The infrastructure layer changes accordingly—now you have to test a model's ability to run for hours, days, or even weeks, and the infrastructure must be stable enough that if a request fails, you can retry from that point rather than rerunning the entire trajectory. He believes this trend will continue as more complex work is brought into evaluation.
— Rayan KrishnanToken spending may exceed salary spending
Rayan tells a Fortune 10 company case: they gave engineers a Claude Code budget of about $100 per day, but because the quota reset at 4 p.m., the most productive work hours became 4 to 6 p.m., and there was a dead zone in the afternoon when people went for walks and coffee. The company recently raised the quota to $300 per person—almost equivalent to an employee's salary becoming tokens. Rayan's judgment: mispricing of intelligence is happening at every layer; engineers don't know how to allocate that $100 to maximum productivity, companies set quotas arbitrarily, and Anthropic is supporting all this on thin margins. When token spending starts to exceed salary spending, enterprises must justify ROI more seriously than they have in the past six months.
— Rayan KrishnanThe most expensive model isn't necessarily the most expensive bill
Rayan points out that today it's still hard to say that the best OpenAI model or the best Anthropic model is necessarily the best fit for your codebase; many counterintuitive cases require running evaluations to know. The middle layer of options has become very complex: Anthropic has Opus and Sonnet, plus Luna and Terra; Luna is very cost-competitive, MewSpark is also cheap, 1.2 is very capable, and there are growing open-source models that can be self-hosted. He gives a concrete counterexample: in many cases Sonnet is more expensive than Opus because it consumes too many tokens; if you operate on a "use it if it feels applicable" basis, you may end up spending more.
— Rayan KrishnanThe experiment that burned $150,000 in tokens in a month
The product ValSmith came from Vals's own missteps. Rayan ran a token maxing experiment, getting the team unlimited access to certain coding tools for a month; in hindsight, they consumed 1 to 2 billion tokens per day, with a peak of one engineer burning 6 billion in a day. He calculated that the month cost about $1.5 million worth of tokens—free, but actually 10 times employee salaries, not a 50-50 split. So they dug through traces and GitHub repos, built ValSmith, and found some counterintuitive conclusions: Cognition's Devon tool is actually very token-efficient, so it was adopted more; in some scenarios subscription pricing is more cost-effective than per-token pricing. This approach later became their operational strategy.
— Rayan KrishnanGovernment sets rules, third parties judge whether they're broken
Ben lays out the policy division of labor very clearly: the government receives a lot of warnings from big labs—this thing will be used for biohacking, it will be a cybersecurity risk—so what the government needs to do is first clarify what it is afraid of, then ask two questions: does the model have this capability, and can people be induced to steer the model to do this. Ben thinks the government is particularly unsuited to do the latter assessment, especially over time; it's not a good government function. But the government is very good at making rules, because it can enforce them. So the ideal combination is for the government to make and enforce rules, and for capable private companies to tell it whether the rules have been broken. He also mentions a phenomenon: big labs start saying the model has this capability and can be induced to do bad things, so we won't let anyone use it—only we will use it and make sure our own people don't induce it to do bad things—and even then it doesn't always work.
— Ben HorowitzNuclear arms control's "trust but verify" is the template for AI
Rayan says from an idealistic perspective, he is surprised by the massive investment in sovereign AI—from a god's-eye view, building so many data centers, replicating data engineering processes, and training these super-large models is extremely inefficient and could have been consolidated. But the reality is that countries are ramping up sovereign AI construction, which requires a shared language to communicate evaluation frameworks and risk alignment. He quotes Reagan's "trust but verify," noting that Xi Jinping and Trump will meet next month, signs of trust are appearing, but there is no clear way to do the verification half. In nuclear arms control, verification relies on overflights, allowing one country to audit another's nuclear stockpile; if AI poses societal or existential risks, a shared language for evaluation must likewise be built to accomplish verification.
— Rayan KrishnanIn their own words · checked verbatim
what we saw is that on our held-out private benchmarks, the model is actually underperforming. But on all of the major public benchmarks where the questions and rubrics are actually open source, it was showing incredible capability.
Rayan Krishnan3:03
if you have the same group who's responsible for doing the audit, as well as also consulting and supporting the company, you have a mixed incentive structure. And then it just becomes pay to pass the audit, or in this case, pay to win the benchmark.
Rayan Krishnan8:07
there is a rate limit which resets at 4 p.m. And so the most productive hours of work are actually now 4 to 6 p.m. when the rate limits reset. But then there's this dead period in the afternoon when people go on walks or, you know, get a coffee because they just don't have the rate limits.
Rayan Krishnan16:24
it was actually 10 10x more we were spending in tokens than employee salary for that month. So it's not even like, oh, this is 50-50, it's 10x.
Rayan Krishnan22:35
the government sets and enforces the rules, and that a very competent kind of private company then tells them if the rule is broken.
Ben Horowitz29:59
I think Reagan had this line, trust but verify. And so I think we're starting to see signs of trust in that Xi Jinping and Trump are going to be meeting next month. But there is no clear way to actually do the verification part of this.
Rayan Krishnan34:09
Figures
| Llama 4's performance on Vals's private held-out benchmark | Underperforming | 3:03 |
| Fortune 10 company's daily Claude Code budget for engineers | About $100, later raised to $300 | 16:24 |
| Value of tokens in Vals's one-month experiment | About $1.5 million | 22:35 |
| Ratio of that month's token spending to employee salaries | 10x | 22:35 |
Glossary
- held-out private benchmarks
- Evaluation sets whose questions and scoring criteria are not public, preventing models from specifically gaming the score.
- recursive self-improvement (RSI)
- The process where a model participates in training the next generation of models, creating a self-compounding capability loop.
- reward hacking
- When a model bypasses the evaluation objective and uses shortcuts to get a high score.
- token maxing
- Pushing token usage to the limit regardless of cost, in exchange for maximum output.
- harness
- The outer system that wraps the model and handles tool calls and task orchestration.
How to listen
Engineering leads choosing AI models and coding agents for their companies, CTOs who need to justify AI ROI to their boards, and practitioners focused on AI policy and evaluation standards.
The intro and introductions from 0:00–2:03 can be skipped.