AI detection isn't about style, it's about whether the information is watered down
The tell for AI writing isn't style, it's the "information inflation" of output exceeding input. A hedge fund titan's AI-written op-ed and his public admission became the turning point — Pangram's fourth generation has a false positive rate of about 1 in 24,000, and as models converge they actually get easier to catch.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
"Did you use AI" is the wrong question; ask where the ideas came from
The event: Stanley Druckenmiller published an op-ed in the WSJ that any sharp reader could see was AI-written, and someone ran it through Pangram and got a flat 100% AI. The turning point was that he didn't hide it — he openly admitted "I'm fundamentally not a good writer, I just used AI" — and the WSJ refused to retract. Bryan pushes one step further: if you're owning up to it, show the prompt. "Did you use AI" is a false binary; the real question is where the ideas came from — was it "write me an op-ed against the Treasury's action," or "here are my five pages of notes, polish them into prose."
— Max SperoDetecting AI is theoretically impossible, and practically good enough
In 2023 Max asked research friends whether AI text could be detected, and the answer was unanimous: "impossible, don't try, and the models will only get stronger." Pangram's response was to abandon theoretical counterexamples ("just have ChatGPT paraphrase any text and you've disproven it") and care only about the practical case: when the information in the input is smaller than the output, that's the trace of AI involvement, which Bryan calls word inflation. That definition also explains why early Yelp/Amazon fake reviews never became the wedge — platforms figured AI reviews were only 3% and didn't hurt enough to matter.
— Max SperoCatching AI isn't about em dashes, it's about synthetic mirror pairs
Pangram is a classifier model, not surface rules like an em dash detector. The approach: cut off an open-source LLM's next-token-prediction head and replace it with a classification head that outputs a 0-15 AI involvement score. Training data is built with "synthetic mirroring": take a human-written article and have an LLM compress it into a prompt ("write 200 words covering these three points"), then feed that to a random frontier model to get a matched human/AI pair on the same topic. The second layer is editing prompts, then clause-level labeling: text left as-is is human, text that was changed is AI-assisted, and new sentences conjured from nothing are AI-generated.
— Max SperoA detector's lifeline is its false positive rate, not its accuracy
Pangram 4's false positive rate is about 1/24,000, measured by running millions of pre-2022 documents (an era when AI ghostwriting was essentially impossible); the false negative rate is about 1/300, measured by taking real user prompts from WildChat and regenerating them with frontier models. Not perfect, but pointed the right way. Bryan's experience is that false positives are extremely low, and that is exactly the lifeline for whether a product can be used seriously. Pangram 4's step change came from moving from a 512-token window to token-level output: a whole human-written document with two AI sentences stuck in the middle can be pointed out word by word.
— Max SperoA detector doesn't need to be 100%, it just needs to make cheating cost something
Max says education is a huge market, and this is the first year anyone has realized "there exist AI detectors that actually work." The product logic isn't 100% precision, it's creating friction on the shortcut — turning "throw the assignment at AI" from zero cost into something with a cost, so students take the next easiest path, which is doing it themselves. The same judgment applies to enterprise settings: hiring at Oxide is a writing-intensive process, and an LLM-parroted "why I want to join Oxide" is recognizable at a glance; the detector just makes it explicit.
— Max SperoMost "I was falsely flagged" sob stories are astroturf ads
Max distinguishes two kinds of opposition. One group genuinely wishes detectors didn't exist: they let AI text produced in their profession be authoritatively identified. The other is marketing from the humanizer ecosystem: these tools claim to rewrite AI text so it fools Turnitin and Pangram, and they sell to students caught cheating on assignments; they usually ship with their own "AI detector" that first declares your text AI, then sells you the rewriting service. Many of the Reddit sob posts about "my teacher flagged me at 76% AI and gave me a zero" are astroturf for this kind of lead gen — run them through Pangram and they're all AI-written.
— Max SperoStronger models haven't made detection harder, they've made it easier
Asked whether he's forever chasing the frontier model, Max gives a counterintuitive answer: two things happen at once — models get smarter, but after post-training repeatedly teaches "correctness" and preferences, the output distribution is far narrower than in the GPT-2 era. GPT-2 tried to model the full distribution of human output; today's models only give the "correct" token, not what an ordinary person would say. He and the three Oxide hosts agree from experience: new models talk with a "strangely erudite" flavor, and detection hasn't gotten harder, it's gotten easier to pick up.
— Max SperoThe next fight is over how much the human actually put in
Pangram can already guess the model family internally: the probe's top-1 accuracy is 90%, but to productize it they'd need to get it to 98%. The further direction is "reverse Pangram" — inferring what prompt might have generated this text and how much context the model was given: two bullet points or fifteen? That dimension of information changes the characterization of Druckenmiller's op-ed entirely — giving five pages of notes to polish counts as collaboration, giving five bullet points to expand is ghostwriting. The next argument in detection isn't "was AI used," it's "how much did the human actually put in."
— Max SperoIn their own words · checked verbatim
He says, yes. Of course I used AI to write this. I'm not a good writer, essentially.
Max Spero3:15
I actually think that, like, doing one's own writing actually helps you form your thoughts.
Bryan Cantrill7:30
We don't we don't care about, like, solving it in a, like, theoretical sense. We care about solving it in a practical sense.
Max Spero13:00
And so how we create the training data for Pangram is a method that we call synthetic mirroring.
Max Spero34:30
For Pangrom four, it's about one in twenty four thousand.
Max Spero39:30
creating friction on the, like, the easy path just makes people do the the next easiest path, which is doing the assignment themselves.
Max Spero46:00
Their claim is that it will evade any AI detector, like Turnitin, now Pangram.
Max Spero50:00
today's LLMs have a much more narrow distribution of what they output than, like, g p t two.
Max Spero59:30
Figures
| Pangram false negative rate | about 1/300 (measured with real WildChat prompts regenerated by frontier models) | 40:30 |
| Corpus size for false positive rate measurement | millions of pre-2022 documents | 39:30 |
| First Pangram model release | February 2024 | 41:00 |
| Pangram 4 release | a little over a month before this episode was recorded | 42:30 |
| Classification granularity before Pangram 4 | 512 token window | 43:30 |
| Internal probe top-1 accuracy for identifying model family | 90% | 1:01:00 |
| API spend by someone trying to fool Pangram | $700 | 28:00 |
| Early estimate by large platforms of AI reviews | 3% | 15:30 |
Glossary
- synthetic mirroring
- Reverse-engineering a prompt from a human article and having a model rewrite it, to construct paired human/AI samples.
- humanizer
- A service claiming to rewrite AI text so it evades detectors, sold mainly to students caught cheating on assignments.
- classifier head
- The classification scoring head that replaces a large model's original next-token-prediction output head.
- WildChat
- A dataset of real user-AI conversations, used by Pangram to estimate its false negative rate.
How to listen
Product and engineering leads doing content moderation, hiring screening, and education anti-cheating, plus content teams producing large volumes of text with LLMs who need to know where the line is.
The Shell Game podcast and philosophical rambling at 53:00-57:00 can be skipped without losing the main thread.