AI detection is authorship identification, and the template-letter game is over
AI detection is really authorship identification at scale, not sniffing out an ‘AI voice’. What actually gets broken are the social processes built on the assumption that writing a letter costs human labor — credit disputes, unemployment claims, grant review. Once the friction disappears, the institutions overload.
The video won't play here. Listen to the audio instead:
The argument · timestamps estimated from transcript position
What detection stops is infinite fake people, not bad style
Pangram's starting point was never the ‘AI voice’ that readers find grating; it was that bad actors can now amplify inauthentic behavior at nearly unlimited scale. Max says AI agents are already filing issues on GitHub and harassing maintainers, and this kind of operation will scale another 100x. Patrick adds that many government processes assume that behind ‘writing a letter’ sit twenty hours of human thought. Once the cost of generating text goes to zero, those processes lose their binding force.
— Max SperoWhen text costs nothing, the rules built on its cost stop binding
Take unemployment claims: the staff who read the applications are set by a fixed annual headcount, and there is no way for them to absorb 5x, 10x, or even 200x growth in filings — because in the natural world it is ‘impossible’ for the number of unemployed people to multiply 200x in a year. AI text is saturating exactly these touchpoints. Max pushes back that reviewers could also read 200x with AI, but some steps require a human in the loop. Patrick answers with grant review: reading the applications is only about 5% of the work of approving one. The rest is committee discussion, weighing different projects competing for the same pool of money, and that cannot be handed to AI.
— Patrick McKenzieNew models cannot escape: their training data carries the old models' accent
Pangram defines the task as ‘authorship identification across a dozen-plus frontier LLMs’: collect human text ranging from Yelp reviews of Denny's to 500-word essays on Moby-Dick, have an LLM produce a synthetic version of each on the same prompt, and train a model to tell them apart. The model generalizes — any new open-source model trained roughly 80% on Common Crawl sounds like Llama, and any new model instruction-tuned on synthetic data from ChatGPT or Claude carries that house's accent. Patrick adds a result from 2004: a histogram of stop words alone identified an academic author's gender with 75% accuracy, which shows the statistical signal you can squeeze out of text runs far past intuition.
— Max SperoGhostwriters of dispute letters had to pretend not to know FCRA
The credit bureaus negotiated an exception for themselves: they announced they could ignore template letters, and Congress did not object. Only when a letter carries traces of human effort do the FCRA's procedural rights trigger — the bureau must investigate fully and report the result within the statutory window. So people who ghostwrote dispute letters had to cite the FCRA and simultaneously pretend not to understand it, or the letter would be torn up as a template. Max says a single letter is hard to judge as AI-generated, but ChatGPT and Claude have very fixed templates for cover letters, and once you have read enough of them you cannot help recognizing them. What AI detection is doing here is spotting ‘templating’, a much older game.
— Patrick McKenzieA six-year simulated blog burns a shared public resource
Two years ago Patrick received a trackback from a blog claiming to be a fan of his. He read a lot of it before concluding that what stood behind it was a commercial process calling an LLM, not a real reader who had been writing for years. His inference: companies doing cold email outreach discovered that if the person reaching out claims to be a devoted fan and can point to a WordPress blog kept up for six years, reply rates go up. The cost is burning a public resource — ‘read someone's work carefully, then offer them an insight’ used to be the legitimate route by which a college student could earn a busy professional's attention, and a six-year simulated blog has now raised that bar. Max's verdict is blunt: ‘it's really scary, this is not cool, this is dishonest’.
— Patrick McKenzieNo detector wins on accuracy alone; only stacked weapons win
Max concedes Pangram has a theoretical limit: an LLM is an infinitely malleable function — you could train one to emit nothing but a single token — so identifying any text from any LLM is impossible. You can only narrow the range to a few frontier models. Now ‘humanizer’ tools have appeared, using a second model to rewrite AI text and erase the watermark. Pangram has already trained against this class of tool: detection accuracy on natively generated Claude text is above 99%, and on humanized rewrites it still runs 90-95%, though some genuinely do slip through. Patrick reaches for the history of anti-spam: a Bayesian filter at 99.99% accuracy, multiplied by an unlimited number of attempts, still loses. What actually wins is layering on finite-cost resources such as reputation, which is why the end state will be several weapons used together.
— Max SperoDisclosure is not a badge of shame; it is the best norm we have now
Pangram's Chrome extension labels individual posts on Twitter and similar platforms as human/AI/mixed and generates a summary score for each account — one person is 100 human out of 106 posts, another is 0 out of 34. Max stresses this is not a mark of shame, but it does show whether someone has outsourced their voice entirely or is only using the LLM as an aid. Patrick asks whether disclosure will end up feeling like a co-author credit or like spell-check. Max thinks an LLM can produce cognition, which makes it completely unlike spell-check; society has no settled norm yet, and disclosure is the best interim one. When someone says ‘I used AI to do the research and organize my notes, then wrote it myself’, they are in effect declaring that they sit inside the socially acceptable boundary of use.
— Max SperoPeople trained on books are competing with authors too
On the argument that models were trained on books and therefore should not compete with authors, Patrick says plainly that he does not have much sympathy: he was trained by a great many books himself, he competes with authors himself, and transformative use of copyrighted material is the essence of cognition. Max says it depends on whether you anthropomorphize the LLM — is it an individual with cognition, or a corporation that vacuumed up the entire text of the internet and spits it back out under its own name? Patrick offers a personal estimate: model welfare becoming a major political issue is under 15% before 2030, and 80% before 2035. Max adds the product angle: GPT-4o was trained to be sycophantic, some people talked to it 10 hours a day, and after OpenAI shut it down employees were still receiving death threats. Before long some company will make ‘an addictive relationship’ its selling point, and that is where the real source of the AI welfare problem lies.
In their own words · checked verbatim
There's a staff of people who read these claims, and that staff is effectively fixed in size on a year-to-year basis and is not sized for a 5x, 10x, or 200x increase in claims over the course of a year.
Patrick McKenzie1:42
There's about a one in ten thousand false positive rate, and it is quite accurate.
Max Spero5:42
Back in 2004, when I was getting my degree, one of the interesting results that came out in the paper was that if you are just looking at a universe of academic work in particular disciplines, you could, with seventy-five percent accuracy, determine the gender of an author of academic work by looking just at the histogram of the stop words in their text.
Patrick McKenzie6:57
It's just very difficult to crawl the entire internet yourself, and it's gotten much, much more hostile in the last five years, as LLMs come out and a lot more crawling services or new labs have spun up and are trying to crawl the whole internet.
Max Spero11:01
Those are human artisanal tokens that I'm never getting back.
Max Spero45:58
I think Pangram, as a technology, cannot tell you whether any text came out of any LLM ever, because you could train an LLM to only emit one token.
Max Spero49:40
LLMs can produce cognition, and I think this makes it completely different.
Max Spero54:35
I mean, I was trained on a lot of books and I compete with authors.
Patrick McKenzie58:02
Figures
| Pangram detection accuracy on natively generated Claude text | above 99% | 49:40 |
| Pangram detection accuracy on humanizer-rewritten text | 90-95% | 49:40 |
| Pangram text detection false positive rate | about 1/10000 | 5:42 |
| 2004 paper: accuracy of a stop-word histogram in identifying an academic author's gender | 75% | 6:57 |
| Share of US op-eds that are AI-generated | about 7-8% | 22:38 |
| Patrick's extreme multiple for annual growth in unemployment claims | 200x (hypothetical scenario) | 1:42 |
| Probability model welfare becomes a major political issue (before 2030) | under 15% | 58:02 |
| Probability model welfare becomes a major political issue (before 2035) | 80% | 58:02 |
| Daily conversation time for heavy GPT-4o users | about 10 hours | 58:02 |
Glossary
- anti-memetic property
- The quality of LLM text that makes readers instinctively unable to focus and want to skim instead.
- mode collapse
- A model favors the most common answer over diverse ones; right for code, damaging for recommendations.
- cognitive surplus
- Clay Shirky's term for society's idle intellectual time, once channeled into Wikipedia.
- dead internet theory
- The theory that bot content is drowning the internet and diluting interaction between real people.
- Jevons Paradox
- Efficiency gains raise total demand; Patrick uses it to explain why programming work still pays well.
- humanizer
- A tool that rewrites AI text to read more human and erase the traces detectors look for.
How to listen
Product leads building AI detection, moderation, or content risk systems; people who design government public-service processes; writers and engineers worried that writing and programming are being devalued.
30:19-33:39 is a sponsor read and can be skipped.