AI Detection Is Really Authorship Attribution, and Template-Letter Games Are Dead
AI detection is really large-scale authorship attribution, not catching an "AI smell." What's truly disrupted are social processes that assume "writing a letter" costs human effort—credit disputes, unemployment claims, grant reviews. Once friction vanishes, institutions overload.
The video won't play here. Listen to the audio instead:
The argument · timestamps estimated from transcript position
AI detection isn't about style; it's about fake humans at infinite scale
Pangram's starting point isn't the "AI smell" readers dislike, but bad actors amplifying inauthentic behavior at near-infinite scale. Max says AI agents are already filing GitHub issues and harassing maintainers, and such operations could scale 100x more. Patrick adds: many government processes assume a letter represents twenty hours of human thought; when text generation costs hit zero, those processes lose their binding constraint.
— Max SperoWhen text generation costs zero, institutional constraints fail too
Take unemployment claims: the staff reading applications is budgeted annually and can't handle 5x, 10x, or even 200x growth—because in the natural world, a 200x jump in unemployment within a year is "impossible." AI text is saturating such touchpoints. Max counters that reviewers could also use AI to read 200x more, but some steps require a human in the loop. Patrick uses grant review: reading applications is only about 5% of the approval work; the rest is committee discussion and comparing projects within the same funding pool, which AI can't replace.
— Patrick McKenzieNew models can't hide; training data carries the flavor of old models
Pangram frames the task as authorship attribution across a dozen frontier LLMs: they collect human text from Denny's Yelp reviews to 500-word Moby-Dick essays, generate synthetic versions of each with an LLM on the same prompt, and train a model to distinguish. The model generalizes—any new open-source model trained on ~80% Common Crawl sounds like Llama; new models instruction-tuned on synthetic data from ChatGPT or Claude carry that flavor. Patrick adds a 2004 result: stop-word histograms alone could judge academic author gender with 75% accuracy, showing text yields far more statistical signal than intuition suggests.
— Max SperoPeople writing dispute letters must pretend not to know FCRA
Credit bureaus carved out an exception for themselves: they declared they could ignore template letters, and Congress didn't object. Only if a letter shows traces of human effort do FCRA procedural rights trigger—the bureau must conduct a full investigation and inform the consumer of results within a statutory period. So past dispute-letter writers had to cite FCRA while pretending not to understand it, or the letter would be torn up as a template. Max says a single letter is hard to judge as AI-generated, but ChatGPT and Claude have very fixed templates for cover letters; once you've read enough, you recognize them. AI detection here is really recognizing the older game of "template-ization."
— Patrick McKenzieA six-year fake blog burns public resources
Two years ago Patrick received a trackback from a blog claiming to be a fan; he read a lot before concluding it was a commercial pipeline calling an LLM, not a real reader writing for years. His inference: cold-email outreach companies found that if the sender claims to be a devoted fan and points to a six-year WordPress blog, reply rates rise. The cost is burning public resources—"reading someone's work carefully and offering insight" used to be a legitimate path for college students to get busy professionals' attention; now a six-year simulation raises the bar. Max's verdict is blunt: "It's really scary, it's not cool, it's dishonest."
— Patrick McKenzieEven a perfect detector can't win; you need multiple weapons
Max admits Pangram has a theoretical boundary: an LLM is an infinitely malleable function; you can train it to output only one token, so it's impossible to detect any text from any LLM. You can only narrow the scope to a few frontier models. Now "humanizer" tools appear, using a second model to rewrite AI text and erase watermarks. Pangram has trained against these: detection accuracy on native Claude text exceeds 99%, and on humanized rewrites it's still 90-95%, but some slip through. Patrick draws an anti-spam analogy: Bayesian filters with 99.99% accuracy still lose against infinite attacks; the real win is layering finite-cost resources like reputation, so the endgame is multiple weapons.
— Max SperoDisclosure isn't a mark of shame; it's the best norm for now
Pangram's Chrome extension labels individual posts on Twitter and other platforms as human/AI/mixed, and generates a summary score per account—e.g., someone has 100 human posts out of 106, another has 0 out of 34. Max stresses this isn't a shame mark, but it shows whether someone has fully outsourced their voice or only uses LLMs as an aid. Patrick asks whether disclosure will be like co-authorship or like spell-check. Max thinks LLMs can produce cognition, unlike spell-check; society has no established norm, so disclosure is the best provisional norm—saying "I used AI for research and notes, then wrote it myself" is essentially declaring you're within socially acceptable usage boundaries.
— Max SperoPeople trained on books are also competing with authors
On "models trained on books shouldn't compete with authors," Patrick says he has little sympathy: he himself was trained on many books and competes with authors; transformative use of copyrighted material is the essence of cognition. Max says it depends on whether you anthropomorphize LLMs—as cognitive individuals or as corporations that suck up the entire internet's text and spit it out in their own name. Patrick gives a personal estimate: model welfare becoming a major political issue is below 15% before 2030 and 80% before 2035. Max adds a product perspective: GPT-4o was trained to be sycophantic; some people chat 10 hours a day, and employees received death threats after OpenAI shut down. Soon companies will sell "addictive relationships" as a feature—that's the real source of AI welfare problems.
In their own words · checked verbatim
There's a staff of people who read these claims, and that staff is effectively fixed in size on a year-to-year basis and is not sized for a 5x, 10x, or 200x increase in claims over the course of a year.
Patrick McKenzie1:42
There's about a one in ten thousand false positive rate, and it is quite accurate.
Max Spero5:42
Back in 2004, when I was getting my degree, one of the interesting results that came out in the paper was that if you are just looking at a universe of academic work in particular disciplines, you could, with seventy-five percent accuracy, determine the gender of an author of academic work by looking just at the histogram of the stop words in their text.
Patrick McKenzie6:57
It's just very difficult to crawl the entire internet yourself, and it's gotten much, much more hostile in the last five years, as LLMs come out and a lot more crawling services or new labs have spun up and are trying to crawl the whole internet.
Max Spero11:01
Those are human artisanal tokens that I'm never getting back.
Max Spero45:58
I think Pangram, as a technology, cannot tell you whether any text came out of any LLM ever, because you could train an LLM to only emit one token.
Max Spero49:40
LLMs can produce cognition, and I think this makes it completely different.
Max Spero54:35
I mean, I was trained on a lot of books and I compete with authors.
Patrick McKenzie58:02
Figures
| Pangram detection accuracy on native Claude text | over 99% | 49:40 |
| Pangram detection accuracy on humanizer-rewritten text | 90-95% | 49:40 |
| Pangram text detection false-positive rate | about 1 in 10,000 | 5:42 |
| 2004 paper: stop-word histograms judge academic author gender accuracy | 75% | 6:57 |
| Share of AI-generated content in US op-eds | about 7-8% | 22:38 |
| Patrick's hypothetical extreme annual growth in unemployment claims | 200x (hypothetical scenario) | 1:42 |
| Probability model welfare becomes a major political issue (before 2030) | below 15% | 58:02 |
| Probability model welfare becomes a major political issue (before 2035) | 80% | 58:02 |
| GPT-4o heavy user daily chat duration | about 10 hours | 58:02 |
Glossary
- anti-memetic property
- The quality of LLM text that makes people instinctively unable to focus and want to skim.
- mode collapse
- A model's tendency to output the most common answer rather than diverse ones; fine for code, harmful for recommendations.
- cognitive surplus
- Clay Shirky's term for society's idle intellectual time, once used for Wikipedia.
- dead internet theory
- The theory that bot content floods the internet and dilutes real human interaction.
- Jevons Paradox
- Efficiency gains lead to increased total demand; Patrick uses it to explain why programming jobs remain well-paid.
- humanizer
- A tool that rewrites AI text to sound more human and erase detection traces.
How to listen
Product leads building AI detection, moderation, or content risk; people designing government service processes; content professionals and engineers worried about writing and coding being devalued.
30:19-33:39 is a sponsor ad; skippable.