Models identify chemists from molecules — but learn style, not chemistry
Classifying 1,815 chemists' molecules, AI predicts authorship with 60% top-5 accuracy based on style alone, yet fails to learn actual binding physics — why Leash trains models exclusively on proprietary DEL data.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
DNA barcodes enable screening millions of molecules in one tube
DNA-encoded libraries (DEL) tag each small molecule with a unique DNA barcode, allowing hundreds of millions to mix in a single tube. Target proteins act like hooks — magnetic beads pull out bound molecules, then sequencing identifies them by barcode — eliminating well-by-well screening costs. The largest DEL library built contains 40 trillion molecules, vastly exceeding traditional screening at only the million-molecule scale. The industry is infamous for hard-won scars: false positives (protein binding DNA instead of the molecule) and building-block labeling errors running 20-30%.
— Ian QuigleyOn the hardest test set, no one beats random guessing
Two years ago Leash ran a Kaggle competition to test whether the industry could predict molecular binding. They designed three difficulty tiers: shared building blocks (easy), shared triazine scaffold only (medium), completely different chemical libraries (hard, unrevealed before competition). After 2,000 teams competed, on the hardest tier no one beat random guessing — including teams attempting advanced methods like 3D graphs and spherical convolutional networks. Simple SMILES text representation outperformed fancier molecule encodings. Leash concluded the field is near zero understanding of binding prediction across different chemical spaces.
— Ian QuigleyModels identify chemists, not binding sites
At Recursion, Ian and his co-founder watched a chemist recognize a molecule at a glance and identify who made it — raising suspicion that models learn chemist style rather than chemical rules. They downloaded all data from Chemble for 1,815 chemists with 30+ published papers and 600+ molecules each, trained a classifier to predict authorship from molecular structure alone, and achieved 60% top-5 accuracy. More damning: using only the 1,800-dimensional ‘author probability vector’ as the molecule's sole representation predicted binding activity as well as real chemical structure. Combining both showed no improvement — evidence models learn chemist signature and sociological traces, not binding physics.
— Ian QuigleyPerfect-scoring structure models might just memorize the crystal database
Leash built Hermes, a minimal cross-attention model using protein and molecule embeddings, to run controlled experiments quickly. Clustering the training-test split by building-block similarity showed XGBoost performance degrading steadily as distribution shift increased, while Hermes stayed relatively stable — evidence of superior generalization. But a parallel control exposed a different problem: when test molecules and protein (BRD4) happened to appear in the PDB crystal database, the structure model Boltz scored nearly perfect while Hermes performed poorly. Boltz likely memorized crystal structures rather than understanding binding physics.
— Ian QuigleyMutating binding pockets proves whether models understand physics
To prove models understand physics rather than memorization, Leash deliberately disrupts binding pockets. First they screen wild-type protein to confirm which molecules bind, then add a known competitor for that pocket at high concentration to see which molecules get displaced, finally mutate the pocket amino acids at the DNA level, re-express the protein, and screen again. If the same molecules displaced by competition also vanish after pocket mutation — while molecules binding elsewhere remain — it's strong evidence those molecules truly bind that specific pocket. They run this at scale: hundreds of experiments per week, compared to traditional medicinal chemistry validating a handful of examples in a lifetime.
— Ian QuigleyAttention mechanisms nickname proteins instead of learning chemistry
The team expected Hermes' cross-attention to focus on the binding pocket. Instead, the model fixated on a small, non-conserved, protein-specific amino acid sequence — a region unrelated to binding function, merely unique to that protein. Hermes was giving each protein a nickname, not learning binding chemistry. The fix: generate many mutants. When two proteins have almost identical sequence except for one pocket amino acid — yet behave completely differently — the nickname strategy fails entirely. The model is forced to examine all amino acids; retraining makes attention far more uniform.
— Ian QuigleyShifting to natural language for chemistry reasoning via reinforcement learning
Recently Leash shifted to using natural language rather than numerical vectors for chemical reasoning. They train open-source language models to describe molecular structures in English and articulate differences between molecules, give them binder/non-binder comparisons for reasoning, then let the model autonomously pick molecules from a real database to test, update hypotheses after results, and iterate toward the best binder. Leash believes natural language is a stronger memory regularizer than stacking parameters — harder to exploit — and creates a physics-grounded, scalable reinforcement learning environment, unlike traditional autoregressive molecular generation methods.
— Ian QuigleyMoat is institutional process culture, not algorithms
When asked why models matter if massive screening can find good molecules, Ian explains: finding chemicals for a single target doesn't require models, but teaching models to generalize across proteins and chemical space does. Leash has built roughly 2,000 protein constructs, run about 5,000 screens covering at least 6.5 million molecules. A contract research organization would charge ~$100,000 per screen. The real moat isn't algorithm — it's process culture built by hitting every conceivable ‘experimental cheat mode’ over years. For instance, they found a class of molecules resembling the protein-folding drug Lumacaftor that exclusively hit misfolded proteins. Only teams staring at data long-term catch those traps; money cannot buy institutional memory.
— Ian QuigleyIn their own words · checked verbatim
Mike was like, hey, looks like a Stuart Shribert molecule. And then we were like, oh no, you can tell just by walking by? You're like, who made this? That's horrifying.
Ian Quigley0:00
When you train models on the exhaust, you might be in trouble.
Ian Quigley1:00
And then when you go to the most challenging eval, no one did better than chance.
Ian Quigley31:53
It's well known that things like molecular weight will just – you know, all you have to do is sort by molecular weight. That's like one of the single biggest predictors you can possibly have.
Ian Quigley52:40
Our models did kind of bad on that molecule. It was a distant chemical library. They weren't very good at it. And Bolts was perfect at it.
Ian Quigley58:53
we realized that Hermes was probably coming up with a nickname for the protein. It's not thinking about, oh, this is the functional part. It's like, I don't need to learn that. I can remember, you know, 10 or 12 amino acids that are different here than everyone else. And I can just memorize it.
Ian Quigley1:12:27
And what we are currently doing is we're using readable, interpretable English to represent the small molecules and the proteins and their behaviors.
Ian Quigley1:15:47
We have a bunch of chemicals in our chemical library that looks like Luma-Captor, this thing whose job it is to refold misfolded proteins. Guess what kind of proteins it hits when we screen them?
Ian Quigley1:45:21
Figures
| Chemist identification (top-5 accuracy) | 60% | 47:28 |
| Chemists in Chemble dataset (30+ papers, 600+ molecules each) | 1815 | 47:28 |
| Largest DEL library ever built | 40 trillion molecules | 4:02 |
| Building-block labeling error rate | 20-30% | 26:39 |
| Binding interactions: Leash Kaggle dataset vs BindingDB | ~4 billion vs fewer than 3 million | 27:43 |
| Teams in Kaggle competition | 2000 | 30:48 |
| Leash screening scale to date | ~2000 protein constructs, ~5000 screens, at least 6.5 million molecules per screen | 1:39:10 |
| Time from DEL hit to mouse in vivo PKPD | ~6 months | 1:28:03 |
Glossary
- DNA-encoded library (DEL)
- Millions of small molecules each tagged with a unique DNA barcode, mixed in a single tube to screen binding via magnetic bead selection and barcode sequencing.
- SMILES
- Linear text representation of molecular structure using ASCII characters; encodes connectivity without 3D coordinates.
- Clever Hans effect
- Model appears to solve a task but actually exploits unrelated signals; named after the horse that could ‘count’ by reading human body language, not math.
- Activity cliff
- Small structural change in a molecule produces drastic swing in binding affinity or other biological activity.
- Cross-attention
- Architecture component that explicitly links different parts of two inputs (e.g. protein sequence and molecule structure) so the model weights their relevance.
- ADMET
- Five pharmacokinetic properties — absorption, distribution, metabolism, excretion, toxicity — that determine whether a molecule can become a real drug.
How to listen
Machine learning researchers in drug discovery, those managing biological data quality, and early-stage biotech investors — especially anyone concerned whether models learn artifacts rather than true physical laws.
Sections on wet-lab details within the DEL workflow, such as PCR amplification and barcode design, can be skipped.