The biggest engineering challenge in legal AI is not models—it's data you can never see
Most client data isn't even visible to the law firm itself. Legora's evals team has to guess what lawyers actually do, working only with classified and rewritten synthetic data.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
The hardest hire: lawyers from top firms who can code
Legora's evals team blends engineers and lawyers, plus a distinct group of legal engineers—lawyers with CS backgrounds who spent three to four years at top firms. They're the hardest intersection to fill. Jacob calls them the company's hardest hire, but they're invaluable: they run evals and directly shape product design because they use the product every day and see clients every day. They become force multipliers who can summon a hundred-person expert pool to reshape the product.
— Jacob LauritzenAI may not be able to fix the law firm partnership model
Partnership exists because law firms' only capital is human capital—lose people, lose the firm, so just make people shareholders. Jacob argues AI should let firms codify experience and methods into reusable systems—say, a self-iterating knowledge graph—building firm-owned assets independent of any person. Instead, partners are paid by eating what they hunt, so encoding experience costs them clients. And those assets don't follow you when you leave partnership. The incentive misalignment has no answer yet.
— Jacob LauritzenJunior lawyers aren't disappearing—they're splitting into two tiers
The host suggests paralegals are obsolete, junior lawyers are obsolete. Jacob pushes back: that's not what they're seeing. He uses engineers as analogy: a great engineer is worth far more than two years ago because AI multiplies output; same for lawyers. Good junior lawyers become more valuable because AI lets them produce more, higher-quality work. This split always existed—now it's amplified, not the whole group gone.
— Jacob LauritzenBuild evals when you cannot see a single user's real data
Legora's biggest engineering problem isn't model ability—it's that you can't see what customers do with the product. Data stays under lock. Often it doesn't even belong to the law firm; it belongs to their clients. Looking is forbidden. The company's only move is its evals team: guess, approximate real scenarios, build those guesses into test sets. Rewritten, anonymized synthetic traces are the only workaround. The stricter the anonymization, the weaker the signal. Jacob calls this the most maddening knot in the business.
— Jacob LauritzenThree years ago law firms wouldn't touch AWS
Legora runs entirely on Azure. Early in the company, law firm customers flatly didn't trust AWS or Google—no one had audited them. But Microsoft was already Office 365's vendor, so it had passed their audit test. Azure became the only option. In three years, that cloud skepticism nearly evaporated. Talk of private deployment vanished too. Legal's cloud migration was compressed into three years by AI pressure.
— Jacob LauritzenLegal engineers don't write code; frontline engineers do
Legora splits on-site support in two: legal engineers are big-firm alumni who know the product and customer workflows—they configure like power users, no code. Forward deployed engineers actually write code and hit customer APIs, usually serial entrepreneurs or large-company veterans. Big customers' custom needs go to legal engineers first; if they can't handle it and it needs product change or deep integration, escalate to forward engineers. This two-tier split lets the company staff big customers without assigning a full engineering team per customer.
— Jacob LauritzenTwenty engineers to beat everyone—he gave up that idea
Jacob once planned to keep the company to twenty engineers forever and beat much larger competitors. The team is now a hundred; he still thinks it's short. Hard competition and customer demand killed the minimalism. Now his thinking is different: collapse the product to a few core primitives—Lists, Projects, Agent. Customers think the company builds many things. Actually they're combinations of those three. He's even imagining a future where agents are themselves APIs, one agent call instead of a full interface to maintain.
— Jacob LauritzenWill large model companies squeeze out companies like Legora?
Anthropic and OpenAI have already squeezed out startups dependent on their models—once in code, once in search. Will legal be next? Jacob's read: no. Model companies must cover all enterprise knowledge work at once—law, data analysis, sales, finance. They can't go deep enough in one vertical. Small customers can use Claude for light legal work. But a $400 million acquisition? Enterprises pick the specialist.
— Jacob LauritzenIn their own words · checked verbatim
Paralegals, gone. Yeah. Junior lawyers, gone. I don't need them.
something that I'm toying with is, like, how much of our product can just be the agent versus all the other stuff?
Jacob Lauritzen0:00
we had one problem where, like, it wouldn't cite properly, and so it would be difficult to follow, and it was like next model release solved.
Jacob Lauritzen11:00
we can't look at any traces, basically. We can't look at any data 'cause it's all super confidential.
Jacob Lauritzen27:00
when we started, law firms were like, "AWS? No."
Jacob Lauritzen29:00
we're just gonna be twenty engineers for the rest of the company's history, and, like, we're gonna beat everyone who's much bigger. And we're a hundred engineers now, and it feels like we're way too small.
Jacob Lauritzen50:45
But if you're doing a, you know, $400 million M&A transaction, you're gonna want Legora.
Jacob Lauritzen1:04:00
Figures
| Citation error correction accuracy after model upgrade | 70% to 99% | 11:00 |
| Max cells in single Tabular Review run | 1,000,000 | 21:50 |
| Due diligence room example: employment contracts | 10,000 | 9:39 |
| Engineering team size (past vs. now) | from 20 to ~100 | 50:45 |
| New York office headcount | ~100 | 1:05:00 |
| Example acquisition deal | $400 million | 1:04:00 |
Glossary
- Tabular Review
- Batch-evaluate thousands of documents with an agent asking questions on each, then aggregate results into a table.
- Forward Deployed Engineer
- Engineer based at a large customer site who writes code directly for deep-custom integration.
- Legal Engineer
- Lawyer with a CS background who spent years at a top firm; handles product configuration and evals, writes no code.
- Pareto frontier
- Strategy of always choosing the current best model mix for the tradeoff between ability, cost, and latency.
- DPA
- Data processing agreement; contract specifying which suppliers may receive data, determining which AI models or tools can be used.
How to listen
Enterprise AI product managers, legal tech practitioners, engineers concerned with building products and running evals under data-privacy constraints.
The closing discussion of law school scholarships and CS-to-JD programs is low-signal filler.