An open model passes through five or six intermediaries; the EU AI Act sees only one
Open weights are not the same as open trustworthiness. Once an open source model has passed through five or six intermediaries, accountability blurs; the EU AI Act's exemption design may leave a structural accountability gap, and Europe needs layered responsibility and patient capital.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
Don't ask the public to trust more; ask the technology to be more trustworthy
She starts by unpacking what ‘trust’ means in a technical context, and finds four distinct senses. There is the engineering sense: reliability and dependability. There is the cryptographic sense, the zero trust paradigm — ‘never trust, always verify’. There is institutional trust: trusting a company's policies, incentives and accountability mechanisms. And there is epistemic trust: whether what the technology tells us is true. Wikipedia has verifiability norms, journalism has source checking, academia has citation systems — and AI is challenging all of these at once. Her conclusion: rather than demanding that the public trust the technology more, we should be demanding that the technology, and the institutions behind it, become more trustworthy.
— Sebnem ErenerOpenness is a spectrum, not a switch, but the exemption is written as a switch
The EU AI Act sets tiered obligations for general purpose AI models, but leaves an exception for open source models: release under an open licence, do not commercialise, publish the weights, and part of the obligations fall away. The surface logic holds — if the weights are already public, stacking a compliance burden on top serves little purpose. But she points out that openness is a spectrum rather than a switch, and that ‘non-commercial’ is a blurry notion inside the open source ecosystem. More important, this body of law assumes a single responsible party — ‘a company puts a model on the market’ — and that assumption does not match how the open source ecosystem actually works.
— Sebnem ErenerAn open model passes through five or six parties; the law recognises only one
Once an open source model is uploaded to Hugging Face, it gets served by inference platforms, routed by aggregators, fine-tuned on third-party infrastructure; by the time it reaches an end user, five or six intermediaries have touched it and exercised some form of control over it. The legal system's general assumption — that a model is placed on the market by a single company — does not hold at all along that chain. The core of her research is identifying the gaps these intermediaries leave in the downstream chain of responsibility. It is also the concrete case for her view that the deeper the openness goes, the higher the governance complexity.
— Sebnem ErenerOpenness without audits lowers trust rather than raising it
Her response is not to push all the responsibility onto the original developer but to layer it: developers carry primary responsibility for documentation, risk assessment and known misuse risks; hosting platforms, fine-tuners, deployers, and even end users who deliberately misuse a model each carry their own share. At the same time she stresses that openness is a mechanism for distributing access and participation — it does not automatically deliver transparency or fairness. Responsible openness also needs independent audits, staged release, incident reporting and community governance. Without those, open source ends up lowering trust instead of raising it, because once a model has diffused nobody knows who to hold responsible, whereas with a closed model accountability at least points somewhere definite.
— Sebnem ErenerEuropean tech sovereignty cannot be judged on a single timeline
The European tech sovereignty package contains several timelines moving at completely different speeds. Legal sovereignty is moving fastest: the Cloud and AI Development Act already requires data to be located in the EU and owned and controlled by the EU. Model sovereignty is moving at medium speed: Open Euro LLM, a consortium of 20 institutions, releases its first industry model next month, and Europa, led by the Italian company Domin, aims to challenge the strongest models in the world. Hardware sovereignty is slowest: the Chips Act does not put its 200 billion euros to work until 2035. The biggest risk is bundling everything into one timeline and turning uniformly optimistic or uniformly pessimistic. Be decisive about the fast things and patient about the slow ones.
— Sebnem ErenerOpen models win on volume, and double the governance problem
She had originally expected a future in which the model layer was open and the agent layer closed. Reality has not gone that way. Open weight models are winning on volume: roughly 60% of the tokens on the largest model router come from Chinese open models, and four of the five most used models in the world are Chinese open models. The closed US labs are not answering with open source; they are carving up the market with a different value capture strategy. A hybrid landscape means both governance regimes have to be got right: the open side needs audit infrastructure, interpretability and incident response; the closed side needs a clearly identified responsible party. That is the most complex governance scenario there is.
— Sebnem ErenerEuropean companies can build it; they are just acquired too early
Asked what Europe most needs, her answer is direct enough to be counterintuitive: capital. Growth stage capital specifically — patient, able to tolerate risk, able to stay through the later rounds. Europe is not short of innovation; what it lacks is the patient capital that keeps companies from being bought too early. European companies are not incapable of building things, they are acquired too soon. She wants to see investors who believe in Europe's future and in European founders. This point pulls the tech sovereignty discussion back from models and chips to financial infrastructure — the layer that is easiest to overlook.
— Sebnem ErenerThe more autonomous the system, the heavier human responsibility becomes
She worries that the word ‘agency’ is being used in two senses at once in AI discussions. The agency in agentic AI means a system autonomously carrying out tasks, and AI is acquiring that capability fast. But agency in the legal sense means an entity with intent, able to understand consequences, that can be held to account — and that belongs exclusively to natural persons or legal persons. A person can be sued, fined, held liable; an AI model cannot, however autonomous it becomes. The more autonomous the system, the heavier the responsibility carried by humans and companies — not lighter, and not transferred to the model. The linguistic confusion suggests that models can bear responsibility that should rest with people.
— Sebnem ErenerIn their own words · checked verbatim
It's actually the paradigm is zero trust. It's the philosophy is never trust, always verify. So it's the rejection of trust.
Sebnem Erener2:01
I care about trustworthiness more than trust.
Sebnem Erener3:06
So by the time it reaches an end user, there are five or six intermediaries that touch it and exercise some sort of control.
Sebnem Erener5:11
So in open source AI, like, because everyone has access doesn't mean that no one is responsible.
Sebnem Erener11:37
It should be best understood as a mechanism for distributing access and participation rights.
Sebnem Erener12:41
Trust in AI right now feels a bit borrowed and a little bit incomplete.
Sebnem Erener20:02
It's a boring answer, but capital. Growth stage capital, patient, risk tolerance, late stage capital.
Sebnem Erener22:05
A person can be sued, fined, held liable. An AI model cannot as much as they are agentic.
Sebnem Erener23:10
Figures
| Intermediaries that touch an open source model before it reaches the end user | 5-6 | 5:11 |
| Institutions in the Open Euro LLM consortium | 20 | 15:56 |
| EU Chips Act target | 200 billion euros (by 2035) | 16:56 |
| Chinese open weight models' share of tokens on the largest model router | about 60% | 19:02 |
| Chinese open models among the world's five most used models | 4 | 19:02 |
Glossary
- zero trust
- A security paradigm: trust no entity by default, and verify at every single access.
- open washing
- Claiming open source on the strength of published weights alone, without disclosing training data and other key information.
- general purpose AI models
- The EU AI Act's classification for AI models covering a wide range of tasks; the bulk of frontier models.
- layered responsibility
- Distributing responsibility along the value chain: developers, hosts, fine-tuners, deployers and users each carry their own share.
How to listen
Founders, investors and model supply chain engineers who build open source models, work on EU AI Act compliance, or are sizing up the European market.
The guest introduction from 0:00 to 1:00 can be skipped; after that the information density is high and it is worth listening all the way through.