Open-source models pass through five or six intermediaries, who fills the accountability gap in the EU AI Act?
Open weights do not equal open trust; after passing through five or six intermediaries, responsibility for open-source models becomes blurred, and the EU AI Act's exemptions may leave a structural accountability gap. Europe needs layered responsibility and patient capital.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
Don't ask the public to trust more; demand technology be more trustworthy
She first breaks down 'trust' in the technical context into four meanings: engineering reliability and dependability; the zero-trust paradigm in cryptography—'never trust, always verify'; institutional trust, i.e., trusting a company's policies, incentives, and accountability mechanisms; and epistemic trust, i.e., whether what technology tells us is true. Wikipedia has verifiability norms, journalism has source checking, academia has citation systems, but AI is challenging all these mechanisms simultaneously. Her conclusion: rather than asking the public to trust technology more, we should demand that technology and its underlying institutions be more trustworthy.
— Sebnem ErenerOpenness is a spectrum, not a switch, but exemptions are designed as a switch
The EU AI Act sets tiered obligations for general-purpose AI models but carves out exceptions for open-source models: as long as they are released under an open license, not commercialized, and weights are public, some obligations are waived. The surface logic holds—if weights are public, piling on compliance burdens seems pointless. But she points out: openness is a spectrum, not a switch, and 'not commercialized' is ambiguous in the open-source ecosystem. More critically, the law assumes a single responsible party—'one company placing a model on the market'—which does not match the reality of the open-source ecosystem.
— Sebnem ErenerOpen-source models pass through five or six parties, but the law recognizes only one
After an open-source model is uploaded to Hugging Face, it gets served by inference platforms, routed by aggregators, fine-tuned on third-party infrastructure, and by the time it reaches end users, five or six intermediaries have touched it and exerted some form of control. Legal systems generally assume a model is placed on the market by a single company, which does not hold at all along this chain. The core of her research is identifying the gaps in the downstream chain of responsibility among these intermediaries. This is also a concrete example of her view that the deeper the openness, the higher the governance complexity.
— Sebnem ErenerOpenness without audits reduces trust, not increases it
Her proposed response is not to put all responsibility on the original developer but to layer it: developers bear primary responsibility for documentation, risk assessment, and known misuse risks; hosting platforms, fine-tuners, deployers, and even end users who intentionally misuse each bear their own responsibility. She also emphasizes that openness is a mechanism for distributing access and participation rights, not something that automatically brings transparency or fairness. Responsible openness also requires independent audits, staged releases, incident reporting, and community governance. Without these, open source can actually reduce trust, because no one knows who to hold accountable after a model spreads, whereas closed-source models at least have clear responsibility.
— Sebnem ErenerEuropean tech sovereignty cannot be judged on a single timeline
The European tech sovereignty package contains several timelines with very different speeds: legal sovereignty advances fastest—the Cloud and AI Development Act already requires data to be located in the EU, owned and controlled by the EU; model sovereignty is medium-speed—Open Euro LLM, with 20 institutions, releases its first industry model next month, and Europa, led by Italian company Domin, aims to challenge the world's strongest; hardware sovereignty is slowest—the Chips Act only invests 200 billion euros by 2035. The biggest risk is bundling everything into one timeline, generating uniform optimism or pessimism. We should be decisive on fast-moving things and patient on slow ones.
— Sebnem ErenerOpen models win on quantity, but governance difficulty doubles
She originally envisioned a future where 'the model layer is open, the agent layer is closed,' but reality is not like that. Open-weight models dominate in quantity: about 60% of tokens on the largest model router come from Chinese open models, and four of the five most-used models globally are Chinese open models. US closed-source labs do not counter with open source but instead use different value-capture strategies to segment the market. The mixed landscape means both governance systems must be done right: the open-source side needs audit infrastructure, explainability, and incident response; the closed-source side needs clear responsible entities—this is the most complex governance scenario.
— Sebnem ErenerEuropean companies are not unable to build; they are acquired too early
Asked what Europe needs most, her answer is direct and somewhat counterintuitive: capital. Specifically, growth-stage capital—patient, risk-tolerant, able to stay until later stages. Europe does not lack innovation; it lacks patient capital that prevents companies from being bought out too early. European companies are not unable to build; they are acquired too early. She hopes for investors who believe in Europe's future and in European founders. This point pulls the tech sovereignty discussion back from models and chips to financial infrastructure, a layer that is easy to overlook.
— Sebnem ErenerThe more autonomous the system, the heavier human responsibility, not lighter
She worries that the word 'agency' is being conflated in AI discussions. In agentic AI, agency refers to a system's ability to autonomously execute tasks, and AI is rapidly gaining this capability. But in the legal sense, agency refers to an entity that has intent, can understand consequences, and can be held accountable—exclusively natural or legal persons. A person can be sued, fined, and held responsible; an AI model, no matter how autonomous, cannot. The more autonomous the system, the heavier the responsibility on humans and companies, not transferred to the model. Linguistic confusion can imply that models can bear responsibility that should belong to people.
— Sebnem ErenerIn their own words · checked verbatim
It's actually the paradigm is zero trust. It's the philosophy is never trust, always verify. So it's the rejection of trust.
Sebnem Erener2:01
I care about trustworthiness more than trust.
Sebnem Erener3:06
So by the time it reaches an end user, there are five or six intermediaries that touch it and exercise some sort of control.
Sebnem Erener5:11
So in open source AI, like, because everyone has access doesn't mean that no one is responsible.
Sebnem Erener11:37
It should be best understood as a mechanism for distributing access and participation rights.
Sebnem Erener12:41
Trust in AI right now feels a bit borrowed and a little bit incomplete.
Sebnem Erener20:02
It's a boring answer, but capital. Growth stage capital, patient, risk tolerance, late stage capital.
Sebnem Erener22:05
A person can be sued, fined, held liable. An AI model cannot as much as they are agentic.
Sebnem Erener23:10
Figures
| Number of intermediaries an open-source model touches before reaching end users | 5-6 | 5:11 |
| Number of institutions in the Open Euro LLM consortium | 20 | 15:56 |
| EU Chips Act investment target | 200 billion euros (by 2035) | 16:56 |
| Share of tokens from Chinese open-weight models on the largest model router | about 60% | 19:02 |
| Number of Chinese open models among the five most-used models globally | 4 | 19:02 |
Glossary
- zero trust
- A security paradigm: never trust any entity by default; verify every access request.
- open washing
- Claiming open source merely by releasing weights, without disclosing key information such as training data.
- general purpose AI models
- A category in the EU AI Act for AI models covering a broad range of tasks, the main body of frontier models.
- layered responsibility
- Allocating responsibility along the value chain: developers, hosters, fine-tuners, deployers, and users each bear their own responsibility.
How to listen
Founders, investors, and model supply chain engineers working on open-source models, researching EU AI Act compliance, or evaluating European market opportunities.
The guest introduction from 0:00 to 1:00 can be skipped; after that, the information density is high, so it's recommended to listen to the rest.