The Open-Weight Dilemma

1. The question

What happens to cyber risk — and to the regulatory architectures meant to contain it — when the most capable dual-use artifact in computing history can be copied infinitely and modified by anyone? This is the zero-day question at AI scale. Earlier entries in this series established that vulnerability behaves as a commodity (INV-003), that trust architectures fail invisibly through single poisoned components (INV-002), and that capability markets require governance designed from operational ground truth (INV-004). Open-weight LLMs combine all three dynamics: a freely copied capability, a safety regime that fails silently under fine-tuning, and a governance debate conducted largely by parties who have never operated the capabilities under discussion.

2. Method note

Movements. All four movements of the house method were engaged. The conceived adversarial process: assess open-weight models from the operator’s chair — which offensive workflows they actually accelerate — then hold each regulatory provision against that ground truth and ask a single question of it: can this obligation be discharged by an actor who no longer controls the artifact?

Instruments. Operator-perspective threat analysis across the malware lifecycle, social-engineering, and vulnerability-discovery workflows; provision-level mapping of the EU AI Act and the draft GPAI Code of Practice; translation of one body of findings into five registers — research, policy, standards, open-source community, and strategic context.

Exclusions. The analysis deliberately excluded speculative harms in favour of capability uplift that is real and measurable — because regulation aimed at the former while missing the latter is how mitigation gaps are made. The Studio register was deliberately left open (see §7).

3. Findings

1. Open-weight LLMs deliver real, specific, asymmetric cyber-capability uplift — concentrated in malware development, social engineering, and vulnerability discovery — and this uplift is unmonitorable and irrevocable once weights circulate.
2. The EU’s regulatory instruments, as drafted, exhibited a mitigation gap: obligations addressed to original providers cannot operationally reach the downstream actors who create deployment risk by modifying weights.
3. Provider-anchored liability is not only ineffective but counterproductive: it punishes openness, deters European open-model development, and undermines the open strategic autonomy the Union simultaneously pursues.
4. The workable corrective re-anchors responsibility at the point of substantial modification — protecting open release while binding those who alter weights — complemented by standardised evaluation of cyber-capability uplift, a task for technical standardisation bodies, not legislation alone.
5. Governance of uncontrollable artifacts is a security-engineering problem, and security engineering has relevant prior art: threat modelling, trust-boundary analysis, and the lesson of INV-002 — architectures fail at their weakest assumed-trustworthy component.

4. Consequence

Regulatory. Through expert technical consultation to the EU AI Office (2025), the investigation contributed to the substantial modification clause in the General-Purpose AI Code of Practice — protecting open-model providers from undue liability while making downstream actors who tamper with model weights responsible for their modifications.

Standards. The ETSI Security Conference 2025 address led to an invitation from the Chair of ETSI TC SAI (Securing AI) to join the committee’s standardisation work. Status: membership in progress; sponsorship will be disclosed in the Register upon settlement.

Discourse. The mitigation-gap framing entered circulation across the security, policy, and open-source communities, from Sophia Antipolis to FOSDEM Brussels.

5. Artefact record

#	Artefact	Type	Venue / identifier	Date	Access
1	Mitigating Cyber Risk in the Age of Open-Weight LLMs: Policy Gaps and Technical Realities	preprint	arXiv:2505.17109	2025	arxiv.org/abs/2505.17109
2	The Open-Weight Dilemma: Cyber Policy When Control on AI is Lost	article	Pwnshow	2025	report
3	Securing the Open AI Frontier: Addressing Cybersecurity Risks of Open-Weight LLMs through Technical Realities and Standardisation	talk + slides	ETSI Security Conference, Sophia Antipolis	Oct 2025	slides
4	Expert technical consultation on the EU AI Act / GPAI Code of Practice	consultation	EU AI Office	2025	Record of engagement; substance reflected in row 5. input
5	Substantial modification clause, GPAI Code of Practice	policy outcome	GPAI Code of Practice	2025	GPAI Code of Practice
6	ETSI TC SAI participation (invited)	standards contribution	ETSI TC SAI, Sophia Antipolis	2025–	Status: membership in progress; sponsorship will be disclosed in the Register upon settlement.
7	The Open-Weight Dilemma: Mitigating AI Cyber Risks Without Killing Open Source	talk + slides	FOSDEM, Brussels	Jan–Feb 2026	talk
8	Beyond the Digital Fortress: Europe’s Quest for Open Strategic Autonomy	article	Zenodo	2025	doi.org/10.5281/zenodo.18525948
9	Studio register	artwork	—	forthcoming	In development; see §7.
10	Open Weights, Open Societies: Governing AI Without Rebuilding the Walled Garden	talk	Think Twice, Brussels	forthcoming	–

6. Continuity

Inherits: the commodity dynamics of vulnerability (INV-003), the silent failure of trust architectures (INV-002), and governance from operational ground truth (INV-004). Feeds: the longitudinal empirics of INV-006 (Qubes OS, RAID 2026) and this investigation’s agenda converge on a candidate successor — measuring model-assisted vulnerability discovery rather than reasoning about it: the mitigation gap, quantified.

7. What’s next

The Studio register: a cultural treatment of the open-weight dilemma, making the irrevocability of released weights experiential. When delivered, INV-007 becomes the first investigation to traverse all three validation systems — peer review, regulatory process, and curatorial selection — completing the demonstration the agency is built on. Through ETSI TC SAI: evaluation methods for the cyber-capability uplift of open-weight models.

8. Provenance

Provenance. Mitigating Cyber Risk in the Age of Open-Weight LLMs (arXiv:2505.17109) and the related outputs above were produced without external funding. Their subject — the cyber-risk governance of open-weight AI models — is adjacent to but distinct from the vulnerability-acquisition market in which Zeronomi, founded and directed by the author, operates; the positions recommended would have no foreseeable differential effect on Zeronomi’s commercial interests. The analysis relies on the author’s general expertise and on the public sources cited, not on non-public commercial information.

Added 2026/07/07 under Policy v1.0.