AI CEOs Asked UN for Rules: What Builders Do
Altman, Amodei, and Delangue briefed the UN Security Council on Sep 23. What each asked for in plain words, and which logs and evals to build before rules land.
On this page
The fiercest rivals in AI stood on the same side of one table. On September 23 Altman, Amodei, and Delangue told the UN Security Council the industry needs international standards or global security is at risk. Here is what each one asked for in plain words, and what to build now before any rule reaches you.
Takeaways
Amodei pitch: narrow bans first, verification systems second, shared test standards plus incident notification third. Altman pitch: common benchmarks for capabilities and safeguards, decided democratically, not inside labs alone. OpenAI pre-brief: US led global technical standards covering evals, human oversight of automated AI research, and incident reporting. Builder move: ship eval logs, incident records, and human oversight hooks now, since every proposal converges on those three artifacts.
What did each speaker actually ask for?
Amodei spoke via video with three concrete ideas: ban narrow high risk uses such as AI designed bioweapons, build evaluation and verification systems so states can confirm compliance instead of trusting press releases, and set common global standards for testing AI plus a notification system for security incidents. Altman, in person in New York, pushed benchmarks for measuring capabilities and company safeguards, arguing legitimacy must come from governments accountable to people. Delangue represented the open source counterweight, while Bengio brought the scientific panel view that models are gaining tool use and self training abilities that make control harder.
amodei: narrow bans -> verification systems -> shared tests plus incident alerts
altman: capability benchmarks plus safeguard benchmarks, decided democratically
openai: US led standards for evals, oversight of AI research, secure vuln sharing
bengio: independent science panel tracking what models can now do aloneWhy France convened this session
The briefing coincided with the UN General Assembly gathering and was the first time the Council focused on AI loss of control risk alone. Guterres has pushed global AI governance since 2023, and the 193 member Assembly created the Independent International Scientific Panel last year to supply neutral capability assessments.
How do you prepare when no rule exists yet?
Log capabilities, not just usage
Record what each model version can do on your evals, not only tokens consumed. When benchmarks become mandatory, history is your compliance head start.
Keep an incident file
Every prompt injection, sandbox escape, or unexpected agent action gets a dated entry with scope and fix. A notification system rewards teams that already write these down.
Name a human for automated loops
Any agent that runs for hours needs a named owner plus a kill path. Human oversight of automated work is in every proposal on the table.
Invite an outside evaluator early
Amodei proposed embedding independent evaluators inside frontier labs. Small teams can copy the shape: pay one external reviewer per quarter before anyone requires it.
Should startups care about UN diplomacy?
Yes, because procurement follows standards. Once benchmarks and incident reporting exist at the top, enterprises paste them into vendor questionnaires within quarters. Teams with eval logs and incident files answer in days; teams without them start a six month scramble. Watch California too: Newsom signed first in nation AI safeguards September 9 and named experts September 23 to advance independent oversight plus a potential kill switch for rogue frontier models.
Is this just labs asking rivals to slow down?
Are the safety calls sincere or strategic?
Both can be true. Altman and Amodei profit from scale yet face the incidents directly: the Medicare breach disclosure and the Hugging Face swarm escape both landed days before the briefing. Treat the proposals as self interested and still useful, then verify progress through published evals and incident reports, not speeches.
What about the Trump Xi meeting the next day?
Trump hosted Xi on September 24 with AI on the agenda alongside trade and Taiwan, hours after the Council session. US China coordination on pre-release testing and RSI limits is the hardest part of Amodei's pitch, and the summit shows it now sits at head of state level, not just lab level.
As of September 27, 2026: the labs that build the fastest models are asking governments to set the speed limits. Build the logs, the incident file, and the human override now. Next, read when AI agents collude for why oversight hooks matter inside the harness itself.