The Information Machine
Updated todayDay 2in the edition since 9 Nov 20263 sources

Hinton and Bengio's AI oversight paper

The gist

Latest: Hinton, Bengio co-author RSI policy paper; Anthropic lobbying record disputed

Prominent researchers are framing RSI as a near-term policy concern and calling for government visibility into companies' internal automation of AI development. The gap critics identify between Anthropic's safety rhetoric and its actual regulatory positions bears on whether voluntary or enforceable oversight frameworks gain traction.

When stories like this move, I write about them in an email each morning.

One free email each morning, around 8 AM US Eastern.

The full picture

The paper 'What if Automating AI R&D Triggers an Intelligence Explosion?' is co-authored by Geoffrey Hinton, Yoshua Bengio, Jakub Pachocki, Jack Clark, Hilary Greaves, Tom Davidson, and others. It calls on policymakers to demand transparency from AI companies about how far they have automated model development and how close they are to recursive self-improvement (RSI). The Wall Street Journal covered the paper under the headline 'Top AI Researchers Call for Urgent Oversight of Self-Improving Systems.' Google's Astra AI system has demonstrated the ability to directly edit LLM weights, with the edits described as useful rather than merely syntactically valid, cited as an example of recursive self-improvement capability. Separately, Dario Amodei testified under oath that AI is 'the greatest threat humanity has ever faced,' while critics argue Anthropic's regulatory advocacy record does not match its stated strategy of racing to maintain credibility for strong regulation.

How it developed
9 November 2026added 10 Nov

Google's Astra AI system demonstrated direct editing of LLM weights in ways described as useful, cited as a concrete example of the recursive self-improvement capability the paper by Hinton, Bengio, and others asks governments to monitor.

Dario Amodei testified under oath that AI is 'the greatest threat humanity has ever faced,' but Anthropic declined a legally binding voluntary commitment framework California offered, and critics argue its regulatory advocacy record looks similar to a pure capability-racing strategy.

First in the edition · 9 November 2026
8 November 2026in the 9 Nov edition

A paper by Geoffrey Hinton, Yoshua Bengio, Jack Clark, and others asks policymakers to require AI companies to disclose how far they have automated model development and their proximity to recursive self-improvement.

Three congressional proposals run alongside it: the Sanders-Casar Ban Artificial Superintelligence Act, introduced September 2026, would pause advanced AI development with up to 20 years in prison. The Foundation for American Innovation called the FRONTIER Act the most promising AI governance legislation. And a Cruz-Thune-Klobuchar bipartisan bill is in negotiation with no text released.

7 November 2026

Advocacy thread calls on Congress to urgently ban RSI, citing OpenAI and Anthropic timelines of 2027-2028

5 November 2026

Google's Astra AI system reported to directly edit LLM weights in ways described as useful, cited as recursive self-improvement capability

4 November 2026

Sam Altman states human control over the future should serve as an ultimate constraint on AI development

2 November 2026

Dean Ball argues RSI will come from scaling once-mundane research techniques to superhuman scale

28 September 2026

Paper 'What if Automating AI R&D Triggers an Intelligence Explosion?' published, co-authored by Hinton, Bengio, Pachocki, Clark, and others; Wall Street Journal covers it

23 September 2026

Sanders and Casar introduce the Ban Artificial Superintelligence Act with a 20-year prison penalty for violations

Sources

Get the daily email

When stories like this move, I write about them in an email each morning.

One free email each morning, around 8 AM US Eastern.

See the latest edition · Privacy