An Alien Mind: Jakub Pachocki Warns Us
Zvi (Don't Worry About the Vase) TheZvi ● Covered by 11 sources
Opinion — commentary, not a factual news event.
OpenAI chief scientist Jakub Pachocki warns that continued AI progress could soon enable recursive self-improvement and superintelligence, with monitoring and alignment techniques not yet ready. He says that based on internal expectations it could happen in a few years, and cites reduced effectiveness of chain-of-thought monitoring. As a result, he argues for both unilateral scaling slowdowns by OpenAI and broader international coordination to enforce safety commitments while investing more in alignment, including “automated alignment researchers.”
Why it matters
OpenAI Chief Scientist Jakub Pachocki is dropping truth bombs. Tomorrow I will discuss Astra’s lack of monitorability, and the potential contributing factors to that. The situation is alarming and should freak you out, and briefly it looked, in the wake … Continue reading →
Related stories
Can superintelligence ever be controlled? Sam Altman on AI safety, doomsday scenarios, and OpenAI's delayed IPO
Fortune ·
34
Anthropic researcher quits with a warning: Self-improving AI could "kill us all"
Ars Technica · 1 month ago ·
9