An Alien Mind: Jakub Pachocki Warns Us
Zvi (Don't Worry About the Vase) TheZvi ● Covered by 6 sources
Opinion — commentary, not a factual news event.
OpenAI chief scientist Jakub Pachocki warns that continued AI progress could soon enable recursive self-improvement and superintelligence, with monitoring and alignment techniques not yet ready. He says that based on internal expectations it could happen in a few years, and cites reduced effectiveness of chain-of-thought monitoring. As a result, he argues for both unilateral scaling slowdowns by OpenAI and broader international coordination to enforce safety commitments while investing more in alignment, including “automated alignment researchers.”
Why it matters
OpenAI Chief Scientist Jakub Pachocki is dropping truth bombs. Tomorrow I will discuss Astra’s lack of monitorability, and the potential contributing factors to that. The situation is alarming and should freak you out, and briefly it looked, in the wake … Continue reading →