Twenty-five Fields Medal–winning mathematicians sent an open letter saying AI labs are threatening math research by racing to publish solutions and risking attribution and verification problems. They cited 25 signatories and described OpenAI pulling sponsorship of a CalTech math event this week after criticism. The dispute is escalating as labs face growing pressure over how AI-derived proofs are credited, documented, and shared, with fears that secrecy could replace open research.
Employees at major AI labs are raising concerns that advanced AI could destroy humanity. The discussion goes live on Tuesday, September 15 at 16:00 BST (11:00am EST). The event will cover where the extinction fears come from and whether they have merit, plus what actions may be appropriate if they do.
An Anthropic researcher resigned this week after warning on X that the company was moving toward self-improving superintelligence. The company’s alignment lead also co-signed the warning rather than retracting it. The episode frames the resignation as landing at a sensitive moment while Anthropic reportedly prepares for an IPO and the industry pushes for more capable models.
The “Behind the Blog” post discusses AI doomers alongside reactions to 9/11 and a separate reference to Barbie. It emphasizes Joseph’s emotional link to 9/11 as the moment “everything changed.” As a result, the entry frames the week’s topics through personal feelings and points readers to related reporting projects, but the main content is restricted to paid members.
Zvi (Don't Worry About the Vase)·1 month ago·
19
● 2 sources
Jacob Coxon resigned from Anthropic and warned that AI labs are racing toward self-improving superintelligence in a way he said could lead to human extinction. He said he spent the last 3 years doing pretraining research across OpenAI and Anthropic. As a result, employees’ public safety alarms escalated into a broader “preference cascade,” drawing major mainstream media coverage and intensifying calls for slowing or stopping certain progress.
Anthropic reported that its platform was used on five occasions by malign actors trying to build biological weapons, and the company called this one of the most serious risks of frontier AI models. The House and other lawmakers are considering measures including a new federal AI safety agency, mandatory kill-switches, and a House oversight committee. This shifts the focus toward AI safety regulation that can constrain model behavior and increase government oversight following similar disclosures from Anthropic and OpenAI.
AI researchers and executives escalated public safety concerns, including Anthropic researcher Jacob Coxon’s resignation and OpenAI leadership calling for an AI research slowdown. The most concrete detail is that Anthropic’s alignment science lead referenced a more-than-10% chance of AI “kill[ing] all humans” in the next decade. Despite the warnings, the article says AI labs continued launching many new models and apps and that fundraising and AI infrastructure spending (and related data-center opposition) may shape future pace and deployment.
Zvi (Don't Worry About the Vase)·1 month ago·
3
● 2 sources
OpenAI, Anthropic, and Google employees posted and discussed quotes saying they believe AI could soon pose catastrophic human risk unless progress is slowed. One OpenAI employee, Marcus Williams, estimated a 70% chance of human extinction within 3 years without regulation or slowdown. The statements add more urgency and support for coordination and “pacing” safety steps rather than continuing AI development on business-as-usual speed.
Anthropic reported that Claude users made multiple attempts to use its technology for bioweapons-related research by trying to bypass safeguards. Anthropic provided five examples of circumvention and obfuscation attempts. As a result, Anthropic published the cases to spur discussion with the AI industry and governments about emerging biological risks and countermeasures.
Anthropic says it blocked multiple efforts that would have used its AI models for biological-weapon-relevant research. It disrupted “several” plots by applying caution when it couldn’t distinguish legitimate from harmful queries. As a result, some potentially dangerous uses of its models were prevented rather than answered.
Meta is changing the AI chatbot prompts it suggests after a viral example showed the feature asking for personal information about a woman’s young daughters. Meta said the prompt should not have included questions like “Who is the child passenger?”. The feature will no longer present that kind of intrusive suggestion in prompts going forward, after Meta acknowledged it “missed the mark.”
The Character.AI team published an arXiv study tracking how users’ social engagement with AI companions changes over about 12 months and how it relates to well-being. The study surveyed 1,182 participants at baseline and 439 after a mean follow-up of 12 months. It found continued engagement patterns and associated sustained interaction and self-disclosure with lower well-being, mainly explained by reduced in-person social interaction, and argues for designing companions that support human relationships without displacing them.
Anthropic published a threat-intelligence report describing how malicious actors tried to use Claude for cyber, surveillance, influence, scams, biological misuse, conventional weapons, and illicit distillation, and how Anthropic disrupted the operations and updated safeguards. The cases it covers span from December 2025 to August 2026, using Claude Haiku, Sonnet, and Opus models. As a result, Anthropic says safeguards were strengthened based on what was learned and intelligence was shared with authorities and industry partners where appropriate, while noting AI-fueled misuse has evolved since earlier reports.
Anthropic blocked attempts to use its Claude models for malicious activity that could support biological weapons development. It said misuse of Claude Haiku, Sonnet, and Opus was disrupted between December 2025 and August 2026. As a result, Anthropic added safeguards to better prevent, detect, and disrupt these activities and shared intelligence with authorities and industry partners.
Anthropic published a threat intelligence report describing misuse of its Claude model, including five cases tied to biological research that could be adjacent to biological weapons development. The report covers activity over about eight months. Anthropic blocked sensitive requests, denied access to stronger model versions, banned the implicated accounts, and shared the findings with authorities and other AI providers.
Jacob Coxon, a former OpenAI and Anthropic researcher, went viral with warnings that AI could kill all humans by the end of the decade, drawing replies from Anthropic’s alignment leadership and concern from OpenAI research leadership. The post cites a greater than 10% chance of this happening within the next decade. The article says Coxon’s message is too vague to drive concrete action, so it mainly fuels fear rather than prompting specific policy or projects to stop.
A recently resigned Anthropic researcher’s post and comments from OpenAI and other safety figures intensified public and political debate about AI existential risks. The article cites a 159 million view X post and Hubinger’s stated belief that the risk is more than 10% within the next decade. As a result, lawmakers have announced probes and introduced bills to pause or ban superintelligence development while public attention on AI safety shifts toward regulation and oversight.
Every AI story that matters,
in your inbox by 8am.
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the
day in two minutes. Follow companies and topics for alerts, or get the
briefing in Slack. Free, no spam, unsubscribe anytime.
Reading TLDRocket needs no cookies, and the readership counts we rely on come from
our own cookieless analytics. Google Analytics is the exception: it sets cookies and
reports to Google, so it stays switched off until you allow it. You can change your
mind any time from “Cookie settings” in the footer.
Strictly necessary
Session security and form protection (tldrocket-session,
XSRF-TOKEN, 2 hours). The site cannot work without them,
so they need no consent.
Always on
Google Analytics 4 (_ga,
_ga_<id>, up to 2 years). Measures which
stories and sections readers use. Google acts as a third-party processor and may
store the data outside the EU. No advertising, no profiling, no data sold.
