The article argues over when to treat someone as an “epistemic peer,” using an Eliezer Yudkowsky–Leopold Aschenbrenner exchange to argue against “modesty’s bailey” (social status standing in for evidence). Situational Awareness declined from $45 billion in assets to $10 billion after leverage and margin calls. The takeaway is a shift toward evaluating reasoning and outcomes rather than deferring to credentials or perceived authority, while also extending the author’s broader epistemic principles beyond that specific dispute.
Researchers propose a dependence-aware way to aggregate outputs from multiple LLM judges so that agreement counts don’t overstate independent evidence when judges share biases. The method’s strongest results using 10 judge models at temperature 0 reach 0.912 accuracy for relevance, up from 0.820 with weighted majority vote and 0.804 with uniform majority vote. As a result, evaluation scores adjust for learned pairwise correlation between judges rather than assuming independent errors, improving accuracy across three binary tasks.
The article lays out a process for preparing supervised fine-tuning datasets by auditing raw data quality and then formatting examples for training. It highlights that 1,000 carefully curated examples can match models trained on far more data, and notes that filtering to the cleanest 20 percent can improve training speed and scores. As a result, teams should verify correctness, diversity, deduplication, and safety, then structure each example as valid conversational JSONL (including system prompts and, when enabled, reasoning traces and tool-call blocks) before training.
Perplexity shipped Portable Computer, a local-first version of its Computer agent that splits probabilistic action proposal from deterministic, OS-level tool execution authority. It runs on an Nvidia DGX Spark starting at $4,700 and reports 82.6% on 53 held-out Computer tasks with Qwen3.8-27B. For enterprises, this shifts evaluation toward the harness/sandbox that grants and denies tool permissions and changes accuracy behavior as context length increases.
Agentic coding would replace junior engineers only if four conditions hold, but the article argues three key conditions fail. The most specific cited benchmark issue is that OpenAI audited 27.6% of SWE-bench Verified and found flawed tests rejecting correct solutions in 59.4% of the audited problems. The result, it claims, is that agentic coding is more likely to replace junior-level tasks and increase reliance on senior verification and judgment rather than eliminating junior hiring.
AI agent traces that capture observable execution details for review and later debugging increasingly need to be stored as durable application data rather than disposable telemetry. Langfuse reported Postgres IOPS exhaustion during ingestion and prompt API latency rising to 7 seconds under heavy load before moving trace data to ClickHouse. The shift changes storage and data-modeling choices toward analytics-friendly architectures and access-controlled retention aligned with product needs.
AI models have performed poorly on multiple puzzle-based intelligence tests, especially those involving spatial/visual reasoning and small changes that break memorized patterns. In late 2024, top models solved only 18% of the New York Times Connections puzzles. The article changes by reframing puzzles as a way to pinpoint specific model weaknesses and invites readers to try the same challenges that tripped up AI.
Tool-calling distillation pipelines that repeatedly re-run supervised fine-tuning on teacher-generated trajectories keep paying the frontier-teacher cost while generating-and-filtering only passing cases. On τ 2-bench, 57% of teacher trials fail, with two-thirds of those failures being near-misses, so the current process leaves the hardest scenarios without useful training signal. PROOF-Gen changes the post-training approach by turning those optimized data signals (including near-misses) into better distillation for deployable models.
Every AI story that matters,
in your inbox by 8am.
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the
day in two minutes. Follow companies and topics for alerts, or get the
briefing in Slack. Free, no spam, unsubscribe anytime.
Reading TLDRocket needs no cookies, and the readership counts we rely on come from
our own cookieless analytics. Google Analytics is the exception: it sets cookies and
reports to Google, so it stays switched off until you allow it. You can change your
mind any time from “Cookie settings” in the footer.
Strictly necessary
Session security and form protection (tldrocket-session,
XSRF-TOKEN, 2 hours). The site cannot work without them,
so they need no consent.
Always on
Google Analytics 4 (_ga,
_ga_<id>, up to 2 years). Measures which
stories and sections readers use. Google acts as a third-party processor and may
store the data outside the EU. No advertising, no profiling, no data sold.
