Data Machina #257

Text-to-SQL sounds like the easiest AI win imaginable: point an LLM at a database, ask a question in plain English, get back a chart. Data Machina #257 spends most of its space explaining why that pitch keeps falling apart in practice, and why a growing crowd of engineering teams are now building elaborate multi-agent scaffolding just to make it work at all.

Pinterest gets credit here as one of the first companies to run Text-to-SQL at real enterprise scale, and more importantly, one of the first to publish honestly about the journey from academic demo to production system. That transparency matters because most vendors gloss over the ugly middle part. LLMs write syntactically gorgeous SQL because they've swallowed enormous amounts of public SQL code during training. But real company databases are nothing like the clean, well-documented schemas that show up in training data. They're full of legacy naming, undocumented joins, and business logic that lives only in someone's head.

A separate post flagged in the newsletter, from a Pattern team building financial-analysis agents, makes a sharper argument: maybe Text-to-SQL was always the wrong frame. They call it the uncanny valley problem — a query that looks correct but misses the analytical context a human analyst would apply automatically. Their fix is to push toward something closer to Text-to-Analysis, where multiple specialized agents collaborate the way a real analyst team would, rather than expecting one model to nail a single SQL statement on the first try.

That idea connects directly to Meadow, an open-source framework mentioned here that chains together Planner, Executor, Schema Cleaner, Validator, and Router agents into one workflow, plus tools like Dataherald and PandasAI, which open-sourced natural-language interfaces for querying and cleaning data respectively. None of these are single-model tricks. They're compound systems, echoing the argument Berkeley researchers made back in February that production AI increasingly means orchestrating models, retrievers, and APIs together rather than leaning on one static model. Databricks' recent Compound AI System tooling announcement, cited in the roundup, suggests the big platforms are already betting the same way.

What comes through clearly is that nobody credible is claiming Text-to-SQL is solved. It's evolving into something messier and more interesting: a layered agent architecture where SQL generation is just one small piece of a much longer pipeline built to survive contact with real, dirty, constantly-changing enterprise data.