Google Cloud’s Always-On Memory Agent Replaces RAG and Embeddings With Continuous LLM Consolidation on Gemini 3.1 Flash-Lite
MarkTechPost Michal Sutter
Google Cloud released a sample agent that gives AI memory without RAG or vector databases. It runs 24/7, quietly linking your notes, PDFs, and audio into new insights while you sleep.
Based on reporting by MarkTechPost, Michal Sutter — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Most AI agents have the memory of a goldfish. You send a request, get an answer, and the context vanishes the moment the process ends. Google Cloud's generative-ai repo just added something that pushes back on that pattern: the Always-On Memory Agent, a reference build that treats memory as a job that never clocks out rather than a lookup you perform when convenient.
The setup skips the usual RAG toolkit entirely. No vector database, no embeddings model humming in the background. Instead it leans on Gemini 3.1 Flash-Lite, chosen specifically because it's cheap and fast enough to run continuously, and lets the LLM itself read, reason, and write structured rows into a plain SQLite database. An orchestrator built on Google's Agent Development Kit hands every job to one of three specialist sub-agents. The IngestAgent turns incoming files into a summary, extracted entities, topics, and an importance score. The ConsolidateAgent wakes up every 30 minutes on a timer, reviews whatever hasn't been processed yet, and — like a nightly sleep cycle for your data — writes new connections and insights it finds between memories, with nobody asking it to. The QueryAgent then answers questions by reading through memories and consolidation notes, citing the specific memory IDs it pulled from.
What's notable is the breadth of what it'll swallow. Drop a file into an ./inbox folder and the IngestAgent handles 27 extensions spanning text, images, audio, video, and PDFs — everything from .yaml configs to .mkv recordings. That's a wider net than most memory demos bother casting, and it's clearly aimed at people who want one folder to become a working archive rather than a curated dataset.
Google's own comparison table draws a pointed contrast with existing approaches. Vector-DB-plus-RAG setups embed once and retrieve later, but do nothing active in between. Conversation summaries compress but lose detail and never cross-reference anything. Knowledge graphs capture relationships well but need constant manual upkeep to stay useful. The Always-On Memory Agent's pitch is that consolidation happens automatically and continuously, so by the time you ask a question, some of the synthesis work is already done — the query step just reads up to 50 recent memories rather than reconstructing meaning from scratch.
Setup is refreshingly unglamorous: pip install, export an API key, run agent.py. It watches a folder, serves an HTTP API on port 8888 for ingest and query calls, and exposes status, browse, and delete endpoints, with an optional Streamlit dashboard for people who'd rather click than curl. The examples Google offers — a research assistant linking a cost target to a reliability bug from a week of PDFs and meeting audio, a support agent citing old tickets — are modest, but they're the kind of modest that tends to be genuinely useful rather than flashy.
My take — AI-written commentary, not fact-checked reporting
I like this precisely because it's boring in the right way — SQLite and a scheduled LLM pass beat another vector-DB integration nobody wants to maintain. The RAG-industrial-complex has been selling embeddings as the only path to memory for two years, and it's refreshing to see Google Cloud quietly admit that sometimes an LLM just re-reading its own notes on a timer works fine. Whether it scales past a hobby project's worth of memories before costs creep back up is the real open question, and Google conveniently doesn't answer it.
Read more about this at: MarkTechPost
