Beyond RAG: Task-aware knowledge compression for enterprise AI on AWS
AWS Dhananjay Karanjkar
AWS shared a new open-source pattern called TAKC that pre-compresses whole document sets for specific tasks instead of just searching for snippets. It's built to catch cross-document connections that regular RAG search misses, and it can cut token costs way down.
Based on reporting by AWS, Dhananjay Karanjkar — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Retrieval-Augmented Generation has a blind spot, and AWS is trying to patch it. RAG works fine when the answer sits in one paragraph somewhere in your corpus. But ask it something that requires stitching together facts from dozens or hundreds of documents — the kind of question a private equity team might ask while sizing up a $500 million acquisition spread across 12 subsidiaries, 200-plus supplier contracts, environmental reports from 8 facilities, and 50-plus legal cases — and similarity search just shrugs. The pieces it needs share no lexical overlap, so it never finds them together.
AWS's answer is something it calls task-aware knowledge compression, or TAKC, and the idea is almost stubbornly simple: compress the entire knowledge base ahead of time, not per query, and do it differently depending on what the task actually needs. A financial analysis pass keeps revenue, margins, and cash flow. A compliance pass on the exact same document keeps regulatory citations and violation history instead. Generic summarization tries to please everyone and ends up diluting what matters for anyone. TAKC skips that by compressing through the lens of one job at a time, offline, once per document per task type — so by the time a question actually gets asked, the system is pulling from a compressed version of the whole corpus rather than a handful of top-ranked chunks.
What makes this more than a compression trick is the tiering. AWS built four compression rates per task — 8x, 16x, 32x, and 64x — because not every question deserves the same amount of context. A cross-subsidiary risk question needs room to reason;
My take — AI-written commentary, not fact-checked reporting
General.a query complexity analyzer routes each incoming question to the tier that fits, defaulting to medium when it's unsure, and most everyday enterprise questions apparently land in the cheaper, more compressed tiers anyway. The heavy lifting — the expensive Bedrock compression calls — happens once during ingestion, not on every query, which is the whole economic argument here. The reference architecture is straightforwardly serverless: S3 for storage, Lambda for chunking and compression, Bedrock (Claude 3 Haiku, Claude 3 Sonnet, Titan Text) doing the actual compressing and inference, ElastiCache Serverless holding the compressed cache with S3 as backup, API Gateway and Cognito handling the query side, WAF and KMS wrapped around it for security. AWS published cost figures too: on a 100,000-token knowledge base queried 1,000 times a day, full context costs 100% of baseline input tokens, standard RAG with top-10 chunks runs about 10%, and TAKC's tiers range from roughly 12.5% down to about 1.6% at the ultra-compressed end. The catch, which AWS is upfront about, is that this only pays off for knowledge bases that don't change every hour — if your documents are in constant flux, the upfront compression cost stops making sense and plain RAG's per-query retrieval wins. AWS isn't pitching this as a RAG replacement, and the two pair up in the framing given: RAG for narrow factual lookups where a user needs to see the source document, TAKC for the messier analytical questions that require touching the whole corpus and where showing your work matters less than getting the synthesis right. For anyone wanting to poke at it directly, AWS put a full open-source CDK implementation on GitHub, deployable with a handful of commands.Rap-filling — and cheap to tear down again once you're done testing.","opinion":"This is a sensible, if unglamorous, engineering fix rather than a breakthrough — nobody should mistake pre-computed compressed summaries for actual reasoning over documents. What's genuinely useful is the tiered cost model, which quietly admits that most enterprise questions are boring lookups and only a few actually need expensive synthesis; more RAG pitches should be this honest about when their own approach falls apart."}```json omitted correction:
Read more about this at: AWS