Natural language security queries at Orca Security using Amazon Bedrock
Amazon Web Services Deniz Yilmaz
Orca Security turned plain English into cloud security queries with Amazon Bedrock. It cut query cost 90%, but the bigger win was turning a 60-second prototype into a 15-second production system.
Based on reporting by Amazon Web Services, Deniz Yilmaz — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Orca Security had a familiar problem: analysts needed answers fast, but Sonar’s query system spoke in precise schema names, relationship hops, and 18 different operators. With more than 900 cloud models in play, that was fine for specialists and a nuisance for everyone else.
So Orca and AWS built Discovery AI, a natural language agent on Amazon Bedrock that turns a plain question into a Sonar query. The first version worked, and users liked it. But it was expensive, slow, and brittle. The team was feeding the whole schema into one giant prompt, which meant roughly 2 million input tokens per workload and about 60 seconds end to end.
The production system looks nothing like that first draft. Discovery AI uses a nine-stage pipeline: it adds user and cloud context, catches obvious structured inputs with rules, screens for unsafe or unsupported requests, retrieves similar past examples with Amazon Titan embeddings, reranks them with Cohere Rerank 3.5, narrows the schema down to the relevant models, generates a filter with Anthropic Claude Haiku 4.5, and then validates the result against Sonar before sending it back. Amazon Nova 2 Lite handles the guardrail and validation work, where speed and cost matter more than heavyweight reasoning.
The key design choice was not to ask the model to memorize everything. Instead, Orca only feeds the generator the model schemas it actually needs, plus strong examples from previous queries. That matters because example retrieval turned out to be the biggest accuracy lever. Top-20 retrieval followed by reranking beat both plain top-K search and maximal marginal relevance, which is a neat reminder that diversity is nice until the right answer gets pushed out of the room.
The numbers are the point. Compared with the baseline, Discovery AI cut input tokens by 93 percent, from about 2 million to 140,000 per comparable workload. Costs fell 90 percent, and latency dropped 75 percent, from about 60 seconds to 15. It now handles thousands of queries a day in production, with a continuous evaluation harness and a labeling tool that lets Orca’s experts correct outputs and feed those fixes back into the system.
My take — AI-written commentary, not fact-checked reporting
This is the kind of AI product that deserves applause because it does boring work well. The industry has plenty of demos that talk, and too few systems that validate, rerank, and admit when a query is nonsense. Open or closed, the useful models are the ones that keep their receipts.
Read more about this at: Amazon Web Services