Deepgram deepens Amazon SageMaker AI observability with Enhanced Metrics

Deepgram is plugging a familiar blind spot in self-hosted speech AI on Amazon SageMaker AI. Until now, you could see that an endpoint was alive and busy, but the useful questions stayed hidden: what the traffic was actually costing, which features people were using, and how the engine was behaving on each GPU.

The new part is called Deepgram Enhanced Metrics. It pushes billing and usage data into Amazon CloudWatch through the same logging path SageMaker AI already uses, so there’s no agent to install, no sidecar to manage, and no extra IAM permissions to hand out. The numbers are the same consumed-unit values used for AWS Marketplace metered billing, which means teams can line up the AWS bill with real traffic and build budget alarms or finance dashboards around it.

Deepgram is also exposing a separate usage stream that breaks traffic down by method, model tier, and feature. That matters because it answers questions standard endpoint metrics can’t: whether the workload is streaming or pre-recorded, which model tiers are doing the heavy lifting, and whether features like diarization, smart formatting, redaction, or keyterm prompting are actually being used.

Then there’s the deeper observability layer. Deepgram containers now serve Prometheus metrics, and SageMaker AI detailed observability scrapes them through an AWS managed OpenTelemetry Collector on each instance. That brings in per-GPU and host metrics too, so one saturated GPU doesn’t disappear inside a summed average. The Deepgram engine’s own capacity estimate is there as well, alongside active request counts, which gives a more concrete signal for scaling.

The whole setup is built to work under AWS Marketplace network isolation. The container doesn’t need to phone home, and the data lands in the customer’s own CloudWatch account. For teams running speech models on SageMaker AI, that’s the kind of visibility they should have had from day one.