Build visibility for Codex on Amazon Bedrock with OpenTelemetry and Amazon CloudWatch

Once a coding assistant like Codex moves from a few curious engineers to entire departments, the question stops being about capability and starts being about control. Who's actually using this thing? Which teams are burning through tokens? Is it reliable enough to trust for real work? AWS just published a pattern that answers those questions using tools most enterprises already have, rather than bolting on a new proxy layer that sits between developers and the model.

The setup is simple in outline: Codex, when running locally and talking to OpenAI models through Amazon Bedrock, can emit OpenTelemetry metrics. Instead of routing that data through some centralized gateway, AWS has each developer's workstation run a small collector that listens only on localhost, tags the metrics with organizational context like team, department, or cost center, and then signs the payload with temporary AWS credentials before shipping it to CloudWatch. No load balancer, no VPC, no public endpoint — just a dashboard that lights up with active users, token counts, tool-call volume, and request durations, sliceable by whatever attributes IAM Identity Center already knows about a user.

What's notable is the restraint here. AWS is explicit that this is a visibility tool, not a billing system — token counts drift from actual invoiced costs due to pricing changes and credits, so anyone wanting real financial numbers needs to pair this with AWS Cost and Usage Reports 2.0. It's also explicit that this can't enforce anything in real time; because Identity Center hands out credentials directly, there's no way to block a request mid-flight if someone blows through a budget. For that, AWS points people toward a proper in-path gateway, acknowledging the tradeoff in identity attribution and complexity that comes with it.

The rollout advice reads like someone who's watched observability projects go sideways before. Start with one team, confirm the identity tagging is actually correct, make sure the dashboard is answering real questions rather than just existing, and only then push it out fleet-wide through normal endpoint management. AWS also flags the privacy angle directly — per-user dashboards should stay locked to finance, security, or ops roles, not become ambient surveillance for every manager who wants to peek at someone's token usage. And cardinality matters too: dumping in project names or one-off identifiers without a plan just clutters CloudWatch and racks up query costs, since PromQL billing scales with samples scanned.

The whole thing is presented as a reference pattern via a public GitHub repo, complete with shell scripts to deploy the CloudFormation stack, generate per-developer collector configs, and validate the pipeline end to end. It's a fairly unglamorous piece of infrastructure — a five-step runbook and a dashboard — but it's the kind of plumbing that decides whether an AI coding tool stays a pilot forever or actually gets trusted at scale.