EL

Tracing Azure CLI Copilot end to end

At Microsoft, I built multi-handler telemetry and one filterable dashboard for investigating reliability, performance, and cost across environments and deployments.

Contribution
Microsoft · Observability + telemetry
Period
Dec 2025 — Feb 2026
Stack
Python, OpenTelemetry, Grafana
Delivery
Released to UAT

One trace across the workflow.

Consistent signals follow AI-assisted commands across multiple tool handlers before reaching one operational view.

  1. Command
    AI-assisted CLI workflowThe investigation begins at the user action.
  2. Handlers
    Multiple AI tool handlersOne trace preserves context across the workflow.
  3. Instrumentation
    Python + OpenTelemetryConsistent spans carry the operational signals.
  4. Operational view
    Filterable Grafana dashboardEnvironment and deployment remain filterable.
  5. Delivery
    Pull request + CI → UATThe work moved through the normal release path.

The operational gap

Ordinary logs could not cleanly explain a multi-handler AI workflow. The useful questions crossed component boundaries.

  • Where did the workflow spend time?
  • How did model use affect cost?
  • Did the response remain useful?
  • What changed across handlers and deployments?

Signals that made those questions answerable.

Latency
Where the workflow spends time
Token use
Model usage and cost signals
Errors + outcomes
Failure and completion states
Response quality
Whether the result remains useful

From instrumentation to UAT.

I implemented end-to-end Python and OpenTelemetry tracing, then carried it through review and release.

  1. ImplementMulti-handler tracing
  2. ReviewPull request
  3. VerifyContinuous integration
  4. ReleaseUser acceptance testing

One filterable dashboard brought reliability, performance, quality, and cost analysis into a unified operational view across environments and deployments.