Secure, Monitor, and Troubleshoot Azure Solutions · Topic 2 of 2

Tracing with OpenTelemetry and Querying with KQL

AI-200 monitoring: tracing distributed systems with the Azure Monitor OpenTelemetry Distro, Application Insights data, and KQL queries for logs and metrics.

  • 4 min
In a nutshell
  • A trace is made of spans; context propagation joins spans across services.
  • The Azure Monitor OpenTelemetry Distro sends data to Application Insights using a connection string.
  • Tables: AppRequests, AppDependencies, AppExceptions, AppTraces, AppMetrics.

What it is

OpenTelemetry is an open standard for collecting traces, metrics and logs from applications. Azure's Azure Monitor OpenTelemetry Distro sends this data to Application Insights, where you analyze it with Kusto Query Language (KQL). Together they let you follow a request through a distributed AI back end and find what failed or slowed down.

Service AspanService Bchild spanDatabase or model calldependency spanApplication InsightsKQL query

How it works

Traces and spans

TermMeaning
TraceThe whole journey of one request across services
SpanOne operation in the trace, such as an HTTP call or database query, with start time and duration
Context propagationPassing the trace ID between services (W3C Trace Context headers) so spans join into one trace
AttributesKey-value details on a span, such as model name or tokens used

Instrumenting an app

  • In Python, install the Azure Monitor OpenTelemetry Distro (azure-monitor-opentelemetry) and call configure_azure_monitor(). It reads the Application Insights connection string from the APPLICATIONINSIGHTS_CONNECTION_STRING environment variable, or you pass connection_string. For new apps, Learn now also offers the Microsoft OpenTelemetry Distro (microsoft-opentelemetry).
  • Common libraries for HTTP, databases and messaging are instrumented automatically.
  • Add custom spans around important steps, such as embedding generation or a model call, with attributes for tokens and latency.
  • Trace context (the W3C traceparent header) must flow between services so their spans join one trace. When several services send to the same Application Insights resource, give each a cloud role name so the application map tells them apart.

Where the data lands

In a workspace-based Application Insights resource, data is stored in Log Analytics tables such as:

TableContains
AppRequestsIncoming requests to your services
AppDependenciesOutgoing calls: databases, HTTP APIs, queues
AppExceptionsErrors thrown by the app
AppTracesLog messages
AppMetricsCustom metrics (customMetrics in the classic view)

In the Application Insights Logs view, the same data also appears under classic names such as requests, dependencies, exceptions and traces.

KQL essentials

OperatorPurpose
whereFilter rows
projectChoose columns
extendAdd a calculated column
summarizeAggregate, such as count() or avg(), often by bin(TimeGenerated, 5m)
order by / topSort or take the first rows
joinCombine tables, for example requests with exceptions by operation ID
percentile()Latency percentiles, such as the 95th
renderChart the result, such as a time chart

Example: failed requests per 5 minutes in the last hour:

AppRequests
| where TimeGenerated > ago(1h) and Success == false
| summarize failures = count() by bin(TimeGenerated, 5m)
| render timechart

A worked example: users report slow answers. A KQL query on AppDependencies grouped by target shows the vector database call has a 95th percentile of 2 seconds, while the model call is fast, so the team tunes the vector index.

Exam tip

To find which downstream call is slow, query dependencies. To find failing endpoints, query requests where success is false. To see errors, query exceptions.

How the exam asks about this

  • "Correlate one request across three services." Use OpenTelemetry with context propagation, sending to Application Insights.
  • "Count failures in 5-minute buckets." summarize count() by bin(time, 5m).
  • "Find the slowest external calls." Query AppDependencies and use percentile on duration.

What to remember for the exam

  • A trace is made of spans; context propagation joins spans across services.
  • The Azure Monitor OpenTelemetry Distro sends data to Application Insights using a connection string.
  • Tables: AppRequests, AppDependencies, AppExceptions, AppTraces, AppMetrics.
  • KQL: where, project, extend, summarize with bin, order by, join, percentile, render.
  • Add custom spans and attributes for AI steps such as model calls.
Watch out

If a service drops the incoming trace context, its spans start a new trace and you cannot follow the request end to end.

Common mistakes

Querying requests to find slow database calls. Those are dependencies.

Forgetting context propagation. Spans will not join into one trace.

Summarizing without bin for time charts. Use bin to group by time interval.

Check yourself

  • What is a span?

    One timed operation within a trace, such as an HTTP call or database query.

  • Which Application Insights table holds outgoing calls to databases and APIs?

    AppDependencies (dependencies in the classic view).

  • Which KQL function groups timestamps into fixed intervals?

    bin, used inside summarize.

Topic 2 of 2 in Secure, Monitor, and Troubleshoot Azure Solutions

Test yourself on Secure, Monitor, and Troubleshoot Azure Solutions

The full Microsoft AI-200 question bank has 650 questions, with timed practice exams and an explanation for every answer.

SubscribeTry Sample Exam

This study guide is independent and is not affiliated with, endorsed by, or sponsored by Microsoft. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.