OpenTelemetry for .NET: Distributed Tracing & Metrics Guide

Modern applications are distributed, often spanning multiple services, databases, and cloud infrastructure. When a user reports a slow page load or an error, pinpointing the root cause becomes a detective game. Traditional logging gives you isolated messages, but lacks context across service boundaries. This is where OpenTelemetry shines.

OpenTelemetry is an open-source observability framework that provides a unified way to collect traces, metrics, and logs from your applications. It standardizes how telemetry data is generated, collected, and exported, making it easier to monitor and debug complex systems.

In this tutorial, you'll learn how to instrument a .NET application with OpenTelemetry. We'll cover distributed tracing, metrics, and logs, with practical examples for ASP.NET Core. You'll see how to export data to Jaeger for traces and Prometheus for metrics. By the end, you'll have a production-ready setup that helps you understand your application's behavior and quickly diagnose issues.

What is OpenTelemetry?

OpenTelemetry is a collection of APIs, SDKs, and tools for instrumenting, generating, collecting, and exporting telemetry data (traces, metrics, logs) to help you analyze your software's performance and behavior. It is vendor-agnostic and supports multiple backends like Jaeger, Prometheus, Zipkin, and Azure Monitor.

OpenTelemetry standardizes how these signals are generated and exported, allowing you to switch backends without changing your instrumentation code.

using OpenTelemetry;
using OpenTelemetry.Trace;
using OpenTelemetry.Metrics;
using OpenTelemetry.Logs;

var builder = WebApplication.CreateBuilder(args);

// Configure OpenTelemetry tracing
builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddConsoleExporter())
    .WithMetrics(metrics => metrics
        .AddAspNetCoreInstrumentation()
        .AddConsoleExporter());

// Configure OpenTelemetry logging
builder.Logging.AddOpenTelemetry(logging => logging
    .AddConsoleExporter());

var app = builder.Build();

app.MapGet("/", () => "Hello World!");

app.Run();

Setting Up Distributed Tracing

Distributed tracing tracks a single request as it travels across multiple services. Each service creates spans that are linked together via a trace ID. In .NET, you can automatically instrument ASP.NET Core requests, HttpClient calls, and database queries.

To set up tracing, add the OpenTelemetry SDK and configure the tracer provider. Use AddAspNetCoreInstrumentation() to capture incoming HTTP requests, and AddHttpClientInstrumentation() to capture outgoing HTTP calls. You can also add custom spans for business logic.

using OpenTelemetry.Trace;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddEntityFrameworkCoreInstrumentation() // if using EF Core
        .AddOtlpExporter(options => options.Endpoint = new Uri("http://localhost:4317")));

var app = builder.Build();

app.MapGet("/order/{id}", async (int id, HttpClient client) =>
{
    // Custom span
    using var activity = Activity.Current?.Source.StartActivity("ProcessOrder");
    activity?.SetTag("order.id", id);
    
    var response = await client.GetAsync($"http://payment-service/process/{id}");
    return await response.Content.ReadAsStringAsync();
});

app.Run();

Collecting Metrics

Metrics are numerical measurements collected over time, such as request count, latency, and error rate. OpenTelemetry provides instruments like Counter, Histogram, and Gauge. ASP.NET Core instrumentation automatically captures HTTP server metrics (request duration, active requests). You can also create custom metrics.

To export metrics, configure a metric reader and exporter. Prometheus is a popular backend; use AddPrometheusExporter() to expose a /metrics endpoint.

using OpenTelemetry.Metrics;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddOpenTelemetry()
    .WithMetrics(metrics => metrics
        .AddAspNetCoreInstrumentation()
        .AddPrometheusExporter());

var app = builder.Build();

app.MapGet("/", () => "Hello World!");

// Expose metrics endpoint
app.UseOpenTelemetryPrometheusScrapingEndpoint();

app.Run();

Integrating Logs with OpenTelemetry

OpenTelemetry can also collect logs, providing a structured and correlated logging experience. By integrating with .NET's ILogger, you can enrich logs with trace and span IDs, making it easy to correlate logs with traces.

To enable OpenTelemetry logging, add the OpenTelemetry.Extensions.Logging package and configure the logger factory.

using OpenTelemetry.Logs;

var builder = WebApplication.CreateBuilder(args);

builder.Logging.AddOpenTelemetry(logging => logging
    .AddOtlpExporter(options => options.Endpoint = new Uri("http://localhost:4317")));

var app = builder.Build();

app.MapGet("/", (ILogger<Program> logger) =>
{
    logger.LogInformation("Handling request");
    return "Hello World!";
});

app.Run();

Exporting Telemetry to Backends

OpenTelemetry supports multiple exporters: Console, OTLP (OpenTelemetry Protocol), Jaeger, Zipkin, Prometheus, Azure Monitor, and more. The OTLP exporter is the recommended standard, as it can send traces, metrics, and logs to any OTLP-compatible backend.

To export to Jaeger, use the OpenTelemetry.Exporter.Jaeger package. For Prometheus, use OpenTelemetry.Exporter.Prometheus.AspNetCore. For Azure Monitor, use Azure.Monitor.OpenTelemetry.AspNetCore.

// OTLP exporter (traces, metrics, logs)
builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing.AddOtlpExporter())
    .WithMetrics(metrics => metrics.AddOtlpExporter());

// Jaeger exporter (traces only)
builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing
        .AddJaegerExporter(options =>
        {
            options.AgentHost = "localhost";
            options.AgentPort = 6831;
        }));

// Prometheus exporter (metrics only)
builder.Services.AddOpenTelemetry()
    .WithMetrics(metrics => metrics
        .AddPrometheusExporter());

Sampling Strategies

In high-traffic systems, collecting every trace can be expensive. Sampling allows you to collect a representative subset. OpenTelemetry supports head-based sampling (decision made at the start of a trace) and tail-based sampling (decision made after the trace is complete).

Head-based sampling is simpler and can be configured via the sampler. For example, TraceIdRatioBasedSampler samples a fixed percentage of traces.

using OpenTelemetry.Trace;

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing
        .SetSampler(new TraceIdRatioBasedSampler(0.1)) // 10% sampling
        .AddAspNetCoreInstrumentation()
        .AddOtlpExporter());

Security and Best Practices

When instrumenting your application, be mindful of security. Avoid capturing sensitive data in span attributes or log messages. Use attribute filtering to redact personal information.

Also, ensure that your OpenTelemetry endpoints are secured. If using OTLP over gRPC, consider TLS. For Prometheus scraping, protect the /metrics endpoint with authentication if it exposes internal metrics.

// Redact sensitive data from spans
builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation(options =>
        {
            options.Filter = (httpContext) =>
            {
                // Exclude health check endpoints
                return !httpContext.Request.Path.StartsWithSegments("/health");
            };
            options.EnrichWithHttpRequest = (activity, request) =>
            {
                // Remove query string parameters that may contain PII
                activity.SetTag("http.url", request.Scheme + "://" + request.Host + request.Path);
            };
        }));