Logging and Monitoring in ASP.NET Core

Table of Contents

1. ILogger and Log Levels
2. Semantic Logging (Structured Logging)
3. Log Categories and Filters
4. Log Providers
5. Scopes and Context Enrichment
6. Sensitive Data — Protection
7. LoggerMessage Source Generators
8. Exception Handling
9. Request Logging
10. Log Destinations (Sinks)
11. Health Checks
12. Monitoring and Alerts
13. Traceability and OpenTelemetry
14. Summary and Key Points

1. ILogger and Log Levels

Injecting ILogger:

public class OrderService
{
    private readonly ILogger<OrderService> _logger;

    public OrderService(ILogger<OrderService> logger)
    {
        _logger = logger;
    }

    public async Task<Order?> GetOrderAsync(int id)
    {
        _logger.LogDebug("Retrieving order {OrderId}", id);
        
        var order = await _repository.GetByIdAsync(id);
        
        if (order == null)
            _logger.LogWarning("Order {OrderId} not found", id);
        
        return order;
    }
}

The 6 log levels (ascending severity):

Configuring levels in appsettings.json:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information",
      "MyApp.Services": "Debug",
      "System": "Warning"
    }
  }
}

2. Semantic Logging (Structured Logging)

Structured logging = naming parameters in log messages to enable searching.

❌ Bad (string interpolation):

// String interpolation — impossible to filter/search
_logger.LogInformation($"Fetching items for category {category}");
_logger.LogInformation($"User {userId} signed in from {ipAddress}");

✅ Good (message template with named parameters):

// Template with named parameters — searchable and indexable
_logger.LogInformation("Fetching items for {Category}", category);
_logger.LogInformation("User {UserId} signed in from {IpAddress}", userId, ipAddress);

Why it matters:

With structured logging, in Seq or Application Insights:
→ Filter by: Category = "Electronics"
→ Group by: IpAddress
→ Alert on: UserId = "specific-user"
→ Analyze trends by Category

With string interpolation:
→ Can only search raw text (grep)
→ Cannot extract values

Event IDs:

// Define event IDs to categorize logs
private static readonly EventId ItemNotFound = new(1001, "ItemNotFound");
private static readonly EventId OrderProcessed = new(2001, "OrderProcessed");

_logger.LogWarning(ItemNotFound, "Item {ItemId} not found", id);
_logger.LogInformation(OrderProcessed, "Order {OrderId} processed", orderId);

3. Log Categories and Filters

Log Category = identifier for the source of a log.

With ILogger<T>:

// Automatic category = full namespace + class name
// E.g.: "MyApp.Services.OrderService"
private readonly ILogger<OrderService> _logger;

With ILoggerFactory (custom category):

// Custom category (without namespace)
private readonly ILogger _logger;

public OrderService(ILoggerFactory loggerFactory)
{
    _logger = loggerFactory.CreateLogger("Orders");
}

Filters in appsettings.json:

{
  "Logging": {
    "LogLevel": {
      "Default": "Warning",
      "MyApp": "Information",
      "MyApp.Controllers": "Debug",
      "Microsoft.AspNetCore": "Warning",
      "System.Net.Http": "Warning"
    },
    "Console": {
      "LogLevel": {
        "Default": "Information"
      }
    }
  }
}

Environment-specific filters:

// appsettings.Development.json
{
  "Logging": {
    "LogLevel": {
      "Default": "Debug",
      "Microsoft": "Information"
    }
  }
}

// appsettings.Production.json
{
  "Logging": {
    "LogLevel": {
      "Default": "Warning",
      "MyApp": "Information"
    }
  }
}

Filters via environment variables:

# PowerShell
$env:Logging__LogLevel__Default = "Warning"
$env:Logging__LogLevel__MyApp = "Debug"

# Bash
export Logging__LogLevel__Default=Warning
export Logging__LogLevel__MyApp=Debug

Filters in code:

builder.Logging.AddFilter("MyApp", LogLevel.Debug);
builder.Logging.AddFilter<ConsoleLoggerProvider>("Microsoft", LogLevel.Warning);

4. Log Providers

Built-in ASP.NET Core providers:

Registering providers:

// Program.cs
builder.Logging.ClearProviders();  // Remove default providers
builder.Logging.AddConsole();
builder.Logging.AddDebug();
// builder.Logging.AddEventLog(); // Windows only

Application Insights provider:

dotnet add package Microsoft.Extensions.Logging.ApplicationInsights

builder.Logging.AddApplicationInsights(
    configureTelemetryConfiguration: config =>
        config.ConnectionString = builder.Configuration["APPINSIGHTS_CONNECTION_STRING"],
    configureApplicationInsightsLoggerOptions: options => { }
);

5. Scopes and Context Enrichment

Log Scope = add contextual properties to all logs within a block.

Basic usage:

using (_logger.BeginScope("Processing order {OrderId}", orderId))
{
    // All logs in this using block will have OrderId in context
    _logger.LogInformation("Validating order");
    await ValidateOrderAsync(order);
    
    _logger.LogInformation("Processing payment");
    await ProcessPaymentAsync(order);
    
    _logger.LogInformation("Creating shipment");
    await CreateShipmentAsync(order);
}
// Exiting scope — OrderId property is no longer included

Scope with dictionary (more properties):

using (_logger.BeginScope(new Dictionary<string, object>
{
    ["OrderId"] = orderId,
    ["CustomerId"] = customerId,
    ["CorrelationId"] = correlationId
}))
{
    _logger.LogInformation("Processing order");
}

Middleware for user scope:

// UserScopeMiddleware.cs
public class UserScopeMiddleware
{
    private readonly RequestDelegate _next;
    private readonly ILogger<UserScopeMiddleware> _logger;

    public UserScopeMiddleware(RequestDelegate next, ILogger<UserScopeMiddleware> logger)
    {
        _next = next;
        _logger = logger;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        var username = context.User.Identity?.Name ?? "anonymous";
        
        using (_logger.BeginScope(new { Username = username }))
        {
            await _next(context);
        }
    }
}

// Program.cs
app.UseMiddleware<UserScopeMiddleware>();

6. Sensitive Data — Protection

No built-in automatic solution — must be handled manually.

Approaches:

1. Manual redaction:

// Mask sensitive data before logging
public async Task ProcessPayment(string cardNumber, decimal amount)
{
    var maskedCard = MaskCardNumber(cardNumber);
    _logger.LogInformation("Processing payment {MaskedCard} for {Amount}", maskedCard, amount);
}

private string MaskCardNumber(string cardNumber)
{
    // Keep only the last 4 digits
    return "****-****-****-" + cardNumber[^4..];
}

2. Redaction with Regex:

// Intercept and sanitize logs
public class RedactingLogger : ILogger
{
    private readonly ILogger _inner;
    private static readonly Regex CreditCardPattern = 
        new(@"\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b");

    public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, 
        Exception? exception, Func<TState, Exception?, string> formatter)
    {
        var message = formatter(state, exception);
        var redacted = CreditCardPattern.Replace(message, "****-****-****-####");
        _inner.Log(logLevel, eventId, redacted, exception, 
            (s, e) => s.ToString() ?? string.Empty);
    }
}

Rules:

• Never log passwords, tokens, or API keys
• Mask card numbers, SSNs, etc.
• Avoid logging personal data (GDPR compliance)
• Use Event IDs for sensitive logs (to audit who accesses them)

7. LoggerMessage Source Generators

Problem with standard LogInformation():

• Allocates strings and objects on every call
• Degraded performance in hot paths

Solution — Source Generators:

// ❌ Standard method (allocations on every call)
_logger.LogInformation("Processing order {OrderId} for user {UserId}", orderId, userId);

// ✅ Source Generator (high performance, zero allocation)
public partial class OrderService
{
    [LoggerMessage(EventId = 1001, Level = LogLevel.Information, 
        Message = "Processing order {OrderId} for user {UserId}")]
    private partial void LogProcessingOrder(int orderId, string userId);
    
    public void ProcessOrder(int orderId, string userId)
    {
        LogProcessingOrder(orderId, userId);  // High-performance call
    }
}

Benefits of source generators:

• No object allocations at runtime
• Compile-time verification (no runtime errors)
• Pre-compiled log messages
• Recommended for hot paths

With injected ILogger:

public partial class OrderController : ControllerBase
{
    private readonly ILogger<OrderController> _logger;

    public OrderController(ILogger<OrderController> logger)
    {
        _logger = logger;
    }

    [LoggerMessage(Level = LogLevel.Information, 
        Message = "Order {OrderId} created by user {Username}")]
    partial void LogOrderCreated(int orderId, string username);

    [LoggerMessage(Level = LogLevel.Warning, 
        Message = "Order {OrderId} validation failed: {Reason}")]
    partial void LogOrderValidationFailed(int orderId, string reason);
}

8. Exception Handling

Global Exception Handler:

// Program.cs - .NET 8+
app.UseExceptionHandler(exceptionHandlerApp =>
{
    exceptionHandlerApp.Run(async context =>
    {
        context.Response.StatusCode = StatusCodes.Status500InternalServerError;
        context.Response.ContentType = "application/problem+json";
        
        var exception = context.Features.Get<IExceptionHandlerFeature>()?.Error;
        var logger = context.RequestServices.GetRequiredService<ILogger<Program>>();
        
        // Log full details (stack trace)
        logger.LogError(exception, "Unhandled exception");
        
        // Return ProblemDetails to client (without stack trace)
        var problemDetails = new ProblemDetails
        {
            Status = 500,
            Title = "An unexpected error occurred",
            Detail = "Please contact support with the trace ID below",
            Extensions = { ["traceId"] = Activity.Current?.Id }
        };
        
        await context.Response.WriteAsJsonAsync(problemDetails);
    });
});

ProblemDetails (built-in .NET 8):

// Program.cs — configure automatic ProblemDetails
builder.Services.AddProblemDetails();
builder.Services.AddExceptionHandler<GlobalExceptionHandler>();

// Implementation
public class GlobalExceptionHandler : IExceptionHandler
{
    private readonly ILogger<GlobalExceptionHandler> _logger;

    public GlobalExceptionHandler(ILogger<GlobalExceptionHandler> logger)
    {
        _logger = logger;
    }

    public async ValueTask<bool> TryHandleAsync(HttpContext context, 
        Exception exception, CancellationToken ct)
    {
        _logger.LogError(exception, "Exception occurred: {Message}", exception.Message);
        
        var problemDetails = new ProblemDetails
        {
            Status = exception switch
            {
                ArgumentNullException => 400,
                KeyNotFoundException => 404,
                UnauthorizedAccessException => 403,
                _ => 500
            },
            Title = "Request failed",
            Detail = exception.Message
        };
        
        context.Response.StatusCode = problemDetails.Status!.Value;
        await context.Response.WriteAsJsonAsync(problemDetails, ct);
        
        return true;
    }
}

9. Request Logging

HTTP Logging (built-in .NET 6+):

// Program.cs
builder.Services.AddHttpLogging(logging =>
{
    logging.LoggingFields = HttpLoggingFields.RequestMethod 
        | HttpLoggingFields.RequestPath 
        | HttpLoggingFields.ResponseStatusCode
        | HttpLoggingFields.Duration;
    
    // Warning: RequestBody/ResponseBody = lots of data!
    logging.RequestBodyLogLimit = 4096;
    logging.ResponseBodyLogLimit = 4096;
});

app.UseHttpLogging();

W3C Logging (standard log files):

builder.Services.AddW3CLogging(logging =>
{
    logging.LoggingFields = W3CLoggingFields.All;
    logging.FileSizeLimit = 5 * 1024 * 1024;  // 5 MB per file
    logging.RetainedFileCountLimit = 2;
    logging.FileName = "w3clog-";
    logging.LogDirectory = @"C:\logs\w3c";
    logging.FlushInterval = TimeSpan.FromSeconds(2);
});

app.UseW3CLogging();

Differences between HTTP and W3C Logging:

10. Log Destinations (Sinks)

Application Insights:

<PackageReference Include="Microsoft.Extensions.Logging.ApplicationInsights" Version="2.x" />

builder.Logging.AddApplicationInsights(config =>
    config.ConnectionString = builder.Configuration["APPINSIGHTS_CONNECTION_STRING"]);

Serilog + Seq:

<PackageReference Include="Serilog.AspNetCore" Version="8.x" />
<PackageReference Include="Serilog.Sinks.Seq" Version="6.x" />

builder.Host.UseSerilog((context, services, configuration) =>
{
    configuration
        .ReadFrom.Configuration(context.Configuration)
        .ReadFrom.Services(services)
        .Enrich.FromLogContext()
        .WriteTo.Console()
        .WriteTo.Seq(context.Configuration["Seq:ServerUrl"]!);
});

NLog + Splunk:

<PackageReference Include="NLog.Web.AspNetCore" Version="5.x" />

builder.Logging.ClearProviders();
builder.Host.UseNLog();

11. Health Checks

Basic configuration:

// Program.cs
builder.Services.AddHealthChecks()
    .AddCheck("self", () => HealthCheckResult.Healthy())
    .AddDbContextCheck<AppDbContext>("database")
    .AddCheck<CustomHealthCheck>("custom");

app.MapHealthChecks("/health");
app.MapHealthChecks("/health/live", new HealthCheckOptions
{
    Predicate = _ => false  // Liveness: always OK if process is running
});
app.MapHealthChecks("/health/ready", new HealthCheckOptions
{
    Predicate = check => check.Tags.Contains("ready")  // Readiness: DB connected?
});

Custom Health Check:

public class ExternalApiHealthCheck : IHealthCheck
{
    private readonly HttpClient _httpClient;

    public ExternalApiHealthCheck(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context, 
        CancellationToken cancellationToken = default)
    {
        try
        {
            var response = await _httpClient.GetAsync("/ping", cancellationToken);
            
            return response.IsSuccessStatusCode
                ? HealthCheckResult.Healthy("External API is responsive")
                : HealthCheckResult.Degraded($"External API returned {response.StatusCode}");
        }
        catch (Exception ex)
        {
            return HealthCheckResult.Unhealthy("External API unreachable", ex);
        }
    }
}

AspNetCore.Diagnostics.HealthChecks (NuGet):

<PackageReference Include="AspNetCore.HealthChecks.SqlServer" Version="7.x" />
<PackageReference Include="AspNetCore.HealthChecks.UI" Version="7.x" />

builder.Services.AddHealthChecks()
    .AddSqlServer(builder.Configuration.GetConnectionString("Default")!)
    .AddRedis(builder.Configuration["Redis:ConnectionString"]!)
    .AddAzureBlobStorage(builder.Configuration["Storage:ConnectionString"]!);

Liveness vs Readiness:

Liveness: "Is the process alive?"
  → If not: Kubernetes restarts the pod
  → Only check that the application responds

Readiness: "Can the application handle traffic?"
  → If not: Kubernetes removes the pod from the load balancer
  → Check DB, caches, dependencies

12. Monitoring and Alerts

Seq — query-based alerts:

Seq → Alerts → Add Alert
→ Title: "Error rate spike"
→ Signal: @Level >= 'Error'
→ Condition: count() > 10 in the last 5 minutes
→ Action: Webhook / Email

Application Insights — alerts:

Application Insights → Alerts → Create Alert Rule
→ Scope: Application Insights resource
→ Condition: Custom log search
→ Query: exceptions | summarize count() by bin(timestamp, 5m)
→ Threshold: > 5
→ Action Group: email + Teams webhook

13. Traceability and OpenTelemetry

Trace Context (W3C standard):

In ASP.NET Core logs:

{
  "timestamp": "2024-01-15T10:30:00Z",
  "level": "Information",
  "message": "Order processed",
  "traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
  "spanId": "00f067aa0ba902b7",
  "RequestId": "...",
  "ConnectionId": "..."
}

OpenTelemetry:

<PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="1.x" />
<PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.x" />
<PackageReference Include="OpenTelemetry.Instrumentation.Http" Version="1.x" />
<PackageReference Include="OpenTelemetry.Instrumentation.EntityFrameworkCore" Version="1.x" />
<PackageReference Include="OpenTelemetry.Exporter.Otlp" Version="1.x" />

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing =>
    {
        tracing
            .AddAspNetCoreInstrumentation()
            .AddHttpClientInstrumentation()
            .AddEntityFrameworkCoreInstrumentation()
            .AddOtlpExporter(otlp =>
            {
                otlp.Endpoint = new Uri("http://localhost:4317");  // Jaeger
            });
    })
    .WithMetrics(metrics =>
    {
        metrics
            .AddAspNetCoreInstrumentation()
            .AddHttpClientInstrumentation();
    });

Visualize with Jaeger:

# Run Jaeger locally with Docker
docker run -d -p 16686:16686 -p 4317:4317 jaegertracing/all-in-one
# → UI: http://localhost:16686

14. Summary and Key Points

Search Terms

asp.net · logging · monitoring · core · testing · patterns · architecture · c# · .net · development · log