Azure Fundamentals – Monitoring

Module 1 – Azure Monitor

Overview

Azure Monitor is the centralized monitoring platform for all Azure resources.

Data Sources

Types of Collected Data

Module 2 – Metrics and Metrics Explorer

Accessing Metrics

Azure Portal → Resource (e.g., VM) → Monitoring → Metrics
  → Metric: CPU Percentage
  → Aggregation: Average
  → Time range: Last 24 hours

Creating a Chart

• Choose one or more metrics.
• Configure time granularity (1min, 5min, 1h).
• Pin to dashboard.

Metric-Based Alerts

Azure Monitor → Alerts → Create
  → Scope: [Resource]
  → Condition: CPU Percentage > 80% (for 5 min)
  → Action Group: [email, SMS, webhook]
  → Alert rule name: "High CPU Alert"

Module 3 – Log Analytics and KQL

Log Analytics Workspace

• Stores logs from all configured sources.
• Queryable via KQL (Kusto Query Language).

Important Tables

KQL – Common Examples

// Check which VMs are active
Heartbeat
| summarize LastHeartbeat = max(TimeGenerated) by Computer
| order by LastHeartbeat desc

// Errors from the last 24 hours
AppExceptions
| where TimeGenerated > ago(24h)
| summarize count() by ExceptionType
| order by count_ desc

// Administrator activities
AzureActivity
| where Caller == "admin@contoso.com"
| where TimeGenerated > ago(7d)
| project TimeGenerated, OperationNameValue, ResourceGroup, ActivityStatus

// Average latency per endpoint
AppRequests
| where TimeGenerated > ago(1h)
| summarize AvgDuration = avg(DurationMs) by Url
| order by AvgDuration desc

Module 4 – Alerts and Action Groups

Alert Pipeline

Data source (Metric or Log)
  ↓ Condition
Alert triggered
  ↓
Action Group
  ├── Email notifications
  ├── SMS
  ├── Push notification (Azure app)
  ├── Voice call
  ├── Azure Function
  ├── Logic App
  ├── Webhook
  └── ITSM (ServiceNow, Jira)

Action Groups

• Reusable notification group across multiple alert rules.
• A change to the group applies to all rules that use it.

Azure Monitor → Action groups → Create
  → Group name: "Ops Team Notifications"
  → Actions:
    [Email] ops-team@contoso.com
    [SMS] +1-555-0100
    [Webhook] https://hooks.slack.com/...

Alert Severities

Module 5 – Azure Workbooks and Dashboards

Workbooks

• Interactive reports with text, metrics, logs, parameters.
• Pre-built templates for common scenarios (VM insights, Application Insights, etc.).

Azure Monitor → Workbooks → + New
  → Add text (Markdown)
  → Add query (KQL query → Log Analytics)
  → Add metrics (Metrics chart)
  → Add parameters (dropdown, time range)
  → Save & Share

Azure Dashboards

• Custom views with metrics tiles, alerts, workbooks.
• Shareable with the team.

Module 6 – Application Insights

Features

Summary

Advanced Deep-Dive – Azure Monitor In Depth

Section A – Azure Monitor: Complete Architecture

Platform Overview

Azure Monitor is the central nervous system of Azure observability. It ingests telemetry data from all Azure resources, on-premises environments, and hybrid clouds, then makes it available for analysis, visualization, and automated action.

Main Components

Detailed Data Sources

Monitoring Pipeline Architecture

flowchart TD
    subgraph Sources["Data Sources"]
        VM["Azure VMs / Arc"]
        APP["Applications\n(SDK / auto-instrumentation)"]
        AKS["AKS / Containers"]
        INFRA["Azure Infrastructure\n(SQL, Storage, VNet...)"]
        SUB["Subscription / Entra ID"]
    end

    subgraph Collect["Collection"]
        AMA["Azure Monitor Agent\n(AMA)"]
        DCR["Data Collection Rules"]
        DIAG["Diagnostic Settings"]
        SDK["Application Insights SDK"]
    end

    subgraph Store["Storage"]
        METRICS["Metrics Store\n(time-series 93d)"]
        LAW["Log Analytics\nWorkspace"]
        AI["Application Insights\nWorkspace"]
        STG["Storage Account\n(long-term archiving)"]
        EH["Event Hub\n(SIEM streaming)"]
    end

    subgraph Analyze["Analysis"]
        ME["Metrics Explorer"]
        KQL["KQL Queries"]
        WB["Workbooks"]
        DASH["Dashboards"]
    end

    subgraph Act["Action"]
        ALERT["Alerts"]
        AG["Action Groups"]
        AS["Autoscale"]
        LA["Logic Apps / Functions"]
    end

    VM --> AMA --> DCR --> LAW
    APP --> SDK --> AI
    AKS --> DCR --> LAW
    INFRA --> DIAG --> LAW
    INFRA --> DIAG --> STG
    INFRA --> DIAG --> EH
    SUB --> LAW
    METRICS --> ME
    LAW --> KQL
    AI --> KQL
    KQL --> WB
    KQL --> DASH
    ME --> ALERT
    KQL --> ALERT
    ALERT --> AG --> LA

Activity Log

The Activity Log is a subscription-level log that records all control plane operations:

• Creation / modification / deletion of resources
• RBAC changes
• Azure Policy operations
• Service health alerts

Default retention: 90 days. For longer retention → export to Log Analytics or Storage Account.

// All resource deletions in the last 7 days
AzureActivity
| where TimeGenerated > ago(7d)
| where OperationNameValue endswith "delete"
| where ActivityStatusValue == "Success"
| project TimeGenerated, Caller, ResourceGroup, Resource, OperationNameValue
| order by TimeGenerated desc

Diagnostic Settings

Diagnostic Settings allow sending resource logs and platform metrics to one or more destinations:

Azure Resource → Diagnostic Settings → + Add diagnostic setting
  ├── Logs: AllLogs / specific categories
  ├── Metrics: AllMetrics
  └── Destinations:
        ├── Log Analytics Workspace (KQL queries)
        ├── Storage Account (archiving / compliance)
        └── Event Hub (streaming to SIEM / Splunk)

Section B – Log Analytics and Advanced KQL

Workspace Architecture

A Log Analytics Workspace is an isolated data container with:

• Its own access control (RBAC)
• Its own retention (30 days by default, up to 730 days)
• Its own data tables
• Its own solution configuration

Workspace Design Strategies

Data Retention

KQL – Fundamental Operators

where – Filtering

// Filter HTTP 5xx errors from the last 24h
AppRequests
| where TimeGenerated > ago(24h)
| where ResultCode >= 500
| where Success == false

project – Column Selection

// Keep only useful columns
AppRequests
| where TimeGenerated > ago(1h)
| project TimeGenerated, Name, DurationMs, ResultCode, ClientIP

summarize – Aggregation

// Number of requests per HTTP code, sorted
AppRequests
| where TimeGenerated > ago(24h)
| summarize RequestCount = count(), AvgDuration = avg(DurationMs)
    by ResultCode
| order by RequestCount desc

// Error rate per hour
AppRequests
| where TimeGenerated > ago(7d)
| summarize
    Total = count(),
    Errors = countif(Success == false)
    by bin(TimeGenerated, 1h)
| extend ErrorRate = round(100.0 * Errors / Total, 2)

join – Table Joins

// Correlate exceptions with the requests that generated them
AppRequests
| where TimeGenerated > ago(1h)
| join kind=leftouter (
    AppExceptions
    | where TimeGenerated > ago(1h)
    | project ExceptionType, OperationId
) on OperationId
| project TimeGenerated, Name, DurationMs, ExceptionType

extend – Computed Columns

// Categorize latency
AppRequests
| where TimeGenerated > ago(1h)
| extend LatencyCategory = case(
    DurationMs < 200, "Fast",
    DurationMs < 1000, "Normal",
    DurationMs < 3000, "Slow",
    "Critical"
)
| summarize count() by LatencyCategory

render – Inline Visualization

// Latency chart over time
AppRequests
| where TimeGenerated > ago(24h)
| summarize AvgDuration = avg(DurationMs) by bin(TimeGenerated, 15m)
| render timechart

// HTTP status code pie chart
AppRequests
| where TimeGenerated > ago(24h)
| summarize count() by ResultCode
| render piechart

KQL – Useful Advanced Queries

// Top 10 IPs with the most failed requests
AppRequests
| where TimeGenerated > ago(24h)
| where Success == false
| summarize FailedRequests = count() by ClientIP
| top 10 by FailedRequests desc

// Latency percentiles (p50, p95, p99)
AppRequests
| where TimeGenerated > ago(1h)
| summarize
    p50 = percentile(DurationMs, 50),
    p95 = percentile(DurationMs, 95),
    p99 = percentile(DurationMs, 99)
    by Name
| order by p99 desc

// CPU anomaly detection on VMs
Perf
| where TimeGenerated > ago(24h)
| where ObjectName == "Processor" and CounterName == "% Processor Time"
| summarize AvgCPU = avg(CounterValue) by Computer, bin(TimeGenerated, 5m)
| where AvgCPU > 85
| order by TimeGenerated desc

// VMs with no heartbeat for more than 10 minutes (potentially down)
Heartbeat
| summarize LastHeartbeat = max(TimeGenerated) by Computer, ComputerIP
| extend TimeSinceHeartbeat = now() - LastHeartbeat
| where TimeSinceHeartbeat > 10m
| project Computer, ComputerIP, LastHeartbeat, TimeSinceHeartbeat

Log Analytics Solutions

Saved Searches and Log Alerts

Log Analytics Workspace → Logs → [Write KQL query]
  → Save → Save as query
    → Category: "Performance", "Security", etc.
    → Query name: "High CPU VMs"

// Create alert from a query
  → New alert rule
    → Condition: result count > 0 (or custom threshold)
    → Evaluation frequency: Every 5 minutes
    → Action Group: [Ops Team]

Section C – Application Insights In Depth

Integration Modes

SDK Integration by Language

.NET / ASP.NET Core

// Program.cs – ASP.NET Core 8
builder.Services.AddApplicationInsightsTelemetry(
    options => options.ConnectionString = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"]
);

// Custom event
private readonly TelemetryClient _telemetry;
_telemetry.TrackEvent("OrderPlaced", new Dictionary<string, string> {
    { "OrderId", orderId },
    { "UserId", userId }
});

// Custom metric
_telemetry.TrackMetric("CartItemCount", items.Count);

// Custom exception
try { ... }
catch (Exception ex) {
    _telemetry.TrackException(ex, new Dictionary<string, string> {
        { "OrderId", orderId }
    });
    throw;
}

Python

from applicationinsights import TelemetryClient

tc = TelemetryClient(os.environ["APPLICATIONINSIGHTS_CONNECTION_STRING"])

# Event
tc.track_event("UserLogin", {"UserId": user_id, "Method": "OAuth"})

# Metric
tc.track_metric("ProcessingTimeMs", processing_time)

# Exception
try:
    process_order()
except Exception as e:
    tc.track_exception()
    raise

tc.flush()

Java (Spring Boot)

<!-- pom.xml -->
<dependency>
    <groupId>com.microsoft.azure</groupId>
    <artifactId>applicationinsights-spring-boot-starter</artifactId>
    <version>2.6.4</version>
</dependency>

# applicationinsights.json (Java agent auto-instrumentation)
{
  "connectionString": "InstrumentationKey=...",
  "sampling": { "percentage": 100 },
  "customDimensions": {
    "Environment": "Production",
    "AppVersion": "2.4.1"
  }
}

Availability Tests

Application Insights → Availability → + Add test
  → Test name: "Homepage Health Check"
  → URL: https://app.contoso.com
  → Test frequency: Every 5 minutes
  → Test locations: East US, West Europe, Southeast Asia
  → Success criteria: HTTP 200, response < 2000ms
  → Alert: Yes → Action Group: Ops Team

Live Metrics (Real-Time Metrics Stream)

Live Metrics provides a real-time window with less than 1 second delay:

Application Map

The Application Map automatically detects components and their dependencies:

[React Frontend]
    ↓ HTTP
[API Gateway]
    ↓ HTTP          ↓ HTTP
[Orders Service]  [Users Service]
    ↓ SQL              ↓ Redis
[Azure SQL DB]    [Azure Cache]
    ↓ Event
[Service Bus]
    ↓
[Fulfillment Worker]

• Each node shows: error rate, average latency, request volume.
• Red edges indicate dependency issues.
• Clicking a node opens the component’s detailed metrics/logs.

Distributed Tracing

Distributed tracing allows following a request across all microservices:

// Retrieve all calls for a specific operation
union AppRequests, AppDependencies, AppExceptions
| where OperationId == "a1b2c3d4e5f6"
| order by TimeGenerated asc
| project TimeGenerated, itemType, Name, DurationMs, Success, Target

Sampling – Reducing Ingestion Costs

// applicationinsights.json – Fixed rate 25%
{
  "sampling": {
    "percentage": 25
  }
}

Important: Sampling is consistent – if a request is sampled, all its related dependencies and exceptions are too (same OperationId).

Section D – Advanced Azure Monitor Metrics

Metric Dimensions

Dimensions allow segmenting a metric by attribute:

Metrics Explorer → [Select a metric]
  → + Add filter → Dimension: HttpStatusCode → Value: 5xx
  → + Split by → Dimension: InstanceName

Available Aggregations

Custom Metrics

// Publish a custom metric via SDK
var metric = telemetryClient.GetMetric("OrderProcessingTime", "Region");
metric.TrackValue(processingMs, "EastUS");

// Via Azure Monitor REST API
POST https://management.azure.com/subscriptions/{sub}/resourceGroups/{rg}/providers/
     Microsoft.Insights/metrics
Authorization: Bearer {token}
{
  "time": "2025-01-15T14:30:00Z",
  "data": {
    "baseData": {
      "metric": "ActiveConnections",
      "namespace": "MyApp/Performance",
      "dimNames": ["ServiceName"],
      "series": [{ "dimValues": ["OrderAPI"], "sum": 142, "count": 1, "min": 142, "max": 142 }]
    }
  }
}

Prometheus Integration

Azure Monitor can ingest Prometheus metrics via Azure Monitor managed service for Prometheus:

# Prometheus scraping configuration for AKS
apiVersion: v1
kind: ConfigMap
metadata:
  name: ama-metrics-prometheus-config
  namespace: kube-system
data:
  prometheus-config: |-
    global:
      scrape_interval: 30s
    scrape_configs:
      - job_name: myapp
        static_configs:
          - targets: ["myapp-service:8080"]

// Query Prometheus metrics in Log Analytics
InsightsMetrics
| where Namespace == "prometheus"
| where Name == "http_requests_total"
| summarize sum(Val) by bin(TimeGenerated, 5m), Tags

Cross-Resource Queries

// Compare metrics from multiple workspaces
union workspace("workspace-prod").AppRequests,
      workspace("workspace-staging").AppRequests
| where TimeGenerated > ago(1h)
| summarize count() by bin(TimeGenerated, 5m), _ResourceId
| render timechart

Section E – Advanced Azure Monitor Alerts

Alert Types

flowchart LR
    subgraph AlertTypes["Alert Types"]
        MA["Metric Alerts\n(static/dynamic thresholds)"]
        LA["Log Alerts\n(KQL queries)"]
        AA["Activity Log Alerts\n(ARM operations)"]
        SA["Smart Detection\n(Application Insights anomalies)"]
    end

    subgraph Processing["Processing"]
        SG["Smart Groups\n(AI grouping)"]
        DEDUP["Deduplication\nsingle alert"]
    end

    subgraph Actions["Actions"]
        EMAIL["Email / SMS / Push"]
        WH["Webhook / HTTP"]
        FUNC["Azure Function"]
        LOGICAPP["Logic App"]
        ITSM["ITSM (ServiceNow, Jira)"]
        AS["Autoscale"]
    end

    MA --> SG
    LA --> SG
    AA --> SG
    SA --> SG
    SG --> DEDUP --> EMAIL
    DEDUP --> WH
    DEDUP --> FUNC
    DEDUP --> LOGICAPP
    DEDUP --> ITSM
    DEDUP --> AS

Metric Alerts – Static vs Dynamic Threshold

Static Threshold

Condition: CPU Percentage > 80%
Evaluation: Every 1 minute, looking back 5 minutes
Severity: Warning (Sev 2)

Dynamic Threshold

Dynamic thresholds use machine learning to detect anomalies without defining a fixed threshold:

Metric: Requests per second
Threshold: Dynamic
Sensitivity: Medium (reduces false positives)
Look-back period: 4 days (learns weekly patterns)
Aggregation period: 5 minutes

Advantage: automatically adapts to seasonal variations (higher traffic on Fridays, month-end spikes, etc.).

Log Alerts (KQL)

Azure Monitor → Alerts → Create → Log search
  → KQL Query:
      AppExceptions
      | where TimeGenerated > ago(5m)
      | summarize ExceptionCount = count() by ExceptionType
      | where ExceptionCount > 10
  → Evaluation frequency: Every 5 minutes
  → Lookback: Last 5 minutes
  → Threshold: Result count > 0
  → Severity: Error (Sev 1)

Activity Log Alerts

// Alert if someone deletes a Key Vault
AzureActivity
| where OperationNameValue == "Microsoft.KeyVault/vaults/delete"
| where ActivityStatusValue == "Success"

Activity Log Alert Condition:
  → Category: Administrative
  → Operation: Delete Key Vault
  → Status: Succeeded
  → Resource Type: Microsoft.KeyVault/vaults

Action Groups – Complete Configuration

Smart Groups

Smart Groups cluster correlated alerts to reduce noise:

• Grouping by resource, subscription, alert type, or AI-detected patterns.
• A single incident represents multiple correlated alerts.
• Significantly reduces alert fatigue.

Azure Monitor → Alerts → Smart Groups
  → Smart group: "Production API Degradation"
    → 14 alerts grouped
    → Resources: api-gateway, orders-svc, users-svc
    → Pattern: HTTP 503 errors + latency spikes

Section F – Advanced Azure Dashboards and Workbooks

Azure Dashboards – Tile Types

Dashboard Sharing

Dashboard → Share → Publish dashboard
  → Resource group: monitoring-rg
  → Share with: All users in tenant / Specific RBAC role

// RBAC access control for dashboards
Role: "Reader" → View only
Role: "Contributor" → Edit and share

Azure Workbooks – Advanced Features

Workbooks are parameterizable interactive reports:

Available Parameter Types

Workbook → + Add parameter
  → Parameter type: Drop down
  → Data source: KQL query
  → Query:
      Heartbeat
      | distinct Computer
  → Display as: dropdown
  → Parameter name: SelectedVM

Operational Workbook Example

[Title]: Production Health Overview
[Parameter]: Time range (last 4h)
[Section 1]: Services Status
  → KQL: Heartbeat | summarize LastSeen = max(TimeGenerated) by Computer
[Section 2]: Error Rate Trend
  → KQL: AppRequests | summarize Errors=countif(!Success), Total=count()
           by bin(TimeGenerated, 15m)
  → Render: Area chart
[Section 3]: Top Slowest Endpoints
  → KQL: AppRequests | top 10 by DurationMs desc
  → Render: Grid (table)
[Section 4]: Active Alerts
  → Alert data source: critical + error severity

Power BI Integration

// Export a Log Analytics query to Power BI
// In Log Analytics → Export → Export to Power BI

// Query optimized for Power BI (avoid large aggregations)
AppRequests
| where TimeGenerated > ago(30d)
| summarize
    DailyRequests = count(),
    DailyErrors = countif(Success == false),
    AvgLatency = avg(DurationMs)
    by Date = bin(TimeGenerated, 1d), Name

Section G – Azure Service Health

Service Health Components

flowchart TD
    SH["Azure Service Health"]
    SH --> SI["Service Issues\n(ongoing incidents affecting my resources)"]
    SH --> PM["Planned Maintenance\n(Microsoft planned maintenance)"]
    SH --> HA["Health Advisories\n(recommendations & breaking changes)"]
    SH --> RH["Resource Health\n(individual health of each resource)"]
    SH --> SHA["Service Health Alerts\n(proactive notifications)"]

Service Health Alerts

Service Health → Health alerts → + Create service health alert
  → Services: Virtual Machines, App Service, SQL Database
  → Regions: East US, West Europe
  → Event types: Service issue, Planned maintenance, Health advisory
  → Action Group: Executive Notifications

Best practice: Always configure Service Health alerts for critical production services. This is the only way to be proactively informed of an Azure outage before users complain.

Resource Health vs Service Health

Section H – Network Watcher Monitoring

Network Watcher Tools

NSG Flow Logs and Traffic Analytics

Network Watcher → NSG flow logs → [Select NSG]
  → Status: On
  → Flow Log version: Version 2
  → Storage Account: flow-logs-storage
  → Retention: 30 days
  → Traffic Analytics: Enabled
  → Traffic Analytics workspace: [Log Analytics Workspace]
  → Processing interval: Every 10 minutes

// Top source IPs in Traffic Analytics
AzureNetworkAnalytics_CL
| where TimeGenerated > ago(24h)
| where SubType_s == "FlowLog"
| summarize TotalFlows = count() by SrcIP_s
| top 10 by TotalFlows desc

// Traffic blocked by NSG
AzureNetworkAnalytics_CL
| where TimeGenerated > ago(1h)
| where AllowedInFlows_d == 0 and DeniedInFlows_d > 0
| project TimeGenerated, NSGRule_s, SrcIP_s, DestIP_s, DestPort_d

Connection Monitor

Network Watcher → Connection Monitor → + Create
  → Name: "Prod-DB-Connectivity"
  → Sources: [VM: api-server-01]
  → Destinations: [Azure SQL Endpoint: sql.contoso.com:1433]
  → Test configuration:
      → Protocol: TCP
      → Destination port: 1433
      → Probe interval: 30 seconds
  → Alerts: Latency > 50ms or packet loss > 5%

Section I – ITSM Integration

Available ITSM Connectors

ServiceNow Configuration

Azure Monitor → Action Groups → + Add action
  → Action type: ITSM
  → Connection: ServiceNow-Production
  → Work Item Type: Incident
  → Template:
      short_description: "Azure Alert: {AlertName}"
      description: "{AlertDetails}"
      urgency: "2 - High"
      impact: "2 - Medium"
      assignment_group: "Azure Operations"

Complete Alert → ITSM Ticket Pipeline

sequenceDiagram
    participant AM as Azure Monitor
    participant AG as Action Group
    participant SN as ServiceNow
    participant OPS as Ops Team

    AM->>AM: Alert condition triggered\n(CPU > 90% for 5min)
    AM->>AG: Triggers Action Group
    AG->>AG: Deduplication\n(avoids duplicates)
    AG->>SN: POST /api/now/table/incident\n(HTTP POST with JSON payload)
    SN->>SN: Creates incident INC0123456\nAssigns to "Azure Operations"
    SN->>OPS: Email + mobile notification
    OPS->>AM: Acknowledge the alert
    AM->>SN: Update: status = "In Progress"
    OPS->>SN: Resolves the incident
    SN->>AM: Feedback: incident resolved

Generic Webhook Configuration

// Azure Monitor webhook payload (common format)
{
  "schemaId": "AzureMonitorMetricAlert",
  "data": {
    "version": "2.0",
    "status": "Activated",
    "context": {
      "alertRuleName": "High CPU Alert",
      "severity": "2",
      "monitorCondition": "Fired",
      "resourceName": "prod-vm-01",
      "resourceGroupName": "production-rg",
      "subscriptionId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "timestamp": "2025-01-15T14:30:00Z"
    },
    "properties": {
      "metricValue": "92.4",
      "metricName": "Percentage CPU",
      "threshold": "80"
    }
  }
}

Section J – Monitoring Best Practices

The 4 Golden Signals (Google SRE)

The 4 golden signals are the most important metrics to monitor for any service:

quadrantChart
    title Monitoring priority by impact and complexity
    x-axis Low complexity --> High complexity
    y-axis Low impact --> High impact
    quadrant-1 "Do first"
    quadrant-2 "Plan"
    quadrant-3 "Deprioritize"
    quadrant-4 "Automate"
    HTTP Error Rate: [0.2, 0.9]
    CPU Alerts: [0.15, 0.7]
    Latency p99: [0.3, 0.85]
    Log Retention: [0.5, 0.4]
    Distributed Tracing: [0.75, 0.8]
    Custom Dashboards: [0.6, 0.5]
    Prometheus Integration: [0.8, 0.6]
    NSG Flow Logs: [0.55, 0.35]

SLI / SLO / SLA – Definitions

// Real-time SLI calculation (success rate)
AppRequests
| where TimeGenerated > ago(24h)
| summarize
    TotalRequests = count(),
    SuccessfulRequests = countif(Success == true)
    by bin(TimeGenerated, 1h)
| extend SLI = round(100.0 * SuccessfulRequests / TotalRequests, 3)
| extend SLO_Target = 99.9
| extend WithinSLO = SLI >= SLO_Target
| render timechart

Preventing Alert Fatigue

Alert fatigue occurs when a team receives too many irrelevant alerts and starts ignoring them.

Causes and Solutions

Recommended Severity Hierarchy

Monitoring Checklist for a New Application

□ Application Insights enabled (SDK or auto-instrumentation)
□ Log Analytics Workspace configured
□ Diagnostic Settings enabled on all resources
□ Metric alerts: CPU, memory, HTTP errors
□ KQL log alerts: critical exceptions, error patterns
□ Availability tests (URL ping at minimum)
□ Operational dashboard shared with the team
□ Action Groups configured (email + Teams/Slack webhook)
□ Service Health Alerts enabled
□ Log retention defined according to compliance requirements
□ Runbooks documented for each Sev 0/1 alert
□ SLOs defined and SLI/Error Budget dashboard created

Recommended Monitoring Architecture

flowchart TD
    subgraph Prod["Production"]
        APP["Application\n(Application Insights SDK)"]
        VM["VMs / AKS\n(Azure Monitor Agent)"]
        DB["Azure SQL / Cosmos"]
        NET["Virtual Network\n(NSG Flow Logs)"]
    end

    subgraph Monitoring["Monitoring Platform"]
        AI_WS["Application Insights\nWorkspace"]
        LAW["Log Analytics\nWorkspace"]
        METRICS["Azure Monitor\nMetrics Store"]
    end

    subgraph Alerting["Alerts & Actions"]
        RULES["Alert Rules\n(Metric + Log + Activity)"]
        SG["Smart Groups"]
        AG_OPS["Action Group\nOps Team"]
        AG_EXEC["Action Group\nExec / Stakeholders"]
    end

    subgraph Visualize["Visualization"]
        WB["Azure Workbooks\n(Ops Dashboard)"]
        DASH["Shared Dashboard\n(Executive View)"]
        PBI["Power BI\n(Monthly Reporting)"]
    end

    subgraph Respond["Response"]
        SN["ServiceNow\nIncident"]
        RUNBOOK["Automation\nRunbook"]
        AS["Autoscale"]
    end

    APP --> AI_WS
    VM --> LAW
    DB --> LAW
    NET --> LAW
    APP --> METRICS
    VM --> METRICS

    AI_WS --> RULES
    LAW --> RULES
    METRICS --> RULES
    RULES --> SG --> AG_OPS --> SN
    SG --> AG_OPS --> RUNBOOK
    METRICS --> AS

    AI_WS --> WB
    LAW --> WB
    WB --> DASH
    LAW --> PBI

Section K – Review Questions

KQL and Monitoring Questions

Q1. What KQL query finds the 5 most time-consuming operations in Application Insights?

AppRequests
| where TimeGenerated > ago(24h)
| top 5 by DurationMs desc
| project TimeGenerated, Name, DurationMs, ResultCode, ClientIP

Q2. How do you calculate the hourly error rate over the last 24 hours with KQL?

AppRequests
| where TimeGenerated > ago(24h)
| summarize
    Total = count(),
    Errors = countif(Success == false)
    by bin(TimeGenerated, 1h)
| extend ErrorRate = round(100.0 * Errors / Total, 2)
| render timechart

Q3. What is the difference between a metric alert and a log alert in Azure Monitor?

Q4. What is a Data Collection Rule (DCR) and why use it?

Q5. You want to be alerted if an Azure VM has not sent a heartbeat for more than 15 minutes. Write the KQL query and describe the alert configuration.

// KQL query for the alert
Heartbeat
| summarize LastHeartbeat = max(TimeGenerated) by Computer
| extend TimeSinceLast = now() - LastHeartbeat
| where TimeSinceLast > 15m
| project Computer, LastHeartbeat, TimeSinceLast

Q6. What is the difference between Azure Service Health and Azure Resource Health?

Q7. How do you configure Application Insights to reduce ingestion costs while maintaining data representativeness?

// applicationinsights.json
{
  "sampling": {
    "percentage": 10
  }
}

builder.Services.AddApplicationInsightsTelemetry(options => {
    options.EnableAdaptiveSampling = true;
    options.MaxTelemetryItemsPerSecond = 5;
});

Q8. Explain the concept of Error Budget in the SRE/SLO context.

Q9. What KQL command joins Application Insights exceptions with the HTTP requests that generated them?

AppRequests
| where TimeGenerated > ago(1h)
| where Success == false
| join kind=inner (
    AppExceptions
    | where TimeGenerated > ago(1h)
    | project ExceptionType, Message, OperationId, SeverityLevel
) on OperationId
| project TimeGenerated, Name, DurationMs, ResultCode, ExceptionType, Message
| order by TimeGenerated desc

Q10. How do you pin an Azure Monitor metrics chart to a shared dashboard?

Q11. What is distributed tracing in Application Insights and how does it work?

// See the full call chain for a transaction
union AppRequests, AppDependencies, AppExceptions, AppTraces
| where OperationId == "abc123def456"
| order by TimeGenerated asc
| project TimeGenerated, itemType, Name, DurationMs, Success, Target

Q12. Your team receives 200 alerts per day and has started ignoring them. What are the 5 priority actions to address alert fatigue?

Final Summary – Key Enriched Concepts

Search Terms

azure · fundamentals · monitoring · core · infrastructure · microsoft · alerts · log · kql · alert · analytics · integration · metrics · monitor · application · architecture · dashboards · data · groups · health · service · types · workbooks · action