Azure AI Fundamentals, Planning and Management

Certification: Microsoft Azure AI Engineer Associate — AI‑102
Prerequisites: Familiarity with Azure Fundamentals and the Azure portal

Table of Contents

1. Course and AI‑102 Exam Overview
2. What Is AI?
   • Artificial Intelligence vs Machine Learning
   • The ML Model Lifecycle
3. Machine Learning Fundamentals
   • Types of Machine Learning
   • Supervised vs Unsupervised Learning
   • Deep Learning
   • Data Preparation
   • Algorithms and Evaluation
   • Azure Automated Machine Learning — Demo Bike Rentals
   • Azure Machine Learning Designer — Demo Automobile Prices
   • Classification Pipeline — Demo Census Income
   • Confusion Matrix and Metrics
   • Inference Pipelines and Deployment
4. Selecting the Right Azure AI Service
   • Single-Service Instance
   • Multi-Service Instance
   • Choosing Between Single‑Service and Multi‑Service
   • Azure OpenAI Service
   • Azure AI Search
5. Planning, Creating, and Deploying an Azure AI Service
   • Responsible AI Principles
   • Endpoints and Access Keys
   • Integration into a CI/CD Pipeline
   • Infrastructure as Code
   • Container Deployment
6. Managing, Monitoring, and Securing an Azure AI Service
   • Monitoring an Azure AI Resource
   • Configure Diagnostic Logging
   • Managing Costs
   • Managing Account Keys
   • Protecting Keys with Azure Key Vault
7. AI‑102 Exam Tips
8. References

1. Course and AI‑102 Exam Overview

The Azure AI Engineer Role

The AI‑102 exam targets Azure AI Engineers who have the following responsibilities:

mindmap
  root((Azure AI Engineer))
    Understand Requirements
      Map to appropriate Azure AI services
    Develop and Deploy
      Azure AI Resources
    Assist Architects
      Integrate AI services into applications
    Maintain and Monitor
      Tune deployed AI resources

Exam Skill Distribution

pie title Skill Distribution — AI-102 Exam
    "Planning and management" : 17
    "Content moderation solutions" : 12
    "Computer vision solutions" : 17
    "NLP solutions" : 32
    "Knowledge Mining and Document Intelligence" : 12
    "Generative AI solutions" : 12

Exam Format

Recommended Resources

• Official Microsoft study guide for AI‑102
• Microsoft Learn modules (free + labs included)
• Exam sandbox (to familiarize with the interface)
• YouTube study cram (quick synthesis review)

2. What Is AI?

Artificial Intelligence encompasses capabilities that allow software to emulate certain human skills.

Human Capabilities Emulated by AI

mindmap
  root((AI))
    Vision
      Image Classification
      Object Detection
      OCR - Text Reading
      Facial Recognition
      Video Analysis
    Language and Speech
      Speech Understanding
      Translation
      Sentiment Analysis
      Text Summarization
    Decision
      Data Analysis
      Anomaly Detection
    Interaction
      Two-way Flows
      Knowledge Base
      Conversation Scripts
    Creation
      Text Generation
      Image Generation
      Embeddings

Artificial Intelligence vs Machine Learning

graph TD
    DS[Data Science\nData processes and analysis\nPattern discovery] --> ML
    ML[Machine Learning\nTraining predictive models\nStatistical algorithms] --> AI
    AI[Artificial Intelligence\nEmulating human capabilities\nBuilt on ML]

    style DS fill:#F4D03F,color:#000
    style ML fill:#85C1E9,color:#000
    style AI fill:#82E0AA,color:#000

Machine Learning is a subset of Data Science, and AI is built on Machine Learning, but the two terms are not interchangeable.

The ML Model Lifecycle

sequenceDiagram
    participant D as 📊 Labeled Data
    participant T as 🤖 Training
    participant V as ✅ Validation
    participant P as 🚀 Production

    D->>T: Training data (with known labels)
    T->>V: Trained model
    V->>V: Test data (without labels)
    V->>V: Compare predictions vs actual labels
    V-->>T: Adjustments if needed
    V->>P: Validated and deployed model
    P->>P: New real data (without labels)
    P-->>P: Inferences with confidence score

Key Concepts:

3. Machine Learning Fundamentals

Types of Machine Learning

mindmap
  root((Machine Learning))
    Supervised
      Regression
        Predict a numeric value
        Ex: Car price
      Classification
        Binary - 2 classes 0 or 1
          Ex: Diabetic or not
        Multi-class - N classes
          Ex: Mood type
    Unsupervised
      Clustering
        Group similar elements
        Ex: Grouping flowers
    Specialized
      Deep Learning
        Artificial neural networks
        Unstructured data
        Images - video - audio - text

Regression

Regression uses historical data with features to predict a numeric value (label).

Example — Car dataset:

Rule to remember: Regression = predict a numeric value

Classification

Classification predicts which category or class an element belongs to.

Binary classification — Diabetes example:

0 = negative / not diabetic · 1 = positive / diabetic

Multi-class classification: N possible classes (e.g., mood — happy, sad, angry, worried)

Clustering

Clustering groups similar elements into clusters. There is no label to predict.

Example — Penguin dataset:

Supervised vs Unsupervised Learning

graph LR
    subgraph Supervised["🎓 Supervised Learning"]
        S1[Known features] --> S3[Predicts a value\nor a category]
        S2[Known labels] --> S3
    end
    subgraph Unsupervised["🔍 Unsupervised Learning"]
        N1[Known features] --> N3[Groups similar\nelements into clusters]
        N2[No labels] --> N3
    end
    Supervised --> Reg[Regression]
    Supervised --> Class[Classification]
    Unsupervised --> Clust[Clustering]

    style Supervised fill:#E8F4FD
    style Unsupervised fill:#FEF9E7

Deep Learning

Deep Learning uses artificial neural networks with multiple layers. The model learns features itself from data.

graph LR
    subgraph Input["Input Layer"]
        I1[Pixel 1]
        I2[Pixel 2]
        I3[Pixel N]
    end
    subgraph Hidden["Hidden Layers"]
        H1[Neuron 1]
        H2[Neuron 2]
        H3[Neuron 3]
        H4[Neuron 4]
    end
    subgraph Output["Output Layer"]
        O1[Result]
    end
    I1 --> H1; I1 --> H2
    I2 --> H2; I2 --> H3
    I3 --> H3; I3 --> H4
    H1 --> O1; H2 --> O1; H3 --> O1; H4 --> O1

    style Input fill:#AED6F1
    style Hidden fill:#A9DFBF
    style Output fill:#FAD7A0

Machine Learning vs Deep Learning:

Data Preparation

flowchart TD
    A[📥 Raw Dataset] --> B{Missing values?}
    B -- Yes --> C[Cleaning\nRemove or replace NaN]
    B -- No --> D{Very different scales?}
    C --> D
    D -- Yes --> E[Normalization\nScaling 0 to 1]
    D -- No --> F[Split Data]
    E --> F
    F --> G[70% Training Set]
    F --> H[30% Validation Set]
    G --> I[Model Training]
    H --> J[Validation / Evaluation]

    style A fill:#85C1E9
    style G fill:#82E0AA
    style H fill:#F9E79F

Normalization example:

Algorithms and Evaluation

Microsoft Algorithm Cheat Sheet

flowchart TD
    A[What is your objective?] --> B[Predict a numeric value]
    A --> C[Predict between 2 categories]
    A --> D[Predict between N categories]
    A --> E[Group elements]

    B --> B1[Linear Regression\nFast Forest Quantile Regression\nNeural Network Regression]
    C --> C1[Two-Class Logistic Regression\nTwo-Class Boosted Decision Tree\nTwo-Class SVM]
    D --> D1[Multiclass Decision Forest\nMulticlass Neural Network]
    E --> E1[K-Means Clustering]

    style A fill:#0078D4,color:#fff
    style B1 fill:#E8F4FD
    style C1 fill:#E8F4FD
    style D1 fill:#E8F4FD
    style E1 fill:#FEF9E7

Recommended algorithms:

Evaluation Metrics — Regression

$$R^2 = 1 - \frac{\sum(y_i - \hat{y}_i)^2}{\sum(y_i - \bar{y})^2}$$

R² = 1 → Perfect predictions · R² = 0 → No better than the mean

Azure Automated Machine Learning — Demo Bike Rentals

Objective: Predict the number of bike rentals per day based on weather, season, day, etc.

Dataset — Main columns:

Steps in Azure ML Studio:

1. Azure ML Studio → Automated ML → New Automated ML job
2. Task type: Regression
3. Dataset: bikerentals (web URL, Tabular type)
   URL: provided Microsoft dataset
4. Feature selection: remove the "atemp" column (not relevant)
5. Target column: rentals
6. Limits:
   - Experiment timeout = 15 min
   - Enable early termination ✅
7. Compute type: Compute Cluster
8. Submit → wait ~30-40 min

Deployment configuration:

Deploy → Web service
Name   : [service name]
Compute type         : Azure Container Instance
CPU reserve capacity : 1
Memory capacity      : 1 GB

Results obtained:

Actual value: 331 rentals · Predicted value: 380 rentals (acceptable result)

Automated ML flow:

sequenceDiagram
    participant U as User
    participant AML as Azure Automated ML
    participant C as Compute Cluster

    U->>AML: 1. Provide a dataset
    U->>AML: 2. Choose the label (rentals)
    U->>AML: 3. Configure limits (15 min timeout)
    AML->>C: 4. Launch multiple runs with different algorithms
    C-->>AML: 5. Results from each algorithm
    AML-->>U: 6. Best model selected (VotingEnsemble)
    U->>AML: 7. Deploy the model
    AML-->>U: 8. REST endpoint available for testing

Azure Machine Learning Designer — Demo Automobile Prices

Objective: Predict a car’s price based on its characteristics.

flowchart TD
    A[📂 Automobile Price Data\nDataset] --> B[Select Columns in Dataset\nExclude: normalized-losses]
    B --> C[Clean Missing Data\nRemove rows with NaN\nAll columns]
    C --> D[Split Data\n70% Training / 30% Validation]
    D -- 70% Training --> E[Train Model\nLabel: price]
    D -- 30% Validation --> F[Score Model]
    G[Linear Regression\nAlgorithm] --> E
    E --> F
    F --> H[Evaluate Model\nMetrics: R², RMSE]

    style A fill:#AED6F1
    style G fill:#A9DFBF
    style H fill:#FAD7A0

Module configuration:

Classification Pipeline — Demo Census Income

Objective: Predict whether income is ≤ $50,000 or > $50,000

flowchart TD
    A[📂 Adult Census Income\nDataset] --> B[Select Columns in Dataset\nExclude: race, sex]
    B --> C[Clean Missing Data\nAll columns except income\nRemove missing rows]
    C --> D[Normalize Data\nColumns: fnlwgt, capital-gain, capital-loss]
    D --> E[Split Data\n70% Training / 30% Validation]
    E -- 70% Training --> F[Train Model\nLabel: income]
    E -- 30% Validation --> G[Score Model]
    H[Two-Class Logistic Regression\nAlgorithm] --> F
    F --> G
    G --> I[Evaluate Model\nConfusion Matrix, AUC, Accuracy]

    style A fill:#AED6F1
    style H fill:#A9DFBF
    style I fill:#FAD7A0

Why exclude race and sex? → To avoid biases in the model.
Why exclude income from Clean Missing Data? → Because during inference, income is unknown — it’s what we’re trying to predict.

Confusion Matrix and Metrics

The confusion matrix is the main tool for evaluating a classification model.

                    ┌───────────────────────────────────────────┐
                    │           ACTUAL VALUE                     │
                    │    Positive (1)     │    Negative (0)      │
┌───────────────────┼─────────────────────┼──────────────────────┤
│ PREDICTED Pos.(1) │  True Positives     │   False Positives    │
│                   │       (TP)          │        (FP)          │
│           Neg.(0) │  False Negatives    │   True Negatives     │
│                   │       (FN)          │        (TN)          │
└───────────────────┴─────────────────────┴──────────────────────┘

Concrete example — Diabetes prediction:

                    ┌──────────────────────────────────────────┐
                    │           ACTUAL VALUE                    │
                    │  Diabetic (1)  │  Not Diabetic (0)       │
┌───────────────────┼────────────────┼─────────────────────────┤
│ PREDICTED Diab.(1)│   TP = 3,500   │    FP = 1,200           │
│           Non (0) │   FN = 1,500   │    TN = 8,000           │
└───────────────────┴────────────────┴─────────────────────────┘

Typical exam question: “How many people are actually diabetic but the model predicted them as not diabetic?” → False Negatives (FN) = 1,500

Classification metrics:

$$\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN} \quad \text{Precision} = \frac{TP}{TP + FP} \quad \text{Recall} = \frac{TP}{TP + FN}$$

Inference Pipelines and Deployment

graph TD
    A[Training Pipeline] --> B[Create an Inference Pipeline]
    B --> C[Real-time Inference]
    B --> D[Batch Inference]
    C --> C1[A few predictions at a time]
    C --> C2[Deployment:\nAzure Container Instance\nor Azure Kubernetes Service]
    D --> D1[Many predictions at once]

    style A fill:#AED6F1
    style C fill:#A9DFBF
    style D fill:#FAD7A0

Differences between Training vs Inference Pipeline:

Inference pipeline deployment steps:

1. After running the Training Pipeline in Designer:
   → Select (...) → Create inference pipeline → Real-time inference pipeline

2. Fix the connection:
   Web Service Input → connect to: Apply Transformation

3. Exclude the label from Select Columns:
   Add "income" to the exclusion list
   (the pipeline will no longer ask for the value to predict as input)

4. Remove the "Evaluate Model" module

5. Configure & Submit → Create a new experiment

6. Deploy:
   Compute type: Azure Container Instance (testing) or Azure Kubernetes Service (production)

7. Test in Endpoints → Test tab
   Developer access: REST API + authentication key

Deployment options:

Exam rule: Mention of “test” → ACI · Otherwise → AKS

4. Selecting the Right Azure AI Service

Service Types Overview

graph TD
    A[Azure AI Services] --> B[Single-Service\nInstance]
    A --> C[Multi-Service\nInstance]
    A --> D[Azure OpenAI\nService]
    A --> E[Azure AI Search]

    B --> B1[One AI type only\nDedicated endpoint\nIsolated billing\nFree SKU available]
    C --> C1[Almost all AI types\nSingle endpoint\nConsolidated billing\nPaid SKU only]
    D --> D1[OpenAI models\nGPT, DALL-E, ADA\nDeployment per model\nSpecialized GPU hardware]
    E --> E1[Hybrid search\nLexical and semantic\nVectors and embeddings\nProvisioned SKU billing]

    style A fill:#0078D4,color:#fff
    style B fill:#50E6FF,color:#000
    style C fill:#50E6FF,color:#000
    style D fill:#50E6FF,color:#000
    style E fill:#50E6FF,color:#000

Single-Service Instance

Each instance can only perform one type of AI capability.

Available categories:

Decision and Content Safety

⚠️ Content Moderator is retired — use Content Safety instead.

Knowledge Mining and Document Intelligence

Generative AI

Computer Vision

Natural Language Processing (NLP)

Multi-Service Instance

A single instance capable of performing almost all types of AI capabilities.

graph LR
    App[Application] -->|Single endpoint\nSingle key| MS[Multi-Service\nInstance]
    MS --> V[Vision]
    MS --> L[Language]
    MS --> S[Speech]
    MS --> CS[Content Safety]
    MS --> T[Translator]
    MS --> DI[Document Intelligence]

    style MS fill:#0078D4,color:#fff
    style App fill:#82E0AA

Demo — Using the OCR service via Python:

import os
from azure.cognitiveservices.vision.computervision import ComputerVisionClient
from msrest.authentication import CognitiveServicesCredentials

# Configuration with environment variables (secure)
endpoint = os.environ["AZURE_AI_ENDPOINT"]
api_key = os.environ["AZURE_AI_KEY"]

# Create the Computer Vision client
client = ComputerVisionClient(endpoint, CognitiveServicesCredentials(api_key))

# Send the image and read the text (OCR)
image_path = "whiteboard.png"
with open(image_path, "rb") as image_stream:
    read_response = client.read_in_stream(image_stream, raw=True)

# Retrieve the operation ID from the header
operation_id = read_response.headers["Operation-Location"].split("/")[-1]

# Wait for the result
import time
while True:
    result = client.get_read_result(operation_id)
    if result.status not in ["notStarted", "running"]:
        break
    time.sleep(1)

# Display results (words and coordinates)
for page in result.analyze_result.read_results:
    for line in page.lines:
        for word in line.words:
            print(f"Word: '{word.text}' | Coordinates: {word.bounding_box}")

Result obtained on a hand-drawn whiteboard:
The service identified words such as “semantic index”, “retrieval augmented generation”, “OK2”, “me”, “plan” with their precise coordinates in the image.

Choosing Between Single‑Service and Multi‑Service

graph TD
    A{What is your need?} --> B[Experimentation\nat no cost]
    A --> C[Use many\ntypes of AI]
    A --> D[Isolated billing\nper AI type]
    A --> E[Access to all\navailable services]

    B --> B1[Single-Service\nFree SKU available]
    C --> C1[Multi-Service\nSingle endpoint and key]
    D --> D1[Single-Service\nGranular billing]
    E --> E1[Single-Service\nAll services available]

    style B1 fill:#82E0AA
    style C1 fill:#82E0AA
    style D1 fill:#82E0AA
    style E1 fill:#82E0AA

Azure OpenAI Service

Microsoft provides copies of OpenAI models hosted on Azure.

graph TD
    AO[Azure OpenAI Service\nInstance] --> D[Model Deployments]
    D --> GPT[GPT\nText generation\nPrompt responses\nMicrosoft Copilot]
    D --> DALLE[DALL-E v3\nImage generation\nfrom text]
    D --> ADA[ADA v002\nEmbedding\nSemantic vector\nrepresentation]
    D --> WH[Whisper\nTranscription and translation\nSpeech-to-text]
    D --> TTS[Text-to-Speech\nVoice synthesis]

    style AO fill:#0078D4,color:#fff
    style GPT fill:#FFF3CD
    style DALLE fill:#FFF3CD
    style ADA fill:#FFF3CD

Flow for creating and using an Azure OpenAI Service:

sequenceDiagram
    participant U as User
    participant AZ as Azure Portal
    participant AOS as Azure OpenAI Studio

    U->>AZ: 1. Create an Azure OpenAI instance\n(Resource Group, Region, Name)
    AZ-->>U: Instance created (endpoint + keys)
    U->>AOS: 2. Access via Azure OpenAI Studio
    U->>AOS: 3. Model deployments → New deployment
    AOS-->>U: 4. Model deployed (e.g.: gpt-4, ada-002)
    U->>AOS: 5. Use via endpoint + deployment name

Building the URL to call an OpenAI model:

https://{endpoint_base}/openai/deployments/{deployment_name}/completions?api-version=2024-02-01

Concrete example:

https://my-openai.openai.azure.com/openai/deployments/gpt-4-deployment/completions?api-version=2024-02-01

Unlike other AI services, the URL must include the deployment name because a single instance can host multiple different models.

Provisioned Throughput Units (PTUs):

If PTU capacity is exceeded → HTTP 429 response → Route to a non-PTU deployment as fallback.

Token calculation with GPT:

import os
from openai import AzureOpenAI

client = AzureOpenAI(
    api_key=os.environ["AZURE_OPENAI_KEY"],
    api_version="2024-02-01",
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"]
)

response = client.chat.completions.create(
    model="gpt-4-deployment",  # Deployment name (not model name)
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain RAG in 3 sentences."}
    ]
)

print(response.choices[0].message.content)
print(f"Tokens used - Prompt: {response.usage.prompt_tokens}, "
      f"Completion: {response.usage.completion_tokens}")

Azure AI Search

Hybrid search (lexical + semantic) service primarily used with LLMs.

graph LR
    subgraph Sources["Data Sources"]
        BS[Azure Blob Storage]
        CDB[Cosmos DB]
        SQL[Azure SQL Database]
        SP[SharePoint]
        More[And many more...]
    end

    subgraph Search["Azure AI Search"]
        Idx[Indexer\nText extraction]
        Vec[Vectorization\nEmbeddings via ADA]
        LexIdx[Lexical index\nExact search]
        VecIdx[Vector index\nSemantic search]
        Rank[Semantic Ranker\nScore 0 to 4]
    end

    subgraph Output["Results"]
        Fused[Reciprocal Rank Fusion\nResult merging]
        LLM[GPT / LLM\nFinal response via RAG]
    end

    Sources --> Idx
    Idx --> Vec
    Idx --> LexIdx
    Vec --> VecIdx
    LexIdx --> Fused
    VecIdx --> Fused
    Fused --> Rank
    Rank --> LLM

    style Search fill:#E8F4FD
    style Output fill:#E8F8E8

How hybrid search works:

1. Lexical search: exact keyword matching (useful for product names, SKUs)
2. Semantic search: based on vectors (embeddings) that represent semantic meaning
3. Reciprocal Rank Fusion: merges the two sets of results
4. Semantic Ranker: score from 0 to 4 (4 = most relevant) — available from Basic SKU

Azure AI Search billing (exception: provisioned SKU, not consumption):

Billing is per hour (not per transaction), based on the SKU and number of provisioned compute units.

Supported data sources:

Azure Blob Storage / Data Lake Gen2
Azure Cosmos DB (SQL, Gremlin, MongoDB)
Azure SQL Database / Table Storage
SharePoint / MySQL / Azure Files
+ many partner connectors

5. Planning, Creating, and Deploying an Azure AI Service

Responsible AI Principles

Since AI is used for decisions impacting people’s lives (loans, healthcare, job applications), the 6 Responsible AI Principles are essential and concern all levels of an organization.

graph TD
    Account[Accountability] --> Fair
    Account --> Trans

    subgraph Core["Core Principles"]
        Fair[Fairness and Inclusiveness]
        Rel[Reliability and Safety]
        Priv[Privacy and Security]
        Trans[Transparency]
    end

    Fair --> Trans
    Rel --> Trans
    Priv --> Trans

    style Account fill:#E74C3C,color:#fff
    style Trans fill:#F39C12,color:#fff
    style Fair fill:#3498DB,color:#fff
    style Rel fill:#27AE60,color:#fff
    style Priv fill:#8E44AD,color:#fff

Detail of the 6 principles:

Critical note: The quality of training data directly determines the presence or absence of bias in the model.

Endpoints and Access Keys

Structure for standard services (Single or Multi-Service):

Keys and Endpoint (in the Azure portal):
├── Endpoint : https://{service-name}.cognitiveservices.azure.com/
├── KEY 1   : [32 hex characters]
└── KEY 2   : [32 hex characters]

Structure for Azure OpenAI (endpoint + deployment name required):

https://{endpoint_base}/openai/deployments/{deployment_name}/{action}?api-version=2024-02-01

GPT example (completions):
https://my-openai.openai.azure.com/openai/deployments/gpt-4/completions?api-version=2024-02-01

ADA example (embeddings):
https://my-openai.openai.azure.com/openai/deployments/ada-002/embeddings?api-version=2024-02-01

Each OpenAI model type supports different actions (completions, embeddings, chat/completions).

REST API call (curl example):

curl https://my-service.cognitiveservices.azure.com/vision/v3.2/read/analyze \
  -H "Ocp-Apim-Subscription-Key: {API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/image.jpg"}'

SDK call via Python (Computer Vision):

from azure.cognitiveservices.vision.computervision import ComputerVisionClient
from msrest.authentication import CognitiveServicesCredentials

client = ComputerVisionClient(
    endpoint=os.environ["AZURE_AI_ENDPOINT"],
    credentials=CognitiveServicesCredentials(os.environ["AZURE_AI_KEY"])
)

Integration into a CI/CD Pipeline

DevOps philosophy:

graph LR
    Dev[Developers\nEmbrace change] <-->|Collaboration| Ops[Operations\nStability]
    Dev --> CICD[CI/CD Pipeline]
    Ops --> CICD
    CICD --> Value[Continuous\nIncremental Value]

    style CICD fill:#0078D4,color:#fff
    style Value fill:#27AE60,color:#fff

Complete DevOps pipeline flow with Azure AI services:

flowchart TD
    Git[Git Repository\nSource code + IaC templates\n+ Artifacts] --> Trigger[Trigger\nCommit or merge]
    Trigger --> IaC[1. Infrastructure as Code\nBicep/Terraform deployment\nCreate Azure AI resources]
    IaC --> Build[2. Code compilation\nBuild and store artifacts]
    Build --> Deploy[3. Deployment\nTest environment]
    Deploy --> Test[4. Automated tests\nFunctional, smoke, load\nAzure Load Testing]
    Test --> Fault[5. Fault injection\nAzure Chaos Studio]
    Fault --> Gate{6. Quality gate\nFault threshold / tickets\nWait time}
    Gate -- Success --> Prod[7. Production deployment\nIaC + Artifacts]
    Gate -- Failure --> Git
    Prod --> Monitor[8. Continuous monitoring\nSynthetic transactions\nAlerts and feedback]

    style Git fill:#F4D03F
    style Prod fill:#82E0AA
    style Gate fill:#E74C3C,color:#fff

Infrastructure as Code

Azure AI resources are standard Azure resources → they must be deployed with Infrastructure as Code.

Available options:

Bicep example — Create an Azure AI service:

// Deploy an Azure AI account (Multi-Service)
resource aiService 'Microsoft.CognitiveServices/accounts@2023-05-01' = {
  name: 'my-ai-service'
  location: 'eastus'
  kind: 'CognitiveServices'  // Multi-service
  sku: {
    name: 'S0'
  }
  properties: {
    publicNetworkAccess: 'Enabled'
    networkAcls: {
      defaultAction: 'Allow'
    }
  }
}

Bicep example — Create specific services:

// Computer Vision service (single-service)
resource computerVision 'Microsoft.CognitiveServices/accounts@2023-05-01' = {
  name: 'my-computer-vision'
  location: resourceGroup().location
  kind: 'ComputerVision'
  sku: {
    name: 'F0'  // Free SKU available for single-service
  }
  properties: {}
}

// Speech service (single-service)
resource speechService 'Microsoft.CognitiveServices/accounts@2023-05-01' = {
  name: 'my-speech-service'
  location: resourceGroup().location
  kind: 'SpeechServices'
  sku: {
    name: 'F0'
  }
  properties: {}
}

Terraform example — Create an Azure AI service:

# Terraform - Azure Cognitive Services (Multi-Service)
resource "azurerm_cognitive_account" "ai_service" {
  name                = "my-ai-service"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  kind                = "CognitiveServices"
  sku_name            = "S0"

  identity {
    type = "SystemAssigned"
  }

  tags = {
    Environment = "Production"
  }
}

# Terraform - Azure OpenAI
resource "azurerm_cognitive_account" "openai" {
  name                = "my-openai-service"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  kind                = "OpenAI"
  sku_name            = "S0"
}

resource "azurerm_cognitive_deployment" "gpt4" {
  name                 = "gpt-4-deployment"
  cognitive_account_id = azurerm_cognitive_account.openai.id
  model {
    format  = "OpenAI"
    name    = "gpt-4"
    version = "0613"
  }
  scale {
    type = "Standard"
  }
}

Multi-environment parameterization (single template, separate parameter files):

// Single template with parameters
param serviceName string
param environment string
param keyVaultName string

resource aiService 'Microsoft.CognitiveServices/accounts@2023-05-01' = {
  name: '${serviceName}-${environment}'
  // ...
}

// dev.parameters.json
{
  "serviceName": { "value": "my-ai" },
  "environment": { "value": "dev" },
  "keyVaultName": { "value": "kv-dev-123" }
}

// prod.parameters.json
{
  "serviceName": { "value": "my-ai" },
  "environment": { "value": "prod" },
  "keyVaultName": { "value": "kv-prod-456" }
}

Container Deployment

Many Azure AI services can run outside Azure via Docker containers — to reduce latency, meet data sovereignty requirements, or comply with regulatory constraints.

graph LR
    subgraph Azure["Azure (billing and license)"]
        AIS[Azure AI Instance\nEndpoint + API Key\nConsumption billing]
    end
    subgraph Local["Local / on-premises environment"]
        Docker[Docker / Kubernetes\nAzure AI Container]
        App[Application]
    end

    AIS -- Key + Endpoint --> Docker
    App --> Docker
    Docker -- Consumption metrics --> AIS

    style Azure fill:#E8F4FD
    style Local fill:#E8F8E8

⚠️ Azure AI Search and Azure OpenAI are not available in containers (specialized GPU infrastructure required).

Azure AI container deployment steps:

1. Deploy an Azure AI instance for the desired service
   → Retrieve the endpoint and API key

2. Validate hardware prerequisites
   → Required CPU instruction sets, RAM, number of cores

3. Download the image from the Microsoft Container Registry
   → docker pull mcr.microsoft.com/azure-cognitive-services/{service-name}

4. Start the container with the required parameters

Docker run example — Computer Vision service (OCR):

docker run --rm -it -p 5000:5000 \
  --memory 8g \
  --cpus 4 \
  mcr.microsoft.com/azure-cognitive-services/vision/read:latest \
  Eula=accept \
  Billing=https://my-service.cognitiveservices.azure.com/ \
  ApiKey=YOUR_API_KEY_HERE

Environment variables (recommended method):

# Define variables in the environment
export BILLING_ENDPOINT="https://my-service.cognitiveservices.azure.com/"
export SERVICE_KEY="your_api_key"

# Start the container with environment variables
docker run --rm -it -p 5000:5000 \
  --memory 8g --cpus 4 \
  mcr.microsoft.com/azure-cognitive-services/vision/read:latest \
  Eula=accept \
  Billing=${BILLING_ENDPOINT} \
  ApiKey=${SERVICE_KEY}

⚠️ Security: Azure AI containers are open by default — put an application firewall (WAF) or load balancer with access controls in front of the container.

Available image registry:

mcr.microsoft.com/azure-cognitive-services/
├── vision/read:latest                    ← OCR / Computer Vision
├── vision/face:latest                    ← Face API
├── language/sentiment:latest             ← Sentiment analysis
├── speech/neural-text-to-speech:latest   ← Text-to-Speech
├── speech/speech-to-text:latest          ← Speech recognition
├── translator/text-translation:latest    ← Translation
└── ...

Required parameters for all containers:

6. Managing, Monitoring, and Securing an Azure AI Service

Monitoring an Azure AI Resource

Each Azure resource has metrics and logs available for different uses.

graph TD
    Res[Azure AI Resource] --> AL[Activity Log\n90 days by default\nControl plane actions\ne.g.: key rotation]
    Res --> Met[Azure Monitor Metrics\nTime series DB\nFree - 93 days\nActive by default]
    Res --> Diag[Diagnostic Settings\nAdditional logs\nOptional - configurable]

    Met --> Native[Native visualization\nAzure Portal]
    Met --> Grafana[Grafana / third-party tools]
    Met --> Alerts[Azure Monitor Alerts]

    Diag --> LA[Log Analytics\nWorkspace\nKQL - Advanced analysis]
    Diag --> SA[Storage Account\nLow-cost retention]
    Diag --> EH[Event Hub\nSIEM integration]

    style Res fill:#0078D4,color:#fff
    style AL fill:#F4D03F
    style Met fill:#A9DFBF
    style Diag fill:#AED6F1

Available signal types:

Key metrics for Azure OpenAI:

Example: Splitting metrics by deployment name (Azure OpenAI):

Azure Portal → OpenAI instance → Metrics
→ Select "OpenAI Requests"
→ Apply splitting → Model Deployment Name

Result: Visualize separately the requests for:
- gpt-4-deployment (GPT-4 for completions)
- ada-002-deployment (ADA for Azure AI Search embeddings)

Azure Monitor Metrics Data Plane API:

# Retrieve metrics for up to 50 resources in a single call
# (same region and same subscription)
GET https://management.azure.com/subscriptions/{subscriptionId}/
    providers/microsoft.insights/metrics?
    api-version=2021-05-01&
    resourceids={resource_id_1}&resourceids={resource_id_2}&
    metricnames=TotalCalls&
    aggregation=Count

Configure Azure Monitor alerts:

Azure Portal → Resource → Monitoring → Alerts → Create alert rule

Available signals:
├── Metrics (e.g.: Total Calls > 10,000 / hour)
├── Custom log search (KQL)
└── Activity Log (e.g.: key regenerated)

Threshold types:
├── Static: fixed value (e.g.: > 1,000 requests)
└── Dynamic: ML analyzes history and detects anomalies
    (sensitivity: low / medium / high)

Available actions:
├── Email / SMS / Phone call
├── Azure Function
├── Logic App
├── Webhook
└── ITSM ticket

Configure Diagnostic Logging

Diagnostic Settings allow sending additional logs and metrics to storage destinations.

flowchart LR
    Res[Azure AI Resource] --> DS[Diagnostic Setting\nConfiguration]
    DS --> |Diagnostic logs| LA[Log Analytics\nWorkspace\nKQL Analytics]
    DS --> |Diagnostic logs| SA[Storage Account\nLong-term retention\nLow cost]
    DS --> |Diagnostic logs| EH[Event Hub\nSIEM integration\nPublish / Subscribe]
    DS --> |Metrics| LA
    DS --> |Metrics| SA

    style Res fill:#0078D4,color:#fff
    style DS fill:#F39C12,color:#fff
    style LA fill:#82E0AA
    style SA fill:#85C1E9
    style EH fill:#F4D03F

Configurable log types:

Demo — Enable Diagnostic Settings:

Azure Portal → Resource → Monitoring → Diagnostic Settings
→ Add diagnostic setting

1. Name the setting (e.g., "audit-to-log-analytics")
2. Select log categories:
   ☐ Audit Logs
   ☐ Request and Response Logs
   ☐ Trace Logs
   ☑ All Metrics

3. Select destinations (combination possible):
   ☑ Log Analytics Workspace → Select the workspace
   ☐ Storage Account         → Select the storage account
   ☐ Event Hub               → Select the namespace + policy

4. Save → Logs start flowing

Management: Modify or delete at any time

Destination comparison:

KQL query example in Log Analytics:

// Requests to the AI service in the last 24 hours
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| where TimeGenerated > ago(24h)
| summarize RequestCount = count() by ResultType, bin(TimeGenerated, 1h)
| order by TimeGenerated desc
| render timechart

// Detect errors (4xx / 5xx codes)
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| where ResultType startswith "4" or ResultType startswith "5"
| project TimeGenerated, ResourceType, ResultType, CallerIPAddress
| order by TimeGenerated desc

Managing Costs

Cost estimation (before deployment):

flowchart LR
    A[Estimate expected\nconsumption] --> B[Azure AI pricing page\nBy region and currency]
    A --> C[Azure pricing calculator\nConfigure and get a total]
    B --> D[Estimated monthly budget]
    C --> D
    D --> E[Monitor and adjust\nMetrics + Cost Analysis]

    style D fill:#F39C12,color:#fff
    style E fill:#82E0AA

GPT-4 cost example in the Azure calculator:

Azure OpenAI — GPT-4-32K (East US 2)
├── Prompt tokens       : 1,500,000 tokens → $ X
├── Completion tokens   :   500,000 tokens → $ Y
└── Estimated monthly total: $ X + Y

Cost analysis in Azure Cost Management:

Azure Portal → Subscription → Cost Management → Cost Analysis

Smart Views → Resources:
├── Azure Cognitive Search          : $71.10
│   └── Detail per service...
└── Cognitive Services (OpenAI)     : $X.XX
    ├── GPT-4 inference output tokens   : $ ...
    ├── GPT-4 prompt (input) tokens     : $ ...
    └── ADA embedding tokens            : $0.04

Available filters: by resource group, type, region, tag

Configure a budget with alerts:

Azure Portal → Subscription → Cost Management → Budgets → Add

Configuration:
├── Scope: Subscription (or Resource Group)
├── Filters: Type = CognitiveServices (or any AI service)
├── Amount: e.g. $50 / month (monthly reset)
└── Alert conditions:
    ├── Actual ≥ 80% of budget  → Notify team
    ├── Actual ≥ 100% of budget → Critical alert
    └── Forecast ≥ 110%         → Preventive warning
    → Action Group: Email + Logic App + ITSM ticket

Azure OpenAI quotas:

Azure OpenAI Studio → Quotas

Displays per model:
├── Allocated quota (tokens per minute)
├── Used quota
└── Link → Support ticket to increase quota

Azure Policy to control resource creation:

// Policy to prevent creation of unapproved Cognitive Services
{
  "policyRule": {
    "if": {
      "allOf": [
        {
          "field": "type",
          "equals": "Microsoft.CognitiveServices/accounts"
        },
        {
          "field": "Microsoft.CognitiveServices/accounts/kind",
          "notIn": ["CognitiveServices", "OpenAI"]
        }
      ]
    },
    "then": {
      "effect": "Deny"
    }
  }
}

Managing Account Keys

Each Azure AI service has two keys to allow rotation without interruption.

sequenceDiagram
    participant App as Application
    participant Key1 as Key 1 (active)
    participant Key2 as Key 2 (standby)
    participant Portal as Azure Portal

    Note over App,Key1: Phase 1 — Normal operation
    App->>Key1: Requests with Key 1

    Note over App,Portal: Phase 2 — Key 1 rotation
    Portal->>Key2: Regenerate Key 2
    App->>Key1: Still on Key 1 (no interruption)
    App->>Key2: Switch to Key 2
    Portal->>Key1: Regenerate Key 1 (now unused)

    Note over App,Key1: Phase 3 — Optional return to Key 1
    App->>Key1: Switch to Key 1 (if desired)

Best practice: Define key rotation frequency according to the organization’s security policies.

Portal actions:

Azure Portal → Resource → Keys and Endpoint

Display:
├── KEY 1 : ••••••••••••• [Copy] [Show]
├── KEY 2 : ••••••••••••• [Copy] [Show]
└── Endpoint: https://...

Buttons:
├── [Show Keys]       → Reveal both keys
├── [Regenerate Key1] → Regenerate key 1
└── [Regenerate Key2] → Regenerate key 2

Activity log records each regeneration: visible in Activity Log → list key / regenerate key.

Protecting Keys with Azure Key Vault

Storing keys directly in code or in unsecured environment variables is a bad practice. Azure Key Vault is the recommended solution.

graph LR
    App[Application] --> MI[Managed Identity\nNo secret to manage]
    MI --> KV[Azure Key Vault\nEncrypted secrets]
    KV --> Key[Azure AI API Key]
    App --> AI[Azure AI Service]
    Key -.->|Secure retrieval| App

    style KV fill:#E74C3C,color:#fff
    style MI fill:#8E44AD,color:#fff

Python example — Retrieve a key from Azure Key Vault:

from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient

# Using Managed Identity (no credentials in the code)
credential = DefaultAzureCredential()

key_vault_url = "https://my-keyvault.vault.azure.net/"
secret_client = SecretClient(vault_url=key_vault_url, credential=credential)

# Retrieve the Azure AI API key
ai_api_key = secret_client.get_secret("azure-ai-api-key").value
ai_endpoint = secret_client.get_secret("azure-ai-endpoint").value

# Use the retrieved key
from azure.cognitiveservices.vision.computervision import ComputerVisionClient
from msrest.authentication import CognitiveServicesCredentials

client = ComputerVisionClient(ai_endpoint, CognitiveServicesCredentials(ai_api_key))

C# example — Retrieve a key from Azure Key Vault:

using Azure.Identity;
using Azure.Security.KeyVault.Secrets;
using Azure.AI.Vision.ImageAnalysis;

// Managed Identity — no credentials in the code
var credential = new DefaultAzureCredential();
var secretClient = new SecretClient(
    new Uri("https://my-keyvault.vault.azure.net/"),
    credential
);

// Retrieve secrets
string apiKey = secretClient.GetSecret("azure-ai-api-key").Value.Value;
string endpoint = secretClient.GetSecret("azure-ai-endpoint").Value.Value;

// Use with the Azure AI service
var client = new ImageAnalysisClient(
    new Uri(endpoint),
    new AzureKeyCredential(apiKey)
);

Azure Key Vault advantages:

7. AI‑102 Exam Tips

mindmap
  root((AI-102 Success))
    Preparation
      Review the Microsoft study guide
      Complete Microsoft Learn modules
      Use the exam sandbox
      Practice free labs
      Watch study cram videos
    Key Knowledge
      Endpoint + key for each service
      Full URL for OpenAI with deployment name
      REST API and SDK - both
      Responsible AI principles
      Container deployment sequence
    Practice
      Try all service types
      Create ML pipelines in Designer
      Use Automated ML
      Deploy a model and test the endpoint
    Exam
      Take your time
      Eliminate obviously wrong answers
      Most intuitive answer if unsure
      No stress - retakes are possible

Essential Technical Information

Information required to use an AI service:

Single-Service / Multi-Service:
├── Endpoint  : https://{service}.cognitiveservices.azure.com/
└── API Key   : KEY 1 or KEY 2

Azure OpenAI:
├── Endpoint  : https://{name}.openai.azure.com/
├── API Key   : KEY 1 or KEY 2
└── Deployment Name: name of the deployed model
    └── Full URL: {endpoint}/openai/deployments/{deployment_name}/{action}

AI container deployment sequence:

1. Create the Azure service instance (for billing)
2. Retrieve the endpoint and API key
3. Validate container hardware prerequisites
4. docker pull mcr.microsoft.com/azure-cognitive-services/{service}
5. docker run ... Eula=accept Billing={endpoint} ApiKey={key}
6. Call the local container like a normal Azure service

Recap of Responsible AI Principles:

Typical questions and answers:

Q: What is the difference between AI and Machine Learning?
A: ML is a subset of Data Science focused on predictive models. AI is built on ML to emulate human capabilities. The terms are not interchangeable.

Q: Why use PTUs rather than standard consumption for Azure OpenAI?
A: PTUs offer stable and predictable latency, avoiding peaks when shared capacity is saturated. Potentially cheaper at high usage.

Q: Why is an Azure instance always required even for container deployment?
A: For billing (metering data sent from the container to Azure) and to control access to Microsoft’s intellectual property.

Q: Is Azure AI Search available in a container?
A: No. Neither Azure AI Search nor Azure OpenAI are available in containers.

Q: When to use ACI vs AKS to deploy an ML model?
A: ACI for testing / small deployments · AKS for large-scale production.

8. References

• Official Microsoft documentation — Azure ML model evaluation
• Metrics for classification models
• Metrics for clustering models
• AI-102 study guide — Microsoft
• Azure AI container images — Microsoft Container Registry
• Azure AI Services pricing
• Azure pricing calculator
• Infrastructure as Code Bicep — Azure AI documentation

Search Terms

ai-102 · azure · ai · fundamentals · planning · management · services · artificial · intelligence · generative · service · machine · exam · managing · classification · deployment · evaluation · instance · metrics · monitoring · pipeline · regression · types