Azure Generative AI Solutions

Target Certification: Azure AI Engineer Associate (AI-102)

Table of Contents

1. Course Overview
2. Generative Artificial Intelligence and LLMs
   • What is a Large Language Model?
   • Transformer Architecture and Tokens
   • Inference Process
   • Training LLMs
   • Use Cases
   • Retrieval Augmented Generation (RAG)
   • Types of Microsoft Generative AI Services
   • Responsible Generative AI
   • Content Filters
3. Using Azure OpenAI Service to Generate Content
   • Creating an Azure OpenAI Resource
   • Model Types and API Endpoints
   • Deploying a Model
   • Using the Playground
   • Code Generation with the Playground
   • Image Generation with DALL-E
   • Programmatic Authentication Options
   • Programmatic Interaction with Azure OpenAI
4. Optimizing Generative AI
   • Available Model Parameters
   • Prompt Engineering
   • Using Your Own Data
   • Integration with Azure AI Search
5. Conclusion and Exam Tips

1. Course Overview

This course covers generative AI — the ability of artificial intelligence to be creative, such as writing text, creating code, and generating images — in the context of the Azure AI Engineer Associate (AI-102) certification.

Prerequisites: Knowledge of Azure and AI fundamentals.

Main topics:

• What generative AI and Large Language Models are, and how they work
• How to use Azure-based generative AI resources
• Best practices for tuning and optimizing interactions with LLMs
• Key considerations for using Retrieval Augmented Generation

2. Generative Artificial Intelligence and LLMs

2.1 What is a Large Language Model?

Artificial intelligence (AI) is a vast field that includes machine learning. Here is the hierarchy of these concepts:

AI (Artificial Intelligence)
└── Machine Learning
    └── Deep Learning
        └── Generative AI
            └── Large Language Models (LLMs)

Examples of Generative AI:

• DALL-E: image generation from text
• GPT: text generation, code, translation
• Generation of songs, stories, code documentation

2.2 Transformer Architecture and Tokens

Most LLMs are built on the Transformer architecture, introduced in the paper “Attention Is All You Need”. This transformer:

• Modifies vectors representing each part of the input
• Improves relationships between different words to express their actual semantic meaning
• Resolves word ambiguity (one word = multiple meanings, multiple words = same meaning)

Tokens

LLMs do not work on words but on tokens:

A token can represent a word, part of a word, or a specific symbol.

Example: The sentence “the amazing things we can do with generative AI” = 10 tokens, 47 characters.

• Most words = 1 token each
• The word “generative” = 2 tokens

GPT-4 has a vocabulary of approximately 100,000 unique tokens (twice as many as GPT-3).

Token implications:

┌─────────────────────────────────────────────────────────┐
│                    TOKENS — IMPACT                       │
├─────────────────────────────────────────────────────────┤
│  INPUT token limit   → What we can send                 │
│  OUTPUT token limit  → What the model can generate      │
│  INPUT token cost    → We pay for what we send          │
│  OUTPUT token cost   → We pay for what we receive       │
│                                                          │
│  Ex: GPT-4 16K → supports 16,000 output tokens          │
└─────────────────────────────────────────────────────────┘

2.3 Inference Process

graph LR
    P["Input Prompt\n'There once was a brave\nknight named John that'"]
    P --> I1["Inference\n→ 'travel'"]
    I1 --> I2["Inference\n→ 'led'"]
    I2 --> I3["Inference\n→ 'to'"]
    I3 --> In["...next\ntoken..."]
    In --> End["End token\n= complete response"]

    style P fill:#2196F3,color:#fff
    style End fill:#4CAF50,color:#fff

Core principle: The LLM predicts the most probable next token from a probability distribution, then the next, and so on until the end token.

This is what you observe visually when using Microsoft Copilot — the text appears one token at a time.

Prompt structure:

┌─────────────────────────────────────────────┐
│              META-PROMPT                     │
├─────────────────────────────────────────────┤
│ SYSTEM PROMPT (role = "system")             │
│ Behavior instructions, tone, persona        │
├─────────────────────────────────────────────┤
│ RAG DATA (optional)                         │
│ Context retrieved from sources              │
├─────────────────────────────────────────────┤
│ MESSAGE HISTORY                             │
│ Previous conversation (user/assistant roles)│
├─────────────────────────────────────────────┤
│ USER MESSAGE (role = "user")                │
│ Current user request                        │
└─────────────────────────────────────────────┘

2.4 Training LLMs

Training data sources:

• Internet / Wikipedia
• Common Crawl
• Books1 and Books2
• WebText2 (Reddit pages with upvotes)
• Licensed text corpora

⚠️ Biases present in training data can appear in model outputs → hence the importance of responsible AI.

Training process: Back propagation adjusts all parameters (neuron weights and biases) from training data to optimize next-token prediction.

Major LLMs on the market:

2.5 Use Cases

mindmap
  root((LLMs))
    Agents and Chatbots
      Automated customer service
      Virtual assistants
    Natural Language
      Language translation
      Content localization
      Document summarization
      Sentiment analysis
    Code Generation
      Code creation
      Automatic comments
      Unit testing
      Technical documentation
    Anomaly Detection
      Fraud detection
      Pattern analysis
    Content Generation
      Stories and creative text
      Images with DALL-E
      Report generation

2.6 Retrieval Augmented Generation (RAG)

RAG solves the fundamental problem: “I only know what I was taught.”

Problem: An LLM is trained on data with a cutoff date. It does not know:

• Recent events (post-training)
• The organization’s private data
• Emails, meeting transcripts, etc.

If asked something it doesn’t know → it hallucinates (invents a response that appears factual).

sequenceDiagram
    participant U as User
    participant App as Application / Orchestrator
    participant VS as Vector Store (Azure AI Search)
    participant OAI as Azure OpenAI LLM

    U->>App: "Summarize this meeting"
    App->>VS: Semantic search (embedding vector)
    VS-->>App: Relevant document chunks
    Note over App: Meta-prompt construction
    Note over App: System prompt + User request + Retrieved RAG data
    App->>OAI: Enriched meta-prompt
    OAI-->>App: Generated response (inference)
    App-->>U: Final response with citations

Vectors and Embeddings

To find the most relevant data in natural language, embedding vectors are used:

graph LR
    Text["Input text"] --> Embed["Embedding Function\n(Neural model)"]
    Embed --> Vec["Multi-dimensional vector\n[0.23, -0.45, 0.67, ...\n1536 or 3072 dimensions]"]

    Query["User query"] --> EmbedQ["Embedding Function"]
    EmbedQ --> VecQ["Query vector"]

    VecQ --> NN["Nearest Neighbor Search\n(semantic proximity)"]
    Vec --> NN
    NN --> Result["Most relevant documents"]

    style Text fill:#2196F3,color:#fff
    style Result fill:#4CAF50,color:#fff

Hybrid Search: Azure AI Search combines semantic search (vectors) AND lexical search (exact keywords), then performs semantic reranking for the best results.

2.7 Types of Microsoft Generative AI Services

Microsoft offers different levels of services depending on the user profile:

graph TD
    subgraph "Users"
        U1["Standard user\n(no code)"]
        U2["Power user\n(low-code)"]
        U3["Developer\n(pro-code)"]
    end

    subgraph "Creation Tools"
        C1["Microsoft Copilot\n(built into applications)"]
        C2["Copilot Studio\n(no/low-code)"]
        C3["Azure AI Studio\n(full development)"]
    end

    subgraph "Orchestrators"
        O1["Built-in orchestrator\n(Copilot)"]
        O2["Configured orchestrator\n(Copilot Studio)"]
        O3["LangChain / Semantic Kernel\nPromptFlow"]
    end

    subgraph "Data Sources"
        D1["Microsoft Graph\n(emails, SharePoint, OneDrive)"]
        D2["Custom data\n(APIs, SharePoint)"]
        D3["Databases, Vector stores\nAzure AI Search, Internet"]
    end

    LLM["Large Language Model\n(OpenAI GPT-4o)"]

    U1 --> C1
    U2 --> C2
    U3 --> C3

    C1 --> O1
    C2 --> O2
    C3 --> O3

    O1 --> D1
    O2 --> D2
    O3 --> D3

    O1 --> LLM
    O2 --> LLM
    O3 --> LLM

    style LLM fill:#FF9800,color:#fff

Key principle: In none of these scenarios is the LLM itself modified. Everything relies on the content of the prompt and the data provided to it.

2.8 Responsible Generative AI

The 6 responsible AI principles:

1. Fairness
2. Reliability & Safety
3. Privacy & Security
4. Inclusiveness
5. Transparency
6. Accountability

With generative AI, these principles are even more critical because of:

• Biases inherited from training data
• Risks of hallucinations (the model invents facts convincingly)
• Very broad use across many applications

The 4 Planning Steps for a Responsible Solution

graph LR
    A["🔍 IDENTIFY"]
    B["📊 MEASURE"]
    C["🛡️ MITIGATE"]
    D["⚙️ OPERATE"]

    A --> B --> C --> D

    style A fill:#FF9800,color:#fff
    style B fill:#2196F3,color:#fff
    style C fill:#4CAF50,color:#fff
    style D fill:#9C27B0,color:#fff

Potential risks to identify:

• Incorrect information / hallucinations
• Offensive or discriminatory content
• Support for illegal activities
• Jailbreaking — manipulating the model to bypass its restrictions
• Indirect attacks — vulnerabilities via external documents processed by the model

Red teaming: Ethical attackers attempt to “break” the solution in a controlled environment before production deployment.

Ring-based deployment:

Inner Ring → Testing → Limited Business → Broader Business → Production

2.9 Content Filters

Content filters are applied by default to all models via Azure AI Content Safety.

Filtering categories:

Severity levels: Safe → Low → Medium → High

Additional options for generative AI:

Available actions: Annotate only or Annotate and block

To create a custom content filter in Azure AI Studio:

1. Go to Content filters → Create content filter
2. Select the connection (Azure OpenAI resource)
3. Configure thresholds for each category (Low / Medium / High)
4. Enable additional options if needed
5. Link the filter to a model deployment

The same process is available in Azure OpenAI Studio under Management → Content filters.

3. Using Azure OpenAI Service to Generate Content

3.1 Creating an Azure OpenAI Resource

OpenAI models must be deployed in a specific Azure OpenAI resource.

graph TD
    A["Choose an Azure region"] --> B["Verify model availability\nby region"]
    B --> C["Create the Azure OpenAI resource\n(portal, ARM, CLI, PowerShell)"]
    C --> D["Deploy a model in the resource"]
    D --> E["Obtain endpoint + API key / Entra ID"]
    E --> F["Interact via REST or SDK"]

    style A fill:#2196F3,color:#fff
    style F fill:#4CAF50,color:#fff

Region selection considerations:

• Available models vary by region (check the model availability table)
• Choose a region close to the application to reduce latency
• Pricing varies slightly by region

Creation methods:

• Azure Portal
• ARM / Bicep template
• Azure CLI
• PowerShell / REST API
• Azure AI Studio (when creating a hub)

Pricing

Cost is based on:

Retrieve Connection Information

# Retrieve the endpoint via Azure CLI
az cognitiveservices account show `
    --name <resource-name> `
    --resource-group <resource-group> `
    --query "properties.endpoint"

# Retrieve the API key
az cognitiveservices account keys list `
    --name <resource-name> `
    --resource-group <resource-group> `
    --query "key1"

The endpoint follows the format: https://<resource-name>.openai.azure.com/

This information is also visible in:

• Azure Portal → Keys and Endpoint under Resource Management
• Azure AI Studio → Resources and Keys
• Via connections in an Azure AI Studio hub

3.2 Model Types and API Endpoints

graph LR
    subgraph "Azure OpenAI Resource"
        A["/completions\n(Legacy)"]
        B["/chat/completions\n(Primary ✅)"]
        C["/embeddings"]
        D["/images/generations"]
    end

    A --> A1["Single-turn interaction\nBabbage-002, Davinci-002"]
    B --> B1["Multi-turn conversation\nGPT-3.5, GPT-4, GPT-4o"]
    C --> C1["Semantic vectors\nAda-002, text-embedding-3"]
    D --> D1["Image generation\nDALL-E-3"]

    style B fill:#4CAF50,color:#fff
    style B1 fill:#4CAF50,color:#fff

⚠️ GPT-3.5 and above no longer use /completions — they exclusively require /chat/completions.

3.3 Deploying a Model

Via Azure OpenAI Studio

1. Select the Azure OpenAI resource
2. Go to Management → Deployments
3. Click Create new deployment
4. Choose the model (e.g., gpt-4o)
5. Select the version and Deployment type:
   • Standard — Default
   • Global-Standard — Global deployment
   • Provisioned-Managed — Dedicated capacity
6. Give a deployment name (e.g., gpt-4o-prod)
7. Configure advanced options (content filter, Tokens Per Minute Rate)

Via Azure AI Studio

1. Open the hub → Deployments → Create a deployment
2. Choose the model AND the target Azure OpenAI resource (from the hub’s connections)
3. Configure the same parameters as via Azure OpenAI Studio

Via Azure CLI

# List existing deployments
az cognitiveservices account deployment list \
    --name <resource-name> \
    --resource-group <resource-group> \
    --output table

# Create a new deployment
az cognitiveservices account deployment create \
    --name <resource-name> \
    --resource-group <resource-group> \
    --deployment-name my-gpt4o \
    --model-name gpt-4o \
    --model-version "2024-05-13" \
    --sku-capacity 1 \
    --sku-name Standard \
    --model-format OpenAI

# Delete a deployment
az cognitiveservices account deployment delete \
    --name <resource-name> \
    --resource-group <resource-group> \
    --deployment-name my-gpt4o

3.4 Using the Playground

The playground is an intuitive web interface for testing and prototyping with deployed models.

graph TD
    PG["Playground"]
    PG --> INF["Inference\n(direct interaction with the model)"]
    PG --> PARAM["Parameter adjustment\n(temperature, max tokens, penalties)"]
    PG --> DATA["Data integration\n(Azure AI Search, etc.)"]
    PG --> CODE["Code examples\n(Python, C#, REST)"]
    PG --> TMPL["Prompt templates\n(pre-defined system messages)"]

Two access options:

Types of prompts testable in the playground:

# Question / Answer
"What is the tallest mountain in America?"

# Translation
"Can you translate 'I would like a large cheese pizza' into Italian?"

# Creative writing
"Create a 200 word story about a cloud engineer called Sarah and a magic keyboard."

# Sentiment analysis (zero-shot)
"Tweet: 'This was an amazing course, best ever'
Sentiment:"

# Summary (RAG example)
"Summarize the following into five key points as a bullet list:
[meeting transcription]"

Advanced playground features:

• Prompt samples — Pre-defined templates (Shakespearean assistant, Xbox support agent…)
• Safety system messages — Add guardrails against harmful content, hallucinations, jailbreaks
• Show code — Automatically generates integration code in the chosen language

3.5 Code Generation with the Playground

GPT-3.5+ LLMs natively handle code tasks without a separate model (Codex is gone).

Code capabilities:

Example — Generating a Python function:

# Prompt: "Generate a Python function that calculates the factorial of a number"

def factorial(n):
    """Calculates the factorial of a positive integer."""
    if n == 0 or n == 1:
        return 1
    else:
        return n * factorial(n - 1)

# Iterative version (the LLM often provides both variants)
def factorial_iterative(n):
    result = 1
    for i in range(2, n + 1):
        result *= i
    return result

# Example usage
print(factorial(5))            # Output: 120
print(factorial_iterative(5))  # Output: 120

Example — Adding comments (prompt: “Add comments to the following Python code”):

def factorial(n):
    # Base case: the factorial of 0 or 1 is 1
    if n == 0 or n == 1:
        return 1
    else:
        # Recursive call: n * factorial(n-1)
        return n * factorial(n - 1)

Example — Python → C# conversion (prompt: “Convert it to C#”):

/// <summary>
/// Calculates the factorial of a positive integer.
/// </summary>
public int Factorial(int n)
{
    // Base case
    if (n == 0 || n == 1)
        return 1;
    // Recursive call
    return n * Factorial(n - 1);
}

Example — Generated unit tests:

import unittest

def add(a, b):
    return a + b

class TestAddFunction(unittest.TestCase):
    def test_add_positive_numbers(self):
        self.assertEqual(add(2, 3), 5)

    def test_add_negative_numbers(self):
        self.assertEqual(add(-1, -1), -2)

    def test_add_zero(self):
        self.assertEqual(add(0, 5), 5)

    def test_add_floats(self):
        self.assertAlmostEqual(add(1.5, 2.5), 4.0)

if __name__ == '__main__':
    unittest.main()

💡 If you use GitHub Copilot, it is already natively configured for coding tasks and integrates directly into your IDE.

3.6 Image Generation with DALL-E

DALL-E-3 generates images from text. The internal process:

graph LR
    P["Text prompt\n'A perfect pizza in 3D digital art'"]
    P --> N["Random pixel canvas\n(initial noise)"]
    N --> AI["DALL-E-3 Model\nIterative pixel adjustment\nbased on trained knowledge"]
    AI --> IMG["Generated image"]

    style P fill:#2196F3,color:#fff
    style IMG fill:#4CAF50,color:#fff

The model:

1. Starts with a random pixel canvas (noise)
2. Progressively nudges pixels according to its knowledge of the subject, style, shadows
3. Produces a different image with each generation

Capabilities:

• Photorealism, cartoons, pixel art, retro art, 3D art, different eras
• Iterative modifications by adjusting the prompt

Available parameters:

⚠️ You pay per generated image. HD resolution costs more. Use Standard for testing.

DALL-E prompt examples:

# Ambiguous → unpredictable result
"Create a picture of the most perfect pizza in the world in 3D digital art"

# More precise → better result
"Create a picture of the most perfect cheese pizza, made of LEGO, in 3D digital art"

# Very descriptive → optimal result
"A 3D render of a cute orange monster on a dark blue background, digital art style"

# Anti-ambiguity tip
"A bald Lego man with no facial hair, standing in front of a large digital whiteboard
drawing a cloud, 3D digital art"

Via the Azure AI Studio playground:

1. Project playground → Images
2. Select the DALL-E-3 deployment
3. Enter the prompt → Generate
4. Use View code for integration code
5. Show code supports Python, C#, JavaScript, etc.

3.7 Programmatic Authentication Options

To integrate models into an application, two authentication options are available:

graph TD
    App["Application"] --> Auth{"Authentication\nmethod"}

    Auth --> Key["API Key"]
    Auth --> Entra["Entra ID\n(Recommended ✅)"]

    Key --> KP["• Simple to use\n• Store in Azure Key Vault\n• Never hardcode\n• Regenerate if compromised"]

    Entra --> EP["• More granular (RBAC)\n• No secret to store\n• Compatible with Managed Identity\n• DefaultAzureCredential"]

    Entra --> Roles{"RBAC Roles"}
    Roles --> R1["Cognitive Services OpenAI User\n(inference only) ✅"]
    Roles --> R2["Cognitive Services OpenAI Contributor\n(full access)"]

    style Entra fill:#4CAF50,color:#fff
    style R1 fill:#2196F3,color:#fff

Credential sources supported by DefaultAzureCredential:

• Managed Identity (Azure VMs, App Service, AKS…)
• Service Principal (via environment variables)
• Authenticated Azure CLI (az login)
• Authenticated Azure PowerShell (Connect-AzAccount)
• Visual Studio / VS Code credentials

3.8 Programmatic Interaction with Azure OpenAI

REST API via PowerShell (with Entra ID)

# ── Configuration ──────────────────────────────────────────────────
$endpoint       = $env:AZURE_OPENAI_ENDPOINT      # ex: https://myresource.openai.azure.com/
$deploymentName = $env:DEPLOYMENT_NAME             # ex: gpt-4o-prod
$apiVersion     = "2024-02-01"

# Build the full URL
$ai_url = "$endpoint/openai/deployments/$deploymentName/chat/completions?api-version=$apiVersion"

# ── Entra ID Authentication ─────────────────────────────────────────
# Prerequisite: Connect-AzAccount or az login
$token = (Get-AzAccessToken -ResourceUrl "https://cognitiveservices.azure.com").Token

$headers = @{
    "Authorization" = "Bearer $token"
    "Content-Type"  = "application/json"
}

# ── Build the request body ──────────────────────────────────────────
$messages = @(
    @{ role = "system"; content = "You are a helpful assistant." },
    @{ role = "user";   content = "What is the tallest mountain in the United States?" }
)

$body = @{
    messages    = $messages
    temperature = 0.3
    max_tokens  = 200
} | ConvertTo-Json -Depth 5

# ── REST call ───────────────────────────────────────────────────────
$response = Invoke-RestMethod -Uri $ai_url -Method Post -Headers $headers -Body $body

# Display full response as JSON (debug)
$response | ConvertTo-Json -Depth 5

# Display only the message content
$response.choices[0].message.content

REST response structure:

{
  "choices": [
    {
      "content_filter_results": {
        "hate":      { "filtered": false, "severity": "safe" },
        "violence":  { "filtered": false, "severity": "safe" }
      },
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "role":    "assistant",
        "content": "The tallest mountain in the United States is Denali (also known as Mount McKinley)..."
      }
    }
  ],
  "model": "gpt-4o",
  "usage": {
    "prompt_tokens":     25,
    "completion_tokens": 45,
    "total_tokens":      70
  }
}

Python SDK — API Key Authentication

import os
from openai import AzureOpenAI

# Retrieve values from environment variables
# NEVER put the key directly in the code
api_key         = os.environ.get("AZURE_OPENAI_API_KEY")
endpoint        = os.environ.get("AZURE_OPENAI_ENDPOINT")
deployment_name = os.environ.get("DEPLOYMENT_NAME")

# Create the AzureOpenAI client
client = AzureOpenAI(
    azure_endpoint=endpoint,
    api_key=api_key,
    api_version="2024-02-01"
)

# Build the messages (system + user)
messages = [
    {
        "role":    "system",
        "content": "You are a helpful assistant."
    },
    {
        "role":    "user",
        "content": "What is the tallest mountain in the United States?"
    }
]

# Call the /chat/completions endpoint
completion = client.chat.completions.create(
    model=deployment_name,
    messages=messages,
    max_tokens=200,
    temperature=0.3
)

# Display the response
print(completion.choices[0].message.content)

Python SDK — Entra ID Authentication (Recommended for production)

import os
from openai import AzureOpenAI
from azure.identity import DefaultAzureCredential, get_bearer_token_provider

# Configuration
endpoint        = os.environ.get("AZURE_OPENAI_ENDPOINT")
deployment_name = os.environ.get("DEPLOYMENT_NAME")

# Entra ID authentication — no key needed!
# DefaultAzureCredential automatically tries:
#   1. Managed Identity  2. Service Principal  3. Azure CLI  4. Azure PowerShell
credential     = DefaultAzureCredential()
token_provider = get_bearer_token_provider(
    credential,
    "https://cognitiveservices.azure.com/.default"
)

# Create the client with azure_ad_token_provider (replaces api_key)
client = AzureOpenAI(
    azure_endpoint=endpoint,
    azure_ad_token_provider=token_provider,
    api_version="2024-02-01"
)

# Messages — identical to the API Key case
messages = [
    {
        "role":    "system",
        "content": "You are a helpful assistant."
    },
    {
        "role":    "user",
        "content": "What is the tallest mountain in the United States?"
    }
]

# Identical call — only auth changes
completion = client.chat.completions.create(
    model=deployment_name,
    messages=messages,
    max_tokens=200,
    temperature=0.3
)

# Display the response, usage, and raw response
print(completion.choices[0].message.content)
print(f"Tokens used: {completion.usage}")
print(completion)  # Full raw response

REST API for Embeddings (PowerShell)

# Configuration — embeddings endpoint
$endpoint            = $env:AZURE_OPENAI_ENDPOINT
$embeddingDeployment = $env:EMBEDDING_DEPLOYMENT_NAME  # ex: text-embedding-3-large
$apiVersion          = "2024-02-01"

$embed_url = "$endpoint/openai/deployments/$embeddingDeployment/embeddings?api-version=$apiVersion"

# API key authentication (example)
$apiKey  = $env:AZURE_OPENAI_API_KEY
$headers = @{
    "api-key"      = $apiKey
    "Content-Type" = "application/json"
}

# Text to transform into a semantic vector
$body = @{
    input = "Azure OpenAI provides REST API access to OpenAI's powerful language models"
} | ConvertTo-Json

# Call
$response = Invoke-RestMethod -Uri $embed_url -Method Post -Headers $headers -Body $body

# The response contains the vector (3072 dimensions for text-embedding-3-large)
$response.data[0].embedding.Count   # Displays: 3072
$response.data[0].embedding         # The float array

4. Optimizing Generative AI

4.1 Available Model Parameters

Why parameters?

graph TD
    subgraph "Primary Parameters — /chat/completions"
        T["temperature\n0.0 → 1.0"]
        TP["top_p\n0.0 → 1.0"]
        FP["frequency_penalty\n-2.0 → +2.0"]
        PP["presence_penalty\n-2.0 → +2.0"]
        MT["max_tokens\nPositive integer"]
    end

    T --> Det["Low (0–0.3)\n→ Deterministic, factual"]
    T --> Bal["Medium (0.4–0.6)\n→ Balanced"]
    T --> Crea["High (0.7–1.0)\n→ Creative, varied"]

    TP --> Tresh["Probability threshold\n→ Filters low-probability tokens"]

    FP --> Freq["Scaling penalty\non each repetition\n→ Avoids repeated words"]
    PP --> Pres["Immediate constant penalty\nfrom first use\n→ Encourages diversity"]

    MT --> MaxT["Limits OUTPUT only\n→ Controls length and cost"]

    style T fill:#FF9800,color:#fff
    style TP fill:#9C27B0,color:#fff
    style FP fill:#F44336,color:#fff
    style PP fill:#2196F3,color:#fff
    style MT fill:#4CAF50,color:#fff

Quick parameter reference:

⚠️ Never combine temperature AND top_p — the effects cancel out and the result becomes unpredictable. The Azure AI Studio playground only shows temperature to avoid this confusion.

Using parameters in Python code:

completion = client.chat.completions.create(
    model=deployment_name,
    messages=messages,

    # Creativity: low=deterministic, high=creative
    temperature=0.7,

    # Limit output token count (not input!)
    max_tokens=800,

    # Scaling penalty for already-used words
    frequency_penalty=0.5,

    # Immediate penalty for any already-used word
    presence_penalty=0.3,

    # DO NOT use with temperature!
    # top_p=0.9,
)

Parameters available in Azure AI Studio playground:

• Max response (max_tokens) — visible by default
• Temperature — visible by default
• Frequency penalty — visible by default
• Presence penalty — visible by default
• Number of messages in history (contextual memory)

4.2 Prompt Engineering

Key principle: The quality of the output is proportional to the quality of the input. Garbage in → Garbage out. Champagne in → Champagne out.

Prompt engineering encompasses the strategies for formulating prompts to get the best results from inference.

mindmap
  root((Prompt Engineering))
    1. Be Specific and Clear
      Avoid ambiguity and jargon
      Specify exact output format
      Define assistant persona
      Indicate target audience
      Repeat instructions after data
    2. Provide Examples
      Zero-shot - no examples
      One-shot - 1 example
      Few-shot - 2 to 3 examples
      Prime the response
    3. Steps and Chain of Thought
      Provide explicit steps
      Break into sub-tasks
      Ask to explain reasoning
      Avoid very complex requests
    4. External Data and Tools
      RAG - additional data
      Citations for reliability
      Function calling
      Code execution

Important: The majority of behavior-based prompt engineering is done in the system prompt (role system), not in user messages.

Strategy 1 — Be Specific and Clear

# ❌ Bad prompt (ambiguous)
"Tell me about Paris"

# ✅ Good system prompt (precise)
"You are an expert tourist guide for Paris. You respond in English,
in a professional but accessible tone. You provide practical information
including hours, prices, and public transport. You always structure
your response with titles and bullet lists. If you are not certain
about something, say so explicitly."

System prompt templates in Azure studios:

# Azure OpenAI Studio — Available templates:
- Xbox customer support agent
- Shakespearean writing assistant
- JSON output formatter
- Safety system messages (guardrails)

# Safety system messages cover:
- Harmful content
- Ungrounded content (hallucinations)
- Copyright
- Jailbreaks and manipulation

Strategy 2 — Provide Examples (Few-shot Learning)

# Few-shot example for sentiment analysis
messages = [
    {
        "role":    "system",
        "content": "Analyze the sentiment of tweets. Respond ONLY with: Positive, Negative, or Neutral."
    },
    # Example 1 (one-shot)
    {
        "role":    "user",
        "content": "Tweet: 'I love this new product, it's absolutely fantastic!'\nSentiment:"
    },
    {
        "role":    "assistant",
        "content": "Positive"
    },
    # Example 2 (few-shot)
    {
        "role":    "user",
        "content": "Tweet: 'This customer service is absolutely terrible'\nSentiment:"
    },
    {
        "role":    "assistant",
        "content": "Negative"
    },
    # Real request (zero-shot → few-shot thanks to preceding examples)
    {
        "role":    "user",
        "content": "Tweet: 'This was an amazing course, best ever'\nSentiment:"
    }
]
# Expected result: "Positive"

Types of examples:

Strategy 3 — Steps and Chain of Thought

# Chain of Thought example — road trip planning
messages = [
    {
        "role":    "system",
        "content": "You are a travel planning assistant."
    },
    {
        "role":    "user",
        "content": """I'm planning a road trip from Seattle to San Francisco.
I want to make 3 stops: Portland, Sacramento, and San Jose.
Considering I want to minimize driving time,
can you determine the optimal order of stops
and EXPLAIN YOUR REASONING step by step?"""
        # "explain your reasoning" → triggers chain of thought
    }
]

Decomposing complex tasks:

# System prompt with explicit steps
"Perform the following steps in order:
1. Summarize the provided text in 3 key points (max 50 words each)
2. Translate the summary into French
3. Generate 5 relevant hashtags for social media
4. Format the final response as JSON with the keys:
   'summary_en', 'summary_fr', 'hashtags'"

Strategy 4 — External Data and Function Calling

import json

# Define available functions for the LLM
tools = [
    {
        "type": "function",
        "function": {
            "name":        "get_weather",
            "description": "Gets the current weather for a given city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type":        "string",
                        "description": "The name of the city"
                    },
                    "unit": {
                        "type":        "string",
                        "enum":        ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["city"]
            }
        }
    }
]

# Call with available tools
response = client.chat.completions.create(
    model=deployment_name,
    messages=[
        {"role": "user", "content": "What's the weather like in London today?"}
    ],
    tools=tools,
    tool_choice="auto"
)

# The LLM may return a "plan" requesting function execution
if response.choices[0].finish_reason == "tool_calls":
    tool_call     = response.choices[0].message.tool_calls[0]
    function_name = tool_call.function.name
    function_args = json.loads(tool_call.function.arguments)

    # We execute the real function (our code)
    weather_result = get_weather(**function_args)

    # We return the result to the LLM for the final response
    messages = [
        {"role": "user",      "content": "What's the weather like in London today?"},
        response.choices[0].message,
        {
            "role":         "tool",
            "tool_call_id": tool_call.id,
            "content":      json.dumps(weather_result)
        }
    ]

    final_response = client.chat.completions.create(
        model=deployment_name,
        messages=messages
    )
    print(final_response.choices[0].message.content)

💡 Cost tip: Consider the size of your prompt — you pay for each input token! There are prompt compression tools available to optimize costs.

4.3 Using Your Own Data

Why additional data?

• The model has a cutoff date — recent data unavailable
• Private data not included in training (emails, internal documents, transcripts)
• Increase accuracy on a specific domain
• Reduce hallucinations by grounding responses in verified data
• Control the information source (no pre-training bias)

Key RAG considerations:

graph TD
    subgraph "3 main considerations"
        T["1️⃣  TOKENS"]
        R["2️⃣  RELEVANCE"]
        S["3️⃣  SECURITY"]
    end

    T --> T1["• Max tokens supported by the model\n• Input token cost\n• Flood risk with unlimited input\n• Optimize chunk size"]
    R --> R1["• Vector semantic search\n• Hybrid search (vectors + lexical)\n• Pre-filters on metadata\n• Chunking quality and overlap"]
    S --> S1["• Model can only access data sent to it\n• Semantic indexes can find unexpected data\n• Verify permissions on ALL data\n• Use Microsoft Purview for classification"]

    style T fill:#FF9800,color:#fff
    style R fill:#2196F3,color:#fff
    style S fill:#F44336,color:#fff

Databases with native vector capabilities:

4.4 Integration with Azure AI Search

Azure AI Search (formerly Azure Cognitive Search) is the recommended service for scenarios with:

• Multiple data sources
• Mixed structured and unstructured data
• Need for hybrid search (semantic + lexical)

graph TD
    subgraph "Data Sources"
        BS["Azure Blob Storage"]
        CDB["Azure Cosmos DB"]
        SQL["Azure SQL Database"]
        DL["Azure Data Lake Gen2"]
        EXT["Partner sources\n(Amazon S3, Salesforce, etc.)"]
    end

    subgraph "Azure AI Search"
        IDX["Indexer\n(Data extraction)"]
        SK["Skillset\n(Chunking + Embedding)"]
        AIDX["Index\n(Vector + text storage)"]

        subgraph "Search Types"
            FS["Full-text Search\n(Lexical — exact keywords)"]
            VS["Vector Search\n(Semantic — vector proximity)"]
            HS["Hybrid Search\n+ Semantic Reranking ✅"]
        end
    end

    EMB["Azure OpenAI\nEmbedding Model\n(ada-002 or text-embedding-3)"]

    BS & CDB & SQL & DL & EXT --> IDX
    IDX --> SK
    SK --> |"Chunking (2000 chars)"| AIDX
    SK --> |"Vectorization"| EMB
    EMB --> AIDX
    AIDX --> FS & VS
    FS & VS --> HS

    style HS fill:#4CAF50,color:#fff
    style EMB fill:#FF9800,color:#fff

Chunking configuration in the skillset:

Why chunking? A single large vector for an entire document loses its semantic meaning. Dividing into small chunks with overlap provides better search accuracy.

Azure AI Search SKUs:

The Semantic Reranker (Standard+ only) combines full-text and vector results, then re-ranks them by actual relevance — essential for production quality.

RAG Integration with Azure AI Search in Python

import os
from openai import AzureOpenAI
from azure.identity import DefaultAzureCredential, get_bearer_token_provider

# ── Configuration ───────────────────────────────────────────────────
endpoint        = os.environ.get("AZURE_OPENAI_ENDPOINT")
deployment_name = os.environ.get("DEPLOYMENT_NAME")
search_endpoint = os.environ.get("AZURE_SEARCH_ENDPOINT")
search_key      = os.environ.get("AZURE_SEARCH_KEY")
search_index    = os.environ.get("AZURE_SEARCH_INDEX")

# ── Entra ID Authentication ─────────────────────────────────────────
credential     = DefaultAzureCredential()
token_provider = get_bearer_token_provider(
    credential,
    "https://cognitiveservices.azure.com/.default"
)

client = AzureOpenAI(
    azure_endpoint=endpoint,
    azure_ad_token_provider=token_provider,
    api_version="2024-02-01"
)

# ── Messages ────────────────────────────────────────────────────────
messages = [
    {
        "role":    "system",
        "content": "You are an assistant that answers ONLY based on the provided data. "
                   "If the information is not in the data, say so explicitly."
    },
    {
        "role":    "user",
        "content": "What is the advantage of private endpoints over service endpoints?"
    }
]

# ── Call with Azure AI Search as RAG source ─────────────────────────
completion = client.chat.completions.create(
    model=deployment_name,
    messages=messages,
    max_tokens=800,
    temperature=0.3,
    extra_body={
        "data_sources": [
            {
                "type": "azure_search",
                "parameters": {
                    "endpoint":   search_endpoint,
                    "index_name": search_index,
                    "authentication": {
                        "type": "api_key",
                        "key":  search_key
                    },
                    # Result quality threshold (1=loose, 5=strict)
                    "strictness":      3,
                    # Max RAG documents included in the prompt
                    "top_n_documents": 5
                }
            }
        ]
    }
)

# ── Display the response ────────────────────────────────────────────
print(completion.choices[0].message.content)

# Access source document citations
if hasattr(completion.choices[0].message, 'context'):
    citations = completion.choices[0].message.context.get('citations', [])
    for i, citation in enumerate(citations, 1):
        print(f"\nSource [{i}]: {citation.get('title', 'N/A')}")
        print(f"  File: {citation.get('filepath', 'N/A')}")

Configuration via the Playground (without code):

1. In Azure AI Studio → Playground for the chosen model
2. Click Add your data → select the Azure AI Search index
3. Configure:
   • Limit responses to your data only — prevents using pre-trained knowledge (recommended for pure RAG scenarios)
   • Strictness (1–5) — minimum quality required from search results
   • Number of documents — number of chunks included in the prompt
4. Test in the chat — the playground queries Azure AI Search automatically
5. Citations appear in the response with references to source documents

5. Conclusion and Exam Tips

Visual Summary of Modules

The course is structured in 3 main modules, summarized here visually:

╔══════════════════════════════════════════════════════════════════════════════╗
║                    MODULE 1 — GENERATIVE AI AND LLMs                        ║
╠═══════════════════════════╦════════════════════════════════════════════════╗ ║
║  Generative AI enables    ║  There are many LLMs with a                   ║ ║
║  creativity               ║  large number of use cases                    ║ ║
╠═══════════════════════════╬════════════════════════════════════════════════╣ ║
║  RAG is essential         ║  Responsible generative AI                    ║ ║
║  to add data              ║  is critical                                   ║ ║
╚═══════════════════════════╩════════════════════════════════════════════════╝ ║
╚═══════════════════════════════════════════════════════════════════════════════╝

╔══════════════════════════════════════════════════════════════════════════════╗
║           MODULE 2 — GENERATING CONTENT WITH AZURE OPENAI                   ║
╠══════════════════════════════════════════════════════════════════════════════╣
║  • Models are deployed in an Azure OpenAI resource                          ║
║  • The resource provides the endpoint, RBAC, and key                        ║
║  • Playgrounds allow experimenting with parameters and prompts              ║
║  • Authentication is possible via Entra ID or by key                        ║
║  • REST and SDKs can be used for LLM interactions                           ║
╚══════════════════════════════════════════════════════════════════════════════╝

╔══════════════════════════════════════════════════════════════════════════════╗
║                    MODULE 3 — OPTIMIZING GENERATIVE AI                      ║
╠═══════════════════╦════════════════════╦═══════════════╦════════════════════╣
║  Parameters       ║  Prompt            ║     RAG       ║  Azure AI Search  ║
║  and history      ║  Engineering       ║               ║                   ║
╚═══════════════════╩════════════════════╩═══════════════╩════════════════════╝

Top 5 Exam Tips

┌─────────────────────────────────────────────────────────────────────────┐
│                        TOP TIPS — AI-102 EXAM                            │
├───────────────────────────┬─────────────────────────┬───────────────────┤
│  Hands-on with            │  Understand the         │  List the key     │
│  everything!              │  importance of          │  points of        │
│                           │  token limits           │  responsible AI   │
├───────────────────────────┴─────────────────────────┴───────────────────┤
│         Practice calling               │  Know how to explain RAG        │
│         from your applications         │  and its benefits               │
└────────────────────────────────────────┴─────────────────────────────────┘

Course Summary

graph TD
    subgraph "Module 1 — Fundamentals"
        LLM["LLMs and Generative AI\nTransformer · Tokens · Inference"]
        RAG_F["RAG\nRetrieval Augmented Generation"]
        RESP["Responsible AI\nIdentify→Measure→Mitigate→Operate"]
        CF["Content Filters\nCategories + Customization"]
    end

    subgraph "Module 2 — Azure OpenAI Service"
        RES["Azure OpenAI Resource\nRegion + Endpoint + Key / Entra ID"]
        DEPLOY["Model Deployment\nPortal / CLI / Azure AI Studio"]
        PG["Playground\nTesting, prototyping, code generation"]
        AUTH["Authentication\nAPI Key vs Entra ID (preferred)"]
        PROG["Programmatic Integration\nREST + Python SDK"]
    end

    subgraph "Module 3 — Optimization"
        PARAM["Parameters\nTemperature · top_p · penalties · max_tokens"]
        PE["Prompt Engineering\n4 key strategies"]
        OWN["Custom Data\nRAG: Tokens · Relevance · Security"]
        AIS["Azure AI Search\nHybrid Search + Semantic Reranking"]
    end

    LLM --> RES
    RAG_F --> AIS
    RESP --> CF
    RES --> DEPLOY
    DEPLOY --> PG
    PG --> AUTH
    AUTH --> PROG
    PROG --> PARAM
    PARAM --> PE
    PE --> OWN
    OWN --> AIS

Key Points to Remember

Fundamentals:

• An LLM predicts the most probable next token, one at a time — it doesn’t actually “understand”
• The quality of the prompt is directly proportional to the quality of the output
• Tokens have limits (input and output) — and cost money for both
• Models have a knowledge cutoff date → use RAG to supplement

Azure OpenAI:

• The resource is created in a specific region (model availability varies by region)
• The endpoint + deployment name + API version form the complete URL
• Entra ID is the preferred authentication (RBAC + Managed Identity, no secret)
• The primary endpoint is /chat/completions for GPT-3.5+

Optimization:

• Never combine temperature AND top_p
• The system prompt is the main lever of prompt engineering (behavior, tone, format)
• RAG: priorities → Tokens → Relevance → Security
• Azure AI Search: chunking + overlap + hybrid search + semantic reranking = optimal quality

Exam Tips (AI-102)

Points not to confuse:

temperature  ≠  top_p              (do not combine)
frequency_penalty ≠ presence_penalty  (scaling vs immediate)
/completions  ≠  /chat/completions  (legacy vs current)
API Key  ≠  Entra ID               (simple vs recommended)
LLM  ≠  embedding model            (generation vs vectorization)

Certification target: Azure AI Engineer Associate (AI-102)

Search Terms

ai-102 · azure · generative · ai · services · artificial · intelligence · api · model · openai · strategy · via · authentication · exam · generation · python · content · data · embeddings · entra · integration · llms · playground · powershell