Azure AI Fundamentals – Complete Overview

Level: Beginner
Objective: Master the Azure AI services ecosystem for the AI-900 certification and to build intelligent applications

Table of Contents

1. The Azure AI Ecosystem – Overview
2. Azure AI Services – Complete Catalog
3. Azure AI Search – Intelligent Search
4. Azure AI Foundry – Generative AI Platform
5. Microsoft Copilot for Azure
6. Practical Implementation
7. Architecture of a Complete AI Solution
8. Azure AI Governance and Security
9. Summary and Key Points
10. Glossary

1. The Azure AI Ecosystem – Overview

1.1 The 3 AI Categories on Azure

Microsoft has organized its AI offerings into three major categories that address different needs:

flowchart TD
    AZURE_AI["☁️ Azure AI\nComplete Ecosystem"] --> AIS["🔷 Azure AI Services\n(AI as a Service)\n\nReady-to-use APIs\nNo ML expertise required\nPay-per-use billing"]
    AZURE_AI --> AML["🔬 Azure Machine Learning\n\nFull ML platform\nTraining + Deployment\nProfessional MLOps"]
    AZURE_AI --> COPILOTS["🤖 Copilots / AI Assistants\n\nGenerative AI assistants\nIntegrated in M365\nMicrosoft Copilot for Azure"]
    
    AIS --> AIS_EX["• Azure OpenAI\n• Document Intelligence\n• AI Vision\n• AI Speech\n• AI Language\n• AI Search\n• Content Safety"]
    
    AML --> AML_EX["• Azure ML Studio\n• AutoML\n• Designer (No-code)\n• MLflow\n• Pipelines"]
    
    COPILOTS --> COP_EX["• Copilot for Azure\n• Copilot in Word/Excel\n• GitHub Copilot\n• Copilot Studio"]

1.2 Comparison of the 3 Categories

1.3 How to Choose the Right Category?

flowchart TD
    Q1{"Need to integrate\nAI into an app?"}
    Q1 -->|Yes| Q2{"Do you have\nproprietary data\nto train on?"}
    Q2 -->|No| AIS2["✅ Azure AI Services\n(Pre-trained APIs)"]
    Q2 -->|Yes| Q3{"ML expertise\navailable?"}
    Q3 -->|No| AIS3["✅ Azure AI Services\n+ Custom Vision/CLU"]
    Q3 -->|Yes| AML2["✅ Azure Machine Learning\n(custom models)"]
    Q1 -->|No, need\nproductivity| COPILOTS2["✅ Microsoft Copilots\n(assistants in M365/Azure)"]

2. Azure AI Services – Complete Catalog

2.1 Services Overview

mindmap
  root((Azure AI Services))
    Vision
      Azure AI Vision
        Image Analysis
        OCR Read Engine
        Smart Crop
        Spatial Analysis
      Azure AI Custom Vision
        Image Classification
        Object Detection
      Azure AI Face
        Facial Detection
        Facial Analysis
        Verification
    Language
      Azure AI Language
        Sentiment Analysis
        Key Phrases
        NER
        CLU
        Question Answering
      Azure AI Speech
        Speech-to-Text
        Text-to-Speech
        Speech Translation
      Azure AI Translator
        90+ languages
        Documents
    Documents
      Azure AI Document Intelligence
        Prebuilt Models
        Custom Models
        Layout Analysis
    Search
      Azure AI Search
        Full-text Search
        Vector Search
        Hybrid Search
        Knowledge Mining
    Generative AI
      Azure OpenAI Service
        GPT-4 GPT-4o
        DALL-E 3
        Embeddings
        Whisper
    Safety
      Azure AI Content Safety
        Text Filtering
        Image Filtering
        Groundedness Detection

2.2 Complete Reference Table

2.3 How to Access Azure AI Services

Option 1: Standalone service (dedicated endpoint + key per service)

from azure.ai.vision.imageanalysis import ImageAnalysisClient
from azure.core.credentials import AzureKeyCredential

# Endpoint and key specific to Azure AI Vision
client = ImageAnalysisClient(
    endpoint="https://my-vision.cognitiveservices.azure.com/",
    credential=AzureKeyCredential("my_vision_key")
)

Option 2: Azure AI Services multi-service (one endpoint + one key for everything)

# A single endpoint for multiple services
endpoint = "https://my-ai-services.cognitiveservices.azure.com/"
key = "my_multiservice_key"

# Vision
vision_client = ImageAnalysisClient(endpoint=endpoint, credential=AzureKeyCredential(key))

# Language
lang_client = TextAnalyticsClient(endpoint=endpoint, credential=AzureKeyCredential(key))

# Speech (requires region as well)
speech_config = speechsdk.SpeechConfig(subscription=key, region="eastus")

For the exam: “Simplify administration” or “a single endpoint for multiple services” → Azure AI Services (multi-service account), formerly called Cognitive Services.

3. Azure AI Search – Intelligent Search

3.1 What Is Azure AI Search?

Azure AI Search (formerly Azure Cognitive Search) is a cloud search service that enables indexing and searching through structured and unstructured content, enriched by AI capabilities (OCR, NLP, Vision).

flowchart LR
    subgraph "Data Sources"
        BLOB["Azure Blob Storage\n(PDFs, Word, HTML...)"]
        SQL["Azure SQL\n(Tables, views)"]
        COSMOS["Cosmos DB\n(JSON)"]
        SHAREPOINT["SharePoint\nOnline"]
    end
    
    subgraph "Indexing Pipeline"
        INDEXER["Indexer\n(Crawler)"]
        SKILLSET["AI Skillset\n(OCR, NLP, Vision,\nKey Phrases, NER...)"]
        INDEX["Index\n(Enriched data)"]
    end
    
    subgraph "Search"
        QUERY["Query\n(Keywords, Semantic,\nVector, Hybrid)"]
        RESULTS["Results\n(Documents + Score)"]
    end
    
    BLOB --> INDEXER
    SQL --> INDEXER
    COSMOS --> INDEXER
    SHAREPOINT --> INDEXER
    
    INDEXER --> SKILLSET
    SKILLSET --> INDEX
    
    INDEX --> QUERY
    QUERY --> RESULTS

3.2 Available Search Types

3.3 AI Enrichment (Skillsets)

Skillsets apply AI transformations during indexing:

3.4 Complete Configuration with Python

# Azure AI Search configuration with indexing and search
from azure.search.documents import SearchClient
from azure.search.documents.indexes import SearchIndexClient
from azure.search.documents.indexes.models import (
    SearchIndex, SimpleField, SearchableField, 
    SearchFieldDataType, VectorSearch,
    HnswAlgorithmConfiguration, VectorSearchProfile,
    SemanticConfiguration, SemanticSearch, SemanticPrioritizedFields,
    SemanticField
)
from azure.search.documents.models import VectorizedQuery
from azure.core.credentials import AzureKeyCredential
import os

# Configuration
SEARCH_ENDPOINT = os.environ["AZURE_SEARCH_ENDPOINT"]
SEARCH_KEY = os.environ["AZURE_SEARCH_KEY"]
INDEX_NAME = "documents-index"

# Administration client (create/manage indexes)
index_client = SearchIndexClient(
    endpoint=SEARCH_ENDPOINT,
    credential=AzureKeyCredential(SEARCH_KEY)
)

def create_index_with_vectors():
    """
    Creates an Azure AI Search index with vector support (for RAG).
    """
    # Vector search configuration
    vector_search = VectorSearch(
        algorithms=[
            HnswAlgorithmConfiguration(name="hnsw-config")
        ],
        profiles=[
            VectorSearchProfile(
                name="vector-profile",
                algorithm_configuration_name="hnsw-config"
            )
        ]
    )
    
    # Semantic configuration
    semantic_config = SemanticConfiguration(
        name="semantic-config",
        prioritized_fields=SemanticPrioritizedFields(
            title_field=SemanticField(field_name="title"),
            content_fields=[SemanticField(field_name="content")],
            keywords_fields=[SemanticField(field_name="keywords")]
        )
    )
    
    # Field definitions
    fields = [
        SimpleField(name="id", type=SearchFieldDataType.String, key=True),
        SearchableField(name="title", type=SearchFieldDataType.String),
        SearchableField(name="content", type=SearchFieldDataType.String),
        SimpleField(name="source", type=SearchFieldDataType.String, filterable=True),
        SimpleField(name="date", type=SearchFieldDataType.DateTimeOffset, filterable=True, sortable=True),
        SearchableField(name="keywords", type=SearchFieldDataType.String, collection=True),
        # Vector field for semantic search
        {
            "name": "content_vector",
            "type": "Collection(Edm.Single)",
            "searchable": True,
            "vector_search_dimensions": 1536,  # text-embedding-3-small embedding size
            "vector_search_profile_name": "vector-profile"
        }
    ]
    
    index = SearchIndex(
        name=INDEX_NAME,
        fields=fields,
        vector_search=vector_search,
        semantic_search=SemanticSearch(configurations=[semantic_config])
    )
    
    index_client.create_or_update_index(index)
    print(f"✅ Index '{INDEX_NAME}' created with vector support")

def index_documents(documents: list[dict], embedder) -> None:
    """
    Indexes documents with their embeddings.
    
    Args:
        documents: List of dicts {id, title, content, source, date}
        embedder: Function that generates embeddings
    """
    search_client = SearchClient(
        endpoint=SEARCH_ENDPOINT,
        index_name=INDEX_NAME,
        credential=AzureKeyCredential(SEARCH_KEY)
    )
    
    docs_to_index = []
    
    for doc in documents:
        # Generate content embedding
        embedding = embedder(doc["content"])
        
        docs_to_index.append({
            "id": doc["id"],
            "title": doc["title"],
            "content": doc["content"],
            "source": doc.get("source", "internal"),
            "date": doc.get("date"),
            "keywords": doc.get("keywords", []),
            "content_vector": embedding
        })
    
    # Index in batch
    result = search_client.upload_documents(docs_to_index)
    
    success = sum(1 for r in result if r.succeeded)
    print(f"✅ {success}/{len(docs_to_index)} documents indexed")

def hybrid_search(
    query: str,
    query_vector: list[float],
    top_k: int = 5,
    filters: str = None
) -> list[dict]:
    """
    Performs a hybrid search (full-text + vector).
    
    Args:
        query: Query text
        query_vector: Query embedding
        top_k: Number of results
        filters: OData filter (e.g.: "source eq 'HR'")
    
    Returns:
        List of relevant documents with scores
    """
    search_client = SearchClient(
        endpoint=SEARCH_ENDPOINT,
        index_name=INDEX_NAME,
        credential=AzureKeyCredential(SEARCH_KEY)
    )
    
    # Vector query
    vector_query = VectorizedQuery(
        vector=query_vector,
        k_nearest_neighbors=top_k,
        fields="content_vector"
    )
    
    # Hybrid search
    results = search_client.search(
        search_text=query,            # Full-text
        vector_queries=[vector_query], # Vector
        filter=filters,
        query_type="semantic",         # Semantic
        semantic_configuration_name="semantic-config",
        select=["id", "title", "content", "source"],
        top=top_k
    )
    
    return [
        {
            "id": r["id"],
            "title": r["title"],
            "content_excerpt": r["content"][:300] + "...",
            "source": r["source"],
            "search_score": r["@search.score"]
        }
        for r in results
    ]

# Usage example
print("=== Azure AI Search Configuration ===")
create_index_with_vectors()

# Simulate indexing HR documents
hr_docs = [
    {
        "id": "1",
        "title": "Remote Work Policy",
        "content": "Employees can work remotely up to 3 days per week...",
        "source": "HR",
        "keywords": ["remote work", "flexibility", "policy"]
    },
    {
        "id": "2",
        "title": "Travel Expense Reimbursement",
        "content": "Professional travel expenses are reimbursed upon presentation of receipts...",
        "source": "HR",
        "keywords": ["reimbursement", "expenses", "travel"]
    }
]

# Note: embedder would be your Azure OpenAI embedding function
# index_documents(hr_docs, embedder=generate_embedding)

4. Azure AI Foundry – Generative AI Platform

4.1 Hub/Project Architecture

Azure AI Foundry is organized into two hierarchical levels:

flowchart TD
    HUB["🏢 Hub\n(Top-level container)\n\n• Shared configuration\n• Resource connections\n• Centralized governance\n• Network and security"]
    
    HUB --> P1["📋 Project 1\n(Isolated workspace)\n\n• Application A\n• Team A\n• Data A"]
    HUB --> P2["📋 Project 2\n(Isolated workspace)\n\n• Application B\n• Team B\n• Data B"]
    HUB --> P3["📋 Project 3\n(Isolated workspace)\n\n• Application C\n• Team C\n• Data C"]
    
    subgraph "Shared Resources (Hub-level)"
        STORAGE["Azure Storage"]
        KEYVAULT["Azure Key Vault"]
        MONITOR["Azure Monitor"]
        AI_SERVICES["Azure AI Services"]
    end
    
    HUB -.-> STORAGE
    HUB -.-> KEYVAULT
    HUB -.-> MONITOR
    HUB -.-> AI_SERVICES

Hub vs Project differences:

4.2 Microsoft Foundry Capabilities

# Using Microsoft Foundry via Python SDK
from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential
import os

# Connect to the Foundry project
project_client = AIProjectClient.from_connection_string(
    conn_str=os.environ["AZURE_AI_PROJECT_CONNECTION_STRING"],
    credential=DefaultAzureCredential()
)

# Deploy and use a model
def use_foundry_model(prompt: str) -> str:
    """Uses a model deployed in Microsoft Foundry."""
    
    # Get the OpenAI client from Foundry
    openai_client = project_client.inference.get_azure_openai_client()
    
    response = openai_client.chat.completions.create(
        model="gpt-4o",  # Deployment name in Foundry
        messages=[
            {"role": "system", "content": "You are an expert Azure assistant."},
            {"role": "user", "content": prompt}
        ],
        max_tokens=500
    )
    
    return response.choices[0].message.content

# List available models
def list_available_models() -> list[dict]:
    """Lists models deployed in the Foundry project."""
    models = []
    
    for deployment in project_client.inference.list_deployments():
        models.append({
            "name": deployment.name,
            "model": deployment.model_name,
            "type": deployment.model_type,
            "status": deployment.provisioning_state
        })
    
    return models

print("=== Available Models in Foundry ===")
for model in list_available_models():
    print(f"  {model['name']} ({model['model']}) - {model['status']}")

4.3 Building a RAG App with Foundry

# Complete RAG application with Azure AI Foundry
# Combines Azure AI Search + Azure OpenAI
from azure.ai.projects import AIProjectClient
from azure.search.documents import SearchClient
from azure.search.documents.models import VectorizedQuery
from azure.core.credentials import AzureKeyCredential
from azure.identity import DefaultAzureCredential
import os

class RAGAppFoundry:
    """
    RAG application using Microsoft Foundry.
    Answers questions by searching in a document knowledge base.
    """
    
    def __init__(self):
        # Foundry client
        self.project = AIProjectClient.from_connection_string(
            conn_str=os.environ["AZURE_AI_PROJECT_CONNECTION_STRING"],
            credential=DefaultAzureCredential()
        )
        self.openai = self.project.inference.get_azure_openai_client()
        
        # Azure AI Search client
        self.search = SearchClient(
            endpoint=os.environ["AZURE_SEARCH_ENDPOINT"],
            index_name="knowledge-base",
            credential=AzureKeyCredential(os.environ["AZURE_SEARCH_KEY"])
        )
    
    def _generate_embedding(self, text: str) -> list[float]:
        """Generates the embedding of a text."""
        response = self.openai.embeddings.create(
            model="text-embedding-3-small",
            input=text
        )
        return response.data[0].embedding
    
    def _search_context(self, question: str, top_k: int = 3) -> str:
        """Searches for relevant documents."""
        vector = self._generate_embedding(question)
        
        query_vector = VectorizedQuery(
            vector=vector,
            k_nearest_neighbors=top_k,
            fields="content_vector"
        )
        
        results = self.search.search(
            search_text=question,
            vector_queries=[query_vector],
            select=["title", "content"],
            top=top_k
        )
        
        context_parts = []
        for r in results:
            context_parts.append(f"### {r['title']}\n{r['content'][:500]}")
        
        return "\n\n".join(context_parts)
    
    def answer(self, question: str) -> dict:
        """
        Answers a question using RAG.
        
        Returns:
            dict with response, sources and metadata
        """
        # Retrieve context
        context = self._search_context(question)
        
        # Generate response
        response = self.openai.chat.completions.create(
            model="gpt-4o",
            messages=[
                {
                    "role": "system",
                    "content": f"""You are an expert enterprise assistant.
Answer ONLY based on the context below.
If the information is not in the context, say so clearly.

CONTEXT:
{context}"""
                },
                {"role": "user", "content": question}
            ],
            temperature=0.1,
            max_tokens=800
        )
        
        return {
            "question": question,
            "answer": response.choices[0].message.content,
            "tokens_used": response.usage.total_tokens,
            "context_used": context[:200] + "..."
        }

# Application test
app = RAGAppFoundry()

test_questions = [
    "What is the company's vacation policy?",
    "How do I submit an expense report?",
    "What are the IT security procedures?"
]

for question in test_questions:
    result = app.answer(question)
    print(f"\n❓ {result['question']}")
    print(f"💬 {result['answer'][:200]}...")
    print(f"🔢 Tokens: {result['tokens_used']}")

5. Microsoft Copilot for Azure

5.1 What Is Microsoft Copilot for Azure?

Microsoft Copilot for Azure is an AI assistant integrated directly into the Azure portal. It helps developers and administrators manage their cloud resources more efficiently using natural language.

flowchart LR
    USER["👤 Developer\n/ Azure Admin"] -->|Natural language\nquestion| COPILOT["🤖 Copilot for Azure\n(Azure Portal)"]
    
    COPILOT --> INSIGHT["📊 Insights\n(Costs, usage,\nperformance)"]
    COPILOT --> SCRIPT["📝 Script generation\n(Azure CLI, PowerShell,\nTerraform, Bicep)"]
    COPILOT --> SECURITY["🔒 Security & Compliance\n(Non-compliant resources,\nvulnerabilities)"]
    COPILOT --> TROUBLE["🔧 Troubleshooting\n(Network diagnostics,\nservice issues)"]
    COPILOT --> MONITOR["📈 Monitoring\n(Alerts, metrics,\nlogs)"]

5.2 Copilot for Azure Prompt Examples

💬 Cost management:
"What is our Azure budget for this month vs last month?"
"Which services are consuming the most resources?"
"Generate a cost report by team for Q4 2024"

💬 Script generation:
"Generate an Azure CLI script to create an Azure Function with a storage account"
"Write a Bicep template to deploy an Azure Container App"
"Generate a PowerShell script to list all stopped VMs"

💬 Security and compliance:
"Which resources don't comply with our security policy?"
"Are there any unauthorized public accesses in our subscription?"
"List resources without governance tags"

💬 Troubleshooting:
"Why is my Function App responding slowly?"
"Are there any network connectivity issues between my services?"
"What are the recent errors in my App Service logs?"

💬 Architecture:
"How do I get started with Azure Functions and Azure OpenAI?"
"What architecture do you recommend for a RAG chatbot?"
"How do I migrate this application to a managed Azure service?"

5.3 Access and Usage

# Copilot for Azure is accessible via:
# 1. Azure Portal: button at the top of the page
# 2. Azure CLI: az copilot (preview)
# 3. Azure SDK: via Azure AI Foundry

# Example of programmatic interaction with Azure Management APIs
# (Copilot for Azure uses these same APIs under the hood)
from azure.mgmt.resource import ResourceManagementClient
from azure.identity import DefaultAzureCredential
import os

credential = DefaultAzureCredential()
subscription_id = os.environ["AZURE_SUBSCRIPTION_ID"]

# Analyze resources without tags (governance)
def find_untagged_resources() -> list[dict]:
    """
    Lists resources without tags.
    Equivalent to asking Copilot:
    "Which resources have no tags?"
    """
    resource_client = ResourceManagementClient(credential, subscription_id)
    
    untagged_resources = []
    
    for resource in resource_client.resources.list():
        if not resource.tags:
            untagged_resources.append({
                "name": resource.name,
                "type": resource.type,
                "group": resource.id.split("/")[4],
                "location": resource.location
            })
    
    return untagged_resources

# Usage example
print("=== Resources without tags ===")
resources = find_untagged_resources()
print(f"Found: {len(resources)} resources without tags")
for r in resources[:5]:
    print(f"  {r['name']} ({r['type']}) - {r['group']}")

6. Practical Implementation

6.1 Decision Table: Which Service to Use?

flowchart TD
    START["AI Need"] --> Q1{"Type of data?"}
    
    Q1 -->|"Images/Photos"| Q_IMG{"Specific need?"}
    Q_IMG -->|"Generic analysis"| AIV["Azure AI Vision"]
    Q_IMG -->|"Your own categories"| AICV["Azure AI Custom Vision"]
    Q_IMG -->|"Facial analysis"| AIF["Azure AI Face"]
    Q_IMG -->|"Scanned documents"| AIDI["Azure AI Document Intelligence"]
    
    Q1 -->|"Text/Speech"| Q_TXT{"Specific need?"}
    Q_TXT -->|"Analyze sentiment/entities"| AIL["Azure AI Language"]
    Q_TXT -->|"Recognize/synthesize speech"| AIS["Azure AI Speech"]
    Q_TXT -->|"Translate"| AIT["Azure AI Translator"]
    Q_TXT -->|"Understand intent"| AIL2["Azure AI Language (CLU)"]
    
    Q1 -->|"Documents (PDF/forms)"| AIDI2["Azure AI Document Intelligence"]
    
    Q1 -->|"Create new content"| AOAI["Azure OpenAI Service\n(via Microsoft Foundry)"]
    
    Q1 -->|"Search through\ndocuments"| AISEARCH["Azure AI Search"]
    
    Q1 -->|"Detect anomalies"| AIAD["Azure AI Anomaly Detector"]

6.2 Complete Multi-Service Application

# Application demonstrating integration of multiple Azure AI Services
# Scenario: Automatically analyze incoming emails and route them

import os
import json
from azure.ai.textanalytics import TextAnalyticsClient
from azure.ai.translation.text import TextTranslationClient
from azure.core.credentials import AzureKeyCredential
from openai import AzureOpenAI

class IntelligentEmailAnalyzer:
    """
    Analyzes incoming emails with multiple Azure AI services:
    1. Language detection (Azure AI Language)
    2. Translation if needed (Azure AI Translator)
    3. Sentiment analysis (Azure AI Language)
    4. Entity and intent extraction (Azure AI Language)
    5. Automated response generation (Azure OpenAI)
    6. Routing based on analysis
    """
    
    def __init__(self):
        lang_endpoint = os.environ["AZURE_LANGUAGE_ENDPOINT"]
        lang_key = os.environ["AZURE_LANGUAGE_KEY"]
        trans_key = os.environ["AZURE_TRANSLATOR_KEY"]
        region = os.environ["AZURE_TRANSLATOR_REGION"]
        
        # Azure AI Language
        self.lang_client = TextAnalyticsClient(
            endpoint=lang_endpoint,
            credential=AzureKeyCredential(lang_key)
        )
        
        # Azure AI Translator
        self.trans_client = TextTranslationClient(
            credential=AzureKeyCredential(trans_key),
            region=region
        )
        
        # Azure OpenAI
        self.openai_client = AzureOpenAI(
            azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
            api_key=os.environ["AZURE_OPENAI_API_KEY"],
            api_version="2024-02-01"
        )
    
    def analyze_email(self, subject: str, body: str, sender: str) -> dict:
        """
        Complete analysis of an incoming email.
        
        Returns:
            Complete analysis with routing and suggested response
        """
        full_text = f"{subject}\n\n{body}"
        
        # 1. Detect language
        detect_result = self.lang_client.detect_language(documents=[full_text])
        language = detect_result[0].primary_language
        
        # 2. Translate to English if needed
        text_en = full_text
        if language.iso6391_name != "en":
            trans_result = self.trans_client.translate(
                body=[{"text": full_text}],
                to_language=["en"]
            )
            text_en = trans_result[0].translations[0].text
        
        # 3. Analyze sentiment
        sent_result = self.lang_client.analyze_sentiment(
            documents=[text_en], language="en",
            show_opinion_mining=True
        )
        sentiment = sent_result[0]
        
        # 4. Extract entities
        ner_result = self.lang_client.recognize_entities(
            documents=[text_en], language="en"
        )
        entities = ner_result[0].entities
        
        # 5. Extract key phrases
        kp_result = self.lang_client.extract_key_phrases(
            documents=[text_en], language="en"
        )
        key_phrases = list(kp_result[0].key_phrases)
        
        # 6. Classify urgency and department
        classification = self._classify_urgency(
            sentiment.sentiment,
            sentiment.confidence_scores.negative,
            key_phrases
        )
        
        # 7. Generate automatic response if possible
        auto_response = self._generate_response(
            text_en,
            classification["department"],
            sender
        ) if classification["auto_respondable"] else None
        
        return {
            "sender": sender,
            "original_language": language.name,
            "translated_text": text_en[:200] + "...",
            "sentiment": sentiment.sentiment,
            "negative_score": round(sentiment.confidence_scores.negative, 3),
            "urgency": classification["urgency"],
            "department": classification["department"],
            "key_phrases": key_phrases[:5],
            "entities": [f"{e.text} ({e.category})" for e in entities[:5]],
            "auto_respondable": classification["auto_respondable"],
            "suggested_response": auto_response
        }
    
    def _classify_urgency(self, 
                           sentiment: str,
                           negative_score: float,
                           key_phrases: list) -> dict:
        """Determines urgency and department based on the analysis."""
        urgency = "normal"
        department = "customer_service"
        auto_respondable = False
        
        # Keywords by department
        billing_keywords = ["invoice", "payment", "refund", "price", "cost"]
        technical_keywords = ["bug", "error", "outage", "technical issue", "not working"]
        order_keywords = ["order", "delivery", "tracking", "shipping", "delay"]
        
        phrases_lower = " ".join(key_phrases).lower()
        
        if any(kw in phrases_lower for kw in billing_keywords):
            department = "billing"
        elif any(kw in phrases_lower for kw in technical_keywords):
            department = "technical_support"
        elif any(kw in phrases_lower for kw in order_keywords):
            department = "logistics"
            auto_respondable = True  # Auto-response for orders
        
        # Calculate urgency
        if negative_score > 0.8:
            urgency = "high"
        elif negative_score > 0.5:
            urgency = "medium"
        
        return {
            "urgency": urgency,
            "department": department,
            "auto_respondable": auto_respondable
        }
    
    def _generate_response(self, email_text: str, department: str, sender: str) -> str:
        """Generates a basic automatic response."""
        response = self.openai_client.chat.completions.create(
            model="gpt-4",
            messages=[
                {
                    "role": "system",
                    "content": f"""You are a customer service assistant ({department}).
Generate a polite response in English.
Start by thanking the customer, confirm their request has been received,
and give an estimated processing time (24-48h).
Sign as "The Customer Service Team"."""
                },
                {
                    "role": "user",
                    "content": f"Email from {sender}:\n{email_text[:500]}"
                }
            ],
            max_tokens=200,
            temperature=0.5
        )
        return response.choices[0].message.content

# Test the analyzer
analyzer = IntelligentEmailAnalyzer()

test_emails = [
    {
        "subject": "Problem with my order #12345",
        "body": "Hello, I placed an order 5 days ago and I still haven't received it. The tracking shows it's been stuck for 3 days. I need this urgently for a work meeting. Very disappointed!",
        "sender": "john.smith@example.com"
    },
    {
        "subject": "Question about my November invoice",
        "body": "Hi, I received my bill and I don't understand the additional charges of $45. Could you explain? Thank you",
        "sender": "mary.johnson@example.com"
    }
]

print("=== Incoming Email Analysis ===\n")
for email in test_emails:
    print(f"📧 From: {email['sender']}")
    print(f"   Subject: {email['subject']}")
    
    analysis = analyzer.analyze_email(
        email["subject"], 
        email["body"], 
        email["sender"]
    )
    
    print(f"   Language: {analysis['original_language']}")
    print(f"   Sentiment: {analysis['sentiment']} (negative: {analysis['negative_score']:.0%})")
    print(f"   Urgency: {analysis['urgency'].upper()}")
    print(f"   → Department: {analysis['department']}")
    print(f"   Key phrases: {', '.join(analysis['key_phrases'][:3])}")
    
    if analysis['suggested_response']:
        print(f"   📝 Auto-response generated")
    
    print()

7. Architecture of a Complete AI Solution

7.1 Enterprise Reference Architecture

flowchart TB
    subgraph "Clients"
        WEB["🌐 Web App"]
        MOBILE["📱 Mobile"]
        TEAMS["💼 Teams Bot"]
        API_EXT["🔗 External API"]
    end
    
    subgraph "API Gateway and Security"
        APIM["Azure API Management\n(Rate limiting, Auth, Cache)"]
        AAD["Azure AD B2C\n(Identity)"]
    end
    
    subgraph "Orchestration Layer"
        FUNC["Azure Functions\n(Logic App)"]
        SERVICE_BUS["Azure Service Bus\n(Message queues)"]
    end
    
    subgraph "Azure AI Services"
        OPENAI["Azure OpenAI\n(GPT-4, DALL-E)"]
        LANG["Azure AI Language\n(NLP)"]
        SPEECH["Azure AI Speech\n(STT/TTS)"]
        VISION["Azure AI Vision\n(Images)"]
        DI["Azure AI Document Intelligence"]
    end
    
    subgraph "Data"
        SEARCH["Azure AI Search\n(RAG)"]
        COSMOS["Azure Cosmos DB"]
        BLOB["Azure Blob Storage"]
        SQL["Azure SQL"]
    end
    
    subgraph "Monitoring and Governance"
        MONITOR["Azure Monitor"]
        APPINSIGHTS["Application Insights"]
        CONTENT_SAFETY["Azure AI Content Safety"]
        KV["Azure Key Vault"]
    end
    
    WEB --> APIM
    MOBILE --> APIM
    TEAMS --> APIM
    API_EXT --> APIM
    
    APIM --> AAD
    APIM --> FUNC
    FUNC --> SERVICE_BUS
    
    FUNC --> OPENAI
    FUNC --> LANG
    FUNC --> SPEECH
    FUNC --> VISION
    FUNC --> DI
    
    OPENAI --> SEARCH
    OPENAI --> COSMOS
    FUNC --> BLOB
    FUNC --> SQL
    
    FUNC -.-> MONITOR
    OPENAI -.-> APPINSIGHTS
    FUNC -.-> CONTENT_SAFETY
    FUNC -.-> KV

8. Azure AI Governance and Security

8.1 Azure AI Security Principles

8.2 Key and Secret Management

# Secure credential management for Azure AI
from azure.identity import DefaultAzureCredential, ManagedIdentityCredential
from azure.keyvault.secrets import SecretClient
import os

class AISecretsManager:
    """
    Secure management of Azure AI credentials.
    Uses Azure Key Vault with Managed Identity.
    """
    
    def __init__(self, keyvault_url: str = None):
        # In production: use Managed Identity (no key in code!)
        self.credential = DefaultAzureCredential()
        
        keyvault_url = keyvault_url or os.environ.get("AZURE_KEYVAULT_URL")
        if keyvault_url:
            self.kv_client = SecretClient(
                vault_url=keyvault_url,
                credential=self.credential
            )
        else:
            self.kv_client = None
    
    def get_secret(self, name: str) -> str:
        """Retrieves a secret from Key Vault or environment variables."""
        if self.kv_client:
            try:
                return self.kv_client.get_secret(name).value
            except Exception:
                pass
        return os.environ.get(name, "")
    
    def get_ai_services_config(self) -> dict:
        """Retrieves the full AI services configuration."""
        return {
            "openai_endpoint": self.get_secret("AZURE-OPENAI-ENDPOINT"),
            "openai_key": self.get_secret("AZURE-OPENAI-KEY"),
            "language_endpoint": self.get_secret("AZURE-LANGUAGE-ENDPOINT"),
            "language_key": self.get_secret("AZURE-LANGUAGE-KEY"),
            "search_endpoint": self.get_secret("AZURE-SEARCH-ENDPOINT"),
            "search_key": self.get_secret("AZURE-SEARCH-KEY"),
        }

# Recommended pattern in production
# Use Managed Identity directly (no key required!)
from azure.ai.vision.imageanalysis import ImageAnalysisClient

# ✅ RECOMMENDED in production - no key in code
vision_client = ImageAnalysisClient(
    endpoint="https://my-service.cognitiveservices.azure.com/",
    credential=DefaultAzureCredential()  # Uses the VM/App Service managed identity
)

# ✅ ACCEPTABLE in development - key in environment variable
vision_client_dev = ImageAnalysisClient(
    endpoint=os.environ["AZURE_VISION_ENDPOINT"],
    credential=AzureKeyCredential(os.environ["AZURE_VISION_KEY"])
)

# ❌ NEVER - hard-coded key in code
# vision_client_bad = ImageAnalysisClient(
#     endpoint="https://xxx.cognitiveservices.azure.com/",
#     credential=AzureKeyCredential("abc123...")
# )

9. Summary and Key Points

9.1 Condensed Overview

mindmap
  root((Azure AI\nFundamentals))
    3 Categories
      Azure AI Services Ready-to-use APIs
      Azure Machine Learning Custom ML
      Copilots AI Assistants
    Key Services
      OpenAI GPT DALL-E
      Document Intelligence Forms
      AI Search RAG Knowledge Mining
      AI Language NLP Sentiment
      AI Speech STT TTS
      AI Vision Images OCR
      Content Safety Filtering
    Platforms
      Microsoft Foundry Hub Project
      Azure ML Studio
      Azure Portal
      Copilot for Azure
    Architecture
      Multi-service Account
      Azure Key Vault Secrets
      Managed Identity Security
      Azure Monitor Observability

9.2 Quick Decision Table

10. Glossary

Additional Resources:

Document generated for Azure AI Fundamentals – Ecosystem Overview

Module 1 – Discovering the Azure AI Services Ecosystem

3 Azure AI Categories

Main Azure AI Services

Module 1 – Demo: Azure AI Search

Azure AI Search Architecture

Data Source (Blob Storage, SQL, etc.)
      ↓
  [Indexer]
      ↓
   [Index]  ←→  [Skillset (AI enrichment)]
      ↓
  Search Service ← Queries

Configuration Steps

1. Create the Azure AI Search service in the Azure portal.
2. Connect a data source (e.g., Azure Blob Storage).
   • Name the source (e.g., invoices-data).
   • Select the storage account and container.
3. Create an Index:
   • The index defines searchable fields (e.g., content, metadata_author).
4. Configure an Indexer:
   • Reads documents from the source and indexes them.
   • Can be triggered manually or on a schedule.
5. Query the index via the “Search Explorer” interface or REST/SDK queries.

Example Supported Documents

• PDFs (e.g., 7 invoice PDFs).
• Excel (e.g., 1 invoice data file).
• Text, HTML, JSON, CSV formats.

Module 2 – Azure AI Foundry and Generative AI Applications

What Is Azure AI Foundry?

• Complete platform for designing, customizing, and managing AI applications.
• Integrates models (Azure OpenAI, Azure Search, etc.) and development tools.
• Built-in governance and collaboration features.
• Accessible at: ai.azure.com.

Key Concepts: Hub and Project

Hub

• Project container: multiple isolated projects under a single hub.
• Pre-configures connections to shared resources (storage, AI models).
• Hub-level connections are shared with all the projects it contains.
• Facilitates governance and collaboration without repeated reconfiguration.

Project

• Dedicated workspace with access to AI app development tools.
• Includes reusable components: datasets, models.
• Isolated environment: each project can secure its data and configure its resources.
• A project belongs to a hub.

Demo: Create a Hub and a Project

1. Go to ai.azure.com.
2. Click Create project.
3. Give the project a name.
4. Create a new hub or select an existing one (click “Create new hub”).
5. Give the hub a name.
6. Access the project from the Azure AI Foundry portal.

Building a Generative AI Application with Azure AI Foundry

Using the Chat Playground

1. In the project → left menu → Playgrounds → Chat playground.
2. Create a model deployment (e.g., GPT-4).
3. Configure a System Message (instructions for the model).
4. Optional: connect an Azure AI Search data source for RAG (Retrieval-Augmented Generation).

RAG (Retrieval-Augmented Generation)

• The model retrieves information from an external source (Azure AI Search) before generating a response.
• Enables answering questions about private documents (e.g., invoices in Blob Storage).
• Flow:

  User question
       ↓
  Azure AI Search → find relevant documents
       ↓
  GPT-4 + context → generate answer
  

Steps to Connect Azure AI Search to the Chat Playground

1. In the Chat Playground → Add data source.
2. Select Azure AI Search as the source type.
3. Choose the Azure AI Search service and the previously created index.
4. Configure the search type (vector, hybrid, text).
5. Test in the chat: responses will include references to source documents.

Module 2 – Microsoft Copilot for Azure

What Is Microsoft Copilot for Azure?

• AI assistance integrated into the Azure portal.
• Helps developers and IT admins work faster and more efficiently.
• Accessible via the button at the top of the Azure portal toolbar.

Use Cases

How to Use It

1. Click the Copilot button in the top bar of the Azure portal.
2. A chat window opens.
3. Ask questions in natural language about your Azure environment.

Course Summary

What You Learned

1. Azure AI Services: panorama of AI services available on Azure (OpenAI, Search, Document Intelligence, Speech, etc.).
2. Azure AI Foundry: creating generative AI applications with hubs and projects. Connecting models (GPT-4) to data sources (Azure AI Search) via RAG.
3. Microsoft Copilot for Azure: optimizing and troubleshooting cloud operations with AI integrated into the Azure portal.

Key Points

• Azure AI Foundry is the unified platform for building AI apps on Azure.
• Hub = project container sharing resources. Project = isolated workspace.
• RAG = the AI model enriches its responses with data from an external source (e.g., Azure AI Search).
• Azure AI Search indexes structured and unstructured content for intelligent search.
• Copilot for Azure generates scripts, analyzes costs, detects non-compliance from the Azure portal.
• This course is an introductory overview of Azure AI services. Each service warrants a dedicated course for full mastery.

Enriched Section 1 – Complete Azure AI Services Portfolio

Overview — Service Map

mindmap
  root((Azure AI))
    Cognitive Services
      Vision
        Image Analysis
        Custom Vision
        Face API
        Spatial Analysis
      Speech
        Speech-to-Text
        Text-to-Speech
        Speech Translation
        Speaker Recognition
        Custom Speech
        Custom Voice
      Language
        Text Analytics
        Question Answering
        CLU
        Translation
        LUIS legacy
      Decision
        Content Safety
        Personalizer
        Anomaly Detector
    Azure OpenAI
      GPT-4 / GPT-4o
      DALL-E 3
      Whisper
      Embeddings
    Azure Machine Learning
      AutoML
      Designer
      MLflow
      Managed Endpoints
    Microsoft AI Foundry
      Model Catalog
      Fine-tuning
      Evaluation
      Azure AI Projects
    Document Intelligence
      Prebuilt Models
      Layout Model
      Custom Models
    Bot Service
      Bot Framework SDK
      Copilot Studio
      Channels

Service Summary Table

Enriched Section 2 – Azure AI Language Service

Main Features

Azure AI Language groups all Natural Language Processing (NLP) capabilities available via REST API or SDK.

2.1 Text Analytics

REST call example — Sentiment analysis:

POST https://<endpoint>.cognitiveservices.azure.com/language/:analyze-text?api-version=2023-04-01
Ocp-Apim-Subscription-Key: <your-key>
Content-Type: application/json

{
  "kind": "SentimentAnalysis",
  "analysisInput": {
    "documents": [
      { "id": "1", "language": "en", "text": "The Azure service is excellent!" }
    ]
  }
}

2.2 Question Answering (QnA)

• Successor to QnA Maker (retired March 2025).
• Creates a knowledge base (question/answer pairs) from:
  • FAQ documents, URLs, Word/PDF files.
• Exposes a REST endpoint queryable by a bot or application.
• Supports multi-turn conversations (context tracking).

2.3 Conversational Language Understanding (CLU)

Successor to LUIS (Language Understanding Intelligent Service).

CLU flow:

User text → CLU → Intent + Entities → Application logic
"Book a flight to Paris tomorrow" → BookFlight + {destination: Paris, date: tomorrow}

2.4 Orchestration Workflow

• Routes requests to the right service (CLU, Question Answering, or LUIS).
• Ideal for bots combining multiple response sources.
• Single unified endpoint for the bot.

2.5 Custom NER and Classification

Enriched Section 3 – Azure AI Speech Service

Capabilities Overview

┌─────────────────────────────────────────────────────┐
│                  Azure AI Speech                     │
├──────────────┬──────────────┬────────────────────────┤
│  Speech-to-  │  Text-to-   │  Speech                │
│  Text (STT)  │  Speech(TTS)│  Translation           │
├──────────────┴──────────────┴────────────────────────┤
│  Speaker Recognition │ Custom Speech │ Custom Voice  │
└─────────────────────────────────────────────────────┘

3.1 Speech-to-Text (STT)

Python SDK example:

import azure.cognitiveservices.speech as speechsdk

speech_config = speechsdk.SpeechConfig(
    subscription="<key>",
    region="eastus"
)
speech_config.speech_recognition_language = "en-US"

recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config)
result = recognizer.recognize_once_async().get()

if result.reason == speechsdk.ResultReason.RecognizedSpeech:
    print(f"Recognized: {result.text}")

3.2 Text-to-Speech (TTS)

• Neural voices: natural, expressive sounds (e.g., en-US-JennyNeural).
• SSML (Speech Synthesis Markup Language): fine control of pronunciation, speed, pitch, pauses.

SSML example:

<speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis" xml:lang="en-US">
  <voice name="en-US-JennyNeural">
    Hello! <break time="500ms"/>
    Welcome to <emphasis level="strong">Azure AI Speech</emphasis>.
    <prosody rate="slow" pitch="+2st">Have a great day!</prosody>
  </voice>
</speak>

3.3 Speech Translation

• Translates audio in real-time to another text or audio.
• Supports more than 60 source and target languages.
• Use: simultaneous interpretation, live subtitles.

3.4 Custom Speech & Custom Voice

3.5 Speaker Recognition

• Speaker verification: “Is this the same person?”
• Speaker identification: “Who is speaking among N registered people?”
• Limited Access feature: mandatory approval request required.

Enriched Section 4 – Azure AI Vision

Service Capabilities

4.1 Image Analysis (API v4)

REST example — Analyze an image:

POST https://<endpoint>.cognitiveservices.azure.com/computervision/imageanalysis:analyze
  ?api-version=2023-10-01
  &features=caption,tags,objects
Ocp-Apim-Subscription-Key: <your-key>
Content-Type: application/json

{
  "url": "https://example.com/photo.jpg"
}

4.2 OCR — Read API

• Extraction of printed and handwritten text from images or PDFs.
• Supports more than 150 languages.
• Returns position (bounding box), content, and confidence for each line/word.
• Async API for long documents (polling).

Image/PDF → Read API → Operation-Location (polling) → JSON result

4.3 Face API (Limited Access)

Important: Since June 2023, access to Face API for face identification and attribute recognition is restricted and requires Microsoft approval.

4.4 Custom Vision

4.5 Spatial Analysis

• Real-time video analysis to understand movements and interactions of people in a physical space.
• Use cases: people counting, zone compliance, store traffic flow.
• Deployment on Azure Stack Edge (edge computing).
• Limited Access feature (responsible AI policy).

Enriched Section 5 – Azure AI Document Intelligence

Prebuilt Models

Layout Model

• Extracts the complete structure of the document: tables, paragraphs, titles, lists.
• Returns bounding boxes, content of each cell, semantic roles.
• Used as a base for custom models.

General Document Model

• Generalist key-value extraction without training.
• Ideal for exploring unknown forms.

Custom Models

Call example — Analyze an invoice:

from azure.ai.formrecognizer import DocumentAnalysisClient
from azure.core.credentials import AzureKeyCredential

client = DocumentAnalysisClient(
    endpoint="https://<endpoint>.cognitiveservices.azure.com",
    credential=AzureKeyCredential("<key>")
)

with open("invoice.pdf", "rb") as f:
    poller = client.begin_analyze_document("prebuilt-invoice", f)

result = poller.result()

for invoice in result.documents:
    vendor = invoice.fields.get("VendorName")
    total = invoice.fields.get("InvoiceTotal")
    print(f"Vendor: {vendor.value}, Total: {total.value}")

Document Intelligence Architecture

Document (PDF/Image)
        ↓
  Analyze Document API
        ↓
  ┌─────────────────┐
  │  Layout Engine  │ ← OCR + Structure
  └────────┬────────┘
           ↓
  ┌─────────────────┐
  │  Model (prebuilt│ ← Business intelligence
  │  or custom)     │
  └────────┬────────┘
           ↓
  Structured JSON (fields, values, confidence scores)

Enriched Section 6 – Bot Service and Copilot Studio

Bot Framework SDK vs Copilot Studio

Deployment Channels

flowchart LR
    Bot[Azure Bot / Copilot Studio] --> Teams[Microsoft Teams]
    Bot --> Webchat[Web Chat iFrame]
    Bot --> Slack[Slack]
    Bot --> FB[Facebook Messenger]
    Bot --> Email[Email]
    Bot --> Twilio[Twilio SMS]
    Bot --> DL[Direct Line API]

Question Answering Integration

1. Create a knowledge base in Azure AI Language → Question Answering.
2. Publish the project (REST endpoint available).
3. Connect the endpoint to the bot via the QnA Maker recognizer (Bot Framework) or directly in Copilot Studio.
4. The bot answers user questions by searching the KB.

Complete Bot Architecture

User (Teams/Web)
       ↓
  Azure Bot Service (Channel Registration)
       ↓
  Bot Application (App Service / Container)
       ├── CLU → Intent detection
       ├── Question Answering → FAQ responses
       └── Azure OpenAI → Generative responses

Enriched Section 7 – Microsoft AI Foundry (Azure AI Foundry)

Unified Platform for AI Development

Azure AI Foundry is the central platform for designing, customizing, evaluating, and deploying AI applications on Azure.

flowchart TD
    A[Azure AI Foundry Portal - ai.azure.com] --> B[Hub]
    B --> C[Project 1]
    B --> D[Project 2]
    C --> E[Model Catalog]
    C --> F[Playground - Chat/Image/Speech]
    C --> G[Fine-tuning]
    C --> H[Evaluation]
    C --> I[Deployment - Managed Endpoint]
    C --> J[Monitoring - App Insights]
    E --> K[Azure OpenAI Models]
    E --> L[Open Source - Llama, Mistral, Phi]
    E --> M[Hugging Face Models]

Model Catalog

Fine-tuning

• Adapts a base model (e.g., GPT-3.5-turbo) to your specific use case.
• Requires training data in JSONL format (prompt/completion pairs).
• Improves model consistency, tone, and behaviors.

Training file example in JSONL:

{"messages": [{"role": "system", "content": "You are an IT support assistant."}, {"role": "user", "content": "My computer won't start."}, {"role": "assistant", "content": "Let's first check the power supply..."}]}
{"messages": [{"role": "system", "content": "You are an IT support assistant."}, {"role": "user", "content": "My VPN isn't working."}, {"role": "assistant", "content": "Check that the VPN service is active in Windows services..."}]}

Evaluation

Azure AI Projects (SDK)

from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

client = AIProjectClient(
    subscription_id="<sub-id>",
    resource_group_name="<rg>",
    project_name="<project>",
    credential=DefaultAzureCredential()
)

# Get an Azure OpenAI client from the project
openai_client = client.inference.get_azure_openai_client(api_version="2024-06-01")
response = openai_client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Explain Azure AI Foundry in 3 sentences."}]
)
print(response.choices[0].message.content)

Enriched Section 8 – Responsible AI in Azure Services

Microsoft Responsible AI Principles

Azure AI Content Safety

Content Safety call example:

from azure.ai.contentsafety import ContentSafetyClient
from azure.ai.contentsafety.models import AnalyzeTextOptions
from azure.core.credentials import AzureKeyCredential

client = ContentSafetyClient(
    endpoint="https://<endpoint>.cognitiveservices.azure.com",
    credential=AzureKeyCredential("<key>")
)

request = AnalyzeTextOptions(text="Content to analyze...")
response = client.analyze_text(request)

for category in response.categories_analysis:
    print(f"{category.category}: severity {category.severity}")

Transparency Notes and Impact Assessments

• Transparency Notes: documents published by Microsoft explaining the operation, limitations, and appropriate use cases of each AI service.
• Responsible AI Impact Assessment: mandatory evaluation process for certain sensitive deployments.
• Available at: learn.microsoft.com/azure/ai-services/responsible-use-of-ai-overview

Limited Access Features

Some features require explicit Microsoft approval before use:

Watermarking and AI-Generated Content Detection

• Azure AI Content Safety Groundedness: verifies if generated content is grounded in sources.
• Content provenance: Microsoft is working on watermarking images generated by DALL-E.
• C2PA (Coalition for Content Provenance and Authenticity): standard supported by Microsoft to trace content origin.

Enriched Section 9 – Pricing and Service Limits

Pricing Model

Most Azure AI services offer two tiers:

Quotas and Throttling by Service

Regional Availability

Not all services are available in all Azure regions. Always verify at aka.ms/azure-ai-regions.

Latency Considerations

Enriched Section 10 – Complete AI Solution Architecture

Reference Architecture Diagram

flowchart TB
    subgraph Clients
        A[Web / Mobile Application]
        B[Teams Bot / Webchat]
        C[Batch Job / ETL]
    end

    subgraph SecurityGovernance[Security and Governance]
        D[Azure API Management\nAuthentication, Throttling, Monitoring]
        E[Azure Entra ID\nOAuth2 / Managed Identity]
    end

    subgraph AzureAIServices[Azure AI Services]
        F[Azure OpenAI\nGPT-4o Chat]
        G[Azure AI Language\nNLP / CLU / QnA]
        H[Azure AI Speech\nSTT / TTS]
        I[Azure AI Vision\nOCR / Image Analysis]
        J[Azure Document Intelligence\nForm extraction]
        K[Azure AI Content Safety\nModeration]
    end

    subgraph StorageData[Storage and Data]
        L[Azure Blob Storage\nDocuments, Audio, Images]
        M[Azure Cosmos DB\nConversation history]
        N[Azure AI Search\nVector index RAG]
    end

    subgraph Observability
        O[Application Insights\nLogs, Metrics, Traces]
        P[Azure Monitor\nAlerts, Dashboards]
    end

    A --> D
    B --> D
    C --> D
    D --> E
    D --> F
    D --> G
    D --> H
    D --> I
    D --> J
    D --> K
    F --> N
    I --> L
    J --> L
    G --> M
    F --> M
    F --> O
    G --> O
    H --> O
    D --> O
    O --> P

Architecture Best Practices

Terraform Configuration Example (excerpt)

resource "azurerm_cognitive_account" "language" {
  name                = "ai-language-prod"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  kind                = "TextAnalytics"
  sku_name            = "S"

  identity {
    type = "SystemAssigned"
  }

  network_acls {
    default_action = "Deny"
    virtual_network_rules {
      subnet_id = azurerm_subnet.ai_subnet.id
    }
  }

  tags = {
    environment = "production"
    team        = "ai-platform"
  }
}

Enriched Section 11 – Review Questions

12 Questions Covering All Azure AI Services

Question 1: What is the difference between Azure AI Language and Azure OpenAI?

Answer: Azure AI Language offers specialized and deterministic NLP APIs (NER, sentiment, QnA, CLU) with predictable quotas and per-transaction pricing. Azure OpenAI gives access to large generative language models (GPT-4) for free-form text generation, reasoning, and complex understanding. Language is ideal for structured NLP tasks; OpenAI for creative generation and complex conversations.

Question 2: When is the “Neural” model used instead of “Template” in Document Intelligence?

Answer: The Template model suits documents with fixed, positional structure (e.g., always the same form with the same fields in the same places). The Neural model is preferred for documents with variable layouts (e.g., invoices from different vendors), as it understands semantic context without relying on fixed positions. The neural model requires more training data (50+ examples vs 5+).

Question 3: What is RAG and how does Azure AI Search contribute to it?

Answer: RAG (Retrieval-Augmented Generation) is a technique where a generative model (GPT) enriches its responses with data retrieved from an external source. Azure AI Search plays the role of the retriever: it indexes documents (with vector embeddings), searches for the most relevant passages for the user’s question, and passes them as context to the GPT model which generates the final response. This enables answering questions about private data without retraining the model.

Question 4: Which Face API features require Microsoft approval to access?

Answer: Face identification (recognizing a person in a known group), face verification (comparing two faces), detection of sensitive attributes (emotions, age), and liveness detection are all limited access features. Only basic face detection (locating faces in an image) remains publicly accessible. This restriction aims to prevent privacy and surveillance abuses.

Question 5: How does Conversational Language Understanding (CLU) work and how does it differ from LUIS?

Answer: CLU is LUIS’s successor. It detects intent and extracts entities from a natural language user utterance. The main difference: CLU uses modern transformers for better accuracy, supports native multilingualism (a single model for multiple languages), and integrates into Azure AI Language (same endpoint). LUIS required a separate endpoint and dedicated service.

Question 6: Describe the three regional availability modes to consider when deploying Azure OpenAI.

Answer: (1) Pay-As-You-Go (PAYG): immediate access in supported regions, shared quota, variable latency. (2) Provisioned Throughput Units (PTU): reserved throughput for predictable latency, ideal for production. (3) Global deployments: Microsoft automatically routes to the available region to maximize availability, useful for traffic peaks. Always check regional availability in the official documentation before choosing an architecture.

Question 7: What is the difference between Custom Speech and Custom Voice?

Answer: Custom Speech improves speech recognition (STT) by adapting the acoustic and language model to specific vocabulary or accent (e.g., medical terminology, regional accent). Custom Voice creates a unique synthetic voice (TTS) from recordings of a human speaker, producing a recognizable brand voice. Custom Speech = improving audio input; Custom Voice = personalizing audio output.

Question 8: What is Azure AI Foundry and what is the relationship between Hub and Project?

Answer: Azure AI Foundry (ai.azure.com) is the unified platform for developing AI applications on Azure. The Hub is a container of shared resources (service connections, storage, identity) that can be used by multiple projects. The Project is the isolated workspace for a developer or team, with access to the model catalog, playgrounds, fine-tuning, and deployments. A hub can contain multiple projects; each project inherits the hub’s connections.

Question 9: What are Microsoft’s 6 Responsible AI Principles?

Answer: (1) Fairness: no discrimination. (2) Reliability and safety: predictable and robust operation. (3) Privacy and security: data protection. (4) Inclusiveness: accessible to everyone. (5) Transparency: explainability of AI decisions. (6) Accountability: humans remain responsible for final decisions.

Question 10: How does the Bot Framework SDK differ from Copilot Studio for building a chatbot?

Answer: The Bot Framework SDK is a developer-oriented solution (C#/Python/JS) offering full control over bot logic, ideal for complex scenarios with multi-service integration. Copilot Studio (formerly Power Virtual Agents) is a low-code solution allowing business analysts to create bots without writing code, with native Microsoft 365 connectors. The choice depends on required complexity and team profile.

Question 11: In a complete AI architecture, why go through Azure API Management rather than calling AI services directly?

Answer: Azure API Management adds several essential layers: centralized authentication (API keys, OAuth2), throttling (limiting calls per user/app to control costs), monitoring (centralized logs, alerts), API versioning (migration without breaking changes), request/response transformation, and caching to reduce redundant calls. It also decouples clients from AI endpoints (if you change the model, only APIM changes, not the clients).

Question 12: What are the differences between real-time and batch text analysis for Azure AI Language, and when to use each approach?

Answer: Synchronous (real-time) analysis returns the result immediately for short to medium texts (max 5,120 characters per document). It is ideal for interactive applications requiring an instant response (chatbot, on-the-fly analysis). Asynchronous (batch) analysis via the analyze API allows processing longer documents, multiple documents simultaneously, and multiple operations in a single request (NER + sentiment + key phrases together). It is ideal for overnight batch processing, ETL, and large data volumes.

Visual Recap of Enriched Sections

mindmap
  root((Azure AI\nComplete Training))
    Portfolio
      9 service families
      Cognitive Services
      Azure OpenAI
      AI Foundry
    Language
      Text Analytics
      CLU replaces LUIS
      Question Answering
      Custom NER/Classification
    Voice
      STT real-time and batch
      TTS Neural + SSML
      Custom Speech and Voice
      Speaker Recognition limited
    Vision
      Image Analysis v4
      OCR Read API
      Face API limited access
      Custom Vision edge export
    Documents
      6 prebuilt models
      Layout and General
      Template vs Neural custom
    Bots
      SDK vs Copilot Studio
      8 deployment channels
      QnA and CLU integrated
    AI Foundry
      Hub and Project
      Model Catalog 6 families
      Fine-tuning JSONL
      Evaluation 6 metrics
    Responsible AI
      6 Microsoft principles
      Content Safety 4 categories
      Limited Access features
      Transparency Notes
    Pricing
      Free F0 vs Standard S0
      Throttling per service
      Key regions
      Latency and PTU
    Architecture
      Central APIM
      Managed Identity
      Private Endpoints
      App Insights
    Review
      12 questions
      All services covered

Search Terms

azure · ai · fundamentals · services · artificial · intelligence · generative · enriched · architecture · service · copilot · foundry · microsoft · search · access · custom · hub · model · api · bot · capabilities · categories · configuration · document