AI-900: Computer Vision Workloads on Azure

Certification: AI-900 Azure AI Fundamentals Level: Beginner to intermediate

Table of Contents

1. Introduction to Computer Vision
2. Computer Vision Capabilities on Azure
3. Azure AI Vision – In-Depth Image Analysis
4. Custom Vision – Building Custom Models
5. Azure AI Face – Facial Detection and Recognition
6. OCR – Read OCR Engine in Detail
7. Azure AI Document Intelligence
8. Azure Video Indexer
9. Practical Implementation with the Python SDK
10. Architecture and Deployment Considerations
11. Security, Ethics, and Responsible AI
12. Exam Tips and Common Pitfalls
13. Practical Exercises and Scenarios
14. Summary and Key Points
15. Glossary

1. Introduction to Computer Vision

1.1 What Is Computer Vision?

Computer Vision is a field of artificial intelligence that enables machines to “see” and interpret the visual world. Just as humans use their eyes and brain to understand what they see, computer vision systems use cameras, digital images, and machine learning algorithms to process and interpret visual data.

On Azure, Computer Vision is made possible through a series of dedicated cognitive services that allow developers to integrate advanced visual capabilities into their applications without requiring deep machine learning expertise.

mindmap
  root((Computer Vision))
    Image Analysis
      Captioning
      Tags
      Detected Objects
      Categories
    Facial Analysis
      Detection
      Attributes
      Verification
      Identification
    Text Extraction
      Image OCR
      Document OCR
      Document Intelligence
    Video
      Transcription
      Scene Detection
      Person Tracking
    Custom Models
      Custom Classification
      Custom Object Detection

1.2 The Evolution of Computer Vision

Before the advent of deep learning, computer vision systems relied on traditional image processing algorithms: edge detection, color histograms, descriptors like SIFT and HOG. These approaches were effective under controlled conditions but often failed in complex real-world scenarios.

With the introduction of convolutional neural networks (CNNs) in 2012 (AlexNet), vision model performance exploded. Today, foundation models like Florence (used by Azure AI Vision) offer generalized capabilities through training on billions of images and text-image pairs.

1.3 AI-900 Exam Positioning

Computer Vision represents approximately 15–20% of questions on the AI-900 exam. Key topics to master:

2. Computer Vision Capabilities on Azure

2.1 Service Overview

Azure provides a comprehensive ecosystem of computer vision services, organized in layers:

flowchart TB
    subgraph "Azure AI Services (Multi-service)"
        direction TB
        AIS["🔷 Azure AI Services\n(formerly Cognitive Services)\nSingle endpoint + key"]
    end
    
    subgraph "Specialized Services"
        direction LR
        AV["Azure AI Vision\nImage analysis + OCR"]
        CV["Azure AI Custom Vision\nCustom models"]
        AF["Azure AI Face\nFacial analysis"]
        DI["Azure AI Document Intelligence\nForms and documents"]
        VI["Azure Video Indexer\nVideo analysis"]
    end
    
    AIS --> AV
    AIS --> CV
    AIS --> AF
    AIS --> DI
    AIS --> VI
    
    AV --> A1["Caption / Dense Captions"]
    AV --> A2["Tags / Objects"]
    AV --> A3["Read OCR"]
    AV --> A4["Smart Crop"]
    
    CV --> C1["Custom Classification"]
    CV --> C2["Custom Object Detection"]
    
    AF --> F1["Facial Detection"]
    AF --> F2["Facial Analysis"]
    AF --> F3["Facial Verification"]
    
    DI --> D1["Prebuilt Models"]
    DI --> D2["Custom Models"]
    
    VI --> V1["Video Insights"]
    VI --> V2["Audio Insights"]

2.2 Service Comparison Table

2.3 Reference Architecture for a CV Application

flowchart LR
    USER["👤 User"] -->|Upload image| APP["Web Application\n(Azure App Service)"]
    APP -->|REST / SDK call| ENDPOINT["Azure AI Vision\nEndpoint"]
    ENDPOINT -->|JSON results| APP
    APP -->|Store results| DB["Azure Cosmos DB\nor Azure Storage"]
    APP -->|Display| USER
    
    subgraph "Security"
        KV["Azure Key Vault\n(API keys)"]
        MSI["Managed Identity"]
    end
    
    APP -.->|Retrieve key| KV
    APP -.->|Uses| MSI

3. Azure AI Vision – In-Depth Image Analysis

3.1 The Florence Model

Azure AI Vision is powered by Florence, a Microsoft foundation model trained on billions of text-image pairs. This model provides rich semantic understanding of images, enabling nuanced descriptions and precise classifications without fine-tuning.

Key concept: Florence is a multi-modal model that understands both natural language and images. This joint understanding enables capabilities like captioning and contextual tags.

3.2 Image Captioning and Dense Captions

Image Captioning generates a single sentence describing the image as a whole.

Dense Captions generates up to 10 descriptions of distinct regions of the image, each with its coordinates.

# Complete example with Azure AI Vision SDK v2 (Python)
from azure.ai.vision.imageanalysis import ImageAnalysisClient
from azure.ai.vision.imageanalysis.models import VisualFeatures
from azure.core.credentials import AzureKeyCredential
import os

# Configuration
endpoint = os.environ["AZURE_AI_VISION_ENDPOINT"]
key = os.environ["AZURE_AI_VISION_KEY"]

# Initialize the client
client = ImageAnalysisClient(
    endpoint=endpoint,
    credential=AzureKeyCredential(key)
)

# Analyze an image from a URL
result = client.analyze_from_url(
    image_url="https://example.com/sample-image.jpg",
    visual_features=[
        VisualFeatures.CAPTION,
        VisualFeatures.DENSE_CAPTIONS,
        VisualFeatures.TAGS,
        VisualFeatures.OBJECTS,
        VisualFeatures.READ
    ],
    language="en"  # Result language
)

# Display the main caption
if result.caption:
    print(f"Description: {result.caption.text}")
    print(f"Confidence: {result.caption.confidence:.3f}")

# Display dense captions
if result.dense_captions:
    print("\n=== Dense Captions ===")
    for caption in result.dense_captions.list:
        bb = caption.bounding_box
        print(f"  '{caption.text}' (confidence: {caption.confidence:.3f})")
        print(f"  Position: x={bb.x}, y={bb.y}, w={bb.width}, h={bb.height}")

# Display tags
if result.tags:
    print("\n=== Tags ===")
    for tag in result.tags.list:
        print(f"  {tag.name}: {tag.confidence:.3f}")

Sample raw JSON output from the REST API:

{
  "captionResult": {
    "text": "A golden retriever running in a lush green park",
    "confidence": 0.9456
  },
  "denseCaptionsResult": {
    "values": [
      {
        "text": "A golden dog running on the grass",
        "confidence": 0.9523,
        "boundingBox": { "x": 0, "y": 0, "w": 800, "h": 600 }
      },
      {
        "text": "Green trees in the background",
        "confidence": 0.8734,
        "boundingBox": { "x": 400, "y": 0, "w": 400, "h": 300 }
      },
      {
        "text": "A red leash on the ground",
        "confidence": 0.7234,
        "boundingBox": { "x": 100, "y": 400, "w": 200, "h": 100 }
      }
    ]
  }
}

3.3 Image Tags (Keywords)

Tags are descriptive keywords automatically extracted from the image. Unlike captioning which produces a full sentence, tags are individual terms ranked by descending confidence.

Tag characteristics:

• Can describe objects, concepts, actions, colors, materials
• Confidence score between 0 and 1
• Can be in the language of your choice (via language parameter)
• Around 200+ possible tag categories

# Analyze and filter tags by confidence threshold
def analyze_image_tags(image_url: str, confidence_threshold: float = 0.8) -> list[dict]:
    """
    Analyzes tags in an image and returns those above the threshold.
    
    Args:
        image_url: Public URL of the image to analyze
        confidence_threshold: Minimum confidence score (0.0 to 1.0)
    
    Returns:
        List of dictionaries {name, confidence}
    """
    client = ImageAnalysisClient(
        endpoint=os.environ["AZURE_AI_VISION_ENDPOINT"],
        credential=AzureKeyCredential(os.environ["AZURE_AI_VISION_KEY"])
    )
    
    result = client.analyze_from_url(
        image_url=image_url,
        visual_features=[VisualFeatures.TAGS],
        language="en"
    )
    
    filtered_tags = []
    if result.tags:
        for tag in result.tags.list:
            if tag.confidence >= confidence_threshold:
                filtered_tags.append({
                    "name": tag.name,
                    "confidence": round(tag.confidence, 4)
                })
    
    # Sort by descending confidence
    return sorted(filtered_tags, key=lambda x: x["confidence"], reverse=True)

# Usage
tags = analyze_image_tags("https://example.com/meal.jpg", confidence_threshold=0.7)
for tag in tags:
    print(f"  {tag['name']}: {tag['confidence']:.1%}")

3.4 Object Detection (Built-in)

The object detection built into Azure AI Vision identifies and localizes predefined objects in images, returning bounding boxes for each detected object.

Fundamental difference from Image Classification:

┌─────────────────────────────────────────────────────────┐
│                  IMAGE CLASSIFICATION                    │
│                                                         │
│  Input: street photo                                    │
│  Output: "This image contains a street with            │
│           vehicles" → single label for the image       │
│                                                         │
│  → No object localization                               │
│  → One (or several) label(s) for the entire image      │
└─────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────┐
│                   OBJECT DETECTION                      │
│                                                         │
│  Input: street photo                                    │
│  Output: [                                              │
│    { object: "car", x:120, y:80, w:200, h:150 },       │
│    { object: "person", x:340, y:60, w:80, h:200 },     │
│    { object: "bicycle", x:500, y:100, w:120, h:180 }   │
│  ]                                                      │
│                                                         │
│  → Precise localization with coordinates                │
│  → Multiple objects identified per image                │
└─────────────────────────────────────────────────────────┘

# Object detection with Azure AI Vision
from azure.ai.vision.imageanalysis import ImageAnalysisClient
from azure.ai.vision.imageanalysis.models import VisualFeatures
from azure.core.credentials import AzureKeyCredential
import json

def detect_objects(image_url: str) -> dict:
    """Detects and localizes objects in an image."""
    client = ImageAnalysisClient(
        endpoint=os.environ["AZURE_AI_VISION_ENDPOINT"],
        credential=AzureKeyCredential(os.environ["AZURE_AI_VISION_KEY"])
    )
    
    result = client.analyze_from_url(
        image_url=image_url,
        visual_features=[VisualFeatures.OBJECTS]
    )
    
    detected_objects = []
    if result.objects:
        for obj in result.objects.list:
            bb = obj.bounding_box
            object_info = {
                "name": obj.tags[0].name if obj.tags else "unknown",
                "confidence": round(obj.tags[0].confidence, 4) if obj.tags else 0.0,
                "position": {
                    "x": bb.x,
                    "y": bb.y,
                    "width": bb.width,
                    "height": bb.height
                }
            }
            detected_objects.append(object_info)
    
    return {
        "object_count": len(detected_objects),
        "objects": detected_objects
    }

# Example usage
results = detect_objects("https://example.com/street.jpg")
print(json.dumps(results, indent=2))

3.5 Smart Crop

Smart Crop generates an intelligent crop suggestion that preserves the most important area of the image according to a desired aspect ratio.

Use case: Create consistent thumbnails for an e-commerce catalog, adapt images to different display formats.

# Smart Crop with Azure AI Vision
result = client.analyze_from_url(
    image_url=image_url,
    visual_features=[VisualFeatures.SMART_CROPS],
    smart_crops_aspect_ratios=[0.9, 1.33, 1.78]  # Portrait, 4:3, 16:9
)

if result.smart_crops:
    for crop in result.smart_crops.list:
        bb = crop.bounding_box
        print(f"Ratio {crop.aspect_ratio:.2f}: "
              f"x={bb.x}, y={bb.y}, w={bb.width}, h={bb.height}")

3.6 Direct REST API Call

For environments without an SDK, here is how to call the REST API directly:

import requests
import json
import os

def analyze_image_rest(image_url: str) -> dict:
    """Direct call to the Azure AI Vision REST API."""
    
    endpoint = os.environ["AZURE_AI_VISION_ENDPOINT"]
    api_key = os.environ["AZURE_AI_VISION_KEY"]
    
    # v4.0 API URL
    api_url = f"{endpoint}/computervision/imageanalysis:analyze"
    
    # Request headers
    headers = {
        "Ocp-Apim-Subscription-Key": api_key,
        "Content-Type": "application/json"
    }
    
    # Request body
    body = {
        "url": image_url
    }
    
    # Query parameters
    params = {
        "api-version": "2024-02-01",
        "features": "caption,denseCaptions,tags,objects,read",
        "language": "en",
        "gender-neutral-caption": "true"
    }
    
    # Send request
    response = requests.post(
        api_url,
        headers=headers,
        params=params,
        json=body,
        timeout=30
    )
    
    # Check status
    response.raise_for_status()
    
    return response.json()

# Usage
results = analyze_image_rest("https://example.com/image.jpg")
print(json.dumps(results, indent=2))

4. Custom Vision – Building Custom Models

4.1 Why Custom Vision?

Azure AI Vision uses generic categories. But what if your company needs to recognize specific products, manufacturing defects, or any other category that doesn’t exist in generic models? That’s where Azure AI Custom Vision comes in.

flowchart TD
    QUESTION{"Do your categories\nexist in\nAzure AI Vision?"}
    QUESTION -->|Yes| AIVISION["Azure AI Vision\nPretrained model"]
    QUESTION -->|No| CUSTOMVISION["Azure AI Custom Vision\nTrain your own model"]
    
    CUSTOMVISION --> CLASSIFICATION["Classification\n(categorize the entire image)"]
    CUSTOMVISION --> DETECTION["Object Detection\n(locate objects)"]
    
    CLASSIFICATION --> CL1["Single-label\n(one category per image)"]
    CLASSIFICATION --> CL2["Multi-label\n(multiple categories)"]

4.2 Complete Custom Vision Workflow

flowchart LR
    A["1️⃣ Create\na project"] --> B["2️⃣ Upload\nimages"]
    B --> C["3️⃣ Annotate\nimages"]
    C --> D["4️⃣ Train\nthe model"]
    D --> E["5️⃣ Evaluate\nmetrics"]
    E --> F{Sufficient?}
    F -->|No| G["Add more\nimages or correct"]
    G --> B
    F -->|Yes| H["6️⃣ Publish\nthe endpoint"]
    H --> I["7️⃣ Integrate\ninto the app"]

4.3 Image Classification – Detailed Guide

Practical scenario: A grocery store wants to automatically identify fruits at its self-checkout machines.

Minimum requirements:

• 5 images minimum per category
• 2 categories minimum
• Images in JPEG, PNG, BMP, GIF, TIFF format
• Maximum size: 6 MB per image

Practical tip: For a reliable production model, aim for 50 to 100 images per category with good diversity (angles, lighting, varied backgrounds). With only 5 images, the model will be inaccurate.

# Creating a Custom Vision project with the Python SDK
from azure.cognitiveservices.vision.customvision.training import CustomVisionTrainingClient
from azure.cognitiveservices.vision.customvision.training.models import ImageFileCreateBatch, ImageFileCreateEntry, Tag
from msrest.authentication import ApiKeyCredentials
import os
import glob

# Configuration
training_endpoint = os.environ["CUSTOM_VISION_TRAINING_ENDPOINT"]
training_key = os.environ["CUSTOM_VISION_TRAINING_KEY"]

# Initialize the training client
credentials = ApiKeyCredentials(in_headers={"Training-key": training_key})
trainer = CustomVisionTrainingClient(training_endpoint, credentials)

# Retrieve available domains
domains = trainer.get_domains()
for domain in domains:
    print(f"  {domain.name}: {domain.id}")
# Example: Food, General, General (compact), Landmarks, Logo, Retail, ...

# Create a project ("Food" domain for fruits)
food_domain = next(d for d in domains if d.name == "Food")
project = trainer.create_project(
    name="Fruit-Classification",
    description="Classifies fruits for self-checkout machines",
    domain_id=food_domain.id,
    classification_type="Multiclass"  # or "Multilabel"
)
print(f"Project created: {project.name} (ID: {project.id})")

# Create tags (categories)
tag_apple  = trainer.create_tag(project.id, "apple")
tag_banana = trainer.create_tag(project.id, "banana")
tag_orange = trainer.create_tag(project.id, "orange")
print(f"Tags created: apple={tag_apple.id}, banana={tag_banana.id}, orange={tag_orange.id}")

# Upload images with their tags
def upload_images_from_folder(trainer, project_id, folder: str, tag: Tag):
    """Uploads all images in a folder with a given tag."""
    images_data = []
    
    for image_path in glob.glob(os.path.join(folder, "*.jpg")):
        with open(image_path, "rb") as f:
            image_data = f.read()
            images_data.append(
                ImageFileCreateEntry(
                    name=os.path.basename(image_path),
                    contents=image_data,
                    tag_ids=[tag.id]
                )
            )
    
    if images_data:
        batch = ImageFileCreateBatch(images=images_data)
        result = trainer.create_images_from_files(project_id, batch)
        if result.is_batch_successful:
            print(f"✅ {len(images_data)} images uploaded for '{tag.name}'")
        else:
            print(f"❌ Upload error for '{tag.name}'")
            for image in result.images:
                print(f"   {image.source_url}: {image.status}")

# Upload images
upload_images_from_folder(trainer, project.id, "./images/apples",  tag_apple)
upload_images_from_folder(trainer, project.id, "./images/bananas", tag_banana)
upload_images_from_folder(trainer, project.id, "./images/oranges", tag_orange)

# Train the model
print("\nStarting training...")
iteration = trainer.train_project(project.id)

# Wait for training to complete
import time
while iteration.status != "Completed":
    iteration = trainer.get_iteration(project.id, iteration.id)
    print(f"Status: {iteration.status}...")
    time.sleep(5)

print(f"✅ Training complete!")

# Publish the model
prediction_resource_id = os.environ["CUSTOM_VISION_PREDICTION_RESOURCE_ID"]
trainer.publish_iteration(
    project.id,
    iteration.id,
    "fruit-model-v1",
    prediction_resource_id
)
print("✅ Model published!")

4.4 Custom Object Detection – Detailed Guide

For Object Detection, annotation is more complex: you must draw bounding boxes around each object in each image.

# Programmatic annotation for Object Detection
from azure.cognitiveservices.vision.customvision.training.models import (
    ImageFileCreateEntry, Region
)

# Create an Object Detection project
project_od = trainer.create_project(
    name="Production-Defect-Detection",
    description="Detects defects on the production line",
    domain_id=next(d for d in domains if d.name == "General (compact)").id,
    target_export_platforms=["CoreML", "TensorFlow"]
)

# Create tags for defects
tag_defect = trainer.create_tag(project_od.id, "defect")
tag_ok     = trainer.create_tag(project_od.id, "part_ok")

# Upload with annotations (bounding boxes)
# Coordinates are normalized between 0 and 1 (percentage of the image)
with open("part_image_01.jpg", "rb") as f:
    image_data = f.read()

batch = ImageFileCreateBatch(images=[
    ImageFileCreateEntry(
        name="part_image_01.jpg",
        contents=image_data,
        regions=[
            Region(
                tag_id=tag_defect.id,
                left=0.23,    # 23% from the left
                top=0.15,     # 15% from the top
                width=0.12,   # 12% wide
                height=0.18   # 18% tall
            )
        ]
    )
])

result = trainer.create_images_from_files(project_od.id, batch)
print(f"Upload: {'success' if result.is_batch_successful else 'failed'}")

4.5 Evaluation Metrics

After training, Custom Vision provides detailed metrics:

flowchart LR
    subgraph "Confusion Matrix"
        TP["TP (True Positives)\nPredicted positive ✓"]
        FP["FP (False Positives)\nPredicted positive ✗"]
        FN["FN (False Negatives)\nPredicted negative ✗"]
        TN["TN (True Negatives)\nPredicted negative ✓"]
    end
    
    PRECISION["Precision\nTP / (TP + FP)\n\nOf all positive predictions,\nhow many are\ncorrect?"]
    RECALL["Recall\nTP / (TP + FN)\n\nOf all actual positive cases,\nhow many were\ndetected?"]
    AP["AP (Average Precision)\nArea under the\nPrecision-Recall curve"]
    
    TP --> PRECISION
    FP --> PRECISION
    TP --> RECALL
    FN --> RECALL
    PRECISION --> AP
    RECALL --> AP

Practical interpretation:

Scenario 1: High Precision, Low Recall
→ The model is "cautious": when it predicts, it's often correct
  but misses many real cases
→ Problem: false negatives (missing production defects = dangerous)

Scenario 2: Low Precision, High Recall
→ The model is "too generous": it detects a lot but with
  many false alarms
→ Problem: false positives (spurious alerts = costly)

Ideal scenario: Both must be > 0.85 for production use

4.6 Using the Published Model (Prediction)

# Using a Custom Vision model for prediction
from azure.cognitiveservices.vision.customvision.prediction import CustomVisionPredictionClient

prediction_endpoint = os.environ["CUSTOM_VISION_PREDICTION_ENDPOINT"]
prediction_key      = os.environ["CUSTOM_VISION_PREDICTION_KEY"]
project_id          = os.environ["CUSTOM_VISION_PROJECT_ID"]
model_name          = "fruit-model-v1"

# Initialize the prediction client
pred_credentials = ApiKeyCredentials(in_headers={"Prediction-key": prediction_key})
predictor = CustomVisionPredictionClient(prediction_endpoint, pred_credentials)

# Prediction from a local image
with open("fruit_test.jpg", "rb") as image_file:
    results = predictor.classify_image(
        project_id=project_id,
        published_name=model_name,
        image_data=image_file.read()
    )

# Display results sorted by probability
print("Classification results:")
for prediction in sorted(results.predictions, 
                          key=lambda x: x.probability, reverse=True):
    bar = "█" * int(prediction.probability * 20)
    print(f"  {prediction.tag_name:15}: {prediction.probability:6.1%}  {bar}")

# Prediction from a URL
results_url = predictor.classify_image_url(
    project_id=project_id,
    published_name=model_name,
    url="https://example.com/fruit.jpg"
)

5. Azure AI Face – Facial Detection and Recognition

5.1 Face Service Architecture

flowchart TD
    IMAGE["📷 Image / Video stream"] --> FACE_DETECT["Facial detection\n(locate faces)"]
    
    FACE_DETECT --> ATTRIB["Attribute analysis\n(accessories, quality...)"]
    FACE_DETECT --> VERIFY["Facial verification\n(same person?)"]
    FACE_DETECT --> IDENTIFY["Identification\n(who is this person?)"]
    FACE_DETECT --> GROUP["Grouping\n(similar faces)"]
    
    VERIFY --> VERIFY_OUT["isIdentical: bool\nconfidence: 0-1"]
    
    IDENTIFY --> PERSON_DB["Person database\nPersonGroup"]
    PERSON_DB --> IDENTIFY_OUT["Name + confidence"]
    
    GROUP --> GROUP_OUT["Clusters of similar\nfaces"]

5.2 Azure AI Vision vs Azure AI Face Comparison

⚠️ Important (exam + practice): Microsoft has restricted access to certain sensitive attributes of Azure AI Face following its responsible AI commitments. Attributes like emotion, estimated age, gender, and race require special approval. Freely available attributes include accessories (glasses, mask), image quality (blur, exposure, noise), head pose (pitch/roll/yaw angles), and occlusions.

5.3 Available Facial Attributes

# Facial detection and analysis with Azure AI Face
from azure.ai.vision.face import FaceClient
from azure.ai.vision.face.models import (
    FaceDetectionModel,
    FaceRecognitionModel,
    FaceAttributeTypeDetection03,
    QualityForRecognition
)
from azure.core.credentials import AzureKeyCredential

face_endpoint = os.environ["AZURE_FACE_ENDPOINT"]
face_key      = os.environ["AZURE_FACE_KEY"]

# Initialize the client
face_client = FaceClient(
    endpoint=face_endpoint,
    credential=AzureKeyCredential(face_key)
)

# Detect faces with attributes
with open("group_photo.jpg", "rb") as f:
    image_data = f.read()

faces = face_client.detect(
    image_content=image_data,
    detection_model=FaceDetectionModel.DETECTION03,
    recognition_model=FaceRecognitionModel.RECOGNITION04,
    return_face_id=True,
    return_face_landmarks=True,
    return_face_attributes=[
        FaceAttributeTypeDetection03.HEAD_POSE,
        FaceAttributeTypeDetection03.GLASSES,
        FaceAttributeTypeDetection03.BLUR,
        FaceAttributeTypeDetection03.EXPOSURE,
        FaceAttributeTypeDetection03.NOISE,
        FaceAttributeTypeDetection03.MASK,
        FaceAttributeTypeDetection03.QUALITY_FOR_RECOGNITION,
        FaceAttributeTypeDetection03.OCCLUSION,
    ]
)

print(f"Faces detected: {len(faces)}")

for i, face in enumerate(faces):
    print(f"\n=== Face {i+1} (ID: {face.face_id}) ===")
    
    # Face position
    rect = face.face_rectangle
    print(f"  Position: top={rect.top}, left={rect.left}, "
          f"width={rect.width}, height={rect.height}")
    
    # Attributes
    attrs = face.face_attributes
    if attrs:
        # Head pose (degrees)
        pose = attrs.head_pose
        print(f"  Head pose: pitch={pose.pitch:.1f}°, "
              f"roll={pose.roll:.1f}°, yaw={pose.yaw:.1f}°")
        
        # Quality for recognition
        print(f"  Quality: {attrs.quality_for_recognition}")
        
        # Accessories
        print(f"  Glasses: {attrs.glasses}")
        
        # Image blur
        blur = attrs.blur
        print(f"  Blur: level={blur.blur_level}, value={blur.value:.3f}")
        
        # Exposure
        exposure = attrs.exposure
        print(f"  Exposure: {exposure.exposure_level}")
        
        # Occlusion
        occlusion = attrs.occlusion
        print(f"  Occlusion: forehead={occlusion.forehead_occluded}, "
              f"eyes={occlusion.eye_occluded}, mouth={occlusion.mouth_occluded}")
        
        # Mask
        mask = attrs.mask
        print(f"  Mask: {mask.type} (nose & mouth covered: {mask.nose_and_mouth_covered})")

5.4 Facial Verification

Verification answers the question: “Do these two photos show the same person?“

# Facial verification: compare two faces
def verify_same_person(photo_path_1: str, photo_path_2: str) -> dict:
    """
    Compares two photos to determine if they are the same person.
    
    Returns:
        dict with is_identical (bool) and confidence (float)
    """
    
    # Detect face in photo 1
    with open(photo_path_1, "rb") as f:
        faces_1 = face_client.detect(
            image_content=f.read(),
            detection_model=FaceDetectionModel.DETECTION03,
            recognition_model=FaceRecognitionModel.RECOGNITION04,
            return_face_id=True
        )
    
    # Detect face in photo 2
    with open(photo_path_2, "rb") as f:
        faces_2 = face_client.detect(
            image_content=f.read(),
            detection_model=FaceDetectionModel.DETECTION03,
            recognition_model=FaceRecognitionModel.RECOGNITION04,
            return_face_id=True
        )
    
    if not faces_1 or not faces_2:
        raise ValueError("No face detected in one of the photos")
    
    # Verification
    verification = face_client.verify_face_to_face(
        face_id1=faces_1[0].face_id,
        face_id2=faces_2[0].face_id
    )
    
    return {
        "same_person": verification.is_identical,
        "confidence": round(verification.confidence, 4),
        "conclusion": "✅ Same person" if verification.is_identical else "❌ Different people"
    }

# Typical usage: identity card verification
result = verify_same_person("customer_selfie.jpg", "id_card_photo.jpg")
print(f"Result: {result['conclusion']} (confidence: {result['confidence']:.1%})")

5.5 Person Groups – Person Identification

The identification workflow requires creating a database of known persons in advance:

sequenceDiagram
    participant DEV as Developer
    participant FACE as Azure AI Face
    participant DB as PersonGroup Database
    
    Note over DEV,DB: Training phase
    DEV->>FACE: CreatePersonGroup("employees")
    FACE->>DB: Creates the group
    DEV->>FACE: CreatePerson("Alice")
    FACE->>DB: Adds person
    DEV->>FACE: AddPersonFace(Alice, photo1.jpg)
    DEV->>FACE: AddPersonFace(Alice, photo2.jpg)
    DEV->>FACE: AddPersonFace(Alice, photo3.jpg)
    DEV->>FACE: TrainPersonGroup("employees")
    FACE->>DB: Trains the model
    
    Note over DEV,DB: Identification phase
    DEV->>FACE: Detect(unknown_photo.jpg)
    FACE-->>DEV: faceId = "xyz-123"
    DEV->>FACE: Identify(faceId, "employees")
    FACE->>DB: Searches in the group
    FACE-->>DEV: person Alice confidence 0.94

6. OCR – Read OCR Engine in Detail

6.1 Read OCR Engine Foundations

The Read OCR Engine is Azure’s optical character recognition engine, based on a deep learning model that far surpasses traditional OCR in terms of accuracy, especially on:

• Handwritten text (cursive, printed)
• Poor quality images (blur, angle, illumination)
• Dense documents with multiple columns
• Text in natural scenes (signs, packaging)

6.2 Two Usage Modes

flowchart TD
    OCR["Read OCR Engine"] --> IMG["Image Mode\n(Image Edition)"]
    OCR --> DOC["Document Mode\n(Document Edition)"]
    
    IMG --> IMG_USE["Use cases:\n• Handwritten notes\n• Sign photos\n• Screenshots\n• Text in natural scenes"]
    
    DOC --> DOC_USE["Use cases:\n• Scanned PDFs\n• Long articles\n• Multi-page reports\n• Dense forms"]
    
    IMG --> IMG_LIMIT["Limits:\n• 50 pages max\n• Optimized for sparse text"]
    DOC --> DOC_LIMIT["Advantages:\n• Handles layout\n• Recognizes columns\n• Handles tables"]

6.3 Complete OCR Response Structure

{
  "status": "succeeded",
  "createdDateTime": "2024-01-15T10:30:00Z",
  "lastUpdatedDateTime": "2024-01-15T10:30:02Z",
  "analyzeResult": {
    "version": "3.2",
    "modelVersion": "2022-04-30",
    "readResults": [
      {
        "page": 1,
        "angle": -1.5,
        "width": 1600,
        "height": 900,
        "unit": "pixel",
        "lines": [
          {
            "boundingBox": [100, 50, 500, 50, 500, 90, 100, 90],
            "text": "Make today an exceptional day",
            "words": [
              {
                "boundingBox": [100, 52, 185, 52, 185, 88, 100, 88],
                "text": "Make",
                "confidence": 0.999
              },
              {
                "boundingBox": [192, 52, 250, 52, 250, 88, 192, 88],
                "text": "today",
                "confidence": 0.998
              }
            ]
          }
        ]
      }
    ]
  }
}

Structure anatomy:

• page: page number (starts at 1)
• angle: page rotation in degrees
• unit: “pixel” or “inch” depending on input
• lines: list of detected lines
  • boundingBox: 8 values = coordinates of the 4 corners of the rectangle
  • text: text recognized on this line
  • words: list of words with their individual positions

6.4 Complete OCR Implementation

# Complete OCR with Azure AI Vision (Read API)
import time
import requests
import json
import os
from pathlib import Path

class AzureOCR:
    """OCR client for Azure AI Vision."""
    
    def __init__(self):
        self.endpoint = os.environ["AZURE_AI_VISION_ENDPOINT"]
        self.key      = os.environ["AZURE_AI_VISION_KEY"]
        self.headers  = {
            "Ocp-Apim-Subscription-Key": self.key
        }
    
    def extract_text(self, source: str, is_url: bool = True) -> dict:
        """
        Extracts text from an image or document.
        
        Args:
            source: URL or file path
            is_url: True if source is a URL, False if it's a local file
        
        Returns:
            Structured dictionary with pages, lines and words
        """
        if is_url:
            submit_url = f"{self.endpoint}/vision/v3.2/read/analyze"
            headers = {**self.headers, "Content-Type": "application/json"}
            body = {"url": source}
            response = requests.post(submit_url, headers=headers, json=body)
        else:
            submit_url = f"{self.endpoint}/vision/v3.2/read/analyze"
            with open(source, "rb") as f:
                image_data = f.read()
            
            ext = Path(source).suffix.lower()
            content_types = {
                ".jpg": "image/jpeg",
                ".jpeg": "image/jpeg",
                ".png": "image/png",
                ".pdf": "application/pdf",
                ".tiff": "image/tiff"
            }
            headers = {
                **self.headers,
                "Content-Type": content_types.get(ext, "application/octet-stream")
            }
            response = requests.post(submit_url, headers=headers, data=image_data)
        
        response.raise_for_status()
        operation_url = response.headers.get("Operation-Location")
        
        if not operation_url:
            raise ValueError("No Operation-Location in response")
        
        # Poll for results
        max_attempts = 30
        for _ in range(max_attempts):
            time.sleep(1)
            result_response = requests.get(operation_url, headers=self.headers)
            result_response.raise_for_status()
            result = result_response.json()
            
            if result["status"] == "succeeded":
                break
            elif result["status"] == "failed":
                raise Exception(f"OCR failed: {result.get('error', {}).get('message')}")
        else:
            raise TimeoutError("OCR timeout after 30 attempts")
        
        return self._structure_results(result)
    
    def _structure_results(self, raw_result: dict) -> dict:
        """Transforms raw results into a readable structure."""
        results = {"full_text": "", "pages": []}
        text_lines = []
        
        for page_data in raw_result["analyzeResult"]["readResults"]:
            page = {
                "number": page_data["page"],
                "angle": page_data.get("angle", 0),
                "lines": []
            }
            
            for line_data in page_data.get("lines", []):
                line = {
                    "text": line_data["text"],
                    "bounding_box": line_data["boundingBox"],
                    "words": [
                        {
                            "text": word["text"],
                            "confidence": word.get("confidence", 0.0)
                        }
                        for word in line_data.get("words", [])
                    ]
                }
                page["lines"].append(line)
                text_lines.append(line_data["text"])
            
            results["pages"].append(page)
        
        results["full_text"]   = "\n".join(text_lines)
        results["page_count"]  = len(results["pages"])
        results["line_count"]  = sum(len(p["lines"]) for p in results["pages"])
        
        return results

# Usage
ocr = AzureOCR()
results = ocr.extract_text("https://example.com/document.jpg", is_url=True)
print(f"Pages: {results['page_count']}, Lines: {results['line_count']}")
print(results["full_text"])

7. Azure AI Document Intelligence

7.1 Overview

Azure AI Document Intelligence (formerly Form Recognizer) goes beyond simple OCR: it understands the semantics of documents, meaning the meaning of extracted fields.

flowchart LR
    DOC["📄 Document\n(receipt, invoice, ID...)"] --> DI["Azure AI\nDocument Intelligence"]
    
    DI --> OCR_LAYER["OCR Layer\n(raw text)"]
    DI --> LAYOUT["Layout Layer\n(structure: tables, fields)"]
    DI --> MODEL["Model Layer\n(semantics: field type)"]
    
    MODEL --> OUTPUT["📊 Structured data\n{ merchant: 'Taco House',\n  total: 42.50,\n  tax: 5.75,\n  date: '2024-01-15' }"]

7.2 Available Prebuilt Models

7.3 Using the Python SDK

# Azure AI Document Intelligence - Complete Extraction
from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.ai.documentintelligence.models import AnalyzeDocumentRequest
from azure.core.credentials import AzureKeyCredential
import os

di_endpoint = os.environ["AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT"]
di_key      = os.environ["AZURE_DOCUMENT_INTELLIGENCE_KEY"]

di_client = DocumentIntelligenceClient(
    endpoint=di_endpoint,
    credential=AzureKeyCredential(di_key)
)

# Analyze a receipt
def analyze_receipt(file_path: str) -> dict:
    """Extracts structured information from a receipt."""
    with open(file_path, "rb") as f:
        poller = di_client.begin_analyze_document(
            model_id="prebuilt-receipt",
            analyze_request=f.read(),
            content_type="image/jpeg"
        )
    
    result = poller.result()
    data = {}
    
    for receipt in result.documents:
        for field_name, field in receipt.fields.items():
            if field:
                data[field_name] = {
                    "value": str(field.value or field.content),
                    "confidence": round(field.confidence or 0, 4)
                }
                print(f"  {field_name}: {field.value or field.content} ({field.confidence:.1%})")
    
    return data

# Analyze an invoice
def analyze_invoice(invoice_url: str) -> dict:
    """Extracts data from a commercial invoice."""
    poller = di_client.begin_analyze_document(
        model_id="prebuilt-invoice",
        analyze_request=AnalyzeDocumentRequest(url_source=invoice_url)
    )
    
    result = poller.result()
    data = {}
    
    for invoice in result.documents:
        for field_name, field in invoice.fields.items():
            if field and (field.value or field.content):
                data[field_name] = str(field.value or field.content)
    
    return data

# Usage
print("=== Receipt Analysis ===")
receipt_data = analyze_receipt("restaurant_receipt.jpg")

print("\n=== Invoice Analysis ===")
invoice_data = analyze_invoice("https://example.com/invoice.pdf")
print(f"Vendor: {invoice_data.get('VendorName', 'N/A')}")
print(f"Total amount: {invoice_data.get('TotalAmount', 'N/A')}")
print(f"Invoice number: {invoice_data.get('InvoiceId', 'N/A')}")

8. Azure Video Indexer

8.1 Service Architecture

flowchart TD
    VIDEO["🎬 Video (file or URL)"] --> VI["Azure Video Indexer\nhttps://www.videoindexer.ai"]
    
    VI --> VISION_ENGINE["🔍 Vision Engine"]
    VI --> AUDIO_ENGINE["🔊 Audio Engine"]
    
    VISION_ENGINE --> V1["Face detection\nand celebrity recognition"]
    VISION_ENGINE --> V2["OCR on video"]
    VISION_ENGINE --> V3["Content moderation"]
    VISION_ENGINE --> V4["Scene detection"]
    VISION_ENGINE --> V5["Person tracking"]
    VISION_ENGINE --> V6["Object labels"]
    
    AUDIO_ENGINE --> A1["Transcription"]
    AUDIO_ENGINE --> A2["Language detection"]
    AUDIO_ENGINE --> A3["Translation"]
    AUDIO_ENGINE --> A4["Emotion detection"]
    AUDIO_ENGINE --> A5["Keyword extraction"]

8.2 Detailed Capabilities

Video insights:

Audio insights:

8.3 Using the REST API

# Using the Video Indexer API
import requests
import json
import time
import os

class VideoIndexerClient:
    """Client for Azure Video Indexer."""
    
    BASE_URL = "https://api.videoindexer.ai"
    
    def __init__(self, account_id: str, location: str, api_key: str):
        self.account_id = account_id
        self.location   = location
        self.api_key    = api_key
    
    def _get_access_token(self) -> str:
        url = (f"{self.BASE_URL}/auth/{self.location}"
               f"/Accounts/{self.account_id}/AccessToken")
        headers = {"Ocp-Apim-Subscription-Key": self.api_key}
        response = requests.get(url, headers=headers)
        response.raise_for_status()
        return response.text.strip('"')
    
    def submit_video_url(self, video_url: str, video_name: str,
                          language: str = "en-US") -> str:
        """Submits a video for indexing. Returns the video ID."""
        token = self._get_access_token()
        url = (f"{self.BASE_URL}/{self.location}/Accounts/{self.account_id}"
               f"/Videos?name={video_name}&videoUrl={video_url}"
               f"&language={language}&accessToken={token}")
        
        response = requests.post(url)
        response.raise_for_status()
        return response.json()["id"]
    
    def wait_for_indexing(self, video_id: str, timeout_sec: int = 600) -> dict:
        """Waits for indexing to complete and returns insights."""
        token = self._get_access_token()
        start = time.time()
        
        while time.time() - start < timeout_sec:
            url = (f"{self.BASE_URL}/{self.location}/Accounts/{self.account_id}"
                   f"/Videos/{video_id}/Index?accessToken={token}")
            response = requests.get(url)
            response.raise_for_status()
            data = response.json()
            
            state = data.get("state", "Processing")
            print(f"  Status: {state}...")
            
            if state == "Processed":
                return data
            elif state == "Failed":
                raise Exception(f"Indexing failed: {data.get('failureMessage')}")
            
            time.sleep(10)
        
        raise TimeoutError("Video indexing timeout")
    
    def extract_transcription(self, insights: dict) -> list[dict]:
        """Extracts formatted transcription."""
        transcription = []
        try:
            for block in insights["videos"][0]["insights"]["transcript"]:
                transcription.append({
                    "text": block["text"],
                    "start": block["instances"][0]["start"],
                    "end":   block["instances"][0]["end"]
                })
        except (KeyError, IndexError):
            pass
        return transcription

# Usage
vi_client = VideoIndexerClient(
    account_id=os.environ["VIDEO_INDEXER_ACCOUNT_ID"],
    location=os.environ["VIDEO_INDEXER_LOCATION"],
    api_key=os.environ["VIDEO_INDEXER_API_KEY"]
)

video_id = vi_client.submit_video_url(
    video_url="https://example.com/presentation.mp4",
    video_name="Azure AI Presentation 2024",
    language="en-US"
)
print(f"Video submitted. ID: {video_id}")

insights       = vi_client.wait_for_indexing(video_id)
transcription  = vi_client.extract_transcription(insights)

print("\n=== Transcription ===")
for segment in transcription[:5]:
    print(f"[{segment['start']} → {segment['end']}] {segment['text']}")

9. Practical Implementation with the Python SDK

9.1 Installing Dependencies

# Install Azure AI packages for Computer Vision
pip install azure-ai-vision-imageanalysis
pip install azure-ai-vision-face
pip install azure-cognitiveservices-vision-customvision
pip install azure-ai-documentintelligence
pip install azure-identity
pip install pillow
pip install requests

9.2 Secure Credential Management

# Recommended approach: Azure Key Vault + Managed Identity
from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient
import os

class AzureAIConfiguration:
    """Secure Azure AI credential management."""
    
    def __init__(self, use_keyvault: bool = True):
        self.use_keyvault = use_keyvault
        self._client = None
        
        if use_keyvault:
            keyvault_url = os.environ.get("AZURE_KEYVAULT_URL")
            if keyvault_url:
                credential = DefaultAzureCredential()
                self._client = SecretClient(
                    vault_url=keyvault_url,
                    credential=credential
                )
    
    def get_secret(self, secret_name: str) -> str:
        """Retrieves a secret from Key Vault or environment variables."""
        if self._client:
            try:
                return self._client.get_secret(secret_name).value
            except Exception:
                pass
        
        value = os.environ.get(secret_name)
        if not value:
            raise ValueError(f"Secret '{secret_name}' not found")
        return value
    
    @property
    def vision_endpoint(self) -> str:
        return self.get_secret("AZURE-AI-VISION-ENDPOINT")
    
    @property
    def vision_key(self) -> str:
        return self.get_secret("AZURE-AI-VISION-KEY")

9.3 Complete Application: Product Catalog Analysis System

# Complete image analysis application for e-commerce catalog
import os
import json
from pathlib import Path
from dataclasses import dataclass, asdict
from typing import Optional
from azure.ai.vision.imageanalysis import ImageAnalysisClient
from azure.ai.vision.imageanalysis.models import VisualFeatures
from azure.core.credentials import AzureKeyCredential

@dataclass
class ProductAnalysis:
    """Result of a product analysis."""
    image_path: str
    description: Optional[str] = None
    description_confidence: float = 0.0
    tags: list = None
    objects: list = None
    detected_text: Optional[str] = None
    
    def __post_init__(self):
        if self.tags is None:
            self.tags = []
        if self.objects is None:
            self.objects = []

class AICatalogAnalyzer:
    """Automatic image analysis for catalog using Azure AI Vision."""
    
    def __init__(self):
        self.client = ImageAnalysisClient(
            endpoint=os.environ["AZURE_AI_VISION_ENDPOINT"],
            credential=AzureKeyCredential(os.environ["AZURE_AI_VISION_KEY"])
        )
    
    def analyze_image(self, image_path: str) -> ProductAnalysis:
        """Complete analysis of a product image."""
        analysis = ProductAnalysis(image_path=image_path)
        
        with open(image_path, "rb") as f:
            image_data = f.read()
        
        try:
            result = self.client.analyze(
                image_data=image_data,
                visual_features=[
                    VisualFeatures.CAPTION,
                    VisualFeatures.TAGS,
                    VisualFeatures.OBJECTS,
                    VisualFeatures.READ
                ],
                language="en"
            )
            
            if result.caption:
                analysis.description = result.caption.text
                analysis.description_confidence = result.caption.confidence
            
            if result.tags:
                analysis.tags = [
                    {"name": tag.name, "confidence": round(tag.confidence, 3)}
                    for tag in result.tags.list
                    if tag.confidence > 0.7
                ]
            
            if result.objects:
                analysis.objects = [
                    {
                        "name": obj.tags[0].name if obj.tags else "unknown",
                        "confidence": round(obj.tags[0].confidence, 3) if obj.tags else 0
                    }
                    for obj in result.objects.list
                ]
            
            if result.read and result.read.blocks:
                texts = []
                for block in result.read.blocks:
                    for line in block.lines:
                        texts.append(line.text)
                analysis.detected_text = " | ".join(texts) if texts else None
        
        except Exception as e:
            print(f"⚠️ Error analyzing {image_path}: {e}")
        
        return analysis
    
    def analyze_folder(self, folder: str) -> list[ProductAnalysis]:
        """Analyzes all images in a folder."""
        images = [
            str(p) for p in Path(folder).iterdir()
            if p.suffix.lower() in [".jpg", ".jpeg", ".png"]
        ]
        
        print(f"📸 {len(images)} images found")
        results = []
        
        for i, path in enumerate(images, 1):
            print(f"  [{i}/{len(images)}] {Path(path).name}...")
            analysis = self.analyze_image(path)
            results.append(analysis)
        
        return results
    
    def export_catalog(self, analyses: list[ProductAnalysis],
                        output_file: str = "catalog.json") -> None:
        """Exports analyses to JSON."""
        catalog = {
            "total_products": len(analyses),
            "products": [asdict(a) for a in analyses]
        }
        with open(output_file, "w", encoding="utf-8") as f:
            json.dump(catalog, f, ensure_ascii=False, indent=2)
        print(f"✅ Catalog exported: {output_file}")

# Main script
if __name__ == "__main__":
    analyzer = AICatalogAnalyzer()
    results  = analyzer.analyze_folder("./catalog_images/")
    analyzer.export_catalog(results, "enriched_catalog.json")

10. Architecture and Deployment Considerations

10.1 Azure AI Vision Pricing Tiers

10.2 Error Handling and Resilience

# Robust error handling with exponential retry
import time
from azure.core.exceptions import HttpResponseError, ServiceRequestError

def analyze_with_retry(client, image_url: str,
                        max_attempts: int = 3,
                        base_delay: float = 1.0):
    """Analysis with error handling and exponential retry."""
    
    for attempt in range(1, max_attempts + 1):
        try:
            result = client.analyze_from_url(
                image_url=image_url,
                visual_features=[VisualFeatures.CAPTION, VisualFeatures.TAGS]
            )
            return result
        
        except HttpResponseError as e:
            if e.status_code == 429:  # Rate limit
                delay = base_delay * (2 ** (attempt - 1))
                print(f"  ⏳ Rate limit. Waiting {delay}s... "
                      f"(attempt {attempt}/{max_attempts})")
                time.sleep(delay)
            elif e.status_code in [400, 415]:
                print(f"  ❌ Invalid image: {e.message}")
                return None
            elif e.status_code >= 500:
                if attempt < max_attempts:
                    time.sleep(base_delay * attempt)
                else:
                    return None
            else:
                return None
        
        except ServiceRequestError:
            if attempt < max_attempts:
                time.sleep(base_delay * attempt)
            else:
                return None
    
    return None

11. Security, Ethics, and Responsible AI

11.1 Microsoft’s Responsible AI Principles

Microsoft applies 6 core principles to all its AI services:

flowchart TD
    subgraph "6 Responsible AI Principles"
        FAIR["🤝 Fairness\nNo discriminatory bias"]
        REL["🛡️ Reliability and Safety\nPerforms in all scenarios"]
        PRIV["🔒 Privacy\nData protection"]
        INC["♿ Inclusiveness\nAccessible to everyone"]
        TRANS["🔍 Transparency\nExplainable and understandable"]
        ACC["✅ Accountability\nClear responsibility"]
    end

11.2 Access Restrictions on Sensitive Features

11.3 Security Best Practices

# ✅ GOOD: Use Managed Identity (no keys in code)
from azure.identity import DefaultAzureCredential
from azure.ai.vision.imageanalysis import ImageAnalysisClient

credential = DefaultAzureCredential()
client = ImageAnalysisClient(
    endpoint=os.environ["AZURE_AI_VISION_ENDPOINT"],
    credential=credential
)

# ✅ GOOD: Validate inputs before sending to the API
def validate_image(path: str, max_size_mb: float = 4.0) -> bool:
    """Validates an image before sending to the API."""
    import mimetypes
    
    if not os.path.exists(path):
        raise FileNotFoundError(f"Image not found: {path}")
    
    size_mb = os.path.getsize(path) / (1024 * 1024)
    if size_mb > max_size_mb:
        raise ValueError(f"Image too large: {size_mb:.1f}MB > {max_size_mb}MB")
    
    mime_type, _ = mimetypes.guess_type(path)
    allowed_types = ["image/jpeg", "image/png", "image/bmp", "image/gif", "image/tiff"]
    if mime_type not in allowed_types:
        raise ValueError(f"Unsupported file type: {mime_type}")
    
    return True

# ❌ BAD: Hardcoded keys in the code
# credential = AzureKeyCredential("abc123...")  # Never do this!

12. Exam Tips and Common Pitfalls

12.1 Critical Distinctions for the Exam

flowchart TD
    Q1{"Analyze generic\nimages?"}
    Q1 -->|Yes| AIVISION["Azure AI Vision\n✓ Caption, tags, objects\n✓ OCR Read engine"]
    Q1 -->|No, custom\ncategories| CUSTOMVISION["Azure AI Custom Vision\n✓ Your own categories"]
    
    Q2{"Analyze faces?"}
    Q2 -->|Just detect| AIVISION2["Azure AI Vision\n✓ Basic face detection"]
    Q2 -->|Advanced analysis| AIFACE["Azure AI Face\n✓ Attributes, verification, ID"]
    
    Q3{"Extract text?"}
    Q3 -->|Photos, notes| OCR["Azure AI Vision\nRead OCR - Images Edition"]
    Q3 -->|Scanned documents| OCRDOC["Azure AI Vision\nRead OCR - Documents Edition"]
    Q3 -->|Understand forms| DI["Azure AI Document Intelligence"]
    
    Q4{"Analyze videos?"}
    Q4 -->|Complete insights| VI["Azure Video Indexer"]

12.2 Pitfall Summary Table

12.3 Sample Exam Questions

Q1: A company wants to extract the total amount from scanned receipts. Which service?

A: Azure AI Document Intelligence with prebuilt-receipt.

Q2: A developer wants to recognize their own products (10 categories). Which service?

A: Azure AI Custom Vision – classification project (5 images min per category).

Q3: Difference between Azure AI Face and Azure AI Vision for faces?

A: Vision = basic detection (position). Face = advanced analysis (attributes, verification, Person Groups).

Q4: “Simplify administration” with a single endpoint for multiple AI services?

A: Azure AI Services (formerly Cognitive Services) – multi-service account.

12.4 Final Pre-Exam Checklist

✅ Know the 5 Azure CV services (Vision, Custom Vision, Face, Document Intelligence, Video Indexer)
✅ Understand Detection vs Recognition (facial)
✅ Know when to use Image Classification vs Object Detection
✅ Know Custom Vision requirements (5 images/tag, 2 tags)
✅ Understand Precision and Recall
✅ Know that Cognitive Services = Azure AI Services
✅ Know the prebuilt Document Intelligence models
✅ Understand both video AND audio capabilities of Video Indexer
✅ Know Microsoft's 6 Responsible AI principles
✅ Know that some Face attributes are restricted for ethical reasons

13. Practical Exercises and Scenarios

13.1 Scenario 1: Automated Mail Sorting

Context: Automate the classification and data extraction from thousands of paper documents.

flowchart LR
    SCAN["📄 Scan"] --> BLOB["Azure Blob Storage"]
    BLOB --> FUNC["Azure Function\n(upload trigger)"]
    FUNC --> DI["Azure AI\nDocument Intelligence"]
    DI --> DETECT["Type detection"]
    DETECT --> INVOICE["prebuilt-invoice"]
    DETECT --> RECEIPT["prebuilt-receipt"]
    DETECT --> OTHER["prebuilt-document"]
    INVOICE --> DB["Azure Cosmos DB"]
    RECEIPT --> DB
    OTHER --> DB

13.2 Scenario 2: E-Commerce Content Moderation

Context: Automatically validate product images uploaded by sellers.

# Content moderation with Azure AI Vision
from azure.ai.vision.imageanalysis import ImageAnalysisClient
from azure.ai.vision.imageanalysis.models import VisualFeatures
from azure.core.credentials import AzureKeyCredential
import os

def moderate_product_image(image_data: bytes) -> dict:
    """
    Validates a product image before publishing.
    Checks: quality, relevance, appropriate content.
    """
    client = ImageAnalysisClient(
        endpoint=os.environ["AZURE_AI_VISION_ENDPOINT"],
        credential=AzureKeyCredential(os.environ["AZURE_AI_VISION_KEY"])
    )
    
    result = client.analyze(
        image_data=image_data,
        visual_features=[
            VisualFeatures.CAPTION,
            VisualFeatures.TAGS,
            VisualFeatures.OBJECTS
        ]
    )
    
    report = {
        "approved": True,
        "rejection_reasons": [],
        "description": None,
        "tags": []
    }
    
    # Check description
    if result.caption:
        report["description"] = result.caption.text
        
        # Check minimum confidence
        if result.caption.confidence < 0.5:
            report["approved"] = False
            report["rejection_reasons"].append("Image too blurry or ambiguous")
    
    # Extract tags
    if result.tags:
        high_confidence_tags = [
            t.name for t in result.tags.list
            if t.confidence > 0.8
        ]
        report["tags"] = high_confidence_tags
        
        # Check for inappropriate content
        banned_words = ["nudity", "violence", "adult", "weapon", "drug"]
        for tag in high_confidence_tags:
            if any(banned in tag.lower() for banned in banned_words):
                report["approved"] = False
                report["rejection_reasons"].append(f"Inappropriate content detected: {tag}")
    
    return report

# Usage
with open("product_photo.jpg", "rb") as f:
    image_bytes = f.read()

report = moderate_product_image(image_bytes)

if report["approved"]:
    print(f"✅ Image approved: {report['description']}")
else:
    print(f"❌ Image rejected:")
    for reason in report["rejection_reasons"]:
        print(f"  - {reason}")

14. Summary and Key Points

14.1 Complete Service → Capabilities Mapping

mindmap
  root((Azure CV Services))
    Azure AI Vision
      Image captioning
      Dense captions
      Image tags
      Object detection
      Read OCR Images
      Read OCR Documents
      Smart crop
    Azure AI Custom Vision
      Custom classification
        Single-label
        Multi-label
      Custom object detection
      Export model CoreML TensorFlow ONNX
    Azure AI Face
      Facial detection
      Face attributes
        Glasses
        Head pose
        Blur exposure noise
        Mask
        Quality
      Facial verification
      Person Groups identification
    Azure AI Document Intelligence
      Prebuilt receipt invoice ID
      Custom models
      Layout analysis tables
    Azure Video Indexer
      Video face detection
      OCR on video
      Scene detection
      Transcription
      Language detection
      Translation
      Keywords named entities

14.2 Final Summary Table

15. Glossary

Additional Resources:

Search Terms

ai-900 · computer · vision · workloads · azure · ai · services · artificial · intelligence · generative · service · architecture · custom · exam · image · ocr · capabilities · detailed · detection · face · facial · analysis · api · application