AWS ML Engineer Associate (MLA-C01): Deployment & Orchestration

Table of Contents

1. Module 1: Selecting the Deployment Infrastructure According to Existing Architecture

   • Model Deployment Best Practices
   • Deployment Overview with Amazon SageMaker
   • Demo: Real-time Model Endpoint
   • When to Use Real-time Endpoints
   • When to Use Serverless Endpoints
   • Demo: Serverless Model Endpoint
   • When to Use Asynchronous Endpoints
   • When to Use Batch Inference
   • Demo: Batch Inference with SageMaker
   • Built-in vs Custom Container Images
   • SageMaker Neo
   • Demo: Deployment Orchestration with SageMaker Pipelines
   • Evaluating Performance, Cost and Latency Trade-offs
   • Selecting the Appropriate Compute Resources
   • When to Use Apache Airflow
   • Multi-model and Multi-container Deployments
   • Containerization Concepts and AWS Services
   • Selecting the Right Deployment Target
   • Module 1 Review

2. Module 2: Creating and Scripting Infrastructure

   • On-Demand and Provisioned Resources
   • Scaling Policies Comparison
   • Infrastructure as Code with AWS CloudFormation
   • AWS Cloud Development Kit (CDK)
   • Cross-Stack Communication
   • Auto Scaling Policies for SageMaker Endpoints
   • SageMaker Managed Spot Training
   • Building and Maintaining Containers
   • Demo: Build Your Own Container
   • Configuring SageMaker Endpoints in a VPC
   • SageMaker SDK
   • Selecting the Right Auto Scaling Metrics
   • Module 2 Review

3. Module 3: Automated Orchestration Tools for CI/CD Pipelines

   • CI/CD Principles
   • AWS CodePipeline
   • AWS CodeBuild
   • AWS CodeDeploy
   • Demo: Using CodePipeline
   • Automated Data Integration and Ingestion
   • Version Control with Git
   • How Repositories and Pipelines Work Together
   • Pipeline Invocation: Gitflow and GitHub Flow
   • Blue-Green Deployment Strategy
   • Canary Deployment Strategy
   • Linear Deployment Strategy
   • Automating Model Build and Deployment
   • Automation with Amazon EventBridge
   • SageMaker Pipelines
   • Automated Testing in CI/CD Pipelines
   • Best Practices for Model Retraining
   • Module 3 Review

4. Module 4: Exam Tips

   • Domain Review
   • Review Questions

5. Advanced MLOps Supplements

   • SageMaker Model Monitor
   • SageMaker Model Registry
   • SageMaker Feature Store
   • A/B Testing with SageMaker
   • Shadow Deployment
   • MLflow on AWS
   • AWS Step Functions for MLOps — Deep Dive

6. Enriched Summary

Module 1: Selecting the Deployment Infrastructure According to Existing Architecture

Model Deployment Best Practices

Deploying a machine learning model is not a one-time activity. After deploying a model, it will eventually need to be updated. This means the way you deploy the model must account for these considerations:

• Version management: How do you manage different versions of the model?
• A/B testing: How do you safely test new versions in production?
• Rollback: How do you ensure you can quickly revert to a previous version if a new version doesn’t produce the expected results?

Tip: Think about model deployment the same way you think about software deployment best practices. Automate deployments to ensure consistency and repeatability.

Deployment Overview with Amazon SageMaker

After fine-tuning and evaluating your model, it’s time to deploy it. Several options are available:

flowchart TD
    A[Trained Model] --> B{Deployment Options}
    B --> C[SageMaker Endpoint\nHTTPS endpoint]
    B --> D[EC2 Instance\nDeep learning AMI]
    B --> E[Custom Docker Container]
    E --> F[SageMaker]
    E --> G[Elastic Container Service\nECS]
    E --> H[Elastic Kubernetes Service\nEKS]
    E --> I[Lambda]

SageMaker Endpoint (recommended approach)

• HTTPS endpoint accessible by client applications via the SageMaker API (InvokeEndpoint)
• Uses EC2 instances to host the endpoint and deployed model
• For systems requiring high availability: auto scaling group of EC2 instances distributed across multiple Availability Zones
• SageMaker uses Elastic Container Service (ECS) under the hood to run Docker containers hosting the deployed model (complete abstraction)

SageMaker Endpoint Types

Demo: Real-time Model Endpoint

Objective: Deploy a real-time endpoint with SageMaker and invoke it.

Demo steps:

1. Search for SageMaker in the AWS console
2. Select Studio > Open Studio
3. Create a JupyterLab space (e.g., myJL)
4. Clone the course GitHub repository
5. Navigate to the Real-time_Model_Endpoint folder
6. Open the sagemaker-hosted-endpoint-demo.ipynb file

Typical Python code to deploy a real-time endpoint:

import boto3
import sagemaker
from sagemaker import get_execution_role

role = get_execution_role()
session = sagemaker.Session()

# Deploy a model to a real-time endpoint
from sagemaker.xgboost import XGBoost

xgb_model = sagemaker.Model(
    model_data='s3://my-bucket/model.tar.gz',
    image_uri='<xgboost-image-uri>',
    role=role
)

predictor = xgb_model.deploy(
    initial_instance_count=1,
    instance_type='ml.m5.xlarge',
    endpoint_name='my-realtime-endpoint'
)

Code to invoke the endpoint:

import json

# Invoke the endpoint with test data
response = predictor.predict(test_data)
print(response)

When to Use Real-time Endpoints

Real-time endpoints are suited for the following use cases:

• Low-latency workloads: ideal for fast requests
• Small requests: maximum payload of 6 MB
• Short requests: timeout of 60 seconds
• Individual requests: each request is processed individually (no batches)
• High availability: hosted on one or more EC2 instances with auto scaling
• VPC deployment: control of security groups, NACLs, VPC Flow Logs

Drawback: Continuous EC2 cost, even if the endpoint is idle (instances waiting for the next request).

flowchart LR
    Client -->|HTTPS InvokeEndpoint| ALB[Elastic Load Balancer]
    ALB --> EC2_1[EC2 Instance AZ1\nDocker Container]
    ALB --> EC2_2[EC2 Instance AZ2\nDocker Container]
    ALB --> EC2_3[EC2 Instance AZ3\nDocker Container]
    subgraph VPC
        ALB
        EC2_1
        EC2_2
        EC2_3
    end

When to Use Serverless Endpoints

Serverless endpoints are suited for the following use cases:

• Infrequent workloads: ideal for irregular traffic
• Small requests: maximum payload of 4 MB
• Short requests: timeout of 60 seconds
• Pay-as-you-go: no infrastructure to pay for, billing only on usage
• Scale to zero: reduction to 0 instances in the absence of requests

Cold start: The serverless runtime must be initialized periodically if no requests have occurred, causing a delay of a few milliseconds.

Provisioned Concurrency: To mitigate cold starts, you can purchase provisioned concurrency — compute resources are initialized and ready to respond in milliseconds, but you pay for these resources even without incoming requests.

Lambda integration: Serverless inference integrates with Lambda, offering high availability, built-in fault tolerance, and auto scaling with the ability to scale to zero.

Demo: Serverless Model Endpoint

Objective: Deploy a serverless endpoint with SageMaker.

Model used: Pre-trained XGBoost binary classification model on synthetic automotive insurance claims data.

Python code for a serverless endpoint:

from sagemaker.serverless import ServerlessInferenceConfig

serverless_config = ServerlessInferenceConfig(
    memory_size_in_mb=2048,
    max_concurrency=5,
)

predictor = model.deploy(
    serverless_inference_config=serverless_config,
    endpoint_name='my-serverless-endpoint'
)

When to Use Asynchronous Endpoints

Asynchronous endpoints are suited for the following use cases:

• Large requests: payload up to 1 GB
• Long requests: execution up to 60 minutes
• Near real-time response: requests are queued and processed asynchronously
• Not suitable for low-latency workloads

sequenceDiagram
    participant Client
    participant SageMaker
    participant S3_Input as S3 (Input)
    participant Queue as Internal Queue
    participant S3_Output as S3 (Output)

    Client->>S3_Input: Deposits payload in S3
    Client->>SageMaker: InvokeEndpointAsync
    SageMaker->>Queue: Queued
    Queue->>SageMaker: Asynchronous processing
    SageMaker->>S3_Output: Result stored in S3
    SageMaker-->>Client: Notification (near real-time)

Key feature: Auto scaling possible down to zero instances in the absence of requests (cost reduction).

When to Use Batch Inference

Batch inference (also called batch transform) is suited for the following use cases:

• Offline processing: ad hoc or scheduled jobs
• Bulk inference: processing an entire dataset at once (CSV, JSON with multiple records)
• Large datasets: maximum payload of 100 MB per request
• Long jobs: up to 60 minutes per record, but long jobs can extend over several days

How it works:

flowchart LR
    S3_in[S3\nInput Data] -->|Input| SM[SageMaker\nBatch Transform Job]
    SM -->|Provision| EC2[EC2 Instances\nautomatic]
    EC2 -->|Processing| SM
    SM -->|Results| S3_out[S3\nResults]
    SM -->|After completion| Terminate[Automatic EC2\nTermination]

Advantages:

• Most economical option (no persistent endpoint, no idle time paid)
• No infrastructure to manage (SageMaker automatically provisions and terminates)
• SageMaker manages the complete EC2 instance lifecycle

Demo: Batch Inference with SageMaker

Objective: Train an XGBoost model and run a batch inference job (batch transform).

Python code for a batch transform job:

import sagemaker
from sagemaker import get_execution_role

role = get_execution_role()
session = sagemaker.Session()

# Create a transformer
transformer = model.transformer(
    instance_count=1,
    instance_type='ml.m5.xlarge',
    output_path='s3://bucket/output/',
)

# Launch the batch transform job
transformer.transform(
    data='s3://bucket/input/data.csv',
    data_type='S3Prefix',
    content_type='text/csv',
    split_type='Line'
)

transformer.wait()
print("Batch transform completed!")

Built-in vs Custom Container Images

SageMaker provides pre-built container images for many use cases. Here is an overview of built-in algorithms:

Supervised Learning

Unsupervised Learning

Text Analysis

Image Processing

When to use a custom container: If no pre-built container meets your needs (specific framework, particular version, custom dependencies), you can create your own Docker image and push it to Elastic Container Registry (ECR).

SageMaker Neo

SageMaker Neo is a service that automatically optimizes machine learning models for maximum performance, whether for execution in the cloud or on edge devices.

flowchart LR
    A[ML Model\nbuilt with\na supported framework] --> B[Target platform\nselection]
    B --> C{Platform type}
    C -->|Cloud| D[SageMaker EC2 Instance]
    C -->|Edge| E[Edge device\nAndroid, Linux,\nWindows, Raspberry Pi]
    B --> F[SageMaker Neo\nAutomatic optimization]
    F --> G[Model compiled\nto executable]
    G --> H{Deployment}
    H --> D
    H --> E

Supported processors: Intel, Apple, Nvidia, Texas Instruments (and others — memorization not required for exam).

Use cases:

• Model needing better performance without accuracy reduction
• Deployment on edge devices with limited compute and memory
• Avoids months of manual optimization

Exam tip: SageMaker Neo is used to optimize performance of custom models — don’t confuse with environment customization.

Demo: Deployment Orchestration with SageMaker Pipelines

Objective: Orchestrate a complete machine learning workflow with SageMaker Pipeline.

Steps:

1. Clone the GitHub repository in JupyterLab
2. Install the SageMaker Python SDK
3. Define pipeline steps (preprocessing, training, evaluation, deployment)
4. Execute the pipeline

import sagemaker
from sagemaker.workflow.pipeline import Pipeline
from sagemaker.workflow.steps import ProcessingStep, TrainingStep
from sagemaker.workflow.model_step import ModelStep

# Define a preprocessing step
preprocessing_step = ProcessingStep(
    name="PreprocessTrainingData",
    processor=sklearn_processor,
    inputs=[...],
    outputs=[...]
)

# Define a training step
training_step = TrainingStep(
    name="TrainClassifier",
    estimator=xgb_estimator,
    inputs={
        "train": TrainingInput(
            s3_data=preprocessing_step.properties.ProcessingOutputConfig.Outputs["train"].S3Output.S3Uri
        )
    }
)

# Create and execute the pipeline
pipeline = Pipeline(
    name="MLWorkflowPipeline",
    steps=[preprocessing_step, training_step],
    sagemaker_session=session
)

pipeline.upsert(role_arn=role)
execution = pipeline.start()

Evaluating Performance, Cost and Latency Trade-offs

When designing AWS solutions, understanding trade-offs is essential:

quadrantChart
    title SageMaker Deployment Trade-offs
    x-axis Low Cost --> High Cost
    y-axis High Latency --> Low Latency
    quadrant-1 High Performance, High Cost
    quadrant-2 High Performance, Low Cost
    quadrant-3 Low Performance, Low Cost
    quadrant-4 Low Performance, High Cost
    Real-time Endpoint: [0.85, 0.90]
    Serverless Endpoint: [0.30, 0.70]
    Async Endpoint: [0.45, 0.30]
    Batch Inference: [0.15, 0.05]

Factors influencing decisions:

• Performance: Selecting more powerful EC2 instances → higher cost
• Cost: Cost-effective options may impact latency and performance
• Latency: Latency-sensitive workloads require more expensive options

Performance optimization:

• Select EC2 instance types powerful enough for the workload
• For low-latency requests: real-time endpoints with GPU/Inferentia instances
• Configure auto scaling to handle traffic spikes

Selecting the Appropriate Compute Resources

CPU vs GPU

Specialized AWS Processors

AWS Trainium:

• High-performance machine learning accelerators designed specifically for model training
• EC2 Trn1 instances

AWS Inferentia:

• High-performance machine learning accelerators designed specifically for model inference
• EC2 Inf1 and Inf2 instances
• Offer better performance at lower cost for inference

flowchart TD
    A[ML Workload] --> B{Phase}
    B -->|Training| C{Compute type}
    B -->|Inference| D{Compute type}
    C -->|Specialized high perf| E[AWS Trainium\nTrn1 Instances]
    C -->|GPU high perf| F[P Instances\nNVIDIA Tensor Core]
    C -->|General purpose| G[Standard CPU Instances]
    D -->|Specialized high perf| H[AWS Inferentia\nInf1/Inf2 Instances]
    D -->|GPU high perf| F
    D -->|General purpose| G

Exam tip: Trainium = Training, Inferentia = Inference (the clue is in the name).

When to Use Apache Airflow

Apache Airflow is a leading open-source orchestration environment for building and running ETL (Extract, Transform, Load) workflows.

Use cases:

• Automating data pipelines for ML systems
• Ingestion, transformation and data movement from various sources
• Orchestrating machine learning workflows

flowchart LR
    S3[S3\nSource Data] -->|Athena Query| AF[Apache Airflow\nOrchestration]
    AF -->|Transform| EMR[Elastic MapReduce\nTransformation]
    EMR -->|Transformed Data| SM[SageMaker\nML Training]
    SM -->|Trained Model| Deploy[Deployed Endpoint]

Amazon MWAA (Managed Workflows for Apache Airflow):

• Managed AWS service for Apache Airflow
• Simplified deployment, auto scaling, automatic updates and patches
• Open-source → no vendor lock-in (same Apache Airflow code)

Multi-model and Multi-container Deployments

Multi-model Endpoints

A multi-model endpoint allows deploying multiple models in a single container behind a single endpoint.

flowchart TD
    Client -->|InvokeEndpoint| MME[Multi-Model Endpoint]
    MME --> Container[Shared Container]
    Container --> M1[Model 1\nPyTorch]
    Container --> M2[Model 2\nXGBoost]
    Container --> M3[Model 3\nscikit-learn]
    Container --> Mn[Model N...]

Advantages:

• Scalable solution for hosting many models using the same ML framework
• Cost-effective: reduces hosting costs and improves endpoint utilization
• Ideal for a mix of frequently and infrequently accessed models

Supported frameworks: PyTorch, MXNet, XGBoost, scikit-learn, RandomForest, and others.

Comparison: 10 models on 10 dedicated endpoints vs a single multi-model endpoint

Containerization Concepts and AWS Services

Containerization is an approach that allows rapidly deploying and scaling applications.

flowchart TD
    App[Application Code] --> DC[Docker Container\nCode + Libraries + Virtual Kernel]
    DC --> Docker[Docker Engine]
    Docker --> OS[Host OS]
    OS --> HW[Hardware]
    subgraph AWS Orchestration
        ECS[Elastic Container Service\nManaged EC2 Cluster]
        EKS[Elastic Kubernetes Service\nManaged Kubernetes]
        K8S[Self-managed Kubernetes\nManual management]
    end

AWS containerization services:

SageMaker and containers:

• Uses Docker containers to train ML algorithms
• Uses Docker containers to deploy models for inference
• Under the hood: ECS manages Docker containers

Selecting the Right Deployment Target

flowchart TD
    A[Ready Docker Container] --> B{Choose target}
    B --> C[SageMaker Endpoint\nSimplified deployment\nPre-built and custom images]
    B --> D[Elastic Container Service\nContainer orchestration\nIAM, VPC, SageMaker integration]
    B --> E[Elastic Kubernetes Service\nManaged Kubernetes on AWS]
    B --> F[Self-managed Kubernetes\nFull control but complex]
    B --> G[Lambda\nEvent-driven containers]

Recommendations by use case:

• SageMaker: Priority for ML workloads, simplified deployment, pre-built images
• ECS: Containerized microservices, deep AWS integration
• EKS: Existing Kubernetes workloads, multi-cloud portability
• Lambda: Event-driven containers, lightweight and infrequent workloads

Module 1 Review

Key points for the exam

SageMaker Endpoints:

flowchart LR
    subgraph Endpoint Types
        RT[Real-time\n• Low latency\n• Payload 6MB\n• Timeout 60s\n• Persistent EC2]
        SL[Serverless\n• Infrequent\n• Payload 4MB\n• Timeout 60s\n• Scale to zero]
        AS[Asynchronous\n• Payload 1GB\n• Up to 60min\n• Near real-time\n• Scale to zero]
        BI[Batch Inference\n• Payload 100MB/req\n• Days possible\n• Offline only\n• Cheapest]
    end

Built-in algorithms:

• Supervised: Linear Learner, KNN, XGBoost, Factorization Machines, DeepAR
• Unsupervised: Random Cut Forest, K-Means, LDA, Neural Topic Modeling
• Text: BlazingText, Text Classification TensorFlow
• Images: Image Classification, Object Detection, Semantic Segmentation

SageMaker Neo: Optimizes custom models for EC2 or edge devices (Android, Raspberry Pi, etc.)

Compute:

• P instances (NVIDIA GPU): high ML training performance, graphics
• Trainium: optimized ML training
• Inferentia: optimized ML inference

Serverless Provisioned Concurrency: Keeps endpoints warm to avoid cold starts. Useful for predictable traffic spikes, but costly for unpredictable traffic.

PrivateLink: To connect a VPC to AWS services without using the public internet. Deploys an elastic network interface (ENI) in the private subnet.

Module 2: Creating and Scripting Infrastructure

On-Demand and Provisioned Resources

flowchart LR
    A{Resource type} --> B[On-Demand]
    A --> C[Provisioned / Provisioned Throughput]

    B --> B1[Unpredictable traffic]
    B --> B2[Flexible, no minimum charges]
    B --> B3[No commitment]
    B --> B4[Pay-as-you-go]

    C --> C1[Regular and predictable traffic]
    C --> C2[Commitment required]
    C --> C3[Reserved capacity paid even if unused]
    C --> C4[Improved performance/throughput]

Provisioned Concurrency with SageMaker Serverless Inference:

• Keeps endpoints warm with pre-initialized resources
• Ready to respond in milliseconds
• Ideal for predictable traffic bursts
• Additional cost for using provisioned concurrency

Cold start: When using on-demand serverless inference without provisioned concurrency, if the endpoint hasn’t received traffic for a while, resources must be initialized on the next request.

Scaling Policies Comparison

SageMaker currently supports three auto scaling options:

flowchart TD
    AS[SageMaker Auto Scaling] --> TT[Target Tracking\nBased on a CloudWatch metric\nEx: CPU utilization, invocations/instance]
    AS --> SS[Step Scaling\nTriggered by CloudWatch alarms\nEx: CPU > 70% → add instances]
    AS --> SC[Scheduled Scaling\nBased on a defined schedule\nEx: increase on weekends]

AWS recommendation: Use Target Tracking by default, as it is fully automated.

Infrastructure as Code with AWS CloudFormation

CloudFormation is an Infrastructure as Code (IaC) service that provisions, configures, and manages AWS infrastructure.

flowchart LR
    T[Template\nJSON or YAML] -->|Upload via S3| CF[CloudFormation]
    CF -->|Automatic API calls| Stack[CloudFormation Stack\nCreated resources]
    Stack --> SM_EP[SageMaker Endpoint]
    Stack --> SM_M[SageMaker Model]
    Stack --> Others[Other AWS resources]

CloudFormation template example for SageMaker (YAML):

Resources:
  MyModel:
    Type: AWS::SageMaker::Model
    Properties:
      ModelName: MyMLModel
      ExecutionRoleArn: !Sub "arn:aws:iam::${AWS::AccountId}:role/SageMakerRole"
      PrimaryContainer:
        Image: !Sub "${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/my-model:latest"
        ModelDataUrl: s3://my-bucket/model/model.tar.gz

  MyEndpointConfig:
    Type: AWS::SageMaker::EndpointConfig
    Properties:
      ProductionVariants:
        - ModelName: !GetAtt MyModel.ModelName
          VariantName: AllTraffic
          InitialInstanceCount: 1
          InstanceType: ml.m5.xlarge
          InitialVariantWeight: 1

  MyEndpoint:
    Type: AWS::SageMaker::Endpoint
    Properties:
      EndpointName: MyProductionEndpoint
      EndpointConfigName: !GetAtt MyEndpointConfig.EndpointConfigName

Best practices:

• Store templates in a source code repository (GitHub) for version control and peer review
• Use separate stacks for different layers (networking, data, ML)

AWS Cloud Development Kit (CDK)

The CDK is an open-source framework for defining AWS resources using a programming language of your choice.

Supported languages: TypeScript, Python, C#, Java, Go

Main advantage: No need to learn CloudFormation YAML/JSON syntax.

flowchart LR
    Code[TypeScript/Python/etc. Code] -->|CDK Toolkit| CF_Template[CloudFormation Template\nAuto-generated YAML/JSON]
    CF_Template -->|CloudFormation| Resources[Provisioned AWS Resources]

CDK example in TypeScript:

import * as cdk from 'aws-cdk-lib';
import * as ec2 from 'aws-cdk-lib/aws-ec2';

export class MyMLStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // VPC definition
    const vpc = new ec2.Vpc(this, 'MLVpc', {
      maxAzs: 3
    });

    // EC2 instance definition
    const instance = new ec2.Instance(this, 'MLInstance', {
      vpc,
      instanceType: ec2.InstanceType.of(
        ec2.InstanceClass.P3,
        ec2.InstanceSize.XLARGE2
      ),
      machineImage: ec2.MachineImage.latestAmazonLinux2(),
    });
  }
}

CDK example in Python:

from aws_cdk import Stack
import aws_cdk.aws_s3 as s3
from constructs import Construct

class DataStack(Stack):
    def __init__(self, scope: Construct, id: str, **kwargs):
        super().__init__(scope, id, **kwargs)

        # Create an S3 bucket
        self.bucket = s3.Bucket(self, "MLDataBucket",
            versioned=True,
            encryption=s3.BucketEncryption.S3_MANAGED
        )

class ProcessingStack(Stack):
    def __init__(self, scope: Construct, id: str, data_stack: DataStack, **kwargs):
        super().__init__(scope, id, **kwargs)

        # Reference the bucket from DataStack
        bucket = data_stack.bucket
        # Use the bucket in ProcessingStack...

Cross-Stack Communication

CloudFormation and CDK support cross-stack references — referencing resources existing in a different stack.

CloudFormation example:

# Stack1 - Export values
Outputs:
  SubnetId:
    Value: !Ref MySubnet
    Export:
      Name: "Stack1-SubnetId"
  
  SecurityGroupId:
    Value: !Ref MySecurityGroup
    Export:
      Name: "Stack1-SecurityGroupId"

# Stack2 - Import values
Resources:
  MyInstance:
    Type: AWS::EC2::Instance
    Properties:
      SubnetId: !ImportValue "Stack1-SubnetId"
      SecurityGroupIds:
        - !ImportValue "Stack1-SecurityGroupId"

Auto Scaling Policies for SageMaker Endpoints

flowchart TD
    Policy{Policy type} --> TT[Target Tracking]
    Policy --> SS[Step Scaling]

    TT --> TT1[Specify target value\nEx: 70 invocations/instance/min]
    TT --> TT2[Choose CloudWatch metric\nEx: SageMakerVariantInvocationsPerInstance]
    TT --> TT3[SageMaker automatically adjusts\ninstance count]

    SS --> SS1[Configure CloudWatch alarms]
    SS --> SS2[Define add/remove instance thresholds]

Target Tracking Scaling Policy example:

import boto3

client = boto3.client('application-autoscaling')

# Register the scalable target
client.register_scalable_target(
    ServiceNamespace='sagemaker',
    ResourceId='endpoint/my-endpoint/variant/AllTraffic',
    ScalableDimension='sagemaker:variant:DesiredInstanceCount',
    MinCapacity=1,
    MaxCapacity=10
)

# Create the target tracking policy
client.put_scaling_policy(
    PolicyName='SageMakerTargetTrackingPolicy',
    ServiceNamespace='sagemaker',
    ResourceId='endpoint/my-endpoint/variant/AllTraffic',
    ScalableDimension='sagemaker:variant:DesiredInstanceCount',
    PolicyType='TargetTrackingScaling',
    TargetTrackingScalingPolicyConfiguration={
        'TargetValue': 70.0,
        'PredefinedMetricSpecification': {
            'PredefinedMetricType': 'SageMakerVariantInvocationsPerInstance'
        },
        'ScaleInCooldown': 300,
        'ScaleOutCooldown': 60
    }
)

SageMaker Managed Spot Training

Spot instances allow reducing training costs by using excess EC2 capacity at a reduced price.

Advantages:

• Savings up to 90% compared to on-demand price
• SageMaker handles Spot interruptions

Disadvantages:

• Can be interrupted with 2 minutes notice
• Not recommended for time-sensitive workloads

How SageMaker handles interruptions:

flowchart TD
    Start[Start training job\nManaged Spot Training] --> Train[Training in progress\nOn Spot instance]
    Train --> CP[SageMaker copies\ncheckpoint data to S3]
    Train -->|Spot instance interrupted| Int[Interruption]
    Int --> NewInst[New Spot instance provisioned]
    NewInst --> Restore[SageMaker copies checkpoints\nfrom S3 to new instance]
    Restore --> Resume[Training resumed\nfrom last checkpoint]
    Resume --> Complete[Training complete]

Code configuration:

import sagemaker
from sagemaker.estimator import Estimator

estimator = Estimator(
    image_uri='<image-uri>',
    role=role,
    instance_count=1,
    instance_type='ml.p3.2xlarge',
    use_spot_instances=True,                    # Enable Spot
    max_run=3600,                               # Max runtime in seconds
    max_wait=7200,                              # Max wait in seconds
    checkpoint_s3_uri='s3://bucket/checkpoints' # S3 for checkpoints
)

Building and Maintaining Containers

When SageMaker pre-built containers don’t fit your needs (specific version, custom framework), you can create your own Docker image.

Dockerfile example (based on Python for SageMaker):

FROM ubuntu:20.04

# Install dependencies
RUN apt-get update && apt-get install -y \
    wget \
    python3 \
    python3-pip \
    nginx \
    ca-certificates \
    && rm -rf /var/lib/apt/lists/*

# Create symbolic links
RUN ln -s /usr/bin/python3 /usr/bin/python
RUN ln -s /usr/bin/pip3 /usr/bin/pip

# Install required Python libraries
RUN pip install --no-cache-dir \
    numpy==1.23.5 \
    scipy==1.10.1 \
    scikit-learn==1.2.2

# Environment variables
ENV PYTHONUNBUFFERED=TRUE
ENV PYTHONDONTWRITEBYTECODE=TRUE
ENV PATH="/opt/program:${PATH}"

# Copy model files
COPY model/ /opt/program/
WORKDIR /opt/program

# Entry point
ENTRYPOINT ["python", "train.py"]

Custom SageMaker container structure:

sample-container/
├── Dockerfile
├── build_and_push.sh        # Build and push to ECR
└── model/
    ├── train.py             # Training script
    ├── predictor.py         # Inference script
    └── nginx.conf           # Web server configuration

Demo: Build Your Own Container

Steps:

1. Package and upload the algorithm for SageMaker
2. Build and push the Docker image to ECR with build_and_push.sh
3. Train the model with the custom image
4. Deploy the model to a SageMaker endpoint

build_and_push.sh script:

#!/bin/bash

# Variables
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
REGION=$(aws configure get region)
IMAGE_NAME="my-custom-ml-model"
ECR_REPO="${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com/${IMAGE_NAME}"

# ECR authentication
aws ecr get-login-password --region $REGION | \
    docker login --username AWS --password-stdin ${ACCOUNT_ID}.dkr.ecr.${REGION}.amazonaws.com

# Create repository if necessary
aws ecr describe-repositories --repository-names ${IMAGE_NAME} || \
    aws ecr create-repository --repository-name ${IMAGE_NAME}

# Build and push
docker build -t ${IMAGE_NAME} .
docker tag ${IMAGE_NAME}:latest ${ECR_REPO}:latest
docker push ${ECR_REPO}:latest

echo "Image pushed: ${ECR_REPO}:latest"

Configuring SageMaker Endpoints in a VPC

By default, all AWS services use public endpoints. For production workloads requiring data confidentiality, AWS PrivateLink allows connecting your VPC to AWS services without using the public internet.

flowchart LR
    subgraph Private VPC
        Server[Application Server\nPrivate subnet]
        ENI[Elastic Network Interface\nENI - VPC Interface Endpoint]
    end

    Server -->|SageMaker requests| ENI
    ENI -->|AWS PrivateLink\nAWS internal network| SM[SageMaker\nEndpoint]

    style Server fill:#f9f,stroke:#333
    style ENI fill:#bbf,stroke:#333
    style SM fill:#bfb,stroke:#333

PrivateLink advantages:

• Data never leaves the AWS private network
• No need for NAT gateways or internet gateways
• ENI is deployed in the private subnet and controlled by a security group
• High availability and scalability

SageMaker SDK

SageMaker provides APIs, SDKs, and CLI to create and manage notebook instances, interact with SageMaker, train and deploy models.

Available SDKs: Python, Java, .NET, C++, Go, JavaScript, PHP, Ruby, Spark

Python example with SageMaker JumpStart:

from sagemaker import image_uris, model_uris, script_uris
from sagemaker.model import Model
from sagemaker.predictor import Predictor

# Select a pre-trained model from JumpStart
model_id = "tensorflow-tc-bert-en-uncased-L-2-H-128-A-2-2"
model_version = "2.0.0"
model_scope = "inference"

# Retrieve necessary URIs
base_model_uri = model_uris.retrieve(
    model_id=model_id,
    model_version=model_version,
    model_scope=model_scope
)

image_uri = image_uris.retrieve(
    region=None,
    framework=None,
    model_id=model_id,
    model_version=model_version,
    image_scope="inference",
    instance_type="ml.m5.xlarge"
)

# Create and deploy the model
model = Model(
    image_uri=image_uri,
    model_data=base_model_uri,
    entry_point="inference.py",
    role=role,
    predictor_cls=Predictor,
    sagemaker_session=session
)

predictor = model.deploy(
    initial_instance_count=1,
    instance_type='ml.m5.xlarge'
)

Selecting the Right Auto Scaling Metrics

Standard predefined CloudWatch metrics (emitted every minute)

Custom metrics

You can use any metric proportional to capacity, for example CPUUtilization to scale at 50% average utilization.

High-resolution metrics (emitted every 10 seconds — faster reaction)

Note: High-resolution metrics trigger scale-out much faster than standard metrics.

Module 2 Review

Key points for the exam

On-Demand vs Provisioned:

• On-Demand: unpredictable traffic, flexible, pay-as-you-go
• Provisioned: regular traffic, commitment required, better performance

Auto Scaling:

• Target Tracking: recommended by AWS, fully automated
• Step Scaling: advanced step configuration
• Scheduled Scaling: predefined schedule

CloudFormation: IaC, YAML/JSON templates, automatic API calls, stacks CDK: Open-source framework, code in TypeScript/Python/etc., generates CloudFormation under the hood

Cross-stack references: Export from Stack1, Import in Stack2

Spot Training: Up to 90% savings, S3 checkpoints, SageMaker handles interruptions

PrivateLink/VPC Interface Endpoints: ENI in private subnet, no traffic on public internet

Module 3: Automated Orchestration Tools for CI/CD Pipelines

CI/CD Principles

CI/CD (Continuous Integration / Continuous Delivery) is a software delivery best practice.

Fundamental principles:

• Automate everything: code integration, build, test, deployment
• Easy change: releases, updates, patches
• Result: fast, repeatable and scalable process

Continuous Integration (CI) Workflow

flowchart LR
    Dev1[Developer 1] -->|Clone| Local1[Local code]
    Dev2[Developer 2] -->|Clone| Local2[Local code]
    Dev3[Developer 3] -->|Clone| Local3[Local code]

    Local1 -->|Changes + Local Tests| PR1[Pull Request]
    Local2 -->|Changes + Local Tests| PR2[Pull Request]
    Local3 -->|Changes + Local Tests| PR3[Pull Request]

    PR1 --> Repo[Shared Repository\nIntegrated code]
    PR2 --> Repo
    PR3 --> Repo

MLOps = DevOps applied to machine learning. Key principles:

• Version control: Code, models, data
• Automation: Automated pipeline
• Continuous testing: Validation at each step
• Model governance: Traceability and auditability

AWS CodePipeline

CodePipeline is a fully managed CI/CD service that orchestrates the build, test and deployment process.

flowchart LR
    Source[Source Code\nRepository\nChange detected] -->|Triggers| CB[CodeBuild\nCompilation + Tests\nProduction artifacts]
    CB -->|Artifacts ready| CD[CodeDeploy\nAutomatic deployment]
    CD --> Staging[Staging Environment]
    CD --> Prod[Production Environment]

CodePipeline integrations:

AWS CodeBuild

CodeBuild is a fully managed build service that compiles code, runs tests and produces deployment-ready artifacts.

How it works:

1. New source code → CodeBuild compiles and tests
2. Produces artifacts (compiled code, packages, Docker images)
3. Pushes Docker image to ECR

buildspec.yml file:

version: 0.2

phases:
  pre_build:
    commands:
      - echo Logging in to Amazon ECR...
      - aws ecr get-login-password --region $AWS_DEFAULT_REGION | \
          docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com
  
  build:
    commands:
      - echo Build started on `date`
      - echo Building the Docker image...
      - docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG .
      - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG \
          $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
  
  post_build:
    commands:
      - echo Pushing the Docker image...
      - docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG
      - echo Build completed on `date`

artifacts:
  files:
    - imagedefinitions.json

CodeBuild quotas:

• Maximum build projects per region: 5,000
• Maximum build timeout: 36 hours
• Security groups for VPC configuration per region: 5
• Subnets for VPC configuration per region: 16

AWS CodeDeploy

CodeDeploy enables automated deployment to EC2, ECS and Lambda.

Advantages:

• Rapid deployment of new features
• Avoids downtime during deployments
• Avoids risks from manual processes

CodeDeploy quotas:

• Maximum hours for a Lambda deployment: 50
• Maximum applications per account/region: 1,000
• Maximum deployment groups per application: 1,000
• Maximum auto scaling groups in a deployment group: 10

Demo: Using CodePipeline

Objective: Create a simple pipeline, deploy an application with CodeDeploy, add a test stage with CodeBuild.

Steps:

1. Open CloudShell in the AWS console
2. Download the sample application
3. Create an S3 bucket with versioning enabled
4. Upload code to S3: aws s3 cp webapp.zip s3://my-bucket/
5. Create IAM roles:
   • EC2 role with AmazonEC2RoleforAWSCodeDeploy and AmazonSSMManagedInstanceCore policies
6. Launch an EC2 instance (Amazon Linux, t3.micro) with the IAM role
7. Install the CodeDeploy agent via Systems Manager
8. Create the pipeline in CodePipeline with Source, Build, Deploy stages

Automated Data Integration and Ingestion

For automating data ingestion in an ML pipeline:

flowchart TD
    Sources[Data Sources] --> S3[Amazon S3]
    Sources --> Kinesis[Amazon Kinesis\nReal-time streaming]
    Sources --> Glue[AWS Glue\nETL]

    S3 --> CP[CodePipeline\nCI/CD Automation]
    Kinesis --> EB[EventBridge\nEvent-driven triggers]
    Glue --> SF[Step Functions\nWorkflow orchestration]

    CP --> SF
    EB --> SF

    SF --> Preprocess[Preprocessing]
    SF --> Train[Model Training\nSageMaker]
    SF --> Evaluate[Evaluation]
    SF --> Deploy[Deployment]

Step Functions:

• Serverless orchestration service to manage workflow logic
• Automatically triggers each step and tracks its state
• Output of one step is often the input of the next
• Real-time workflow visualization
• Detailed error messages for debugging

Version Control with Git

Git is an essential version control system in MLOps.

What is versioned:

• Python scripts
• Jupyter Notebooks
• Algorithms and neural network architectures
• Configuration parameters and hyperparameters

Git workflow:

flowchart LR
    Dev1[Developer A] -->|git clone| L1[Local copy]
    Dev2[Developer B] -->|git clone| L2[Local copy]
    L1 -->|Changes| B1[Build + Local test]
    L2 -->|Changes| B2[Build + Local test]
    B1 -->|git commit + push| Remote[Remote Repository\nGitHub/CodeCommit]
    B2 -->|git commit + push| Remote

How Repositories and Pipelines Work Together

flowchart LR
    Repo[Code Repository\nGitHub/CodeCommit] -->|New commit| CP[CodePipeline\nAutomatically triggered]
    CP --> CB[CodeBuild\nCI: Compilation + Tests]
    CB --> CD[CodeDeploy\nCD: Deployment]
    CD --> EC2[EC2 Instance\nor ECS/Lambda]

    CFTemplate[CloudFormation Template\nin repository] -->|New commit| CP
    CP -->|CloudFormation stage| NewResources[New AWS resources\nprovisioned]

CodePipeline pipeline stages:

Pipeline Invocation: Gitflow and GitHub Flow

Gitflow — Structured branch model

gitGraph
    commit id: "Initial commit"
    branch develop
    checkout develop
    commit id: "Development base"
    branch feature/new-model
    checkout feature/new-model
    commit id: "New feature"
    commit id: "Feature tests"
    checkout develop
    merge feature/new-model id: "Merge feature"
    branch release/v1.0
    checkout release/v1.0
    commit id: "Release preparation"
    checkout main
    merge release/v1.0 id: "Release v1.0" tag: "v1.0"
    checkout develop
    merge release/v1.0 id: "Sync develop"
    branch hotfix/critical-fix
    checkout hotfix/critical-fix
    commit id: "Critical fix"
    checkout main
    merge hotfix/critical-fix id: "Hotfix" tag: "v1.0.1"
    checkout develop
    merge hotfix/critical-fix id: "Sync hotfix"

Gitflow branches:

GitHub Flow: Simpler than Gitflow — everything happens on main with short feature branches.

Blue-Green Deployment Strategy

Blue-Green is a safe deployment strategy minimizing service interruptions.

flowchart TD
    LB[Elastic Load Balancer]

    subgraph Blue [Blue Environment - Current version]
        B1[EC2 Instance B1]
        B2[EC2 Instance B2]
    end

    subgraph Green [Green Environment - New version]
        G1[EC2 Instance G1]
        G2[EC2 Instance G2]
    end

    LB -->|100% traffic| B1
    LB -->|100% traffic| B2

    CodeDeploy[CodeDeploy] -->|Deploy new\nversion| G1
    CodeDeploy -->|Deploy new\nversion| G2

    G1 -.->|After validation:\nswitch traffic| LB
    G2 -.->|After validation:\nswitch traffic| LB

Process:

1. Blue = current production environment
2. CodeDeploy provisions new instances (Green) and deploys the new version
3. After validation, Green instances are registered with the Elastic Load Balancer
4. Traffic is redirected from Blue to Green
5. The Blue environment can be kept for easy rollback before being terminated

Rollback: Simply redirect traffic to the Blue environment (if not yet terminated).

Blue-Green with SageMaker (boto3):

import boto3

sm = boto3.client('sagemaker')

# Create a new EndpointConfig v2 (Green environment)
sm.create_endpoint_config(
    EndpointConfigName='my-model-config-v2',
    ProductionVariants=[{
        'VariantName': 'AllTraffic',
        'ModelName': 'my-model-v2',
        'InstanceType': 'ml.m5.xlarge',
        'InitialInstanceCount': 2,
        'InitialVariantWeight': 1
    }]
)

# Update the endpoint with Blue-Green (switch ALL_AT_ONCE)
sm.update_endpoint(
    EndpointName='my-production-endpoint',
    EndpointConfigName='my-model-config-v2',
    DeploymentConfig={
        'BlueGreenUpdatePolicy': {
            'TrafficRoutingConfiguration': {
                'Type': 'ALL_AT_ONCE',
                'WaitIntervalInSeconds': 300   # Wait 5 min before terminating Blue
            },
            'TerminationWaitInSeconds': 600,
            'MaximumExecutionTimeoutInSeconds': 1800
        }
    }
)

Canary Deployment Strategy

flowchart LR
    LB[Elastic Load Balancer] -->|90% traffic| Old[Servers - Current version]
    LB -->|10% traffic| New[Small number of servers\nNew version]

    New -->|No errors?| Full[Full deployment\nto all servers]
    New -->|Errors detected| Rollback[Rollback:\n100% to old version]

Principle:

• Deploy the new version on a small number of servers (e.g., 10%)
• Redirect only a small proportion of traffic to the new version
• Monitor errors and performance
• If all goes well: progressively deploy to 100%
• If problem: immediate rollback

Advantages:

• Early warning system for production issues
• Tests with real customers without impacting everyone
• Reduces the risk of production deployments

Canary with SageMaker (boto3):

# Canary: start with 10% to new version
sm.update_endpoint(
    EndpointName='my-production-endpoint',
    EndpointConfigName='my-model-config-v2',
    DeploymentConfig={
        'BlueGreenUpdatePolicy': {
            'TrafficRoutingConfiguration': {
                'Type': 'CANARY',
                'CanarySize': {
                    'Type': 'CAPACITY_PERCENT',
                    'Value': 10    # 10% traffic to Green first
                },
                'WaitIntervalInSeconds': 300  # Wait 5 min, monitor metrics
            },
            'TerminationWaitInSeconds': 300
        }
    }
)

Linear Deployment Strategy

Linear is available for CodeDeploy with Lambda and ECS, used in combination with Blue-Green.

sequenceDiagram
    participant Old as Current version (Blue)
    participant LB as Load Balancer
    participant New as New version (Green)

    Note over Old,LB: 100% traffic to Blue
    LB->>New: CodeDeploy launches new containers
    Note over Old,New: t=0: 10% Green, 90% Blue
    Note over Old,New: t=1min: 20% Green, 80% Blue
    Note over Old,New: t=2min: 30% Green, 70% Blue
    Note over Old,New: t=Nmin: 100% Green, 0% Blue
    Note over Old: Old version terminated

Predefined options:

• Linear10PercentEvery1Minute: +10% every minute until 100%
• Linear10PercentEvery3Minutes: +10% every 3 minutes until 100%
• Custom configuration possible

Automating Model Build and Deployment

AWS services for automating model build and deployment:

flowchart TD
    Code[Code Repository\nGitHub] -->|New commit| Triggers{Triggers}
    CFTemplate[CloudFormation Template\nin GitHub] -->|New commit| Triggers

    Triggers --> CP[CodePipeline]
    Triggers --> EB[EventBridge]

    CP --> SF[Step Functions\nML Workflow]
    EB --> SF

    SF --> PP[SageMaker Pipelines\nML Automation]
    SF --> CF[CloudFormation\nDeployed infrastructure]

    PP --> Train[Training]
    PP --> Eval[Evaluation]
    PP --> Deploy[Deployment]

Key points:

• Trigger the MLOps pipeline when a new model or a new CloudFormation template appears in the repository
• Use Step Functions to visualize and manage the workflow
• CloudFormation for infrastructure (SageMaker endpoints, etc.)

Automation with Amazon EventBridge

EventBridge allows easily configuring event-driven systems or scheduled tasks.

flowchart LR
    Events[Source events] --> EB[EventBridge\nRouting rules]

    subgraph Event Sources
        S3_E[S3 - New object]
        SageMaker_E[SageMaker - State change]
        Kinesis_E[Kinesis - New data]
        EC2_E[EC2 - Restart]
        CT[CloudTrail]
        CW[CloudWatch]
    end

    S3_E --> EB
    SageMaker_E --> EB
    Kinesis_E --> EB
    EC2_E --> EB

    EB -->|Route to targets| Targets[Targets]

    subgraph Targets
        SF_T[Step Functions - Workflow]
        CP_T[CodePipeline - Pipeline]
        Lambda_T[Lambda - Function]
        SNS_T[SNS - Notification]
    end

    Targets --> SF_T
    Targets --> CP_T
    Targets --> Lambda_T
    Targets --> SNS_T

Use cases with SageMaker:

• Track state changes: models, training jobs, endpoints
• Trigger a Step Functions workflow to retrain a model when new data arrives in S3
• Initiate CodePipeline on data ingestion events

SageMaker Pipelines

SageMaker Pipelines is a workflow orchestration service dedicated to ML pipelines, integrated with all SageMaker features.

Features:

• Automation: preprocessing, training, fine-tuning, evaluation, deployment, monitoring
• Visual drag-and-drop interface in SageMaker Studio
• SDKs, APIs, or JSON definition
• Auditability: complete history of data in the pipeline
• SageMaker Lineage Tracking: analysis of data sources and consumers

flowchart LR
    Input[Input Data\nS3] --> PP[Processing\nPreprocessing]
    PP --> TT[Training\nXGBoost/etc.]
    TT --> EV[Evaluation\nMetrics]
    EV -->|Accuracy OK?| RM[Register Model\nModel Registry]
    EV -->|Insufficient accuracy| Retr[Retrain with\nadjusted parameters]
    RM --> CM[Create Model]
    CM --> DP[Deploy\nBatch or Real-time]

SageMaker Pipeline JSON definition example:

{
  "Version": "2020-12-01",
  "Parameters": [],
  "Steps": [
    {
      "Name": "TrainingStep",
      "Type": "Training",
      "Arguments": {
        "AlgorithmSpecification": {
          "TrainingImage": "683313688378.dkr.ecr.us-east-1.amazonaws.com/sagemaker-xgboost:1.5-1",
          "TrainingInputMode": "File"
        },
        "HyperParameters": {
          "max_depth": "5",
          "eta": "0.2",
          "objective": "binary:logistic",
          "num_round": "100"
        }
      }
    }
  ]
}

Automated Testing in CI/CD Pipelines

Types of automated tests:

Setup with SageMaker + CodePipeline:

flowchart LR
    Source[Source\nGitHub] -->|Change| CB[CodeBuild\nUnit Tests\nIntegration Tests]
    CB -->|Tests OK| CD[CodeDeploy\nDeployment]
    CB -->|Tests fail| Notif[Notification\nEventBridge/CloudWatch]

Best practices:

• Automated tests at each pipeline stage
• Pre-built AWS tests or custom tests
• Notifications and alerts via EventBridge and CloudWatch
• Source control integrated with Git
• Separate environments: development, staging, production
• Deployment strategy with rollback mechanism
• Monitoring and alerting of execution status

Best Practices for Model Retraining

Common reasons for retraining:

1. New data available for training
2. Updated model code to improve accuracy or fix bugs

flowchart TD
    NewData[New data\navailable in S3] --> EB[EventBridge\nEvent detection]
    EB --> SF[Step Functions\nTriggered workflow]

    SF --> PP[Preprocessing\nof new data]
    PP --> TR[Training Job\nSageMaker]
    TR --> EV[Evaluation\nof new model]
    EV -->|Metrics OK| Deploy[Deployment\nnew model]
    EV -->|Metrics KO| Alert[Alert\nto team]

Step Functions advantage for retraining:

• Workflow visualization during execution
• Display of errors with precise details
• Example: an error at the batch transform step indicates a quota issue → the Step Functions interface shows exactly which limit is reached

Module 3 Review

Key points for the exam

MLOps: Application of DevOps practices to machine learning. Principles: version control, automation, continuous testing, model governance.

CodePipeline: Fully managed CI/CD service, build/test/deployment orchestration, triggered on each code change.

CodeBuild: Build and tests. Uses Dockerfile + buildspec.yml to create Docker images and push to ECR.

CodeDeploy: Automated deployment to EC2, ECS, Lambda.

Deployment Strategies:

flowchart LR
    BG[Blue-Green\nTwo environments\nImmediate rollback\nECS/EC2] 
    CAN[Canary\n10% traffic first\nTest with real users\nEasy rollback]
    LIN[Linear\n+10% every N min\nWith Blue-Green\nECS/Lambda]

EventBridge: Rules based on events (S3, SageMaker, Kinesis) → triggers Step Functions, CodePipeline, Lambda.

SageMaker Pipelines: Dedicated ML orchestration, drag-and-drop in Studio, auditability, lineage tracking.

Step Functions: Serverless orchestration with visualization, error handling, state logs.

Module 4: Exam Tips

Domain Review

The “Deployment and Orchestration of ML Workflows” domain is one of the four domains of the MLA-C01 exam:

1. Data Preparation for Machine Learning
2. Machine Learning Model Development
3. Deployment and Orchestration of ML Workflows ← This course
4. Machine Learning Solution Monitoring, Maintenance, and Security

SageMaker Endpoints Summary

flowchart TD
    E[SageMaker Endpoint Types] --> RT
    E --> SL
    E --> AS
    E --> BI

    RT[Real-time\n• Low latency\n• Individual requests\n• Payload 6MB\n• Timeout 60s\n• Persistent EC2\n• Auto scale possible\n• In a VPC]

    SL[Serverless\n• Infrequent\n• Individual requests\n• Payload 4MB\n• Timeout 60s\n• Cold start possible\n• Scale to zero\n• Optional Provisioned Concurrency]

    AS[Asynchronous\n• Large requests\n• Payload 1GB\n• Up to 60 min\n• Near real-time\n• Scale to zero\n• Results in S3]

    BI[Batch Inference/Transform\n• Offline processing\n• Bulk datasets\n• Payload 100MB/req\n• Jobs over several days\n• Most economical\n• No persistent endpoint]

Key takeaways:

• The fastest options are the most expensive and suited for small requests
• Cheaper options handle larger loads but don’t suit latency-sensitive workloads
• Real-time and Asynchronous can both scale to zero in the absence of requests

Multi-model endpoints: Ideal for many models using the same ML framework on a shared container. Reduces administrative overhead and costs.

SageMaker Neo: Optimizes custom models (supported frameworks) for EC2 or edge devices. Compiles to optimized executable.

SageMaker Managed Spot Training: Up to 90% savings, checkpoints in S3, interruptions handled automatically.

PrivateLink + VPC Interface Endpoints: To access SageMaker from a private VPC without public internet. Deploys an ENI in the subnet.

Review Questions

Question 1: Custom Framework in SageMaker

Scenario: Your ML application requires a framework and version not available as a pre-built option in SageMaker. Which approach allows deployment with the least administrative overhead?

Options:

• A) Use a multi-model endpoint to deploy a pre-built model, then add required dependencies as environment parameters
• B) Select the closest pre-trained model, then update the framework and hyperparameters
• C) Create a custom Docker image, push it to ECR, then deploy it on SageMaker
• D) Use SageMaker Neo to modify an existing pre-built model to include the required framework

Answer: C

Explanation:

• SageMaker Neo → performance optimization, not for customizing environments ❌
• Multi-model endpoint → host multiple models with the same framework on one endpoint ❌
• Closest model → doesn’t meet the specific framework requirement ❌
• Custom Docker image → ECR → SageMaker: Correct solution for specific requirements ✅

Question 2: Automated Retraining Pipeline

Scenario: A SageMaker model trained with custom data has been in production for over a year. New data is available. Which two services allow building an automated pipeline to process new data and retrain the model?

Options: Cloud Development Kit, SageMaker Pipelines, CloudFormation, Step Functions

Answer: SageMaker Pipelines + Step Functions

Explanation:

• CloudFormation → deploy AWS resources (IaC), not for ML pipelines ❌
• CDK → defines CloudFormation templates in code, not for ML pipelines ❌
• SageMaker Pipelines → dedicated orchestration for ML pipelines ✅
• Step Functions → can integrate with CodePipeline to orchestrate ML steps ✅

Question 3: Deployment for Continuous Recommendations

Scenario: Product recommendation application used 24/7, latency-sensitive, requiring best performance. Which SageMaker deployment type to choose?

Options: Batch inference, Real-time, Serverless, Asynchronous

Answer: Real-time

Explanation:

• Batch inference → offline processing, not suitable for individual requests ❌
• Asynchronous → queued requests, near real-time only ❌
• Serverless → good performance, but cold starts possible ❌
• Real-time → low latency, persistent 24/7 endpoint, EC2 always ready ✅

Question 4: Optimizing Underutilized Endpoints

Scenario: Multiple ML models (same framework) deployed on separate dedicated endpoints. Endpoints are all underutilized. How to optimize and reduce costs?

Options: Multi-container endpoint, Asynchronous endpoint, Configure auto scaling, Multi-model endpoint

Answer: Multi-model endpoint

Explanation:

• Multi-container → unnecessary if all use the same framework ❌
• Auto scaling → doesn’t help if a single endpoint is already underutilized ❌
• Asynchronous → can scale to zero but the question doesn’t indicate the workload tolerates queuing ❌
• Multi-model endpoint → all models on a single endpoint/container, less unused capacity ✅

Question 5: Deployment Strategy Matching

Scenario: Match the strategy with the description.

Question 6: Instance Type for ML Inference

Scenario: ML application for legal contract analysis (summarization + action list). Which EC2 instance type for inference?

Options: Trainium, Compute optimized, Inferentia, General purpose

Answer: Inferentia

Explanation:

• Trainium → designed for training (clue in the name), not inference ❌
• Compute optimized → general high performance, not optimized for ML/deep learning ❌
• General purpose → balanced compute/memory/network, light for ML, no hardware acceleration ❌
• Inferentia → designed specifically for ML/deep learning inference, high performance, cost-effective ✅

General Summary

SageMaker Endpoint Comparison Table

Deployment Strategy Comparison Table

MLOps Orchestration Services Table

Advanced MLOps Supplements

SageMaker Model Monitor

SageMaker Model Monitor automatically monitors the quality of ML models in production and notifies you of drift or quality issues.

Types of monitoring

flowchart TD
    MM[SageMaker Model Monitor] --> DQ[Data Quality\nInput data drift]
    MM --> MQ[Model Quality\nAccuracy metric drift]
    MM --> BD[Bias Drift\nBias in predictions]
    MM --> FAD[Feature Attribution Drift\nDrift in feature importance]

How Model Monitor works

sequenceDiagram
    participant Client
    participant Endpoint as SageMaker Endpoint
    participant DC as Data Capture S3
    participant Baseline as Baseline S3
    participant MJ as Monitoring Job
    participant CW as CloudWatch

    Client->>Endpoint: Inference request
    Endpoint->>DC: Automatic capture of input + output
    MJ->>DC: Analysis of captured data
    MJ->>Baseline: Compare with baseline
    MJ->>CW: Metrics and violations
    CW-->>Client: SNS alerts if drift detected

Configuration steps:

1. Enable data capture on the endpoint (configurable sampling)
2. Create a baseline from training data
3. Configure a monitoring schedule (e.g., CronExpressionGenerator.hourly())
4. Inspect reports and violations in CloudWatch

Python code — Complete Model Monitor setup:

from sagemaker.model_monitor import DefaultModelMonitor, DataCaptureConfig
from sagemaker.model_monitor.dataset_format import DatasetFormat
from sagemaker.model_monitor import CronExpressionGenerator

# 1. Enable data capture on the endpoint
data_capture_config = DataCaptureConfig(
    enable_capture=True,
    sampling_percentage=100,
    destination_s3_uri='s3://bucket/data-capture/',
    capture_options=['REQUEST', 'RESPONSE']
)

predictor = model.deploy(
    initial_instance_count=1,
    instance_type='ml.m5.xlarge',
    data_capture_config=data_capture_config
)

# 2. Create Monitor and calculate baseline
my_monitor = DefaultModelMonitor(
    role=role,
    instance_count=1,
    instance_type='ml.m5.xlarge',
    volume_size_in_gb=20,
    max_runtime_in_seconds=3600
)

my_monitor.suggest_baseline(
    baseline_dataset='s3://bucket/baseline-data/baseline.csv',
    dataset_format=DatasetFormat.csv(header=True),
    output_s3_uri='s3://bucket/baseline-results/',
    wait=True
)

# 3. Create monitoring schedule (every hour)
my_monitor.create_monitoring_schedule(
    monitor_schedule_name='hourly-data-quality-monitor',
    endpoint_input=predictor.endpoint_name,
    output_s3_uri='s3://bucket/monitor-reports/',
    statistics=my_monitor.baseline_statistics(),
    constraints=my_monitor.suggested_constraints(),
    schedule_cron_expression=CronExpressionGenerator.hourly()
)

Exam points:

• Model Monitor supports real-time endpoints and batch transform jobs
• Does not support multi-model endpoints
• Monitors: data quality, model quality, bias drift, feature attribution drift
• Uses CloudWatch for alerts; can trigger EventBridge → retraining

SageMaker Model Registry

The SageMaker Model Registry is a centralized catalog for managing the complete ML model lifecycle, from registration to production.

Key features

flowchart TD
    MR[Model Registry] --> MG[Model Groups\nGroups by use case]
    MR --> MV[Model Versions\nRegistered versions + metrics]
    MR --> AS[Approval Status\nPending / Approved / Rejected]
    MR --> MC[Model Cards\nModel documentation]
    MR --> LT[Lineage Tracking\nEnd-to-end traceability]
    MR --> CI[CI/CD Integration\nAutomated deployment]

Typical workflow with approval

flowchart LR
    Train[Training Job\nSageMaker] --> Eval[Evaluation\nof metrics]
    Eval -->|Metrics OK| Reg[Register\nin Model Registry\nPendingManualApproval]
    Reg --> Review[Team review\nData scientists]
    Review -->|Approved| CD[CodePipeline\nAuto deployment]
    Review -->|Rejected| Retrain[Retraining\nwith adjustments]
    CD --> Prod[SageMaker Endpoint\nProduction]
    Prod --> Monitor[Model Monitor\nContinuous monitoring]

Python code — Model registration and approval:

import boto3
from sagemaker.workflow.step_collections import RegisterModel
from sagemaker.model_metrics import ModelMetrics, MetricsSource

sm = boto3.client('sagemaker')

# 1. Create a Model Package Group
sm.create_model_package_group(
    ModelPackageGroupName='churn-prediction-models',
    ModelPackageGroupDescription='Customer churn prediction models v1+'
)

# 2. In a SageMaker Pipeline: RegisterModel step
register_step = RegisterModel(
    name='RegisterChurnModel',
    estimator=xgb_estimator,
    model_data=training_step.properties.ModelArtifacts.S3ModelArtifacts,
    content_types=['text/csv'],
    response_types=['text/csv'],
    inference_instances=['ml.m5.xlarge', 'ml.m5.2xlarge'],
    model_package_group_name='churn-prediction-models',
    approval_status='PendingManualApproval',
    model_metrics=ModelMetrics(
        model_statistics=MetricsSource(
            s3_uri='s3://bucket/evaluation/metrics.json',
            content_type='application/json'
        )
    )
)

# 3. Approve the model (CLI, console, or EventBridge trigger)
sm.update_model_package(
    ModelPackageArn='arn:aws:sagemaker:us-east-1:123456789:model-package/churn-prediction-models/2',
    ModelApprovalStatus='Approved'
)

Exam points: Model Registry = catalog models + manage versions + associate training metrics + approval statuses + automate CI/CD deployment. Natively integrates with SageMaker Pipelines.

SageMaker Feature Store

SageMaker Feature Store simplifies the creation, storage, sharing and management of ML features between teams and across lifecycle phases (training and inference).

Feature Store Architecture

flowchart TD
    Raw[Raw Data\nS3 / Kinesis / Kafka] --> FP[Feature Processing\nData Wrangler / Custom code]
    FP -->|PutRecord API\nStreaming or Batch| FS[Feature Store]

    FS --> Online[Online Store\nMillisecond latency\nLatest records]
    FS --> Offline[Offline Store\nS3 Parquet\nComplete history]

    Online --> RT[Real-time Inference\nLow latency]
    Offline --> TR[Model Training\nComplete historical dataset]
    Offline --> BI[Batch Inference\nOffline predictions]
    Offline --> Explore[Exploration\nAthena SQL queries]

Online Store vs Offline Store:

Python code — Complete Feature Store:

import boto3
from sagemaker.feature_store.feature_group import FeatureGroup

sm_session = sagemaker.Session()

# 1. Create a Feature Group
feature_group = FeatureGroup(
    name='customer-churn-features',
    sagemaker_session=sm_session
)

import pandas as pd
df = pd.DataFrame({
    'customer_id': ['C001'],
    'event_time': [1704067200.0],
    'age': [35],
    'tenure_months': [24],
    'monthly_charges': [65.5],
    'total_charges': [1572.0],
    'contract_type': [1]
})

feature_group.load_feature_definitions(data_frame=df)

feature_group.create(
    s3_uri='s3://bucket/feature-store/',
    record_identifier_name='customer_id',
    event_time_feature_name='event_time',
    role_arn=role,
    enable_online_store=True,
    description='Customer features for churn prediction'
)

# 2. Ingest features (batch)
feature_group.ingest(data_frame=df, max_workers=4, wait=True)

# 3. Ingest in streaming (real-time)
featurestore_runtime = boto3.client('sagemaker-featurestore-runtime')

featurestore_runtime.put_record(
    FeatureGroupName='customer-churn-features',
    Record=[
        {'FeatureName': 'customer_id', 'ValueAsString': 'C002'},
        {'FeatureName': 'event_time', 'ValueAsString': '1704067200.0'},
        {'FeatureName': 'monthly_charges', 'ValueAsString': '72.5'},
    ]
)

# 4. Retrieve a feature in real-time (Online Store)
record = featurestore_runtime.get_record(
    FeatureGroupName='customer-churn-features',
    RecordIdentifierValueAsString='C001'
)
print(record['Record'])

Exam points:

• Feature Store reduces training-serving skew (feature difference between training and production)
• Online store: low latency, for real-time inference
• Offline store (S3 Parquet): for training and batch inference
• Streaming ingestion (PutRecord) or batch (Processing Job)
• Resilient: distributed across multiple AZs

A/B Testing with SageMaker

A/B testing allows testing multiple model versions simultaneously in production by distributing traffic between variants.

flowchart LR
    Client -->|100% requests| EP[SageMaker Endpoint\nMulti-Variant Config]
    EP -->|70% traffic| V1[Variant A\nModel v1\nml.m5.xlarge]
    EP -->|30% traffic| V2[Variant B\nModel v2\nml.m5.2xlarge]

    V1 --> CW[CloudWatch\nPer-variant metrics]
    V2 --> CW
    CW --> Analyze[A/B Analysis\nCompare latency, accuracy]
    Analyze -->|V2 better| Shift[Switch 100% to V2]

Python code — Complete A/B test:

import boto3

sm = boto3.client('sagemaker')

# 1. Create an EndpointConfig with two production variants
sm.create_endpoint_config(
    EndpointConfigName='ab-test-churn-config',
    ProductionVariants=[
        {
            'VariantName': 'ModelV1',
            'ModelName': 'churn-model-v1',
            'InstanceType': 'ml.m5.xlarge',
            'InitialInstanceCount': 1,
            'InitialVariantWeight': 7       # 70% traffic
        },
        {
            'VariantName': 'ModelV2',
            'ModelName': 'churn-model-v2',
            'InstanceType': 'ml.m5.2xlarge',
            'InitialInstanceCount': 1,
            'InitialVariantWeight': 3       # 30% traffic
        }
    ]
)

# 2. Force a request to a specific variant
sm_runtime = boto3.client('sagemaker-runtime')
response = sm_runtime.invoke_endpoint(
    EndpointName='churn-ab-test',
    ContentType='text/csv',
    Body='35,24,65.5,1',
    TargetVariant='ModelV2'   # Force ModelV2 for this test
)

# 3. Switch all traffic to the best variant
sm.update_endpoint_weights_and_capacities(
    EndpointName='churn-ab-test',
    DesiredWeightsAndCapacities=[
        {'VariantName': 'ModelV1', 'DesiredWeight': 0},   # Stop V1
        {'VariantName': 'ModelV2', 'DesiredWeight': 1}    # 100% to V2
    ]
)

Shadow Deployment

Shadow deployment (shadow mode) allows testing a new model on real production traffic without impacting end users.

flowchart TD
    Client -->|Request| EP[SageMaker Endpoint]
    EP -->|100% - Response returned to client| Prod[Production Model\nCurrent version]
    EP -.->|100% copied - Response ignored| Shadow[Shadow Model\nNew version to test]

    Prod --> CW1[CloudWatch\nProduction metrics]
    Shadow --> CW2[CloudWatch\nShadow metrics]
    CW1 --> Compare[Performance\ncomparison]
    CW2 --> Compare
    Compare --> Decision{Decision}
    Decision -->|Shadow better| Deploy[Deploy to production]
    Decision -->|Issues detected| Discard[Discard the model]

Python code — Shadow deployment:

sm.create_endpoint_config(
    EndpointConfigName='shadow-test-config',
    ProductionVariants=[
        {
            'VariantName': 'Production',
            'ModelName': 'churn-model-v2',
            'InstanceType': 'ml.m5.xlarge',
            'InitialInstanceCount': 1,
            'InitialVariantWeight': 1
        }
    ],
    ShadowProductionVariants=[
        {
            'VariantName': 'Shadow',
            'ModelName': 'churn-model-v3',   # New version to test silently
            'InstanceType': 'ml.m5.xlarge',
            'InitialInstanceCount': 1,
            'SamplingPercentage': 100         # 100% of traffic copied to shadow
        }
    ]
)

MLflow on AWS

MLflow is an open-source framework for managing the ML lifecycle (experiment tracking, model registry, deployment). It integrates natively with SageMaker.

MLflow + SageMaker Integration

flowchart TD
    Dev[Developer\nPython Code + MLflow] --> MLT[MLflow Tracking\nExperiments / Runs / Metrics / Artifacts]
    MLT --> S3[Artifacts stored\nin S3]
    MLT --> DB[Tracking Server\nRDS Aurora / EC2]

    MLT -->|mlflow.sagemaker.deploy| SM[SageMaker Endpoint]
    MLT -->|Register Model| SMR[SageMaker Model Registry\nAutomatic registration]
    SMR --> Approve[Approval + Deployment\nAutomated CI/CD]

Python code — MLflow with SageMaker:

import mlflow
import mlflow.xgboost
import xgboost as xgb
from sklearn.metrics import accuracy_score, roc_auc_score

mlflow.set_tracking_uri('http://mlflow-server.mycompany.com:5000')
mlflow.set_experiment('churn-prediction-v3')

with mlflow.start_run(run_name='xgboost-optimized') as run:
    params = {'max_depth': 6, 'learning_rate': 0.05, 'n_estimators': 200, 'subsample': 0.8}
    model = xgb.XGBClassifier(**params)
    model.fit(X_train, y_train)

    y_pred = model.predict(X_test)
    y_proba = model.predict_proba(X_test)[:, 1]
    acc = accuracy_score(y_test, y_pred)
    auc = roc_auc_score(y_test, y_proba)

    mlflow.log_params(params)
    mlflow.log_metrics({'accuracy': acc, 'auc': auc})
    mlflow.xgboost.log_model(model, 'xgboost-churn-model')

    print(f"Run ID: {run.info.run_id} | Accuracy: {acc:.4f} | AUC: {auc:.4f}")

# Deploy the MLflow model to SageMaker
mlflow.sagemaker.deploy(
    app_name='churn-mlflow-endpoint',
    model_uri=f'runs:/{run.info.run_id}/xgboost-churn-model',
    region_name='us-east-1',
    mode='create',
    execution_role_arn=role,
    instance_type='ml.m5.xlarge',
    instance_count=1
)

AWS Step Functions for MLOps — Deep Dive

Step Functions allows orchestrating complex ML workflows with graphical visualization, error handling, and native SageMaker integration.

Complete MLOps workflow diagram

stateDiagram-v2
    [*] --> IngestNewData
    IngestNewData --> PreprocessData: Success
    IngestNewData --> HandleError: S3/Glue error
    PreprocessData --> TrainModel: Success
    PreprocessData --> HandleError: Processing error
    TrainModel --> EvaluateModel: Training complete
    TrainModel --> HandleError: Training error
    EvaluateModel --> CheckAccuracy: Metrics calculated
    CheckAccuracy --> RegisterModel: Accuracy >= 85%
    CheckAccuracy --> RetrainWithTuning: Accuracy < 85%
    RegisterModel --> WaitForApproval: Registered
    WaitForApproval --> DeployToProduction: Approved
    WaitForApproval --> HandleRejection: Rejected
    DeployToProduction --> EnableMonitoring: Deployed
    EnableMonitoring --> [*]: Monitoring configured
    RetrainWithTuning --> TrainModel: Adjusted hyperparameters
    HandleError --> SendNotification: SNS alert
    SendNotification --> [*]
    HandleRejection --> [*]

Python code — Step Functions State Machine for MLOps:

import json
import boto3

sf = boto3.client('stepfunctions')

definition = {
    "Comment": "Automated MLOps Pipeline with SageMaker",
    "StartAt": "PreprocessData",
    "States": {
        "PreprocessData": {
            "Type": "Task",
            "Resource": "arn:aws:states:::sagemaker:createProcessingJob.sync",
            "Parameters": {
                "ProcessingJobName.$": "States.Format('preprocess-{}', $$.Execution.Name)",
                "AppSpecification": {
                    "ImageUri": "<preprocessing-image>",
                    "ContainerEntrypoint": ["python3", "preprocess.py"]
                },
                "ProcessingResources": {
                    "ClusterConfig": {
                        "InstanceCount": 1,
                        "InstanceType": "ml.m5.xlarge",
                        "VolumeSizeInGB": 20
                    }
                },
                "RoleArn": "<sagemaker-role>"
            },
            "Next": "TrainModel",
            "Catch": [{"ErrorEquals": ["States.ALL"], "Next": "HandleError"}]
        },
        "TrainModel": {
            "Type": "Task",
            "Resource": "arn:aws:states:::sagemaker:createTrainingJob.sync",
            "Parameters": {
                "TrainingJobName.$": "States.Format('train-{}', $$.Execution.Name)",
                "AlgorithmSpecification": {
                    "TrainingImage": "<xgboost-image>",
                    "TrainingInputMode": "File"
                },
                "ResourceConfig": {
                    "InstanceType": "ml.m5.xlarge",
                    "InstanceCount": 1,
                    "VolumeSizeInGB": 30
                },
                "RoleArn": "<sagemaker-role>"
            },
            "Next": "CheckAccuracy",
            "Catch": [{"ErrorEquals": ["States.ALL"], "Next": "HandleError"}]
        },
        "CheckAccuracy": {
            "Type": "Choice",
            "Choices": [
                {
                    "Variable": "$.evaluation.accuracy",
                    "NumericGreaterThanEquals": 0.85,
                    "Next": "DeployModel"
                }
            ],
            "Default": "HandleError"
        },
        "DeployModel": {
            "Type": "Task",
            "Resource": "arn:aws:states:::sagemaker:createEndpoint.sync",
            "Parameters": {
                "EndpointName": "churn-production-endpoint",
                "EndpointConfigName": "churn-config-latest"
            },
            "End": True
        },
        "HandleError": {
            "Type": "Task",
            "Resource": "arn:aws:states:::sns:publish",
            "Parameters": {
                "TopicArn": "arn:aws:sns:us-east-1:123456789:MLOpsAlerts",
                "Message.$": "States.Format('MLOps pipeline error: {}', $.Cause)",
                "Subject": "MLOps Alert - Pipeline failure"
            },
            "End": True
        }
    }
}

# Create the State Machine
sf.create_state_machine(
    name='ChurnMLOpsPipeline',
    definition=json.dumps(definition),
    roleArn='arn:aws:iam::123456789:role/StepFunctionsMLRole',
    type='STANDARD'
)

# Start an execution
execution = sf.start_execution(
    stateMachineArn='arn:aws:states:us-east-1:123456789:stateMachine:ChurnMLOpsPipeline',
    name='pipeline-run-2024-06-15',
    input=json.dumps({
        's3_data_uri': 's3://bucket/new-data/june-2024/',
        'target_accuracy': 0.85
    })
)

print(f"Execution started: {execution['executionArn']}")

Enriched Summary

Complete MLOps Architecture — Overview

flowchart TD
    subgraph Source
        Code[Code Repository\nGitHub/CodeCommit] --> CI[CodeBuild\nCI: Tests + Build]
        DataSrc[New data\nS3/Kinesis/Glue]
    end

    subgraph Orchestration
        CI --> CP[CodePipeline\nAutomated CI/CD]
        DataSrc --> EB[EventBridge\nEvent detection]
        EB --> SF[Step Functions\nMLOps Workflow]
        CP --> SF
    end

    subgraph ML Pipeline
        SF --> SMP[SageMaker Pipelines]
        SMP --> FSPre[Feature Store\nIngestion + Enrichment]
        FSPre --> Pre[Processing Job\nPreprocessing]
        Pre --> Train[Training Job\nTrainium / GPU]
        Train --> Eval[Evaluation\nMetrics]
        Eval -->|OK| MR[Model Registry\nPendingApproval]
        Eval -->|NOK| Alert[SNS Alert\nML team]
        MR -->|Approved| Deploy{Deployment\nstrategy}
    end

    subgraph Deployment
        Deploy -->|Continuous traffic| RT[Real-time Endpoint\nA/B Testing / Canary]
        Deploy -->|Irregular traffic| SL[Serverless Endpoint]
        Deploy -->|Large loads| AS[Async Endpoint]
        Deploy -->|Offline processing| BI[Batch Transform]
    end

    subgraph Monitoring
        RT --> MM[Model Monitor\nData Quality\nModel Quality\nBias Drift]
        MM --> CW[CloudWatch\nAlerts + Dashboards]
        CW --> EB
    end

    subgraph IaC Infrastructure
        CF[CloudFormation / CDK] -.->|Provisions| RT
        CF -.->|Provisions| SL
        VPC[PrivateLink / VPC\nInterface Endpoints] -.->|Secures| RT
    end

Key Services Checklist for MLA-C01 Exam

Search Terms

mla-c01 · aws · ml · engineer · associate · deployment · orchestration · ai · machine · amazon · web · services · sagemaker · model · endpoints · question · endpoint · exam · mlops · pipelines · strategy · automated · metrics · selecting