Azure ML Studio and SDK – Overview

Level: Beginner to Intermediate Goal: Master the 3 Azure ML interfaces (Studio, CLI, SDK)

Table of Contents

1. Azure ML Overview
2. Azure ML Studio – Visual Interface
3. Notebooks in Azure ML Studio
4. Azure ML SDK v2 in Python
5. Azure ML CLI v2
6. Studio vs CLI vs SDK – Comparison
7. Compute – Compute Resources
8. Datasets and Data Assets
9. Environments and Reproducibility
10. First End-to-End Job
11. Best Practices
12. Summary and Key Points
13. Glossary

1. Azure ML Overview

1.1 Why Azure ML?

Imagine: you work locally with Jupyter Notebooks. That’s fine to start. But when you need to:

• Scale your experiments to large data volumes
• Collaborate with a geographically distributed team
• Reproduce a result from 3 months ago exactly
• Deploy your model to production reliably
• Monitor model performance over time

…local tools show their limits. That’s where Azure Machine Learning comes in.

flowchart LR
    subgraph "Problems with Local ML"
        P1["❌ No scaling\n(limited RAM/GPU)"]
        P2["❌ No tracking\n('best_model_final_v3_bis.pkl')"]
        P3["❌ No collaboration\n(sharing notebooks is hard)"]
        P4["❌ Manual deployment\n(and fragile)"]
    end
    
    subgraph "Azure ML Solutions"
        S1["✅ Compute Clusters\n(on-demand GPU)"]
        S2["✅ Experiment Tracking\n(integrated MLflow)"]
        S3["✅ Shared Workspace\n(datasets, models)"]
        S4["✅ Managed Endpoints\n(1-line deployment)"]
    end
    
    P1 --> S1
    P2 --> S2
    P3 --> S3
    P4 --> S4

1.2 Azure ML Components

mindmap
  root((Azure ML\nWorkspace))
    Authoring
      Notebooks
      Automated ML
      Designer
    Assets
      Data Assets
      Environments
      Components
      Models
      Endpoints
    Jobs
      Experiments
      Pipelines
      Sweeps
    Compute
      Compute Instances
      Compute Clusters
      Serverless
      Attached Compute
    Monitoring
      MLflow Integration
      Model Monitor
      Alerts

2. Azure ML Studio – Visual Interface

2.1 Navigating Azure ML Studio

Access: ml.azure.com or from Azure Portal → Workspace → Launch Studio

Main menu:

2.2 Creating Your First AutoML Job via Studio

Scenario: Predict whether a customer will subscribe to a term deposit.

Steps in Azure ML Studio:

1. Navigation: Automated ML → New Automated ML Job

2. Basic configuration:
   - Job name: bank-marketing-automl-001
   - Experiment name: bank-marketing-classification
   - Description: "Predict term deposit subscription"

3. Task type selection:
   - Task type: Classification
   
4. Dataset:
   - Create a new dataset
   - Source: Upload local CSV (bank-marketing.csv)
   - Type: Tabular
   - Storage: Azure Blob Storage (workspace default)

5. Model configuration:
   - Target column: y (target variable)
   - Positive class value: yes
   
6. Training parameters:
   - Primary metric: AUC weighted
   - Training timeout: 60 minutes
   - Max concurrent trials: 4
   - Enable deep learning: False (optional)
   
7. Compute:
   - Select existing cluster or create new
   - VM size: Standard_DS3_v2 (4 vCPUs, 14GB)
   
8. Submit → Monitor in Jobs tab

2.3 Jobs Interface – Real-Time Monitoring

In the Jobs tab, you can:

3. Notebooks in Azure ML Studio

3.1 Advantages of Integrated Notebooks

Azure ML Studio integrates a Jupyter Notebook environment directly in the browser, without local installation.

flowchart LR
    subgraph "Azure ML Notebooks Advantages"
        C1["⚡ Flexible compute\n(Attach CPU/GPU as needed)"]
        C2["🔄 Native integration\n(MLClient, direct SDK)"]
        C3["📁 Persistent storage\n(Files in Azure)"]
        C4["🔧 Managed environments\n(No local installation)"]
        C5["👥 Collaboration\n(Workspace sharing)"]
    end

Available Compute types:

3.2 Typical Workflow in a Notebook

# 1. Import and connection
from azure.ai.ml import MLClient
from azure.identity import DefaultAzureCredential

# Connect to workspace from Studio notebook
# (Credentials are automatically managed)
ml_client = MLClient.from_config(credential=DefaultAzureCredential())

workspace = ml_client.workspaces.get(ml_client.workspace_name)
print(f"✅ Connected: {workspace.name} ({workspace.location})")

# 2. Data exploration
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

# Load a registered dataset
dataset = ml_client.data.get("bank-marketing", version="1")
print(f"Dataset: {dataset.name} ({dataset.type})")

# 3. Interactive training
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.metrics import classification_report, confusion_matrix
import numpy as np

# ... training code ...

# 4. MLflow tracking (automatically integrated)
import mlflow

mlflow.set_experiment("notebook-explorations")

with mlflow.start_run(run_name="notebook-rf-exploration"):
    # Train
    model = RandomForestClassifier(n_estimators=100, random_state=42)
    model.fit(X_train, y_train)
    
    # Evaluate
    accuracy = model.score(X_test, y_test)
    cv_mean = cross_val_score(model, X_train, y_train, cv=5).mean()
    
    # Log
    mlflow.log_param("n_estimators", 100)
    mlflow.log_metric("accuracy", accuracy)
    mlflow.log_metric("cv_accuracy_mean", cv_mean)
    
    print(f"Accuracy: {accuracy:.4f}")
    print(f"CV Mean: {cv_mean:.4f}")

# 5. Submit a job from the notebook
from azure.ai.ml import command, Input

job = command(
    code="./src",
    command="python train.py --data ${{inputs.data}}",
    inputs={"data": Input(path="azureml:bank-marketing:1")},
    compute="cpu-cluster",
    environment="AzureML-sklearn-1.0-ubuntu20.04-py38-cpu@latest"
)

submitted = ml_client.jobs.create_or_update(job)
print(f"Job submitted from notebook: {submitted.name}")

3.3 Complete Example: Classification in a Notebook

# Complete classification example in an Azure ML Studio Notebook
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.metrics import (
    accuracy_score, classification_report, 
    confusion_matrix, ConfusionMatrixDisplay
)
from sklearn.preprocessing import StandardScaler
import mlflow
import mlflow.sklearn

# === 1. DATA LOADING ===
print("=== 1. Loading data ===")
iris = load_iris()
df = pd.DataFrame(
    iris.data, 
    columns=iris.feature_names
)
df['species'] = pd.Categorical.from_codes(iris.target, iris.target_names)

print(f"Shape: {df.shape}")
print(f"Species: {df['species'].value_counts().to_dict()}")
print(df.describe().round(2))

# === 2. VISUAL EXPLORATION ===
fig, axes = plt.subplots(2, 2, figsize=(12, 10))

for idx, feature in enumerate(iris.feature_names):
    ax = axes[idx // 2][idx % 2]
    for species in iris.target_names:
        mask = df['species'] == species
        ax.hist(df.loc[mask, feature], alpha=0.7, label=species, bins=15)
    ax.set_title(feature)
    ax.legend()

plt.suptitle("Feature distribution by species", fontsize=14)
plt.tight_layout()
plt.savefig("feature_distributions.png", dpi=150)
plt.show()

# === 3. PREPARATION ===
X = df[iris.feature_names].values
y = iris.target

# Normalization
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

X_train, X_test, y_train, y_test = train_test_split(
    X_scaled, y, test_size=0.2, random_state=42, stratify=y
)

print(f"\nTrain: {X_train.shape[0]} | Test: {X_test.shape[0]}")

# === 4. COMPARE 3 MODELS ===
mlflow.set_experiment("iris-classification-comparison")

models_to_test = {
    "Logistic Regression": LogisticRegression(max_iter=200),
    "Random Forest": RandomForestClassifier(n_estimators=100, random_state=42),
    "Gradient Boosting": GradientBoostingClassifier(n_estimators=100, random_state=42)
}

results = {}

for name, model in models_to_test.items():
    with mlflow.start_run(run_name=f"iris-{name.lower().replace(' ', '-')}"):
        # Train
        model.fit(X_train, y_train)
        
        # Predict
        y_pred = model.predict(X_test)
        
        # Metrics
        accuracy = accuracy_score(y_test, y_pred)
        
        # Log
        mlflow.log_param("model_type", name)
        mlflow.log_metric("test_accuracy", accuracy)
        mlflow.sklearn.log_model(model, "model")
        
        results[name] = {
            "model": model,
            "accuracy": accuracy,
            "report": classification_report(
                y_test, y_pred, 
                target_names=iris.target_names,
                output_dict=True
            )
        }
        
        print(f"{name}: {accuracy:.4f}")

# === 5. RESULTS VISUALIZATION ===
# Find the best model
best_name = max(results, key=lambda k: results[k]["accuracy"])
best_model = results[best_name]["model"]
y_pred_best = best_model.predict(X_test)

# Confusion matrix
fig, ax = plt.subplots(figsize=(8, 6))
cm = confusion_matrix(y_test, y_pred_best)
disp = ConfusionMatrixDisplay(
    confusion_matrix=cm,
    display_labels=iris.target_names
)
disp.plot(ax=ax, cmap='Blues')
ax.set_title(f"Confusion Matrix\n({best_name} - Accuracy: {results[best_name]['accuracy']:.2%})")
plt.tight_layout()
plt.savefig("confusion_matrix.png", dpi=150)
plt.show()

# Model comparison
fig, ax = plt.subplots(figsize=(8, 5))
names = list(results.keys())
scores = [results[n]["accuracy"] for n in names]
colors = ['#2ecc71' if n == best_name else '#3498db' for n in names]

bars = ax.bar(names, scores, color=colors)
ax.set_ylim(0.9, 1.02)
ax.set_ylabel("Accuracy")
ax.set_title("Model Comparison")
ax.bar_label(bars, labels=[f"{s:.2%}" for s in scores], padding=3)

plt.tight_layout()
plt.savefig("model_comparison.png", dpi=150)
plt.show()

print(f"\n🏆 Best model: {best_name} ({results[best_name]['accuracy']:.2%})")

4. Azure ML SDK v2 in Python

4.1 SDK v2 Architecture

flowchart TD
    SDK["Azure ML SDK v2\n(azure-ai-ml)"] --> MLCLIENT["MLClient\n(Main entry point)"]
    
    MLCLIENT --> JOBS["jobs\n(Create, Submit,\nMonitor)"]
    MLCLIENT --> DATA["data\n(Datasets,\nDatastores)"]
    MLCLIENT --> COMPUTE["compute\n(Clusters,\nInstances)"]
    MLCLIENT --> MODELS["models\n(Registry,\nVersions)"]
    MLCLIENT --> ENVIRONMENTS["environments\n(Dependencies)"]
    MLCLIENT --> ENDPOINTS["online_endpoints\nbatch_endpoints\n(Deployment)"]
    MLCLIENT --> COMPONENTS["components\n(Step\nReuse)"]

4.2 Installation and Configuration

# SDK v2 installation
pip install azure-ai-ml
pip install azure-identity  # For authentication
pip install mlflow          # For tracking
pip install azureml-mlflow  # Azure ML + MLflow integration

# Verify
python -c "import azure.ai.ml; print(azure.ai.ml.__version__)"

# MLClient configuration and initialization
from azure.ai.ml import MLClient
from azure.identity import (
    DefaultAzureCredential,
    InteractiveBrowserCredential,
    ClientSecretCredential
)
import json
import os

def create_ml_client(auth_method: str = "default") -> MLClient:
    """
    Creates an MLClient with the specified authentication method.
    
    Args:
        auth_method: "default", "interactive", "service_principal"
    
    Returns:
        MLClient connected to the workspace
    """
    # Choose authentication method
    if auth_method == "default":
        # Tries in order: Managed Identity, CLI, VS Code, Interactive
        credential = DefaultAzureCredential()
    
    elif auth_method == "interactive":
        # Opens a browser for sign-in
        credential = InteractiveBrowserCredential()
    
    elif auth_method == "service_principal":
        # For CI/CD (GitHub Actions, Azure DevOps)
        credential = ClientSecretCredential(
            tenant_id=os.environ["AZURE_TENANT_ID"],
            client_id=os.environ["AZURE_CLIENT_ID"],
            client_secret=os.environ["AZURE_CLIENT_SECRET"]
        )
    
    else:
        raise ValueError(f"Unknown auth method: {auth_method}")
    
    # Create the client
    client = MLClient(
        credential=credential,
        subscription_id=os.environ["AZURE_SUBSCRIPTION_ID"],
        resource_group_name=os.environ["AZURE_RESOURCE_GROUP"],
        workspace_name=os.environ["AZURE_ML_WORKSPACE"]
    )
    
    # Validate the connection
    ws = client.workspaces.get(os.environ["AZURE_ML_WORKSPACE"])
    print(f"✅ Connected: {ws.name}")
    print(f"   Region: {ws.location}")
    print(f"   Resource Group: {ws.resource_group}")
    
    return client

# Usage
ml_client = create_ml_client("default")

4.3 Submitting a Job via the SDK

# Submit a complete training job
from azure.ai.ml import command, Input, Output
from azure.ai.ml.constants import AssetTypes
from azure.ai.ml.entities import Environment
import os

# Training file
TRAINING_CODE = """
import argparse
import pandas as pd
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
import mlflow

# Arguments
parser = argparse.ArgumentParser()
parser.add_argument("--max_iter", type=int, default=200)
parser.add_argument("--C", type=float, default=1.0)
parser.add_argument("--model_output", type=str, default="./outputs")
args = parser.parse_args()

# Data
iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(
    iris.data, iris.target, test_size=0.2, random_state=42
)

# Training
mlflow.sklearn.autolog()

with mlflow.start_run():
    model = LogisticRegression(max_iter=args.max_iter, C=args.C)
    model.fit(X_train, y_train)
    
    accuracy = accuracy_score(y_test, model.predict(X_test))
    mlflow.log_metric("test_accuracy", accuracy)
    
    print(f"Test Accuracy: {accuracy:.4f}")

print("Training complete!")
"""

# Create folder and file
os.makedirs("./src_training", exist_ok=True)
with open("./src_training/train_iris.py", "w") as f:
    f.write(TRAINING_CODE)

# Define the job
job = command(
    # Code to execute
    code="./src_training",
    command="python train_iris.py --max_iter ${{inputs.max_iter}} --C ${{inputs.C}}",
    
    # Inputs
    inputs={
        "max_iter": 300,
        "C": 0.5
    },
    
    # Compute
    compute="cpu-cluster-standard",
    
    # Environment
    environment="AzureML-sklearn-1.0-ubuntu20.04-py38-cpu@latest",
    
    # Metadata
    experiment_name="iris-sdk-training",
    display_name="Iris-Classification-SDK-v1",
    description="Train an Iris classification model with Logistic Regression",
    tags={
        "dataset": "iris",
        "algorithm": "logistic_regression",
        "submitted_by": os.environ.get("USER", "unknown")
    }
)

# Submit
print("Submitting job...")
submitted = ml_client.jobs.create_or_update(job)

print(f"✅ Job submitted!")
print(f"   Name: {submitted.name}")
print(f"   Experiment: {submitted.experiment_name}")
print(f"   Status: {submitted.status}")
print(f"   Studio URL: {submitted.studio_url}")

# Wait for completion
print("\nWaiting for completion...")
ml_client.jobs.stream(submitted.name)

# Retrieve the completed job
completed_job = ml_client.jobs.get(submitted.name)
print(f"\nJob completed with status: {completed_job.status}")

4.4 Managing Environments

# Create and manage Azure ML environments
from azure.ai.ml.entities import Environment, BuildContext

# Option 1: Environment from conda YAML
def create_conda_env(name: str, version: str = "1") -> Environment:
    """Creates an environment from a conda file."""
    
    conda_content = """
name: ml-environment
channels:
  - conda-forge
  - defaults
dependencies:
  - python=3.10
  - pip:
    - azure-ai-ml>=1.12.0
    - scikit-learn>=1.3.0
    - xgboost>=2.0.0
    - lightgbm>=4.0.0
    - pandas>=2.0.0
    - numpy>=1.26.0
    - matplotlib>=3.8.0
    - seaborn>=0.13.0
    - mlflow>=2.9.0
    - imbalanced-learn>=0.11.0
    - shap>=0.44.0
"""
    
    with open("conda_env.yml", "w") as f:
        f.write(conda_content)
    
    env = Environment(
        name=name,
        version=version,
        description="Complete ML environment for scikit-learn and XGBoost",
        conda_file="conda_env.yml",
        image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest",
        tags={"type": "ml", "python": "3.10"}
    )
    
    created_env = ml_client.environments.create_or_update(env)
    print(f"✅ Environment created: {created_env.name}:{created_env.version}")
    return created_env

# Option 2: Environment from Dockerfile
def create_docker_env(name: str) -> Environment:
    """Creates an environment from a Dockerfile."""
    
    dockerfile_content = """
FROM mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest

# System dependency installation
RUN apt-get update && apt-get install -y \\
    build-essential \\
    && rm -rf /var/lib/apt/lists/*

# Python packages installation
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Environment variables
ENV PYTHONUNBUFFERED=1
ENV MLFLOW_TRACKING_URI=azureml
"""
    
    os.makedirs("./docker_context", exist_ok=True)
    with open("./docker_context/Dockerfile", "w") as f:
        f.write(dockerfile_content)
    
    with open("./docker_context/requirements.txt", "w") as f:
        f.write("scikit-learn>=1.3.0\nmlflow>=2.9.0\nxgboost>=2.0.0\n")
    
    env = Environment(
        name=name,
        build=BuildContext(path="./docker_context"),
        description="Custom environment from Dockerfile"
    )
    
    created_env = ml_client.environments.create_or_update(env)
    print(f"✅ Docker environment created: {created_env.name}")
    return created_env

# Option 3: Use a curated environment (recommended)
def list_curated_envs() -> list[dict]:
    """Lists Microsoft pre-built environments."""
    curated_envs = []
    
    for env in ml_client.environments.list():
        if env.name.startswith("AzureML-"):
            curated_envs.append({
                "name": env.name,
                "version": env.version,
                "description": (env.description or "")[:80]
            })
    
    return sorted(curated_envs, key=lambda e: e["name"])

print("=== Available curated environments ===")
for curated_env in list_curated_envs()[:10]:
    print(f"  {curated_env['name']}:{curated_env['version']}")

5. Azure ML CLI v2

5.1 Installation and Configuration

# Install Azure CLI and ML extension
az version
az extension add -n ml -y
az extension update -n ml

# Verify installation
az ml --version

# Login
az login
az account set --subscription "My Azure Subscription"
az configure --defaults group=my-resource-group workspace=my-workspace

5.2 Essential CLI Commands

# Workspace management
az ml workspace show
az ml workspace list

# Data management
az ml data list
az ml data create --name my-dataset --type uri_file --path azureml://datastores/workspaceblobstore/paths/data/train.csv
az ml data show --name my-dataset --version 1

# Job management
az ml job create --file job.yml
az ml job show --name job-xxx-xxx
az ml job list --experiment-name my-experiment
az ml job stream --name job-xxx-xxx
az ml job download --name job-xxx-xxx --output-name model

# Model management
az ml model list
az ml model show --name my-model --version 1
az ml model create --name my-model --type mlflow_model --path runs:/run-id/model

# Endpoint management
az ml online-endpoint list
az ml online-endpoint create --file endpoint.yml
az ml online-deployment create --file deployment.yml --endpoint-name my-endpoint
az ml online-endpoint invoke --name my-endpoint --request-file test.json

# Compute management
az ml compute list
az ml compute create --name cpu-cluster --type amlcompute --size Standard_DS3_v2 --min-instances 0 --max-instances 4

5.3 YAML Files for the CLI

# job_classification.yml
$schema: https://azuremlschemas.azureedge.net/latest/commandJob.schema.json

type: command
display_name: Bank Marketing Classification
experiment_name: bank-marketing-classification

code: ./src
command: >-
  python train.py
  --data ${{inputs.data}}
  --learning_rate ${{inputs.learning_rate}}
  --n_estimators ${{inputs.n_estimators}}

inputs:
  data:
    type: uri_file
    path: azureml:bank-marketing:1
  learning_rate: 0.05
  n_estimators: 200

environment: AzureML-sklearn-1.0-ubuntu20.04-py38-cpu@latest
compute: azureml:cpu-cluster

limits:
  timeout: 3600

tags:
  project: bank-marketing
  version: "1.0"

# Submit from CLI
az ml job create --file job_classification.yml

6. Studio vs CLI vs SDK – Comparison

6.1 Comparison Table

6.2 Decision Tree

flowchart TD
    Q["Which interface\nto choose?"] --> Q1{"Do you need\nautomation\nor CI/CD?"}
    Q1 -->|No| Q2{"Do you prefer\na visual\ninterface?"}
    Q2 -->|Yes| STUDIO["✅ Azure ML Studio\n(Ideal for:\n• Interactive exploration\n• First experiments\n• Training)"]
    Q2 -->|No, I\nalready code in Python| SDK["✅ SDK v2 (Python)\n(Ideal for:\n• Complex notebooks\n• Custom workflows\n• Code collaboration)"]
    
    Q1 -->|Yes| Q3{"Is the team more\nDevOps\nor Data Science?"}
    Q3 -->|DevOps| CLI["✅ CLI v2\n(Ideal for:\n• Shell scripts\n• Azure DevOps pipelines\n• Ops without Python)"]
    Q3 -->|Data Science| SDK2["✅ SDK v2 (Python)\n(Ideal for:\n• Complex ML pipelines\n• GitHub Actions\n• Advanced parameterization)"]

7. Compute – Compute Resources

7.1 Azure ML Compute Types

flowchart TD
    COMPUTE["Azure ML Compute"] --> CI["Compute Instance\n\n• Individual VM\n• Interactive (notebooks)\n• Auto-start/stop\n• Dev/exploration"]
    COMPUTE --> CC["Compute Cluster\n\n• Scale from 0 to N nodes\n• Batch jobs\n• Shareable between teams\n• Production"]
    COMPUTE --> SL["Serverless Compute\n\n• No provisioning\n• Spark/Python\n• Pay-per-use\n• Burst workloads"]
    COMPUTE --> AC["Attached Compute\n\n• Kubernetes cluster\n• Other Azure VM\n• External resources"]
    
    CI --> CI_USE["Notebooks, quick tests"]
    CC --> CC_USE["AutoML, Pipelines, Training"]
    SL --> SL_USE["Spark jobs, exploration"]
    AC --> AC_USE["Special GPUs, on-premises"]

7.2 Selecting the Right VM

# Programmatic compute management
from azure.ai.ml.entities import AmlCompute, ComputeInstance

# Create a CPU cluster
def create_cpu_cluster(name: str = "cpu-cluster") -> AmlCompute:
    """Creates a standard CPU cluster for ML training."""
    cluster = AmlCompute(
        name=name,
        type="amlcompute",
        size="Standard_DS3_v2",        # 4 vCPUs, 14 GB RAM
        min_instances=0,               # Scale to zero → economical
        max_instances=6,               # Max 6 parallel nodes
        idle_time_before_scale_down=60, # Scale down after 1 min of inactivity
        tier="Dedicated"               # "LowPriority" to reduce costs
    )
    
    return ml_client.compute.begin_create_or_update(cluster).result()

# Create an instance for development
def create_dev_instance(name: str = "dev-instance") -> ComputeInstance:
    """Creates a Compute Instance for interactive development."""
    instance = ComputeInstance(
        name=name,
        size="Standard_DS3_v2",
        idle_time_before_shutdown="PT30M",  # Shutdown after 30 min
        setup_scripts=None
    )
    
    return ml_client.compute.begin_create_or_update(instance).result()

# Start/Stop instances to save costs
def manage_compute_cost(action: str, instance_name: str):
    """Start or stop a Compute Instance."""
    if action == "start":
        ml_client.compute.begin_start(instance_name).result()
        print(f"✅ Instance {instance_name} started")
    elif action == "stop":
        ml_client.compute.begin_stop(instance_name).result()
        print(f"✅ Instance {instance_name} stopped")

# List current compute and its state
print("=== Compute Status ===")
for compute in ml_client.compute.list():
    print(f"  {compute.name} ({compute.type}) - {compute.provisioning_state}")

8. Datasets and Data Assets

8.1 Azure ML Data Asset Types

# Data Asset management
from azure.ai.ml.entities import Data
from azure.ai.ml.constants import AssetTypes

# Create a Data Asset from a local file
def register_local_dataset(
    local_path: str,
    name: str,
    description: str,
    tags: dict = None
) -> Data:
    """Registers a local file as an Azure ML Data Asset."""
    
    asset = Data(
        path=local_path,
        type=AssetTypes.URI_FILE,
        name=name,
        description=description,
        tags=tags or {}
    )
    
    data_asset = ml_client.data.create_or_update(asset)
    print(f"✅ Data Asset registered: {data_asset.name}:{data_asset.version}")
    print(f"   Azure Path: {data_asset.path}")
    
    return data_asset

# Create a Data Asset from Azure Blob Storage
def register_blob_dataset(
    blob_path: str,   # "azureml://datastores/workspaceblobstore/paths/data/train.csv"
    name: str,
    asset_type: str = AssetTypes.URI_FILE
) -> Data:
    """Registers an Azure Blob file as a Data Asset."""
    
    asset = Data(
        path=blob_path,
        type=asset_type,
        name=name
    )
    
    return ml_client.data.create_or_update(asset)

# List and retrieve datasets
def list_datasets() -> list[dict]:
    """Lists all Data Assets in the workspace."""
    return [
        {
            "name": d.name,
            "version": d.version,
            "type": d.type,
            "date": d.creation_context.created_at if d.creation_context else "N/A"
        }
        for d in ml_client.data.list()
    ]

def get_dataset(name: str, version: str = "latest") -> Data:
    """Retrieves a Data Asset by name and version."""
    if version == "latest":
        return ml_client.data.get(name=name, label="latest")
    return ml_client.data.get(name=name, version=version)

# Usage in a job
dataset = get_dataset("bank-marketing")
print(f"Dataset: {dataset.name} v{dataset.version}")
print(f"Type: {dataset.type}")
print(f"Path: {dataset.path}")

9. Environments and Reproducibility

9.1 Why Environments Are Critical

A model trained with scikit-learn 1.2.0 may give different results or not work at all with scikit-learn 1.3.0. Azure ML environments ensure that each run uses exactly the same dependencies.

# Version and reuse environments
from azure.ai.ml.entities import Environment

# Retrieve a specific environment
def get_environment(name: str, version: str = None) -> Environment:
    """Retrieves an environment by name and version."""
    if version:
        return ml_client.environments.get(name=name, version=version)
    return ml_client.environments.get(name=name, label="latest")

# Compare two environments
def compare_environments(name_env1: str, v1: str, name_env2: str, v2: str) -> dict:
    """Compares the packages of two environments."""
    env1 = get_environment(name_env1, v1)
    env2 = get_environment(name_env2, v2)
    
    return {
        "env1": {"name": name_env1, "version": v1, "image": env1.image},
        "env2": {"name": name_env2, "version": v2, "image": env2.image},
        "compatible": env1.image == env2.image
    }

# Registry of recommended environments
RECOMMENDED_ENVS = {
    "sklearn": "AzureML-sklearn-1.0-ubuntu20.04-py38-cpu@latest",
    "sklearn_gpu": "AzureML-sklearn-1.0-ubuntu20.04-py38-gpu@latest",
    "pytorch": "AzureML-pytorch-2.0-ubuntu20.04-py38-cuda11.8-gpu@latest",
    "tensorflow": "AzureML-tensorflow-2.13-ubuntu20.04-py38-cuda11.8-gpu@latest",
    "minimal": "AzureML-minimal-ubuntu20.04-py38-inference@latest"
}

print("=== Recommended environments ===")
for usage, env_name in RECOMMENDED_ENVS.items():
    print(f"  {usage:15}: {env_name}")

10. First End-to-End Job

10.1 Complete Pipeline from the SDK

# Complete pipeline: data → training → evaluation → registration
# Example with the Iris dataset

import os
from azure.ai.ml import MLClient, command, Input, Output, dsl
from azure.ai.ml.constants import AssetTypes
from azure.identity import DefaultAzureCredential

# 1. Connection
ml_client = MLClient(
    credential=DefaultAzureCredential(),
    subscription_id=os.environ["AZURE_SUBSCRIPTION_ID"],
    resource_group_name=os.environ["AZURE_RESOURCE_GROUP"],
    workspace_name=os.environ["AZURE_ML_WORKSPACE"]
)

# 2. Training script
TRAIN_SCRIPT = '''
import argparse
import pandas as pd
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.metrics import accuracy_score, f1_score
import mlflow
import mlflow.sklearn
import joblib
import os

parser = argparse.ArgumentParser()
parser.add_argument("--n_estimators", type=int, default=100)
parser.add_argument("--learning_rate", type=float, default=0.1)
parser.add_argument("--model_dir", type=str, default="./model")
args = parser.parse_args()

# Data
iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(
    iris.data, iris.target, test_size=0.2, random_state=42, stratify=iris.target
)

# Training
mlflow.sklearn.autolog()
with mlflow.start_run():
    model = GradientBoostingClassifier(
        n_estimators=args.n_estimators,
        learning_rate=args.learning_rate,
        random_state=42
    )
    model.fit(X_train, y_train)
    
    y_pred = model.predict(X_test)
    acc = accuracy_score(y_test, y_pred)
    f1 = f1_score(y_test, y_pred, average="macro")
    
    mlflow.log_metric("test_accuracy", acc)
    mlflow.log_metric("test_f1_macro", f1)
    
    print(f"✅ Accuracy: {acc:.4f} | F1: {f1:.4f}")

# Save the model
os.makedirs(args.model_dir, exist_ok=True)
joblib.dump(model, os.path.join(args.model_dir, "model.joblib"))
print(f"Model saved in {args.model_dir}")
'''

os.makedirs("./first_experiment", exist_ok=True)
with open("./first_experiment/train.py", "w") as f:
    f.write(TRAIN_SCRIPT)

# 3. Create the job
job = command(
    code="./first_experiment",
    command="python train.py --n_estimators ${{inputs.n_estimators}} --learning_rate ${{inputs.learning_rate}}",
    inputs={
        "n_estimators": 150,
        "learning_rate": 0.05
    },
    compute="cpu-cluster",
    environment="AzureML-sklearn-1.0-ubuntu20.04-py38-cpu@latest",
    experiment_name="first-iris-experiment",
    display_name="Iris-GBM-v1"
)

# 4. Submit
print("Submitting job...")
submitted = ml_client.jobs.create_or_update(job)
print(f"✅ Job submitted: {submitted.name}")
print(f"   See: {submitted.studio_url}")

# 5. Wait for completion
ml_client.jobs.stream(submitted.name)

# 6. Retrieve metrics
final_job = ml_client.jobs.get(submitted.name)
print(f"\nStatus: {final_job.status}")

# 7. Register the model if satisfactory
if final_job.status == "Completed":
    from azure.ai.ml.entities import Model
    
    model = Model(
        path=f"azureml://jobs/{submitted.name}/outputs/artifacts/paths/model",
        name="iris-gbm-model",
        description="Gradient Boosting for Iris classification",
        type="mlflow_model"
    )
    
    registered_model = ml_client.models.create_or_update(model)
    print(f"✅ Model registered: {registered_model.name}:{registered_model.version}")

print("\n✅ First experiment completed successfully!")

11. Best Practices

11.1 Checklist for Each Azure ML Project

Organization:
✅ Workspace clearly named (project-env, e.g., bankmarketing-prod)
✅ Dedicated Resource Group per project
✅ Azure tags applied (owner, project, env)

Security:
✅ Managed Identity enabled (no hardcoded keys)
✅ Azure Key Vault for secrets
✅ RBAC access configured by role (Reader, Contributor, Data Scientist)
✅ Private Endpoints for sensitive environments

Data:
✅ Data registered as Data Assets (versionable)
✅ Train/val/test split BEFORE starting
✅ No data leakage (temporal split if necessary)

Compute:
✅ Use min_instances=0 for clusters (scale to zero)
✅ Stop Compute Instances when not in use
✅ Use Low Priority for non-critical jobs

Experiments:
✅ Descriptive experiment names
✅ MLflow logging enabled
✅ Tags on each run (dataset_version, developer...)
✅ Register the best models in the Model Registry

Production:
✅ Blue/Green deployment for updates
✅ Health checks on endpoints
✅ Monitoring enabled (Azure Monitor)
✅ Alerts defined (latency, errors, drift)

12. Summary and Key Points

12.1 When to Use What

mindmap
  root((Azure ML\nInterfaces))
    Studio
      Data exploration
      First prototype
      Team training
      Visual monitoring
    SDK v2
      Complex pipelines
      CI/CD
      Reusable code
      GitHub/DevOps
    CLI v2
      Bash scripts
      Azure DevOps
      Ops without Python
      IaC deployments
    Notebooks
      Interactive exploration
      Report with code
      Step-by-step debug

12.2 Summary Table

13. Glossary

Additional Resources:

5. Azure ML Studio — In-Depth Navigation

5.1 Interface Architecture by Section

Azure ML Studio is organized into four main functional areas accessible via the left navigation panel.

graph TD
    Studio["🏠 Azure ML Studio"] --> Author["✏️ Author"]
    Studio --> Assets["📦 Assets"]
    Studio --> Manage["⚙️ Manage"]
    Studio --> Discover["🔍 Discover"]

    Author --> Notebooks["Notebooks"]
    Author --> Designer["Designer"]
    Author --> AutoML["Automated ML"]
    Author --> PromptFlow["Prompt Flow"]

    Assets --> Data["Data"]
    Assets --> Environments["Environments"]
    Assets --> Experiments["Experiments / Jobs"]
    Assets --> Models["Models"]
    Assets --> Endpoints["Endpoints"]
    Assets --> Components["Components"]

    Manage --> Compute["Compute"]
    Manage --> Datastores["Datastores"]
    Manage --> LinkedSvc["Linked Services"]
    Manage --> Monitoring["Monitoring"]

    Discover --> ModelCatalog["Model Catalog"]
    Discover --> ResponsibleAI["Responsible AI"]

5.2 Author Section — Creation Tools

5.3 Assets Section — Artifact Management

5.4 Manage Section — Infrastructure and Governance

6. Notebooks in Studio — In-Depth Guide

6.1 Integrated Jupyter Environment

Azure ML Studio notebooks offer a complete Jupyter experience directly in the browser, connected to the ML workspace.

graph LR
    Browser["🌐 Browser"] --> Studio["Azure ML Studio\nNotebooks"]
    Studio --> CI["Compute Instance\n(dedicated VM)"]
    Studio --> SC["Serverless Compute\n(Spark)"]
    CI --> Storage["Azure Blob Storage\n(notebook files)"]
    CI --> Workspace["Azure ML Workspace\n(assets, datasets)"]

Advantages over a local Jupyter setup:

• Direct access to workspace Data Assets via ml_client
• File persistence on Azure Blob Storage
• Notebook sharing among team members
• Pre-configured environments with the v2 SDK pre-installed

6.2 Compute Instance — Configuration and Management

Studio → Compute → Compute Instances → + New

Parameters to configure:

6.3 Kernel Selection and Session Management

Each notebook is associated with a kernel that determines the execution runtime:

Session management commands:

# Check the active kernel version
import sys
print(sys.version)

# Check the installed SDK version
import azure.ai.ml
print(azure.ai.ml.__version__)

6.4 IntelliSense, Terminal, and Git

IntelliSense in Studio notebooks:

• SDK function autocompletion via Tab
• Inline documentation via Shift+Tab
• Real-time syntax error detection

Access to the integrated terminal:

Notebook → Terminal (icon in the top right)

Allows executing shell commands directly on the Compute Instance:

# Install an additional package
pip install lightgbm

# Check available resources
df -h
nvidia-smi  # If VM with GPU

Git integration:

Studio → Notebooks → (Git icon) → Clone repository

# In the Compute Instance terminal
git clone https://github.com/your-org/your-ml-repo.git
cd your-ml-repo
git checkout -b feature/experiment-1

7. Azure ML SDK v2 — Complete Guide

7.1 MLClient — Initialization and Authentication

MLClient is the single entry point for all interactions with Azure ML via the v2 SDK.

from azure.ai.ml import MLClient
from azure.identity import (
    DefaultAzureCredential,
    InteractiveBrowserCredential,
    ClientSecretCredential
)

# --- Option 1: DefaultAzureCredential (recommended in production)
# Automatically tries: env vars → managed identity → CLI → browser
ml_client = MLClient(
    credential=DefaultAzureCredential(),
    subscription_id="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    resource_group_name="rg-ml-prod",
    workspace_name="ws-ml-prod"
)

# --- Option 2: Service Principal (CI/CD pipelines)
credential = ClientSecretCredential(
    tenant_id="<TENANT_ID>",
    client_id="<CLIENT_ID>",
    client_secret="<CLIENT_SECRET>"
)
ml_client = MLClient(credential, "<SUB_ID>", "<RG>", "<WS>")

# --- Option 3: From inside a Compute Instance (simpler)
ml_client = MLClient.from_config(credential=DefaultAzureCredential())
# Uses the config.json file in .azureml/config.json

# Verify the connection
ws = ml_client.workspaces.get(ml_client.workspace_name)
print(f"Workspace: {ws.name}, Location: {ws.location}")

7.2 Available Job Types

Azure ML SDK v2 supports five job types to cover all ML scenarios:

graph TD
    Jobs["Azure ML Jobs"] --> Command["command\nBasic job - Python/R script"]
    Jobs --> Parallel["parallel\nDistributed parallel processing"]
    Jobs --> Pipeline["pipeline\nSequential step orchestration"]
    Jobs --> Sweep["sweep\nAutomatic hyperparameter tuning"]
    Jobs --> Spark["spark\nSpark data processing"]

1. Command Job — basic job

from azure.ai.ml import MLClient, command, Input, Output
from azure.ai.ml.entities import Environment
from azure.ai.ml.constants import AssetTypes, InputOutputModes

job = command(
    name="train-sklearn-model",
    display_name="Training Scikit-learn Model",
    description="Training an Iris classification model",
    # Source code to upload
    code="./src",
    # Command to run in the container
    command="python train.py --data ${{inputs.training_data}} --output ${{outputs.model_output}} --learning-rate ${{inputs.lr}}",
    # Inputs
    inputs={
        "training_data": Input(
            type=AssetTypes.URI_FILE,
            path="azureml:iris-dataset:1"
        ),
        "lr": Input(type="number", default=0.01)
    },
    # Outputs
    outputs={
        "model_output": Output(
            type=AssetTypes.URI_FOLDER,
            mode=InputOutputModes.RW_MOUNT
        )
    },
    # Execution environment
    environment="azureml:sklearn-env:1",
    # Compute
    compute="my-compute-cluster",
    # Grouping experiment
    experiment_name="iris-classification",
    # Tags
    tags={"team": "data-science", "version": "1.0"}
)

returned_job = ml_client.jobs.create_or_update(job)
ml_client.jobs.stream(returned_job.name)  # Follow logs in real time

2. Sweep Job — hyperparameter tuning

from azure.ai.ml.sweep import (
    Choice, Uniform, LogUniform,
    BanditPolicy, MedianStoppingPolicy
)
from azure.ai.ml import command

# First define the base command job
base_job = command(
    code="./src",
    command="python train.py --lr ${{inputs.learning_rate}} --n-estimators ${{inputs.n_estimators}}",
    inputs={
        "learning_rate": 0.01,
        "n_estimators": 100
    },
    environment="azureml:sklearn-env:1",
    compute="my-compute-cluster"
)

# Convert to sweep job
sweep_job = base_job.sweep(
    sampling_algorithm="random",  # random | grid | bayesian
    primary_metric="val_accuracy",
    goal="maximize",
    search_space={
        "learning_rate": LogUniform(min_value=-4, max_value=-1),   # 10^-4 to 10^-1
        "n_estimators": Choice(values=[50, 100, 200, 500])
    }
)
sweep_job.set_limits(
    max_total_trials=20,
    max_concurrent_trials=4,
    timeout=7200  # seconds
)
sweep_job.early_termination = BanditPolicy(
    evaluation_interval=2,
    slack_factor=0.1
)

returned_sweep = ml_client.jobs.create_or_update(sweep_job)

7.3 Input/Output Types

Output mount modes:

7.4 Logging with MLflow

# In the training script (train.py)
import mlflow
import mlflow.sklearn
import argparse
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, f1_score, roc_auc_score

parser = argparse.ArgumentParser()
parser.add_argument("--n-estimators", type=int, default=100)
parser.add_argument("--max-depth", type=int, default=5)
args = parser.parse_args()

# MLflow is automatically configured in Azure ML
# No need for mlflow.set_tracking_uri()
mlflow.autolog()  # Automatic logging of all sklearn parameters

with mlflow.start_run():
    # Log parameters manually
    mlflow.log_param("n_estimators", args.n_estimators)
    mlflow.log_param("max_depth", args.max_depth)

    # Train the model
    model = RandomForestClassifier(
        n_estimators=args.n_estimators,
        max_depth=args.max_depth
    )
    model.fit(X_train, y_train)
    y_pred = model.predict(X_test)

    # Log metrics
    mlflow.log_metric("accuracy", accuracy_score(y_test, y_pred))
    mlflow.log_metric("f1_score", f1_score(y_test, y_pred, average="weighted"))

    # Log artifacts (charts, files)
    import matplotlib.pyplot as plt
    from sklearn.metrics import ConfusionMatrixDisplay
    fig, ax = plt.subplots()
    ConfusionMatrixDisplay.from_predictions(y_test, y_pred, ax=ax)
    mlflow.log_figure(fig, "confusion_matrix.png")

    # Log the model
    mlflow.sklearn.log_model(
        model,
        artifact_path="model",
        registered_model_name="iris-rf-model"
    )

8. YAML Job Definitions

8.1 Command Job YAML

# command_job.yaml
$schema: https://azuremlschemas.azureedge.net/latest/commandJob.schema.json
type: command

name: train-iris-model
display_name: "Training Iris Model"
description: "Iris classification with RandomForest"
experiment_name: iris-classification

command: >-
  python train.py
  --data ${{inputs.training_data}}
  --n-estimators ${{inputs.n_estimators}}
  --output ${{outputs.model_output}}

code: ./src

inputs:
  training_data:
    type: uri_file
    path: azureml:iris-dataset:1
  n_estimators:
    type: integer
    default: 100

outputs:
  model_output:
    type: uri_folder
    mode: rw_mount

environment: azureml:sklearn-env:1
compute: azureml:my-compute-cluster

resources:
  instance_count: 1

limits:
  timeout: 3600

tags:
  team: data-science
  framework: sklearn

8.2 Environment YAML

# environment.yaml
$schema: https://azuremlschemas.azureedge.net/latest/environment.schema.json
name: sklearn-env
version: "2"
description: "scikit-learn environment with MLflow"

# Option A: base Docker image + conda
image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest
conda_file: conda.yaml

# Option B: custom Dockerfile
# build:
#   path: ./docker
#   dockerfile_path: Dockerfile

# conda.yaml
name: sklearn-env
channels:
  - defaults
  - conda-forge
dependencies:
  - python=3.10
  - pip
  - pip:
    - azure-ai-ml==1.12.0
    - scikit-learn==1.3.2
    - pandas==2.0.3
    - mlflow==2.9.2
    - azureml-mlflow==1.54.0
    - matplotlib==3.8.0
    - lightgbm==4.1.0

8.3 Component YAML

# components/train_component.yaml
$schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json
type: command

name: train_classifier
display_name: "Train a Classifier"
description: "Reusable training component"
version: "1"

inputs:
  training_data:
    type: uri_folder
    description: "Training data (CSV format)"
  n_estimators:
    type: integer
    default: 100
    min: 10
    max: 1000
  max_depth:
    type: integer
    default: 5

outputs:
  model_output:
    type: uri_folder
    description: "Trained model in MLflow format"
  metrics_output:
    type: uri_file
    description: "JSON file with metrics"

code: ../src
command: >-
  python train_component.py
  --training-data ${{inputs.training_data}}
  --n-estimators ${{inputs.n_estimators}}
  --max-depth ${{inputs.max_depth}}
  --model-output ${{outputs.model_output}}
  --metrics-output ${{outputs.metrics_output}}

environment: azureml:sklearn-env:2

8.4 Pipeline YAML with Components

# pipeline.yaml
$schema: https://azuremlschemas.azureedge.net/latest/pipelineJob.schema.json
type: pipeline

name: iris-ml-pipeline
display_name: "Complete Iris ML Pipeline"
experiment_name: iris-pipeline-experiment

inputs:
  raw_data:
    type: uri_file
    path: azureml:iris-raw:1

outputs:
  final_model:
    type: uri_folder
    mode: rw_mount

jobs:
  # Step 1: Data preparation
  preprocess_step:
    type: command
    component: azureml:preprocess_data:1
    inputs:
      raw_data: ${{parent.inputs.raw_data}}
    outputs:
      processed_data:
        type: uri_folder

  # Step 2: Training
  train_step:
    type: command
    component: azureml:train_classifier:1
    inputs:
      training_data: ${{parent.jobs.preprocess_step.outputs.processed_data}}
      n_estimators: 200
      max_depth: 8
    outputs:
      model_output:
        type: uri_folder
      metrics_output:
        type: uri_file

  # Step 3: Evaluation
  evaluate_step:
    type: command
    component: azureml:evaluate_model:1
    inputs:
      model: ${{parent.jobs.train_step.outputs.model_output}}
      test_data: ${{parent.jobs.preprocess_step.outputs.processed_data}}
    outputs:
      evaluation_report:
        type: uri_folder

settings:
  default_compute: azureml:my-compute-cluster
  default_datastore: azureml:workspaceblobstore
  continue_on_step_failure: false
  force_rerun: false

9. Components and Pipelines — Advanced Guide

9.1 Building a Pipeline with the SDK v2

from azure.ai.ml.dsl import pipeline
from azure.ai.ml import load_component, Input, Output
from azure.ai.ml.constants import AssetTypes

# Load components from the registry
preprocess = load_component("azureml:preprocess_data:1")
train = load_component("azureml:train_classifier:1")
evaluate = load_component("azureml:evaluate_model:1")
register = load_component("azureml:register_model:1")

@pipeline(
    name="end-to-end-ml-pipeline",
    description="End-to-end ML pipeline with reusable components",
    tags={"pipeline": "production", "version": "2.0"}
)
def ml_pipeline(raw_data: Input(type=AssetTypes.URI_FILE)):
    # Step 1: Preprocessing
    preprocess_step = preprocess(
        raw_input=raw_data,
        test_size=0.2
    )

    # Step 2: Training (takes output from step 1)
    train_step = train(
        training_data=preprocess_step.outputs.train_data,
        n_estimators=200,
        max_depth=8
    )

    # Step 3: Evaluation
    eval_step = evaluate(
        model=train_step.outputs.model_output,
        test_data=preprocess_step.outputs.test_data
    )

    # Step 4: Conditional registration
    register_step = register(
        model=train_step.outputs.model_output,
        metrics=eval_step.outputs.metrics,
        model_name="iris-production-model"
    )

    return {
        "trained_model": train_step.outputs.model_output,
        "evaluation_report": eval_step.outputs.evaluation_report
    }

# Instantiate and configure the pipeline
pipeline_job = ml_pipeline(
    raw_data=Input(
        type=AssetTypes.URI_FILE,
        path="azureml:iris-dataset:1"
    )
)
pipeline_job.settings.default_compute = "my-compute-cluster"
pipeline_job.settings.default_datastore = "workspaceblobstore"
pipeline_job.settings.continue_on_step_failure = False
pipeline_job.settings.force_rerun = False   # Cache enabled

# Submit
returned = ml_client.jobs.create_or_update(pipeline_job)
ml_client.jobs.stream(returned.name)
print(f"Pipeline URL: {returned.studio_url}")

9.2 Pipeline Graph and Data Flow

graph TD
    RawData["📂 Data Asset\niris-dataset:1"] --> Prep["⚙️ preprocess_data\n(Step 1)"]
    Prep --> TrainData["train_data\n(80%)"]
    Prep --> TestData["test_data\n(20%)"]
    TrainData --> Train["🤖 train_classifier\n(Step 2)\nn_estimators=200"]
    Train --> ModelOut["model_output\n(MLflow format)"]
    TestData --> Eval["📊 evaluate_model\n(Step 3)"]
    ModelOut --> Eval
    Eval --> Metrics["metrics.json\naccuracy, f1"]
    ModelOut --> Register["📋 register_model\n(Step 4)"]
    Metrics --> Register
    Register --> Registry["🏷️ Model Registry\niris-production-model:v1"]

    style RawData fill:#1e88e5,color:#fff
    style Registry fill:#43a047,color:#fff

9.3 Caching and Step Reuse

Azure ML Pipeline automatically caches step outputs. A step is re-executed only if:

• The component’s source code has changed
• The inputs have changed (Data Asset version)
• The component parameters have changed
• The cache has been explicitly invalidated

# Disable cache for a specific step
train_step.settings.cache = False

# Force global pipeline re-execution
pipeline_job.settings.force_rerun = True

Search Terms

azure · ml · studio · sdk · platforms · deployment · machine · data · science · job · yaml · cli · pipeline · types · components · compute · configuration · environments · interface · management · notebooks · architecture · assets · comparison