Table of Contents
- Overview: AI vs Traditional Approach
- How Do Machines Learn?
- Types of Machine Learning
- The Machine Learning Process
- Key AWS AI Services
- Key AWS Machine Learning Services
- boto3 Code Examples by Service
- AWS ML Reference Architectures
- Comparisons and Decision Guidance
- Key Concepts to Remember
1. Overview: AI vs Traditional Approach
Traditional Approach (Rule-Based)
In the traditional approach, the programmer explicitly defines all system rules. Consider the example of a password validation feature:
IF length < 8 → "Password too short"
IF length > 20 → "Password too long"
ELSE → "Valid"
Characteristics:
| Property | Description |
|---|---|
| Rule-based | Logic is driven by explicit constraints |
| Deterministic | Output is always predictable for the same input |
| Transparent | Logic is easy to understand and debug |
AI Approach (Data-Driven)
With AI, instead of coding rules, you let the system learn patterns from historical data.
Characteristics:
| Property | Description |
|---|---|
| Data-driven | Logic is derived from historical data |
| Adaptable | The system can adapt to evolving trends |
| Less transparent | Internal logic is less directly readable (black box) |
Comparative Diagram
flowchart LR
subgraph Traditional["Traditional Approach"]
direction TB
T1[Programmer defines rules] --> T2[Hard-coded rules]
T2 --> T3[Deterministic result]
end
subgraph AI["AI Approach"]
direction TB
A1[Data collection] --> A2[Model training]
A2 --> A3[Adaptive prediction]
end
Input([User input]) --> Traditional
Input --> AI
2. How Do Machines Learn?
Just as AI seeks to mimic human intelligence, machine learning (ML) seeks to mimic human learning.
Analogy: Learning to Recognize a Shark
- You show a child a picture book about sharks
- By repeating the images, the child learns to recognize a shark
- The child also learns to distinguish sharks from other sea creatures
- By enriching the vocabulary (seaweed, coral, dolphins), the mental model refines
Similarly, if you provide enough data to a machine, it will learn to recognize patterns and correctly identify new elements.
Core Concepts
The Model
The machine or algorithm is called a model. It is the entity that learns and makes predictions.
Training
Presenting data to a model is called the training stage. This process:
- Uses a large and diverse training dataset
- Contains input variables (images, text, numbers, etc.)
- Contains target variables (the correct labels associated with inputs)
Example:
- Input variable: a shark image
- Target variable: the label “shark”
Inference
Once the model is trained, it applies the learned patterns to make predictions on new unknown data. This is called inference (or inferencing).
flowchart LR
subgraph Training["Training Phase"]
TD[Training Data\nInput + Target] --> M[Model]
M --> TM[Trained Model]
end
subgraph Inference["Inference Phase"]
ND[New unlabeled\ndata] --> TM2[Trained Model]
TM2 --> P[Prediction]
end
TM --> TM2
3. Types of Machine Learning
Overview
mindmap
root((Machine Learning))
Supervised Learning
Classification
Multiclass Classification
Binary Classification
Regression
Unsupervised Learning
Clustering
Anomaly Detection
Reinforcement Learning
Reward-based
Agent and Environment
Deep Learning
Neural Networks
CNN
RNN / LSTM
Transformers
3.1 Supervised Learning
Supervised learning uses labeled training data. You explicitly provide target labels for each input variable.
Classification
Classification involves categorizing inputs into distinct classes.
Multiclass Classification:
- Categorize an input into one of several possible classes
- Examples: shark / seaweed / coral / fish, cat / dog / cow, house / condo / townhome
Binary Classification:
- Categorize an input into one of two possible outcomes
- Examples: yes/no, true/false, fraud/not fraud
Regression
Regression involves predicting continuous values (as opposed to discrete categories).
Examples:
- Stock market prices
- Real estate prices
- Rental property rates
Concrete example — rental price estimation:
- Collect data on other rentals
- Plot number of bedrooms (X-axis) vs price per night (Y-axis)
- Draw a regression line that “best fits” the data
- Use this line to estimate the price for a new property
3.2 Unsupervised Learning
Unsupervised learning uses unlabeled data. The model must discover hidden structures or patterns on its own.
Clustering
Clustering groups similar data without needing predefined labels.
Example: Group customers by similar purchasing behaviors to target marketing campaigns.
Anomaly Detection
Anomaly detection identifies data points that deviate significantly from the norm.
Examples:
- Credit card fraud detection
- Identifying defective equipment in a factory
3.3 Reinforcement Learning
Reinforcement learning is inspired by human behavior based on rewards and penalties. The model learns through a system of trial and error.
Key components:
- Agent: the entity that makes decisions
- Environment: the context in which the agent operates
- Reward: positive signal to encourage behavior
- Penalty: negative signal to discourage behavior
Usage examples:
- Video games (AlphaGo, Atari games)
- Autonomous robots
- Recommendation systems
3.4 Deep Learning
Deep learning is a subset of ML using artificial neural networks with multiple layers.
Common architectures:
| Architecture | Abbreviation | Primary Use |
|---|---|---|
| Convolutional Neural Network | CNN | Computer vision, image classification |
| Recurrent Neural Network | RNN / LSTM | Time sequences, NLP |
| Transformer | — | Advanced NLP, LLMs (GPT, BERT) |
| Generative Adversarial Network | GAN | Image generation |
| Autoencoder | AE | Compression, anomaly detection |
Comparative Table of ML Types
| Type | Data | Primary Task | AWS SageMaker Algorithms | Example |
|---|---|---|---|---|
| Supervised Learning | Labeled | Classification, Regression | XGBoost, Linear Learner, KNN | Spam detection, house prices |
| Unsupervised Learning | Unlabeled | Clustering, Anomaly Detection | K-Means, PCA, Random Cut Forest | Customer segmentation, fraud |
| Reinforcement Learning | Feedback (reward/penalty) | Sequential decision making | Coach (RL framework) | Games, robots, trading |
| Deep Learning | Large, complex | Advanced classification, NLP, vision | TensorFlow, PyTorch, MXNet | Image recognition, chatbots |
4. The Machine Learning Process
The ML process is a circular workflow composed of three main stages.
flowchart TD
subgraph Stage1["1. Generate the Data"]
F[Fetch / Collect] --> C[Clean / Prepare]
C --> P[Prepare / Transform]
end
subgraph Stage2["2. Train the Model"]
T[Train] --> E[Evaluate]
end
subgraph Stage3["3. Deploy the Model"]
D[Deploy] --> M[Monitor]
M --> Col[Collect Feedback]
end
Stage1 --> Stage2
Stage2 --> Stage3
Col -->|Refine the model| Stage1
style Stage1 fill:#e8f5e9,stroke:#4caf50
style Stage2 fill:#e3f2fd,stroke:#2196f3
style Stage3 fill:#fff3e0,stroke:#ff9800
Culinary Analogy
| ML Step | Culinary Analogy |
|---|---|
| Fetch (data collection) | Gathering ingredients |
| Clean (cleaning) | Washing vegetables |
| Prepare (transformation) | Cutting vegetables, preparing spices |
| Train (training) | Cooking (sautéing, frying, simmering) |
| Evaluate (evaluation) | Tasting the dish with a spoon |
| Deploy (deployment) | Serving the dish at the table |
| Monitor/Collect (monitoring) | Receiving feedback from diners |
4.1 Stage 1: Generate the Data
Fetch (Collection)
- Gather raw data from various sources (databases, APIs, files, etc.)
- Data quality and quantity directly impact model performance
Clean (Cleaning)
- Handle missing values
- Remove duplicates
- Fix data inconsistencies
- Eliminate unwanted outliers
Prepare (Feature Engineering)
- Transform data into a format suitable for training
- Normalization and standardization of numerical values
- Encoding of categorical variables
- Creation of new relevant features
- Split into training set (70-80%), validation set (10-15%), and test set (10-15%)
4.2 Stage 2: Train the Model
Train
- The model adjusts iteratively on the training dataset
- It learns patterns and relationships between variables
- Hyperparameters (learning rate, number of epochs, etc.) influence learning
Evaluate
Common metrics by task type:
| Metric | Task | Description |
|---|---|---|
| Accuracy | Classification | % of correct predictions |
| Precision | Classification | TP / (TP + FP) — quality of positive predictions |
| Recall | Classification | TP / (TP + FN) — coverage of true positives |
| F1 Score | Classification | Harmonic mean of Precision + Recall |
| AUC-ROC | Binary Classification | Area under the ROC curve |
| RMSE | Regression | Root Mean Square Error |
| MAE | Regression | Mean Absolute Error |
4.3 Stage 3: Deploy the Model
Deploy
- Put the trained model into production
- Expose via an API (REST / gRPC) or integrate into an application
- The model begins processing real data in production (inference)
Monitor / Collect / Evaluate
- Continuous performance monitoring in production
- Collecting user feedback (solicited or unsolicited)
- Detecting model drift (performance degradation over time)
- Return to the collection/cleaning step to refine the model if necessary
Types of drift:
| Drift Type | Description | Example |
|---|---|---|
| Data drift | Distribution of input data has changed | Post-pandemic purchasing habits |
| Concept drift | The relationship between inputs and outputs has changed | Spam definition evolves |
| Model drift | General performance degradation | Aging financial prediction model |
5. Key AWS AI Services
AWS offers a comprehensive range of fully managed AI services, allowing you to integrate artificial intelligence capabilities without needing ML expertise.
Overview
flowchart TB
subgraph Vision["Computer Vision"]
RekImg[Amazon Rekognition\nImage]
RekVid[Amazon Rekognition\nVideo]
Txt[Amazon Textract]
end
subgraph NLP["Natural Language Processing"]
Comp[Amazon Comprehend]
CompMed[Amazon Comprehend\nMedical]
Trans[Amazon Translate]
Transcrb[Amazon Transcribe]
Polly[Amazon Polly]
Lex[Amazon Lex]
end
subgraph Prediction["Prediction and Personalization"]
Fore[Amazon Forecast]
Pers[Amazon Personalize]
end
subgraph Search["Search and Security"]
Kendra[Amazon Kendra]
FD[Amazon Fraud Detector\nDeprecated Nov. 2025]
end
5.1 Vision Services (Computer Vision)
Amazon Rekognition Image
Deep learning-based image analysis service.
Capabilities:
- Recognition and identification of people (e.g., identifying employees at a building entrance)
- Facial expression analysis to detect emotions
- Automatic detection and filtering of inappropriate content (content moderation)
- Object, scene, and activity detection
- Text recognition in images (OCR)
- Face comparison and search in a collection (Face Search)
- Detection and analysis of PPE (personal protective equipment)
Use cases:
Security application:
Camera image → Rekognition Image → Employee identification → Access granted/denied
boto3 example — Detect labels in an image:
import boto3
rekognition = boto3.client('rekognition', region_name='us-east-1')
# Detect labels (objects, scenes) in an image stored in S3
response = rekognition.detect_labels(
Image={
'S3Object': {
'Bucket': 'my-image-bucket',
'Name': 'photo.jpg'
}
},
MaxLabels=10,
MinConfidence=80.0
)
for label in response['Labels']:
print(f"Label: {label['Name']}, Confidence: {label['Confidence']:.2f}%")
for parent in label.get('Parents', []):
print(f" → Parent: {parent['Name']}")
boto3 example — Analyze facial emotions:
import boto3
rekognition = boto3.client('rekognition', region_name='us-east-1')
response = rekognition.detect_faces(
Image={
'S3Object': {
'Bucket': 'my-image-bucket',
'Name': 'portrait.jpg'
}
},
Attributes=['ALL']
)
for face in response['FaceDetails']:
print(f"Estimated age: {face['AgeRange']['Low']}-{face['AgeRange']['High']} years")
for emotion in face['Emotions']:
if emotion['Confidence'] > 50:
print(f"Emotion: {emotion['Type']}, Confidence: {emotion['Confidence']:.1f}%")
boto3 example — Content moderation:
import boto3
rekognition = boto3.client('rekognition', region_name='us-east-1')
response = rekognition.detect_moderation_labels(
Image={
'S3Object': {'Bucket': 'my-bucket', 'Name': 'image_to_moderate.jpg'}
},
MinConfidence=60.0
)
if response['ModerationLabels']:
print("Inappropriate content detected:")
for label in response['ModerationLabels']:
print(f" - {label['Name']} ({label['ParentName']}): {label['Confidence']:.1f}%")
else:
print("No inappropriate content detected")
Amazon Rekognition Video
Same capabilities as Rekognition Image, but applied to real-time or deferred video.
Additional capabilities:
- Search through hours of surveillance footage
- Person tracking across different sequences
- Identification of specific activities
- Detection of particular scenes
- Real-time streaming analysis via Kinesis Video Streams
boto3 example — Launch an asynchronous video analysis:
import boto3
import time
rekognition = boto3.client('rekognition', region_name='us-east-1')
# Start asynchronous analysis
response = rekognition.start_label_detection(
Video={
'S3Object': {
'Bucket': 'my-video-bucket',
'Name': 'surveillance.mp4'
}
},
MinConfidence=80.0,
NotificationChannel={
'SNSTopicArn': 'arn:aws:sns:us-east-1:123456789:rekognition-topic',
'RoleArn': 'arn:aws:iam::123456789:role/RekognitionRole'
}
)
job_id = response['JobId']
print(f"Job started: {job_id}")
# Wait for the job to finish and retrieve results
while True:
result = rekognition.get_label_detection(JobId=job_id)
status = result['JobStatus']
if status in ['SUCCEEDED', 'FAILED']:
break
time.sleep(5)
if status == 'SUCCEEDED':
for label in result['Labels']:
ts = label['Timestamp']
name = label['Label']['Name']
conf = label['Label']['Confidence']
print(f"[{ts}ms] {name}: {conf:.1f}%")
Amazon Textract
Service for extracting text and structured data from documents.
What Textract does:
- Extracts text from scanned documents, images, forms, tables
- Goes beyond simple OCR: understands the structure of forms
- Automatically extracts key information: names, dates, addresses, amounts
- Processes form and table data while preserving the structure
Main APIs:
| API | Description |
|---|---|
DetectDocumentText | Raw text extraction (simple OCR) |
AnalyzeDocument | Form + table extraction |
StartDocumentTextDetection | Asynchronous version for large documents |
AnalyzeExpense | Specialized extraction for invoices |
AnalyzeID | Data extraction from ID documents |
boto3 example — Extract text and key-value pairs from a form:
import boto3
textract = boto3.client('textract', region_name='us-east-1')
response = textract.analyze_document(
Document={
'S3Object': {
'Bucket': 'my-docs-bucket',
'Name': 'registration_form.pdf'
}
},
FeatureTypes=['FORMS', 'TABLES']
)
# Create a block dictionary for quick reference
blocks = {block['Id']: block for block in response['Blocks']}
# Extract key-value pairs from the form
for block in response['Blocks']:
if block['BlockType'] == 'KEY_VALUE_SET' and 'KEY' in block.get('EntityTypes', []):
key_text = ''
value_text = ''
for rel in block.get('Relationships', []):
if rel['Type'] == 'CHILD':
for child_id in rel['Ids']:
child = blocks.get(child_id, {})
if child.get('BlockType') == 'WORD':
key_text += child.get('Text', '') + ' '
elif rel['Type'] == 'VALUE':
for val_id in rel['Ids']:
val_block = blocks.get(val_id, {})
for val_rel in val_block.get('Relationships', []):
if val_rel['Type'] == 'CHILD':
for wid in val_rel['Ids']:
word = blocks.get(wid, {})
if word.get('BlockType') == 'WORD':
value_text += word.get('Text', '') + ' '
if key_text.strip():
print(f"{key_text.strip()} -> {value_text.strip()}")
Use cases:
- Automated invoice processing
- Digitizing medical records
- Insurance form processing
- Data entry automation
5.2 Natural Language Processing (NLP) Services
Amazon Comprehend
NLP service that uses ML to extract meaning from unstructured text.
Features:
- Sentiment analysis: determines if text is positive, negative, neutral, or mixed
- Entity detection (NER): identifies people, places, organizations, dates, amounts
- Language detection: automatically identifies the language of a text (100+ languages)
- Key phrase extraction: extracts key phrases
- Topic modeling: groups documents by theme (LDA)
- Custom classification: trains custom classifiers
- Custom entity recognition: detects business-specific entities
boto3 example — Batch sentiment analysis:
import boto3
comprehend = boto3.client('comprehend', region_name='us-east-1')
reviews = [
"This product is absolutely fantastic, I love it!",
"Catastrophic delivery, I am very disappointed.",
"The order arrived on time."
]
response = comprehend.batch_detect_sentiment(
TextList=reviews,
LanguageCode='en'
)
for i, result in enumerate(response['ResultList']):
sentiment = result['Sentiment']
scores = result['SentimentScore']
print(f"Review {i+1}: {sentiment}")
print(f" Positive: {scores['Positive']:.2%}")
print(f" Negative: {scores['Negative']:.2%}")
print(f" Neutral: {scores['Neutral']:.2%}")
print(f" Mixed: {scores['Mixed']:.2%}")
boto3 example — Entity detection (NER):
import boto3
comprehend = boto3.client('comprehend', region_name='us-east-1')
text = "Amazon Web Services was founded by Jeff Bezos in Seattle in 1994."
response = comprehend.detect_entities(
Text=text,
LanguageCode='en'
)
for entity in response['Entities']:
print(f"Type: {entity['Type']:<20} Text: {entity['Text']:<30} "
f"Confidence: {entity['Score']:.2%}")
boto3 example — Language detection:
import boto3
comprehend = boto3.client('comprehend', region_name='us-east-1')
text = "Hello, how are you doing today?"
response = comprehend.detect_dominant_language(Text=text)
for lang in response['Languages']:
print(f"Language: {lang['LanguageCode']}, Confidence: {lang['Score']:.2%}")
Amazon Comprehend Medical
Specialized version of Comprehend for unstructured medical data.
Specialties:
- Analysis of physician notes, discharge summaries, test results
- Extraction of medical entities: medications, diagnoses, treatments, symptoms
- Detection of PHI (Protected Health Information) via the DetectPHI API
- Linking medical entities (drug-dosage relationships, etc.)
- Integration with medical ontologies: ICD-10-CM, RxNorm, SNOMED CT
boto3 example — Extract medical entities and detect PHI:
import boto3
comprehend_medical = boto3.client('comprehendmedical', region_name='us-east-1')
medical_note = """
Patient Jane Smith, 45 years old, admitted on March 15, 2024.
Diagnosis: Type 2 Diabetes, arterial hypertension.
Treatment: Metformin 500mg twice daily, Lisinopril 10mg.
HbA1c result: 7.8%
"""
# Detect medical entities
entities_response = comprehend_medical.detect_entities_v2(Text=medical_note)
print("=== Detected medical entities ===")
for entity in entities_response['Entities']:
print(f"Type: {entity['Type']:<25} Category: {entity['Category']:<20} "
f"Text: {entity['Text']}")
# Detect PHI (protected health information)
phi_response = comprehend_medical.detect_phi(Text=medical_note)
print("\n=== PHI detected ===")
for phi in phi_response['Entities']:
print(f"Type: {phi['Type']:<20} Text: {phi['Text']}, "
f"Confidence: {phi['Score']:.2%}")
Amazon Translate
Neural machine translation service that maintains tone and text fluency.
Characteristics:
- Translation between 75+ languages
- Does not limit itself to word-for-word conversion: preserves meaning and style
- Real-time or batch translation (files in S3)
- Custom terminology: glossary for domain-specific terms
- Active Custom Translation: fine-tuning on your translation data
boto3 example — Simple translation:
import boto3
translate = boto3.client('translate', region_name='us-east-1')
response = translate.translate_text(
Text="Machine learning is transforming the way we build applications.",
SourceLanguageCode='en',
TargetLanguageCode='fr'
)
print(f"Original text ({response['SourceLanguageCode']}):")
print(f" Machine learning is transforming the way we build applications.")
print(f"\nTranslation ({response['TargetLanguageCode']}):")
print(f" {response['TranslatedText']}")
boto3 example — Translation with custom terminology:
import boto3
translate = boto3.client('translate', region_name='us-east-1')
# Create custom terminology
terminology_csv = b"en,es\nmachine learning,aprendizaje automatico\ndeployment,implementacion\n"
translate.import_terminology(
Name='it-terminology',
MergeStrategy='OVERWRITE',
TerminologyData={
'File': terminology_csv,
'Format': 'CSV'
}
)
# Use the terminology in a translation
response = translate.translate_text(
Text="The machine learning deployment was successful.",
SourceLanguageCode='en',
TargetLanguageCode='es',
TerminologyNames=['it-terminology']
)
print(response['TranslatedText'])
Amazon Transcribe
Speech-to-text service (transcription of speech to text).
Features:
- Automatic transcription of audio and video files (MP3, MP4, WAV, FLAC, etc.)
- Support for 100+ languages
- Speaker diarization: identification of different speakers
- Custom vocabulary: customized vocabulary for business terms
- Custom language model: language model trained on your data
- Inappropriate content filtering (PII redaction)
- Real-time transcription via Transcribe Streaming
boto3 example — Transcribe an audio file:
import boto3
import time
import uuid
transcribe = boto3.client('transcribe', region_name='us-east-1')
job_name = f"transcription-{uuid.uuid4().hex[:8]}"
audio_uri = "s3://my-audio-bucket/meeting.mp3"
# Start the transcription job with speaker identification
transcribe.start_transcription_job(
TranscriptionJobName=job_name,
Media={'MediaFileUri': audio_uri},
MediaFormat='mp3',
LanguageCode='en-US',
Settings={
'ShowSpeakerLabels': True,
'MaxSpeakerLabels': 4
}
)
# Wait for the job to finish
while True:
response = transcribe.get_transcription_job(TranscriptionJobName=job_name)
status = response['TranscriptionJob']['TranscriptionJobStatus']
if status in ['COMPLETED', 'FAILED']:
break
print(f"Status: {status}...")
time.sleep(10)
if status == 'COMPLETED':
transcript_uri = response['TranscriptionJob']['Transcript']['TranscriptFileUri']
print(f"Transcription available: {transcript_uri}")
Amazon Polly
Text-to-speech service (voice synthesis).
Features:
- Converts text to natural speech
- 60+ voices in 30+ languages
- Support for SSML (Speech Synthesis Markup Language) to control pronunciation, pace, tone
- Two engines:
- Standard: concatenative, more economical
- Neural: deep learning, more natural and expressive
- Direct audio stream generation (streaming) or to S3
Available English voices (sample):
| Name | Gender | Engine | Language Code |
|---|---|---|---|
| Joanna | Female | Neural + Standard | en-US |
| Matthew | Male | Neural + Standard | en-US |
| Amy | Female | Neural + Standard | en-GB |
| Brian | Male | Neural + Standard | en-GB |
| Nicole | Female | Standard | en-AU |
| Russell | Male | Standard | en-AU |
boto3 example — Simple voice synthesis:
import boto3
polly = boto3.client('polly', region_name='us-east-1')
response = polly.synthesize_speech(
Text="Hello, welcome to our AWS Machine Learning training.",
OutputFormat='mp3',
VoiceId='Joanna', # Neural US English voice
Engine='neural',
LanguageCode='en-US'
)
# Save the audio file
with open('output.mp3', 'wb') as f:
f.write(response['AudioStream'].read())
print("Audio file created: output.mp3")
boto3 example — Synthesis with SSML for advanced control:
import boto3
polly = boto3.client('polly', region_name='us-east-1')
ssml_text = """
<speak>
Welcome to our tutorial.
<break time="500ms"/>
We will now cover <emphasis level="strong">Amazon SageMaker</emphasis>.
<break time="300ms"/>
<prosody rate="slow">SageMaker is a fully managed service by AWS.</prosody>
</speak>
"""
response = polly.synthesize_speech(
Text=ssml_text,
TextType='ssml',
OutputFormat='mp3',
VoiceId='Joanna',
Engine='neural'
)
with open('tutorial.mp3', 'wb') as f:
f.write(response['AudioStream'].read())
Amazon Lex
Service for building chatbots and conversational interfaces.
Features:
- NLU (Natural Language Understanding): understands the intent behind text
- ASR (Automatic Speech Recognition): integrated speech recognition
- Conversation context management (intents, slots, utterances)
- Multi-turn dialogue: multi-turn conversations with context memory
- Native integration with AWS Lambda, Amazon Connect, Slack, Facebook Messenger
- Lex V2: improved multilingual support, better performance
Key Lex concepts:
| Concept | Description | Example |
|---|---|---|
| Intent | Action the user wants to perform | BookFlight, CheckBalance |
| Utterance | Phrase that triggers an intent | ”I want to book a flight” |
| Slot | Parameter required for the intent | destination, departureDate |
| Slot type | Data type of the slot | AMAZON.City, AMAZON.Date |
| Fulfillment | Action to execute via Lambda | Call to the booking API |
boto3 example — Interact with a Lex V2 bot:
import boto3
lex = boto3.client('lexv2-runtime', region_name='us-east-1')
bot_id = 'MY_BOT_ID'
bot_alias_id = 'TSTALIASID' # Default test alias
locale_id = 'en_US'
session_id = 'user-session-001'
# Send a message to the bot
response = lex.recognize_text(
botId=bot_id,
botAliasId=bot_alias_id,
localeId=locale_id,
sessionId=session_id,
text="I would like to book a flight to New York on July 20th"
)
print(f"Recognized intent: {response['sessionState']['intent']['name']}")
print(f"State: {response['sessionState']['intent']['state']}")
for message in response.get('messages', []):
print(f"Bot: {message['content']}")
# Show filled slots
slots = response['sessionState']['intent'].get('slots', {})
for slot_name, slot_value in slots.items():
if slot_value:
val = slot_value.get('value', {}).get('interpretedValue', 'not provided')
print(f" Slot '{slot_name}': {val}")
5.3 Prediction and Personalization Services
Amazon Forecast
ML-based time series forecasting service.
Features:
- Generates accurate forecasts from historical data (time series)
- Automatically integrates external variables (related time series): weather, promotions, holidays
- Uses the same algorithms developed by Amazon.com
- Available algorithms: DeepAR+, NPTS, CNN-QR, Prophet, ETS, ARIMA
- AutoML: automatic selection of the best algorithm
Amazon Forecast Workflow:
flowchart LR
S3["(Amazon S3\nHistorical Data)"] --> DS[Dataset Group]
DS --> Predictor[Train a\nPredictor / AutoML]
Predictor --> Forecast[Generate a\nForecast]
Forecast --> Query[QueryForecast API\nRetrieve forecasts]
Query --> App[Application]
style S3 fill:#e8f4fd
style App fill:#e8f5e9
boto3 example — Create a dataset and launch a forecast:
import boto3
forecast = boto3.client('forecast', region_name='us-east-1')
role_arn = 'arn:aws:iam::123456789:role/ForecastRole'
# 1. Create a Dataset Group
dsg_response = forecast.create_dataset_group(
DatasetGroupName='product_sales',
Domain='RETAIL'
)
dsg_arn = dsg_response['DatasetGroupArn']
# 2. Create a Dataset
ds_response = forecast.create_dataset(
DatasetName='sales_history',
Domain='RETAIL',
DatasetType='TARGET_TIME_SERIES',
DataFrequency='D', # Daily data
Schema={
'Attributes': [
{'AttributeName': 'item_id', 'AttributeType': 'string'},
{'AttributeName': 'timestamp', 'AttributeType': 'timestamp'},
{'AttributeName': 'demand', 'AttributeType': 'float'}
]
}
)
ds_arn = ds_response['DatasetArn']
# 3. Import data from S3
forecast.create_dataset_import_job(
DatasetImportJobName='import_sales_2024',
DatasetArn=ds_arn,
DataSource={
'S3Config': {
'Path': 's3://my-forecast-bucket/sales/',
'RoleArn': role_arn
}
},
TimestampFormat='yyyy-MM-dd'
)
# 4. Train a Predictor with AutoML
forecast.create_auto_predictor(
PredictorName='auto_sales_predictor',
ForecastHorizon=30, # Forecast over 30 days
ForecastFrequency='D',
DataConfig={
'DatasetGroupArn': dsg_arn
}
)
Amazon Personalize
Real-time personalized recommendation service.
Features:
- Generates individualized recommendations for each user
- Uses the same recommendation technologies as Amazon.com
- Does not require ML expertise
- Adapts in real-time to user behavior (real-time events)
Types of recommendations:
| Recipe | Use | Example |
|---|---|---|
| User-Personalization | Items for a user | ”Recommended for you” |
| SIMS (Similar Items) | Items similar to an item | ”You might also like” |
| Popularity-Count | Most popular items | ”Trending” |
| Personalized-Ranking | Re-rank items for a user | Personalized search results |
boto3 example — Retrieve recommendations and record events:
import boto3
personalize_runtime = boto3.client('personalize-runtime', region_name='us-east-1')
# Get recommendations for a user
response = personalize_runtime.get_recommendations(
campaignArn='arn:aws:personalize:us-east-1:123456789:campaign/my-campaign',
userId='user-42',
numResults=10,
context={
'DEVICE': 'mobile',
'DAYPART': 'evening'
}
)
print("Recommendations for user 42:")
for item in response['itemList']:
print(f" - Item ID: {item['itemId']}, Score: {item.get('score', 'N/A')}")
# Record a real-time event (e.g., click on a product)
personalize_events = boto3.client('personalize-events', region_name='us-east-1')
personalize_events.put_events(
trackingId='my-tracking-id',
userId='user-42',
sessionId='session-abc-123',
eventList=[
{
'eventId': 'evt-001',
'eventType': 'click',
'itemId': 'product-789',
'sentAt': '2024-03-15T14:30:00Z',
'properties': '{"price": 29.99}'
}
]
)
5.4 Intelligent Search Service
Amazon Kendra
ML-powered enterprise search service.
Features:
- Natural language search across enterprise documents
- Understands the meaning of questions, not just keywords
- Connectors for many sources: S3, SharePoint, Salesforce, ServiceNow, Confluence, RDS
- Returns direct answers (FAQ, excerpts) not just links
- Relevance tuning: business-domain relevance adjustment
boto3 example — Query a Kendra index:
import boto3
kendra = boto3.client('kendra', region_name='us-east-1')
response = kendra.query(
IndexId='my-index-id',
QueryText="What is the refund policy?",
QueryResultTypeFilter='ANSWER' # ANSWER | DOCUMENT | QUESTION_ANSWER
)
print(f"Total results: {response['TotalNumberOfResults']}\n")
for result in response['ResultItems']:
print(f"Type: {result['Type']}")
if result.get('DocumentExcerpt'):
print(f"Excerpt: {result['DocumentExcerpt']['Text'][:200]}...")
print(f"Score: {result['ScoreAttributes']['ScoreConfidence']}")
print("---")
5.5 Fraud Detection
Amazon Fraud Detector
Deprecated: Amazon Fraud Detector is no longer available for new customers since November 2025. For similar capabilities, use Amazon SageMaker, AutoGluon, or AWS WAF.
What Fraud Detector was:
- Fully managed service for online fraud detection
- Automatically trained ML models on your historical fraud data
- Based on 20+ years of Amazon.com fraud detection expertise
- Allowed creating decision rules and outcomes (pass, review, block)
Recommended alternative with SageMaker:
import sagemaker
from sagemaker.sklearn import SKLearn
sagemaker_session = sagemaker.Session()
role = sagemaker.get_execution_role()
# Train a fraud detection model (e.g., Random Forest via sklearn)
estimator = SKLearn(
entry_point='fraud_detection.py',
role=role,
instance_type='ml.m5.xlarge',
framework_version='1.2-1',
hyperparameters={
'n_estimators': 200,
'max_depth': 8,
'class_weight': 'balanced' # Handle fraud/non-fraud imbalance
}
)
estimator.fit({'train': 's3://my-bucket/fraud-data/train/'})
# Deploy the model to an endpoint
predictor = estimator.deploy(
initial_instance_count=1,
instance_type='ml.t2.medium',
endpoint_name='fraud-detection-endpoint'
)
Summary Table of AWS AI Services
| Service | Category | Primary Feature | boto3 Client |
|---|---|---|---|
| Amazon Rekognition Image | Vision | Image analysis, face recognition, moderation | rekognition |
| Amazon Rekognition Video | Vision | Video analysis, surveillance, tracking | rekognition |
| Amazon Textract | Vision/Document | Text and structured data extraction | textract |
| Amazon Comprehend | NLP | Sentiment analysis, NER, language detection | comprehend |
| Amazon Comprehend Medical | NLP/Medical | Medical data analysis, PHI detection | comprehendmedical |
| Amazon Translate | NLP | Neural machine translation | translate |
| Amazon Transcribe | Speech | Speech-to-text | transcribe |
| Amazon Polly | Speech | Text-to-speech | polly |
| Amazon Lex | Conversational | Chatbots, voice interfaces | lexv2-runtime |
| Amazon Forecast | Prediction | Time series forecasting | forecast |
| Amazon Personalize | Recommendation | Real-time personalized recommendations | personalize-runtime |
| Amazon Kendra | Search | Intelligent enterprise search | kendra |
| Amazon Fraud Detector | Security | Deprecated Nov. 2025 — Use SageMaker | frauddetector |
6. Key AWS Machine Learning Services
6.1 Amazon SageMaker — Overview
Amazon SageMaker AI is AWS’s flagship ML service. It is a fully managed service that enables developers and data scientists to build end-to-end ML models.
Note: SageMaker was renamed Amazon SageMaker AI in December 2024.
Positioning in the AWS Ecosystem
flowchart TB
subgraph Level1["Level 1: Managed AI Services\n(Turnkey, no ML expertise needed)"]
Rek[Rekognition]
Comp[Comprehend]
Trans[Translate]
Polly[Polly]
Lex[Lex]
end
subgraph Level2["Level 2: ML Platform\n(ML expertise required)"]
SM[Amazon SageMaker AI]
Canvas[SageMaker Canvas\nNo-code]
Autopilot[SageMaker Autopilot\nAutoML]
end
subgraph Level3["Level 3: Compute and Infrastructure\n(Infrastructure expertise required)"]
EC2[EC2 GPU Instances\np3, g4, trn1]
EKS[Amazon EKS\nKubernetes ML]
end
subgraph GenAI["GenAI / Foundation Models"]
Bedrock[Amazon Bedrock\nManaged FMs: Claude, Llama, etc.]
JumpStart[SageMaker JumpStart\nModel Hub]
end
Level1 -.-> Level2
Level2 -.-> Level3
GenAI --- Level2
style Level1 fill:#e8f5e9
style Level2 fill:#e3f2fd
style Level3 fill:#fff3e0
style GenAI fill:#f3e5f5
6.2 SageMaker Studio and Components
SageMaker covers all steps of the ML workflow:
flowchart TD
subgraph SM["Amazon SageMaker AI"]
subgraph DataStage["Step 1: Data"]
DW[SageMaker Data Wrangler\nPreparation and cleaning]
FE[SageMaker Feature Store\nFeature storage]
GL[SageMaker Ground Truth\nData labeling]
Canvas[SageMaker Canvas\nNo-code ML]
end
subgraph TrainStage["Step 2: Training"]
Studio[SageMaker Studio\nIntegrated ML IDE]
AutoML[SageMaker Autopilot\nAutoML]
Training[SageMaker Training\nDistributed training]
HPO[Hyperparameter Tuning\nAutomatic optimization]
Exp[SageMaker Experiments\nExperiment tracking]
JS[SageMaker JumpStart\nPre-trained model hub]
end
subgraph DeployStage["Step 3: Deployment"]
Endpoint[SageMaker Endpoints\nReal-time inference]
Batch[SageMaker Batch Transform\nBatch inference]
Serverless[SageMaker Serverless\nServerless inference]
Monitor[SageMaker Model Monitor\nProduction monitoring]
Registry[SageMaker Model Registry\nModel versioning]
end
subgraph MLOps["MLOps"]
Pipelines[SageMaker Pipelines\nML CI/CD orchestration]
Projects[SageMaker Projects\nMLOps templates]
end
end
DataStage --> TrainStage --> DeployStage
DeployStage --> MLOps
MLOps -->|Retrain| DataStage
Key SageMaker Components
| Component | Role | Level |
|---|---|---|
| SageMaker Canvas | No-code ML (drag & drop) | Beginner |
| SageMaker Studio | Integrated web IDE for the entire ML lifecycle | Intermediate |
| SageMaker Notebooks | Managed Jupyter notebooks | Intermediate |
| SageMaker Autopilot | AutoML — automatically creates the best model | Intermediate |
| SageMaker Data Wrangler | Data preparation and transformation without code | Intermediate |
| SageMaker Feature Store | Centralized ML feature repository (online + offline) | Advanced |
| SageMaker Ground Truth | Data labeling service (human + auto labeling) | Intermediate |
| SageMaker Training | Scalable and distributed training infrastructure | Advanced |
| SageMaker Experiments | ML experiment tracking and comparison (runs, metrics) | Intermediate |
| SageMaker Hyperparameter Tuning | Automatic hyperparameter optimization (HPO) | Advanced |
| SageMaker JumpStart | Pre-trained model hub (FMs, algorithms) | Beginner/Intermediate |
| SageMaker Model Registry | Model catalog and versioning with approval workflow | Advanced |
| SageMaker Endpoints | Real-time inference endpoint deployment | Advanced |
| SageMaker Serverless Inference | Serverless inference, scale to zero | Intermediate |
| SageMaker Batch Transform | Batch inference on large datasets | Advanced |
| SageMaker Model Monitor | Model drift and data quality detection in production | Advanced |
| SageMaker Pipelines | ML workflow orchestration (MLOps CI/CD) | Advanced |
| SageMaker HyperPod | Persistent clusters for large-scale LLMs and FMs | Expert |
Supported ML Paradigms
| Paradigm | Built-in SageMaker Algorithms |
|---|---|
| Supervised Learning | XGBoost, Linear Learner, KNN, Factorization Machines |
| Unsupervised Learning | K-Means, PCA, IP Insights, Random Cut Forest |
| Reinforcement Learning | Coach (RL framework), Ray RLlib |
| Deep Learning | TensorFlow, PyTorch, MXNet, Hugging Face |
| NLP | BlazingText, Seq2Seq, Object2Vec |
| Computer Vision | Image Classification, Object Detection, Semantic Segmentation |
| Time Series | DeepAR Forecasting |
6.3 Complete ML Pipeline with SageMaker
flowchart LR
subgraph Ingestion["1. Ingestion"]
S3["(Amazon S3\nRaw Data)"]
Kinesis[Kinesis Data\nStreams]
end
subgraph Prep["2. Preparation"]
Glue[AWS Glue\nETL]
DW[SageMaker\nData Wrangler]
FS[SageMaker\nFeature Store]
end
subgraph Label["3. Labeling"]
GT[SageMaker\nGround Truth]
end
subgraph Train["4. Training"]
Exp[SageMaker\nExperiments]
HPO[Hyperparameter\nTuning]
Training[SageMaker\nTraining Jobs]
end
subgraph Register["5. Registry"]
Reg[SageMaker\nModel Registry]
CI[CI/CD Pipeline\nApproval Workflow]
end
subgraph Serve["6. Serving"]
RT[Real-time\nEndpoint]
SL[Serverless\nInference]
BT[Batch\nTransform]
end
subgraph Monitor["7. Monitoring"]
MM[Model Monitor\nDrift Detection]
CW[CloudWatch\nMetrics and Alarms]
end
Ingestion --> Prep
Prep --> Label
Label --> Train
Train --> Register
Register --> Serve
Serve --> Monitor
Monitor -->|Retrain trigger| Train
style Ingestion fill:#e8f4fd,stroke:#1976d2
style Prep fill:#f3e5f5,stroke:#7b1fa2
style Label fill:#fff8e1,stroke:#f57f17
style Train fill:#e8f5e9,stroke:#388e3c
style Register fill:#fce4ec,stroke:#c62828
style Serve fill:#e0f2f1,stroke:#00695c
style Monitor fill:#fff3e0,stroke:#e65100
6.4 SageMaker Code Examples
Train an XGBoost model with the SageMaker Python SDK
import sagemaker
from sagemaker.inputs import TrainingInput
from sagemaker.xgboost import XGBoost
session = sagemaker.Session()
role = sagemaker.get_execution_role()
bucket = session.default_bucket()
prefix = 'xgboost-demo'
# Prepare data channels (S3)
train_input = TrainingInput(
s3_data=f's3://{bucket}/{prefix}/train',
content_type='text/csv'
)
validation_input = TrainingInput(
s3_data=f's3://{bucket}/{prefix}/validation',
content_type='text/csv'
)
# Create the XGBoost estimator
xgb = XGBoost(
entry_point='train.py',
framework_version='1.7-1',
instance_type='ml.m5.xlarge',
instance_count=1,
role=role,
hyperparameters={
'max_depth': 6,
'eta': 0.2,
'gamma': 4,
'min_child_weight': 6,
'subsample': 0.8,
'objective': 'binary:logistic',
'num_round': 100
},
output_path=f's3://{bucket}/{prefix}/output'
)
# Start training
xgb.fit({
'train': train_input,
'validation': validation_input
})
print(f"Model trained, artifacts in: {xgb.model_data}")
Deploy a real-time inference endpoint
import boto3
import sagemaker
# Deploy from a trained estimator
predictor = xgb.deploy(
initial_instance_count=1,
instance_type='ml.t2.medium',
endpoint_name='my-xgboost-endpoint'
)
# Make a prediction with the SDK predictor
import numpy as np
sample_data = np.array([[0.5, 1.2, 3.4, 0.8]])
result = predictor.predict(sample_data)
print(f"SDK Prediction: {result}")
# Make a prediction directly with boto3 (without SageMaker SDK)
sagemaker_runtime = boto3.client('sagemaker-runtime', region_name='us-east-1')
response = sagemaker_runtime.invoke_endpoint(
EndpointName='my-xgboost-endpoint',
ContentType='text/csv',
Body="0.5,1.2,3.4,0.8"
)
import json
result_boto3 = json.loads(response['Body'].read().decode())
print(f"boto3 Prediction: {result_boto3}")
SageMaker Autopilot (AutoML)
import boto3
import sagemaker
import time
sm = boto3.client('sagemaker', region_name='us-east-1')
role = sagemaker.get_execution_role()
# Launch an Autopilot job to predict churn
response = sm.create_auto_ml_job(
AutoMLJobName='autopilot-churn-prediction',
InputDataConfig=[
{
'DataSource': {
'S3DataSource': {
'S3DataType': 'S3Prefix',
'S3Uri': 's3://my-bucket/churn-data/train.csv'
}
},
'TargetAttributeName': 'churn' # Target column to predict
}
],
OutputDataConfig={
'S3OutputPath': 's3://my-bucket/autopilot-output/'
},
RoleArn=role,
AutoMLJobConfig={
'CompletionCriteria': {
'MaxCandidates': 20,
'MaxRuntimePerTrainingJobInSeconds': 600
}
},
ProblemType='BinaryClassification'
)
# Monitor the Autopilot job
while True:
status = sm.describe_auto_ml_job(AutoMLJobName='autopilot-churn-prediction')
job_status = status['AutoMLJobStatus']
print(f"Status: {job_status}")
if job_status in ['Completed', 'Failed', 'Stopped']:
break
time.sleep(60)
if job_status == 'Completed':
best = status['BestCandidate']
print(f"Best candidate: {best['CandidateName']}")
print(f"Best metric: {best['FinalAutoMLJobObjectiveMetric']}")
Create a SageMaker Pipeline (MLOps)
import sagemaker
from sagemaker.workflow.steps import TrainingStep, ProcessingStep
from sagemaker.workflow.pipeline import Pipeline
from sagemaker.workflow.parameters import ParameterString
from sagemaker.sklearn.processing import SKLearnProcessor
from sagemaker.sklearn import SKLearn
session = sagemaker.Session()
role = sagemaker.get_execution_role()
# Pipeline parameters
input_data = ParameterString(name="InputData", default_value="s3://my-bucket/data/")
# Step 1: Preprocessing
sklearn_processor = SKLearnProcessor(
framework_version='1.0-1',
instance_type='ml.m5.xlarge',
instance_count=1,
role=role
)
processing_step = ProcessingStep(
name='PreprocessData',
processor=sklearn_processor,
inputs=[
sagemaker.processing.ProcessingInput(
source=input_data,
destination='/opt/ml/processing/input'
)
],
outputs=[
sagemaker.processing.ProcessingOutput(
output_name='train',
source='/opt/ml/processing/train'
)
],
code='preprocessing.py'
)
# Step 2: Training
estimator = SKLearn(
entry_point='train.py',
framework_version='1.0-1',
instance_type='ml.m5.xlarge',
role=role
)
training_step = TrainingStep(
name='TrainModel',
estimator=estimator,
inputs={
'train': sagemaker.workflow.steps.TrainingInput(
s3_data=processing_step.properties.ProcessingOutputConfig
.Outputs['train'].S3Output.S3Uri
)
}
)
# Create and publish the pipeline
pipeline = Pipeline(
name='MyMLPipeline',
parameters=[input_data],
steps=[processing_step, training_step]
)
pipeline.upsert(role_arn=role)
# Start a pipeline execution
execution = pipeline.start()
print(f"Pipeline started: {execution.arn}")
7. boto3 Code Examples by Service
Full Pipeline: Audio → Transcription → Translation → Voice Synthesis
import boto3
import uuid
import time
import json
import urllib.request
def multilingual_audio_pipeline(
audio_s3_uri: str,
source_language: str = 'en-US',
target_language: str = 'es',
output_bucket: str = 'my-output-bucket'
) -> dict:
"""
Complete pipeline:
1. Transcribe : Audio → Text (source_language)
2. Translate : Text → Translated text (target_language)
3. Polly : Translated text → Audio (target_language)
"""
transcribe = boto3.client('transcribe', region_name='us-east-1')
translate = boto3.client('translate', region_name='us-east-1')
polly = boto3.client('polly', region_name='us-east-1')
s3 = boto3.client('s3', region_name='us-east-1')
# --- STEP 1: Transcription ---
job_name = f"transcription-{uuid.uuid4().hex[:8]}"
transcribe.start_transcription_job(
TranscriptionJobName=job_name,
Media={'MediaFileUri': audio_s3_uri},
MediaFormat='mp3',
LanguageCode=source_language
)
while True:
status = transcribe.get_transcription_job(
TranscriptionJobName=job_name
)['TranscriptionJob']['TranscriptionJobStatus']
if status in ['COMPLETED', 'FAILED']:
break
time.sleep(5)
job_result = transcribe.get_transcription_job(TranscriptionJobName=job_name)
transcript_uri = job_result['TranscriptionJob']['Transcript']['TranscriptFileUri']
with urllib.request.urlopen(transcript_uri) as url:
transcript_data = json.loads(url.read().decode())
original_text = transcript_data['results']['transcripts'][0]['transcript']
print(f"Transcribed text: {original_text[:100]}...")
# --- STEP 2: Translation ---
src_lang_code = source_language[:2] # 'en-US' → 'en'
translation = translate.translate_text(
Text=original_text,
SourceLanguageCode=src_lang_code,
TargetLanguageCode=target_language
)
translated_text = translation['TranslatedText']
print(f"Translated text: {translated_text[:100]}...")
# --- STEP 3: Voice synthesis ---
voice_map = {'en': 'Joanna', 'fr': 'Lea', 'de': 'Vicki', 'es': 'Lucia'}
voice_id = voice_map.get(target_language, 'Joanna')
audio_response = polly.synthesize_speech(
Text=translated_text,
OutputFormat='mp3',
VoiceId=voice_id,
Engine='neural'
)
output_key = f"output/translated_audio_{uuid.uuid4().hex[:8]}.mp3"
s3.put_object(
Bucket=output_bucket,
Key=output_key,
Body=audio_response['AudioStream'].read(),
ContentType='audio/mpeg'
)
return {
'original_text': original_text,
'translated_text': translated_text,
'audio_output': f"s3://{output_bucket}/{output_key}"
}
Pipeline: Image → Rekognition Analysis → DynamoDB Storage
import boto3
from datetime import datetime
def analyze_and_store_image(
image_bucket: str,
image_key: str,
dynamodb_table: str
) -> dict:
"""
Analyzes an image with Rekognition and stores results in DynamoDB.
"""
rekognition = boto3.client('rekognition', region_name='us-east-1')
dynamodb = boto3.resource('dynamodb', region_name='us-east-1')
table = dynamodb.Table(dynamodb_table)
img_ref = {'S3Object': {'Bucket': image_bucket, 'Name': image_key}}
labels_response = rekognition.detect_labels(
Image=img_ref, MaxLabels=20, MinConfidence=70.0
)
moderation_response = rekognition.detect_moderation_labels(
Image=img_ref, MinConfidence=60.0
)
faces_response = rekognition.detect_faces(
Image=img_ref, Attributes=['DEFAULT']
)
item = {
'ImageId': f"{image_bucket}/{image_key}",
'AnalysedAt': datetime.utcnow().isoformat(),
'Labels': [
{'Name': l['Name'], 'Confidence': str(round(l['Confidence'], 2))}
for l in labels_response['Labels']
],
'ModerationLabels': [
{'Name': l['Name'], 'Confidence': str(round(l['Confidence'], 2))}
for l in moderation_response['ModerationLabels']
],
'FaceCount': len(faces_response['FaceDetails']),
'IsAppropriate': len(moderation_response['ModerationLabels']) == 0
}
table.put_item(Item=item)
print(f"Stored: {item['ImageId']} — {len(item['Labels'])} labels, "
f"{item['FaceCount']} faces, appropriate: {item['IsAppropriate']}")
return item
Bulk NLP Analysis of Customer Reviews
import boto3
import json
def analyze_customer_reviews(reviews_list: list[str]) -> list[dict]:
"""
Analyzes customer reviews with Comprehend:
- Language detection
- Sentiment analysis
- Key entity extraction
"""
comprehend = boto3.client('comprehend', region_name='us-east-1')
results = []
# Process in batches of 25 (API limit)
for i in range(0, len(reviews_list), 25):
batch = reviews_list[i:i+25]
lang_response = comprehend.batch_detect_dominant_language(TextList=batch)
sentiment_response = comprehend.batch_detect_sentiment(
TextList=batch, LanguageCode='en'
)
for j, review in enumerate(batch):
detected_language = lang_response['ResultList'][j]['Languages'][0]['LanguageCode']
sentiment_data = sentiment_response['ResultList'][j]
results.append({
'review': review[:80] + '...' if len(review) > 80 else review,
'language': detected_language,
'sentiment': sentiment_data['Sentiment'],
'positive_score': round(sentiment_data['SentimentScore']['Positive'], 3),
'negative_score': round(sentiment_data['SentimentScore']['Negative'], 3)
})
return results
# Example usage
customer_reviews = [
"This product is excellent, truly beyond my expectations!",
"Very disappointed, poor quality and no customer service.",
"Fast delivery, product matches description.",
"Great product, highly recommended!",
"Decent product but the price is too high for what it is."
]
results = analyze_customer_reviews(customer_reviews)
for r in results:
print(f"[{r['sentiment']:8}] ({r['language']}) {r['review']}")
8. AWS ML Reference Architectures
Architecture: Real-Time Sentiment Analysis
flowchart LR
UserReview[Customer review\nMobile/web app]
subgraph Ingestion["Ingestion"]
KDS[Kinesis Data\nStreams]
end
subgraph Processing["Processing"]
Lambda[AWS Lambda\nOrchestration]
Comp[Amazon\nComprehend]
end
subgraph Storage["Storage"]
DDB["(Amazon\nDynamoDB)"]
S3["(Amazon S3\nArchive)"]
end
subgraph Visualization["Visualization"]
QB[Amazon\nQuickSight]
CW[CloudWatch\nDashboard]
end
UserReview --> KDS
KDS --> Lambda
Lambda --> Comp
Comp --> Lambda
Lambda --> DDB
Lambda --> S3
DDB --> QB
S3 --> QB
Lambda --> CW
Architecture: Customer Service Chatbot AWS
flowchart TD
User[User] -->|Text / Voice| Connect[Amazon Connect\nCall Center]
Connect --> Lex[Amazon Lex V2\nIntent understanding]
Lex -->|Intent: FAQ| Lambda[AWS Lambda\nBusiness logic]
Lex -->|Intent: Order| Lambda
Lambda --> Kendra[Amazon Kendra\nDocumentation search]
Lambda --> DDB["(DynamoDB\nOrder history)"]
Lambda --> Polly[Amazon Polly\nVoice response]
Polly --> Connect
Connect --> User
style User fill:#e8f5e9
style Polly fill:#e3f2fd
style Lex fill:#fff3e0
Architecture: MLOps Pipeline with SageMaker
flowchart TD
subgraph Dev["Development"]
DS[Data Scientist\nSageMaker Studio]
end
subgraph CICD["CI/CD Pipeline"]
CodeCommit[AWS CodeCommit\nSource code]
CodePipeline[AWS CodePipeline\nOrchestration]
CodeBuild[AWS CodeBuild\nUnit tests]
end
subgraph SMPipeline["SageMaker Pipeline"]
Proc[Processing Step\nData preparation]
Train[Training Step\nModel training]
Eval[Evaluation Step\nMetrics]
Cond{Condition\nAccuracy > 0.90?}
Reg[Register Model\nModel Registry]
Deploy[Deploy Step\nEndpoint]
end
subgraph Production["Production"]
EP[SageMaker Endpoint\nReal-time inference]
MM[Model Monitor\nDrift and quality]
Alarm[CloudWatch Alarm\nAlerts]
end
Dev --> CICD
CodeCommit --> CodePipeline
CodePipeline --> CodeBuild
CodeBuild --> SMPipeline
Proc --> Train --> Eval --> Cond
Cond -->|Yes| Reg --> Deploy --> EP
Cond -->|No| Train
EP --> MM --> Alarm
Alarm -->|Drift detected| SMPipeline
Architecture: Intelligent Document Processing
flowchart LR
Docs[Documents\nPDF, Images, Scans]
subgraph Extraction["Extraction"]
Textract[Amazon Textract\nOCR + Structure]
end
subgraph Enrichment["NLP Enrichment"]
CompText[Comprehend\nEntities, Sentiment]
CompMed[Comprehend Medical\nIf medical doc]
Trans[Translate\nIf translation needed]
end
subgraph Storage["Storage and Index"]
S3Proc["(S3 Processed)"]
OpenSearch[Amazon OpenSearch\nFull-text search]
Kendra[Amazon Kendra\nSemantic search]
end
subgraph Usage["Usage"]
Search[Internal\nsearch portal]
BI[Dashboards\nQuickSight]
end
Docs --> Textract
Textract --> CompText
CompText --> CompMed
CompText --> Trans
Textract --> S3Proc
S3Proc --> OpenSearch & Kendra
OpenSearch & Kendra --> Search & BI
9. Comparisons and Decision Guidance
When to Use Managed AI Services vs SageMaker?
flowchart TD
Q1{Do you need\na custom model?}
Q1 -->|No| Q2{What is your\nuse case?}
Q1 -->|Yes| QCode{Do you have\nML coding skills?}
QCode -->|No| Canvas[SageMaker Canvas\nNo-code]
QCode -->|Some| Autopilot[SageMaker Autopilot\nAutoML]
QCode -->|Yes| SM[Amazon SageMaker\nCustom ML build]
Q2 -->|Image/video analysis| Rek[Amazon Rekognition]
Q2 -->|Text extraction from docs| Txt[Amazon Textract]
Q2 -->|Text analysis / NLP| Comp[Amazon Comprehend]
Q2 -->|Medical data| CompM[Comprehend Medical]
Q2 -->|Translation| Tran[Amazon Translate]
Q2 -->|Audio transcription| Transcrb[Amazon Transcribe]
Q2 -->|Voice synthesis| Polly[Amazon Polly]
Q2 -->|Chatbot| Lex[Amazon Lex]
Q2 -->|Time series forecasting| Fore[Amazon Forecast]
Q2 -->|Recommendations| Pers[Amazon Personalize]
Q2 -->|Intelligent search| Kendra[Amazon Kendra]
Q2 -->|LLMs / GenAI| Bedrock[Amazon Bedrock]
Selection Criteria
| Criterion | Managed AI Services | SageMaker Autopilot | SageMaker (custom) |
|---|---|---|---|
| ML expertise required | No | No | Yes |
| Code expertise required | Minimal (API calls) | No | Yes |
| Customization | Limited | Moderate | Total |
| Training data | Not needed | Required | Required |
| Time to implement | Hours | Days | Weeks/months |
| Use cases | Standard | Classic tabular | Domain-specific |
| Starting cost | Low (pay-per-use) | Moderate | Variable |
| Infrastructure control | None | Low | Total |
SageMaker Inference Types
| Type | Latency | Billing | Ideal Use |
|---|---|---|---|
| Real-time Endpoint | Milliseconds | Permanent instance | Interactive apps, regular traffic |
| Serverless Inference | Seconds (cold start) | Pay-per-use | Sporadic traffic |
| Batch Transform | Minutes-hours | Pay-per-job | Batch processing of large datasets |
| Async Inference | Seconds-minutes | Pay-per-use | Heavy inputs, long inferences |
| Multi-model Endpoint | Milliseconds | Shared instances | Many similar models |
10. Key Concepts to Remember
| Concept | Definition |
|---|---|
| AI (Artificial Intelligence) | Simulation of human intelligence by machines |
| ML (Machine Learning) | AI subset where machines learn from data |
| Deep Learning | ML subset using deep neural networks |
| Model | Trained algorithm capable of making predictions |
| Training | Process of model learning on labeled data |
| Inference | Application of a trained model on new data to make predictions |
| Supervised Learning | Learning with labeled data (input + target) |
| Unsupervised Learning | Learning without labels, pattern discovery |
| Reinforcement Learning | Learning through rewards and penalties |
| Classification | Predicting a category (discrete) |
| Regression | Predicting a continuous value |
| Clustering | Grouping similar data without supervision |
| NLP (Natural Language Processing) | Automatic processing of natural language |
| Model Drift | Performance degradation of a model in production |
| Data Drift | Change in the distribution of input data |
| Feature Engineering | Transformation of raw data into useful ML features |
| Training Dataset | Data set used to train the model |
| Inference Endpoint | API access point exposing a trained model for predictions |
| AutoML | Automation of best algorithm selection and hyperparameter tuning |
| MLOps | DevOps practices applied to the ML lifecycle (CI/CD, monitoring, versioning) |
| Hyperparameter | Model configuration parameter set before training (e.g., learning rate) |
| Overfitting | Model too specialized on training data, poor generalization |
| Underfitting | Model too simple, unable to capture patterns in the data |
| PHI | Protected Health Information — protected personal health data (HIPAA) |
| Speaker Diarization | Identification and attribution of speech segments to each speaker |
| SSML | Speech Synthesis Markup Language — markup for controlling voice synthesis |
| Foundation Model (FM) | Large model pre-trained on massive data, adaptable via fine-tuning |
| Ground Truth | Labeled data serving as the absolute reference for training |
| Feature Store | Centralized repository to store, share, and reuse ML features |
| NER | Named Entity Recognition — extraction of named entities from text |
| Sentiment Analysis | Determining the emotional polarity of a text |
Resources and References
| Resource | URL |
|---|---|
| Amazon SageMaker AI Documentation | https://docs.aws.amazon.com/sagemaker/latest/dg/whatis.html |
| Amazon Rekognition Documentation | https://docs.aws.amazon.com/rekognition/latest/dg/what-is.html |
| Amazon Comprehend Documentation | https://docs.aws.amazon.com/comprehend/latest/dg/what-is.html |
| boto3 Reference — SageMaker | https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html |
| SageMaker Python SDK | https://sagemaker.readthedocs.io/en/stable/ |
| AWS Machine Learning Blog | https://aws.amazon.com/blogs/machine-learning/ |
| AWS ML Immersion Day (Workshop) | https://catalog.workshops.aws/machinelearning/ |
| AWS Code Examples — ML | https://docs.aws.amazon.com/code-library/latest/ug/ml.html |
Search Terms
aws · machine · ai · fundamentals · amazon · web · services · sagemaker · pipeline · architecture · model · analysis · approach · deploy · inference · rekognition · service · stage · train · types · analogy · comparative · components · comprehend