AI-900: NLP Workloads on Azure

Course: Microsoft Azure AI Fundamentals (AI-900) – NLP Workloads on Azure Certification: AI-900 Azure AI Fundamentals Level: Beginner to Intermediate

Table of Contents

1. Introduction to NLP
2. Text Analytics – Azure AI Language
3. Tokenization and Text Processing
4. Azure AI Language vs Azure AI Services
5. Speech – Recognition and Synthesis
6. Translation – Text and Speech Translation
7. Language Understanding – CLU
8. Question Answering – Knowledge Base
9. Practical Implementation with the Python SDK
10. NLP Solution Architecture
11. Exam Tips and Common Pitfalls
12. Practical Exercises and Scenarios
13. Summary and Key Points
14. Glossary

1. Introduction to NLP

1.1 What is NLP?

Natural Language Processing (NLP) is the AI domain that gives computers the ability to understand, interpret, and generate human language. Unlike programming languages which follow strict, unambiguous rules, human language is rich with nuances, metaphors, irony, and contextual ambiguities.

Think about your voice assistants: Cortana, Siri, Alexa. When you say “Play some jazzy music”, these systems must:

1. Hear your words (speech recognition)
2. Understand your intent (you want to listen to jazz)
3. Act accordingly (launch a jazz playlist)
4. Respond vocally if needed (speech synthesis)

All of these steps fall under NLP.

mindmap
  root((NLP on Azure))
    Text Analytics
      Language Detection
      Sentiment Analysis
      Key Phrase Extraction
      NER Entity Recognition
      PII Extraction
      Text Summarization
    Speech
      Speech-to-Text
        Real-time
        Batch
        Custom Models
      Text-to-Speech
        Predefined voices
        Custom voices
        SSML
    Translation
      Azure AI Translator
        90+ languages
        Office PDF Documents
        Custom Translation
      Speech Translation
        Speech-to-Speech
        Translated Speech-to-Text
    Language Understanding
      CLU Conversational
        Utterances
        Entities
        Intents
      Question Answering
        Knowledge Base
        FAQ import
        Chitchat

1.2 Why is NLP Difficult?

Human language presents unique challenges for machines:

1.3 Azure NLP Services – Overview

flowchart TB
    subgraph "Azure AI Services (Multi-service Account)"
        AIS["Single endpoint + key"]
    end

    subgraph "Specialized NLP Services"
        LANG["Azure AI Language\nText Analytics, CLU,\nQuestion Answering"]
        SPEECH["Azure AI Speech\nSTT, TTS,\nSpeech Translation"]
        TRANS["Azure AI Translator\nText translation\nand documents"]
    end

    AIS --> LANG
    AIS --> SPEECH
    AIS --> TRANS

    LANG --> L1["Language Detection"]
    LANG --> L2["Sentiment Analysis"]
    LANG --> L3["Key Phrase Extraction"]
    LANG --> L4["NER - Entities"]
    LANG --> L5["CLU - Understanding"]
    LANG --> L6["Question Answering"]

    SPEECH --> S1["Speech-to-Text"]
    SPEECH --> S2["Text-to-Speech"]
    SPEECH --> S3["Speech Translation"]

    TRANS --> T1["Text translation"]
    TRANS --> T2["Document translation"]
    TRANS --> T3["Custom models"]

1.4 NLP Capabilities – Reference Table

2. Text Analytics – Azure AI Language

2.1 Language Detection

Language detection automatically identifies the language a text is written in. It is particularly useful for:

• Routing messages to the right language support agent
• Applying the correct processing model afterward
• Managing multilingual applications

How it works: The service analyzes statistical patterns in the text: letter frequency, word combinations, grammatical structure. It can identify more than 120 languages.

{
  "documents": [
    {
      "id": "1",
      "detectedLanguage": {
        "name": "English",
        "iso6391Name": "en",
        "confidenceScore": 1.0
      }
    }
  ]
}

Special cases:

• Mixed text: The service returns the dominant language
• Ambiguous or short text: Low confidence score, may return (Unknown)
• Unknown language: Returns name: "(Unknown)" and confidenceScore: NaN

# Language detection with Azure AI Language
from azure.ai.textanalytics import TextAnalyticsClient
from azure.core.credentials import AzureKeyCredential
import os

endpoint = os.environ["AZURE_LANGUAGE_ENDPOINT"]
key = os.environ["AZURE_LANGUAGE_KEY"]

client = TextAnalyticsClient(
    endpoint=endpoint,
    credential=AzureKeyCredential(key)
)

# Texts in different languages
texts = [
    "Hello, how can I help you today?",
    "Bonjour, comment puis-je vous aider aujourd'hui?",
    "Hola, ¿cómo puedo ayudarte hoy?",
    "Guten Tag, wie kann ich Ihnen helfen?",
    "こんにちは、今日はどのようにお手伝いできますか？"
]

# Detect languages
results = client.detect_language(documents=texts)

for idx, result in enumerate(results):
    if not result.is_error:
        lang = result.primary_language
        print(f"Text {idx+1}: {texts[idx][:40]}...")
        print(f"  Language: {lang.name} ({lang.iso6391_name})")
        print(f"  Confidence: {lang.confidence_score:.2%}")
    else:
        print(f"Error: {result.error.message}")

2.2 Sentiment Analysis

Sentiment analysis determines whether a text expresses a positive, negative, or neutral sentiment. Azure AI Language provides scores for each of the three categories, with the category having the highest score as the main result.

Sentiment scores:

• Each category (positive, neutral, negative) receives a score between 0 and 1
• The sum of the three scores ≈ 1
• The final label corresponds to the highest score

Practical example:

Text: "This website is excellent! Delivery was fast and products are quality."
→ Positive: 0.95, Neutral: 0.04, Negative: 0.01
→ Label: POSITIVE

Text: "Customer service is terrible, I waited 3h and no one helped me."
→ Positive: 0.02, Neutral: 0.05, Negative: 0.93
→ Label: NEGATIVE

Text: "I received my order yesterday."
→ Positive: 0.08, Neutral: 0.84, Negative: 0.08
→ Label: NEUTRAL

Opinion Mining (Aspect-Based Sentiment):

An advanced feature allows analyzing sentiment by aspect of a product or service:

{
  "text": "The coffee was delicious but the service was slow.",
  "sentiment": "mixed",
  "opinions": [
    {
      "target": "coffee",
      "sentiment": "positive",
      "confidenceScores": { "positive": 0.96, "negative": 0.04 },
      "assessments": [{ "text": "delicious", "sentiment": "positive" }]
    },
    {
      "target": "service",
      "sentiment": "negative",
      "confidenceScores": { "positive": 0.02, "negative": 0.98 },
      "assessments": [{ "text": "slow", "sentiment": "negative" }]
    }
  ]
}

# Full sentiment analysis with Opinion Mining
def analyze_sentiments(texts: list[str]) -> list[dict]:
    """
    Analyzes the sentiment of multiple texts with aspect-level granularity.

    Args:
        texts: List of texts to analyze

    Returns:
        List of dictionaries with global and aspect-level sentiment
    """
    analysis_results = []

    results = client.analyze_sentiment(
        documents=texts,
        show_opinion_mining=True,  # Enable aspect-based analysis
        language="en"
    )

    for idx, doc in enumerate(results):
        if doc.is_error:
            analysis_results.append({"error": doc.error.message})
            continue

        analysis = {
            "text": texts[idx][:100],
            "global_sentiment": doc.sentiment,
            "scores": {
                "positive": round(doc.confidence_scores.positive, 3),
                "neutral": round(doc.confidence_scores.neutral, 3),
                "negative": round(doc.confidence_scores.negative, 3)
            },
            "sentences": []
        }

        # Analyze each sentence individually
        for sentence in doc.sentences:
            sentence_info = {
                "text": sentence.text,
                "sentiment": sentence.sentiment,
                "aspects": []
            }

            # Analyze aspects (Opinion Mining)
            for opinion in sentence.mined_opinions:
                aspect_info = {
                    "aspect": opinion.target.text,
                    "sentiment": opinion.target.sentiment,
                    "assessments": [
                        {
                            "text": assess.text,
                            "sentiment": assess.sentiment
                        }
                        for assess in opinion.assessments
                    ]
                }
                sentence_info["aspects"].append(aspect_info)

            analysis["sentences"].append(sentence_info)

        analysis_results.append(analysis)

    return analysis_results

# Example usage
customer_reviews = [
    "Excellent product! Quality is top-notch and delivery was very fast.",
    "Disappointed with the service, packaging was damaged and refund was difficult.",
    "Average product, reasonable price. Customer service was responsive but not very competent."
]

analyses = analyze_sentiments(customer_reviews)

for analysis in analyses:
    if "error" in analysis:
        print(f"Error: {analysis['error']}")
        continue

    print(f"\nText: {analysis['text']}")
    print(f"Sentiment: {analysis['global_sentiment'].upper()}")
    print(f"Scores: +{analysis['scores']['positive']:.0%} | "
          f"0{analysis['scores']['neutral']:.0%} | "
          f"-{analysis['scores']['negative']:.0%}")

    for sentence in analysis["sentences"]:
        if sentence["aspects"]:
            print(f"\n  Aspects in: '{sentence['text']}'")
            for aspect in sentence["aspects"]:
                print(f"    [{aspect['sentiment'].upper()}] {aspect['aspect']}: "
                      f"{', '.join(e['text'] for e in aspect['assessments'])}")

2.3 Key Phrase Extraction

Key phrase extraction identifies the most important terms and expressions in a text. Unlike entities which have predefined categories (person, place, organization), key phrases are simply the most significant terms in the document.

Important difference for the exam:

• Key phrases → Returns important terms (without category)
• Entities (NER) → Returns terms with their category (Person, Place, Org, Date…)

# Key phrase extraction
def extract_key_phrases(texts: list[str], language: str = "en") -> list[dict]:
    """
    Extracts key phrases from multiple texts.

    Returns:
        List of dicts {text, key_phrases}
    """
    results = client.extract_key_phrases(
        documents=texts,
        language=language
    )

    analyses = []
    for idx, doc in enumerate(results):
        if doc.is_error:
            analyses.append({"error": doc.error.message})
        else:
            analyses.append({
                "text": texts[idx][:80],
                "key_phrases": list(doc.key_phrases)
            })

    return analyses

# Example
articles = [
    """Microsoft Azure is a cloud computing platform offering artificial intelligence,
    machine learning, data storage, and advanced analytics services
    for global enterprises.""",

    """Climate change represents one of the most important challenges
    of the 21st century. Greenhouse gas emissions contribute to
    rising global temperatures."""
]

results = extract_key_phrases(articles)
for r in results:
    print(f"Text: {r.get('text', 'N/A')}")
    print(f"Key phrases: {', '.join(r.get('key_phrases', []))}")
    print()

Example output:

Text: Microsoft Azure is a cloud computing platform...
Key phrases: Microsoft Azure, cloud computing platform, artificial intelligence services, machine learning, data storage, advanced analytics, global enterprises

Text: Climate change represents one of the most important challenges...
Key phrases: climate change, important challenges, greenhouse gas emissions, global temperatures

2.4 Named Entity Recognition (NER)

Named Entity Recognition (NER) identifies and classifies entities in a text. Each detected entity receives a category and sub-category with a confidence score.

Available entity categories:

PII (Personally Identifiable Information) Extraction:

Azure AI Language has a specialized service to detect and redact personally identifiable information:

# NER and PII extraction
def analyze_entities(texts: list[str]) -> list[dict]:
    """Extracts named entities and PII from a list of texts."""

    # Standard NER
    ner_results = client.recognize_entities(
        documents=texts,
        language="en"
    )

    # PII (Personal Identifiable Information)
    pii_results = client.recognize_pii_entities(
        documents=texts,
        language="en"
    )

    analyses = []
    for idx, (ner_doc, pii_doc) in enumerate(zip(ner_results, pii_results)):
        analysis = {
            "original_text": texts[idx],
            "redacted_text": pii_doc.redacted_text if not pii_doc.is_error else None,
            "entities": [],
            "pii": []
        }

        # NER entities
        if not ner_doc.is_error:
            for entity in ner_doc.entities:
                analysis["entities"].append({
                    "text": entity.text,
                    "category": entity.category,
                    "sub_category": entity.subcategory,
                    "confidence": round(entity.confidence_score, 3),
                    "position": {"start": entity.offset, "length": entity.length}
                })

        # PII entities
        if not pii_doc.is_error:
            for entity in pii_doc.entities:
                analysis["pii"].append({
                    "text": entity.text,
                    "category": entity.category,
                    "confidence": round(entity.confidence_score, 3)
                })

        analyses.append(analysis)

    return analyses

# Example usage
test_texts = [
    """Dr. Sarah Johnson from Seattle contacted Microsoft on March 15, 2024.
    Her email is sarah.johnson@example.com and her phone is 206-555-9876."""
]

results = analyze_entities(test_texts)
for r in results:
    print(f"Original text: {r['original_text'][:80]}")
    print(f"Redacted text: {r['redacted_text'][:80]}")
    print("\nNER Entities:")
    for e in r["entities"]:
        print(f"  [{e['category']}/{e.get('sub_category', '-')}] "
              f"'{e['text']}' ({e['confidence']:.0%})")
    print("\nPII detected:")
    for p in r["pii"]:
        print(f"  [{p['category']}] '{p['text']}'")

2.5 Extractive Text Summarization

Azure AI Language can also automatically summarize long texts:

# Extractive summarization
def summarize_text(long_text: str, num_sentences: int = 3) -> str:
    """
    Generates an extractive summary of a long text.
    Extractive = selects the most representative sentences.
    """
    from azure.ai.textanalytics import ExtractiveSummaryAction

    actions = [ExtractiveSummaryAction(max_sentence_count=num_sentences)]

    poller = client.begin_analyze_actions(
        documents=[long_text],
        actions=actions,
        language="en"
    )

    for page in poller.result():
        for doc in page:
            if not doc.is_error:
                summary_sentences = sorted(
                    doc.sentences,
                    key=lambda s: s.rank_score,
                    reverse=True
                )[:num_sentences]

                return " ".join(sentence.text for sentence in summary_sentences)

    return ""

3. Tokenization and Text Processing

3.1 Concept of Tokenization

Tokenization is the first step of NLP processing. It consists of breaking a text into elementary units called tokens.

flowchart LR
    TEXT["'Welcome to the party!'"] --> TOK["Tokenization"]
    TOK --> T1["Token 1\n'Welcome'\n→ ID: 2156"]
    TOK --> T2["Token 2\n'to'\n→ ID: 58"]
    TOK --> T3["Token 3\n'the'\n→ ID: 145"]
    TOK --> T4["Token 4\n'party'\n→ ID: 3892"]
    TOK --> T5["Token 5\n'!'\n→ ID: 999"]

3.2 Text Preprocessing Techniques

3.3 Impact on Analysis

# Illustration of preprocessing techniques
import re
from collections import Counter

def preprocess_text(text: str,
                    lowercase: bool = True,
                    remove_punctuation: bool = True,
                    remove_stop_words: bool = True) -> list[str]:
    """
    Preprocesses text for NLP analysis.

    Args:
        text: Text to preprocess
        lowercase: Convert to lowercase
        remove_punctuation: Remove punctuation
        remove_stop_words: Remove stop words

    Returns:
        List of preprocessed tokens
    """
    STOP_WORDS_EN = {
        "the", "a", "an", "of", "in", "on", "at", "to", "for", "is", "are",
        "was", "were", "be", "been", "being", "have", "has", "had", "do",
        "does", "did", "will", "would", "could", "should", "may", "might",
        "and", "or", "but", "if", "by", "with", "from", "this", "that", "it"
    }

    # Lowercase
    if lowercase:
        text = text.lower()

    # Remove punctuation
    if remove_punctuation:
        text = re.sub(r'[^\w\s]', '', text)

    # Simple tokenization (by spaces)
    tokens = text.split()

    # Remove stop words
    if remove_stop_words:
        tokens = [t for t in tokens if t not in STOP_WORDS_EN]

    return tokens

def analyze_frequencies(text: str, top_n: int = 10) -> dict:
    """Analyzes term frequencies in a text."""
    tokens = preprocess_text(text)
    frequencies = Counter(tokens)

    return {
        "total_tokens": len(tokens),
        "unique_tokens": len(frequencies),
        "top_terms": frequencies.most_common(top_n)
    }

# Example
sample_text = """
Azure is a cloud platform by Microsoft that offers services
for artificial intelligence and machine learning. Developers
can use Azure to build intelligent applications
that understand natural language and analyze images.
"""

stats = analyze_frequencies(sample_text, top_n=5)
print(f"Total tokens: {stats['total_tokens']}")
print(f"Unique tokens: {stats['unique_tokens']}")
print("Top terms:")
for term, freq in stats["top_terms"]:
    print(f"  {term}: {freq}x")

3.4 N-grams for Context

def generate_ngrams(tokens: list[str], n: int) -> list[tuple]:
    """
    Generates N-grams from a list of tokens.

    Args:
        tokens: List of tokens
        n: N-gram size (2=bigrams, 3=trigrams...)

    Returns:
        List of N-token tuples
    """
    return [tuple(tokens[i:i+n]) for i in range(len(tokens) - n + 1)]

# Example
tokens = ["machine", "learning", "azure", "artificial", "intelligence"]

bigrams = generate_ngrams(tokens, 2)
trigrams = generate_ngrams(tokens, 3)

print("Bigrams:", bigrams)
# [('machine', 'learning'), ('learning', 'azure'), ('azure', 'artificial'), ...]

print("Trigrams:", trigrams)
# [('machine', 'learning', 'azure'), ...]

4. Azure AI Language vs Azure AI Services

4.1 Two Access Modes

flowchart TD
    APP["Application"] --> Q{"Uses multiple\nAI services?"}

    Q -->|No, only\nAzure AI Language| STANDALONE["Azure AI Language\n(Standalone service)\n→ Dedicated endpoint\n→ Dedicated key\n→ Separate billing"]

    Q -->|"Yes, multiple services\n(Language + Speech + Vision...)"| MULTI["Azure AI Services\n(Multi-service account)\n→ Single endpoint\n→ Single key\n→ Consolidated billing\n→ 'Simplify administration'"]

    STANDALONE --> ADVANTAGE1["✅ Granular control\n✅ Isolate costs\n✅ Dedicated security policy"]
    MULTI --> ADVANTAGE2["✅ Easy to manage\n✅ Single access point\n✅ Less configuration"]

4.2 Use Cases and Recommendations

Exam reminder: When the question mentions “simplify administration” or “use multiple services with a single endpoint”, the answer is Azure AI Services (formerly: Cognitive Services).

5. Speech – Recognition and Synthesis

5.1 Speech-to-Text (Speech Recognition)

Azure AI Speech converts speech to text. The process involves two models:

1. Acoustic Model: Converts audio signal to phonemes
2. Language Model: Reconstructs words and sentences from phonemes

flowchart LR
    AUDIO["🎙️ Audio signal"] --> ACOUSTIC["Acoustic model\n(Phonemes)"]
    ACOUSTIC --> LANGUAGE["Language model\n(Words and sentences)"]
    LANGUAGE --> TEXT["📝 Transcribed text"]

    AUDIO --> PREPRO["Preprocessing\n(Noise reduction,\nNormalization)"]
    PREPRO --> ACOUSTIC

Two transcription modes:

# Speech-to-Text with Azure AI Speech SDK
import azure.cognitiveservices.speech as speechsdk
import os
import time

def transcribe_from_microphone() -> str:
    """Transcribes speech from the microphone in real time."""

    # Configuration
    config = speechsdk.SpeechConfig(
        subscription=os.environ["AZURE_SPEECH_KEY"],
        region=os.environ["AZURE_SPEECH_REGION"]
    )
    config.speech_recognition_language = "en-US"

    # Create recognizer with microphone
    audio_config = speechsdk.audio.AudioConfig(use_default_microphone=True)
    recognizer = speechsdk.SpeechRecognizer(
        speech_config=config,
        audio_config=audio_config
    )

    print("🎙️ Speak now (max 15 seconds)...")

    # Synchronous recognition (waits for end of sentence)
    result = recognizer.recognize_once_async().get()

    if result.reason == speechsdk.ResultReason.RecognizedSpeech:
        print(f"Transcribed: {result.text}")
        return result.text

    elif result.reason == speechsdk.ResultReason.NoMatch:
        print("No speech detected")
        return ""

    elif result.reason == speechsdk.ResultReason.Canceled:
        details = speechsdk.CancellationDetails(result)
        print(f"Canceled: {details.reason}")
        if details.reason == speechsdk.CancellationReason.Error:
            print(f"Error: {details.error_details}")
        return ""

def transcribe_audio_file(file_path: str, language: str = "en-US") -> str:
    """Transcribes a complete audio file."""

    config = speechsdk.SpeechConfig(
        subscription=os.environ["AZURE_SPEECH_KEY"],
        region=os.environ["AZURE_SPEECH_REGION"]
    )
    config.speech_recognition_language = language

    # Enable detailed output format (includes confidence)
    config.output_format = speechsdk.OutputFormat.Detailed

    # Audio source from file
    audio_config = speechsdk.audio.AudioConfig(filename=file_path)

    recognizer = speechsdk.SpeechRecognizer(
        speech_config=config,
        audio_config=audio_config
    )

    # Collect results for long transcription
    full_text = []
    done = False

    def recognized(evt):
        if evt.result.reason == speechsdk.ResultReason.RecognizedSpeech:
            full_text.append(evt.result.text)
            print(f"  ✓ {evt.result.text}")

    def finished(evt):
        nonlocal done
        done = True

    def canceled(evt):
        nonlocal done
        done = True
        print(f"Canceled: {evt.result.cancellation_details.reason}")

    recognizer.recognized.connect(recognized)
    recognizer.session_stopped.connect(finished)
    recognizer.canceled.connect(canceled)

    # Start continuous transcription
    recognizer.start_continuous_recognition()

    # Wait for completion
    while not done:
        time.sleep(0.5)

    recognizer.stop_continuous_recognition()

    return " ".join(full_text)

# Example
print("=== Real-time transcription ===")
text = transcribe_from_microphone()

print("\n=== Audio file transcription ===")
transcription = transcribe_audio_file("meeting_2024.wav", language="en-US")
print(f"Full transcription ({len(transcription)} characters):")
print(transcription[:500])

5.2 Text-to-Speech (Speech Synthesis)

Azure AI Speech transforms text into natural speech. More than 400 voices in more than 140 languages are available.

Voice types:

• Standard voices: Acceptable quality, low latency
• Neural voices: Very natural quality, based on deep learning
• Custom voices: Clone a real voice (with authorization)

# Text-to-Speech with Azure AI Speech SDK
import azure.cognitiveservices.speech as speechsdk
import os

def synthesize_speech(text: str,
                       voice_name: str = "en-US-AriaNeural",
                       output_file: str = None) -> bool:
    """
    Synthesizes text into speech.

    Args:
        text: Text to synthesize
        voice_name: Azure voice name (e.g., en-US-AriaNeural)
        output_file: WAV output path (None = direct playback)

    Returns:
        True if successful, False otherwise
    """
    config = speechsdk.SpeechConfig(
        subscription=os.environ["AZURE_SPEECH_KEY"],
        region=os.environ["AZURE_SPEECH_REGION"]
    )

    # Configure voice
    config.speech_synthesis_voice_name = voice_name

    # Configure output (speaker or file)
    if output_file:
        audio_config = speechsdk.audio.AudioOutputConfig(filename=output_file)
    else:
        audio_config = speechsdk.audio.AudioOutputConfig(use_default_speaker=True)

    synthesizer = speechsdk.SpeechSynthesizer(
        speech_config=config,
        audio_config=audio_config
    )

    # Synthesize
    result = synthesizer.speak_text_async(text).get()

    if result.reason == speechsdk.ResultReason.SynthesizingAudioCompleted:
        print(f"✅ Synthesis successful: {len(text)} characters")
        if output_file:
            print(f"   Saved to: {output_file}")
        return True

    elif result.reason == speechsdk.ResultReason.Canceled:
        details = speechsdk.SpeechSynthesisCancellationDetails(result)
        print(f"❌ Error: {details.reason}")
        if details.error_details:
            print(f"   Details: {details.error_details}")
        return False

def synthesize_ssml(ssml: str, output_file: str = None) -> bool:
    """
    Synthesizes text with fine-grained control via SSML.
    SSML = Speech Synthesis Markup Language.
    """
    config = speechsdk.SpeechConfig(
        subscription=os.environ["AZURE_SPEECH_KEY"],
        region=os.environ["AZURE_SPEECH_REGION"]
    )

    if output_file:
        audio_config = speechsdk.audio.AudioOutputConfig(filename=output_file)
    else:
        audio_config = speechsdk.audio.AudioOutputConfig(use_default_speaker=True)

    synthesizer = speechsdk.SpeechSynthesizer(
        speech_config=config,
        audio_config=audio_config
    )

    result = synthesizer.speak_ssml_async(ssml).get()
    return result.reason == speechsdk.ResultReason.SynthesizingAudioCompleted

# Example SSML
ssml_example = """
<speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis" xml:lang="en-US">
    <voice name="en-US-AriaNeural">
        <prosody rate="slow" pitch="+5%">
            Welcome to Azure AI Services.
        </prosody>
        <break time="500ms"/>
        <emphasis level="strong">
            This demonstration illustrates neural speech synthesis.
        </emphasis>
        <break time="300ms"/>
        <prosody volume="soft">
            Thank you for your attention.
        </prosody>
    </voice>
</speak>
"""

# Simple synthesis
synthesize_speech(
    text="Welcome to the world of AI on Azure!",
    voice_name="en-US-AriaNeural",
    output_file="welcome.wav"
)

# Advanced SSML synthesis
synthesize_ssml(ssml_example, output_file="demo_ssml.wav")

Available English neural voices:

6. Translation – Text and Speech Translation

6.1 Azure AI Translator (Text)

Azure AI Translator translates text between more than 90 languages. It is distinct from the translation service in Speech.

flowchart LR
    TEXT["📝 Source text\n(any language)"] --> TRANS["Azure AI Translator"]
    TRANS --> LANG_DETECT["Automatic detection\nof source language"]
    TRANS --> OUT1["French"]
    TRANS --> OUT2["Spanish"]
    TRANS --> OUT3["Japanese"]
    TRANS --> OUT4["... 90+ languages"]

    TRANS --> FEATURES["Advanced features\n• Dictionary\n• Profanity filter\n• Transliteration\n• Language detection"]

Key features:

• Simultaneous translation to multiple languages in a single call
• Automatic detection of the source language
• Document translation (Word, Excel, PowerPoint, PDF, HTML)
• Custom models (Custom Translator) for specialized vocabulary
• Dictionaries for idiomatic expressions
• Profanity filter (NoAction, Deleted, Marked)
• Transliteration (convert from one alphabet to another, e.g., Arabic → Latin)

# Azure AI Translator - REST API call
import requests
import os
import json
import uuid

class AzureTranslator:
    """Client for Azure AI Translator."""

    BASE_URL = "https://api.cognitive.microsofttranslator.com"

    def __init__(self):
        self.key = os.environ["AZURE_TRANSLATOR_KEY"]
        self.region = os.environ["AZURE_TRANSLATOR_REGION"]
        self.headers = {
            "Ocp-Apim-Subscription-Key": self.key,
            "Ocp-Apim-Subscription-Region": self.region,
            "Content-type": "application/json",
            "X-ClientTraceId": str(uuid.uuid4())
        }

    def translate(self,
                   texts: list[str],
                   target_languages: list[str],
                   source_language: str = None,
                   profanity_action: str = "NoAction") -> list[dict]:
        """
        Translates texts to one or more languages.

        Args:
            texts: List of texts to translate
            target_languages: Target language codes (e.g., ["fr", "es", "de"])
            source_language: Source language code (auto-detection if None)
            profanity_action: "NoAction", "Deleted", "Marked"

        Returns:
            List of translation results
        """
        url = f"{self.BASE_URL}/translate"

        # Query parameters
        params = {
            "api-version": "3.0",
            "to": target_languages,
            "profanityAction": profanity_action
        }

        if source_language:
            params["from"] = source_language

        # Request body
        body = [{"text": text} for text in texts]

        response = requests.post(
            url,
            headers=self.headers,
            params=params,
            json=body,
            timeout=30
        )
        response.raise_for_status()

        return response.json()

    def detect_language(self, text: str) -> dict:
        """Detects the language of a text."""
        url = f"{self.BASE_URL}/detect"
        params = {"api-version": "3.0"}
        body = [{"text": text}]

        response = requests.post(
            url,
            headers=self.headers,
            params=params,
            json=body,
            timeout=30
        )
        response.raise_for_status()

        result = response.json()[0]
        return {
            "language": result["language"],
            "confidence": result["score"],
            "supported": result["isTranslationSupported"]
        }

    def get_available_languages(self) -> dict:
        """Retrieves the list of available languages."""
        url = f"{self.BASE_URL}/languages"
        params = {"api-version": "3.0"}

        response = requests.get(url, params=params, timeout=30)
        response.raise_for_status()

        data = response.json()
        return {
            "translation": len(data.get("translation", {})),
            "transliteration": len(data.get("transliteration", {})),
            "detection": len(data.get("detection", {}))
        }

# Usage
translator = AzureTranslator()

# Translate to multiple languages simultaneously
source_text = "Hello, how can I help you today?"

results = translator.translate(
    texts=[source_text],
    target_languages=["fr", "es", "de", "ja", "ar"]
)

print(f"Original text (en): {source_text}")
print("\nTranslations:")
for translation in results[0]["translations"]:
    print(f"  {translation['to'].upper()}: {translation['text']}")

# Language detection
detection = translator.detect_language("Bonjour, comment allez-vous?")
print(f"\nDetected language: {detection['language']} ({detection['confidence']:.0%})")

# Stats
stats = translator.get_available_languages()
print(f"\nAvailable languages: {stats['translation']}")

6.2 Azure AI Speech – Speech Translation

Speech translation allows converting speech from one language to text or speech in another language.

Available modes:

# Real-time speech translation
import azure.cognitiveservices.speech as speechsdk
import os

def translate_speech_realtime(
    source_language: str = "en-US",
    target_languages: list[str] = ["fr", "es"]
) -> None:
    """
    Translates speech in real time from the microphone.

    Args:
        source_language: Language of source speech (e.g., "en-US")
        target_languages: Target languages for translation
    """
    # Translation configuration
    config = speechsdk.translation.SpeechTranslationConfig(
        subscription=os.environ["AZURE_SPEECH_KEY"],
        region=os.environ["AZURE_SPEECH_REGION"]
    )

    config.speech_recognition_language = source_language

    # Add target languages
    for language in target_languages:
        config.add_target_language(language)

    # Configure output voice (TTS) for primary target language
    config.voice_name = "fr-FR-DeniseNeural"

    # Audio source
    audio_config = speechsdk.audio.AudioConfig(use_default_microphone=True)

    # Create translation recognizer
    recognizer = speechsdk.translation.TranslationRecognizer(
        translation_config=config,
        audio_config=audio_config
    )

    def translated_recognition(evt):
        result = evt.result
        if result.reason == speechsdk.ResultReason.TranslatedSpeech:
            print(f"\n🎙️  Original ({source_language}): {result.text}")
            for language, translation in result.translations.items():
                print(f"🌍  {language.upper()}: {translation}")

    def canceled(evt):
        print(f"❌ Canceled: {evt.result.cancellation_details.reason}")

    recognizer.recognized.connect(translated_recognition)
    recognizer.canceled.connect(canceled)

    print(f"🎙️ Speak in {source_language}...")
    print("Ctrl+C to stop")

    recognizer.start_continuous_recognition()

    import time
    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        recognizer.stop_continuous_recognition()
        print("\nTranslation stopped.")

# Launch real-time translation
translate_speech_realtime(
    source_language="en-US",
    target_languages=["fr", "es", "de"]
)

7. Language Understanding – CLU

7.1 Core Concepts

Conversational Language Understanding (CLU) allows creating NLP models that understand a user’s intent and identify relevant entities in their messages.

The three key concepts:

flowchart TD
    UTTERANCE["Utterance\n(What the user says)\n\n'Turn on the living room lights'"] --> PARSE["CLU Analysis"]

    PARSE --> INTENT["Intent\n(The objective/action)\n\nTurnOnLights"]
    PARSE --> ENTITY["Entities\n(Referenced objects)\n\nLocation: living room"]
    PARSE --> NONE["None Intent\n(If not understood)\n\n'I don't understand'"]

    INTENT --> ACTION["Application performs\nthe appropriate action"]
    ENTITY --> ACTION

7.2 CLU Model Creation Workflow

flowchart LR
    A["1️⃣ Create CLU project\nin Language Studio"] --> B["2️⃣ Define\nIntents"]
    B --> C["3️⃣ Define\nEntities"]
    C --> D["4️⃣ Add\nUtterances\n(at least 5/intent)"]
    D --> E["5️⃣ Annotate\nEntities\nin utterances"]
    E --> F["6️⃣ Train\nthe model"]
    F --> G["7️⃣ Evaluate\n(Precision, Recall)"]
    G --> H{Satisfactory?}
    H -->|No| I["Add utterances\nor correct"]
    I --> D
    H -->|Yes| J["8️⃣ Publish\nthe endpoint"]
    J --> K["9️⃣ Integrate\ninto app"]

7.3 Practical Example: Home Automation Assistant

# Using a deployed CLU model
from azure.ai.language.conversations import ConversationAnalysisClient
from azure.core.credentials import AzureKeyCredential
import os

# Configuration
endpoint = os.environ["AZURE_LANGUAGE_ENDPOINT"]
key = os.environ["AZURE_LANGUAGE_KEY"]
project_name = "home_automation"
deployment_name = "production"

client = ConversationAnalysisClient(
    endpoint=endpoint,
    credential=AzureKeyCredential(key)
)

def understand_command(text: str) -> dict:
    """
    Sends a command to the CLU model and returns the analysis.

    Args:
        text: The user's command

    Returns:
        dict with intent, entities, confidence
    """
    from azure.ai.language.conversations.models import (
        CustomConversationalTask,
        ConversationAnalysisOptions,
        CustomConversationTaskParameters,
        TextConversationItem
    )

    result = client.analyze_conversation(
        task=CustomConversationalTask(
            analysis_input=ConversationAnalysisOptions(
                conversation_item=TextConversationItem(
                    id="1",
                    participant_id="user",
                    text=text
                )
            ),
            parameters=CustomConversationTaskParameters(
                project_name=project_name,
                deployment_name=deployment_name
            )
        )
    )

    prediction = result.results.prediction

    # Extract entities
    entities = {}
    for entity in prediction.entities:
        if entity.category not in entities:
            entities[entity.category] = []
        entities[entity.category].append({
            "value": entity.text,
            "confidence": round(entity.confidence_score, 3)
        })

    return {
        "text": text,
        "top_intent": prediction.top_intent,
        "intent_confidence": round(
            next(i.confidence for i in prediction.intents
                 if i.category == prediction.top_intent), 3
        ),
        "all_intents": [
            {"intent": i.category, "confidence": round(i.confidence, 3)}
            for i in sorted(prediction.intents,
                           key=lambda x: x.confidence, reverse=True)
        ],
        "entities": entities
    }

def execute_home_command(analysis: dict) -> str:
    """
    Executes a home automation action based on CLU analysis.

    Args:
        analysis: CLU analysis result

    Returns:
        Confirmation message
    """
    intent = analysis["top_intent"]
    entities = analysis["entities"]

    # Check minimum confidence
    if analysis["intent_confidence"] < 0.7:
        return "Sorry, I didn't quite understand your request."

    # Process based on intent
    if intent == "TurnOnLights":
        location = entities.get("Location", [{"value": "the whole house"}])[0]["value"]
        return f"✅ Lights turned on in: {location}"

    elif intent == "TurnOffLights":
        location = entities.get("Location", [{"value": "the whole house"}])[0]["value"]
        return f"✅ Lights turned off in: {location}"

    elif intent == "SetTemperature":
        temp = entities.get("Temperature", [{"value": "?"}])[0]["value"]
        location = entities.get("Location", [{"value": "the home"}])[0]["value"]
        return f"✅ Temperature set to {temp} in {location}"

    elif intent == "StartAppliance":
        appliance = entities.get("Appliance", [{"value": "unknown device"}])[0]["value"]
        return f"✅ {appliance.capitalize()} started"

    elif intent == "None":
        return "I don't understand this request. Could you rephrase?"

    else:
        return f"Action '{intent}' not handled by this system."

# Test home automation system
test_commands = [
    "Turn on the living room lights",
    "Turn off everything in the bedroom",
    "Set the temperature to 72 degrees",
    "Start the coffee maker",
    "What is the weather tomorrow?"  # None Intent
]

print("=== Home Automation Assistant Test ===\n")
for command in test_commands:
    analysis = understand_command(command)
    action = execute_home_command(analysis)

    print(f"Command: '{command}'")
    print(f"Intent: {analysis['top_intent']} ({analysis['intent_confidence']:.0%})")
    if analysis["entities"]:
        print(f"Entities: {analysis['entities']}")
    print(f"Action: {action}")
    print("-" * 50)

7.4 CLU Evaluation Metrics

Concrete example:

Spam detection model:
- 10 emails predicted as SPAM
- 8 are actually spam (TP=8, FP=2)
- There are 12 true spams in total (FN=4)

Precision = 8 / (8+2) = 80%  → 80% of alerts are valid
Recall    = 8 / (8+4) = 67%  → 67% of true spam is detected
F1-Score  = 2 × (0.80 × 0.67) / (0.80 + 0.67) = 72.7%

8. Question Answering – Knowledge Base

8.1 Q&A Chatbot Architecture

flowchart TD
    USER["👤 User"] -->|Question| CHATBOT["Chatbot interface\n(Teams, Web, Mobile)"]
    CHATBOT -->|Query| QA["Azure AI Language\nCustom Question Answering"]
    QA --> KB["Knowledge Base\n(Q/A Pairs)"]
    KB --> MATCH["Matching\nQuestion → Answer"]
    MATCH -->|Answer + confidence| CHATBOT
    CHATBOT -->|Displays the answer| USER

    subgraph "Knowledge Base Sources"
        FAQ["FAQ Documents"]
        URL["FAQ Web Pages"]
        CHITCHAT["Predefined Chitchat\n(small talk)"]
        MANUAL["Manual Q/A\nEntry"]
    end

    FAQ --> KB
    URL --> KB
    CHITCHAT --> KB
    MANUAL --> KB

8.2 Creating a Knowledge Base

The knowledge base is populated from different sources:

1. FAQ Import:

Source: URL of an FAQ page
Example: https://azure.microsoft.com/en-us/support/faq/

Azure automatically extracts:
- Question: "What is Azure?"
- Answer: "Azure is Microsoft's cloud platform..."

2. Chitchat (small talk): Predefined informal conversations (Professional, Friendly, Witty…):

Q: "How are you?"
R: "I'm doing great, thanks! How can I help you?"

Q: "You're smart"
R: "I do my best! Is there something I can do for you?"

3. Manual entry:

{
  "questions": [
    "How do I reset my password?",
    "I forgot my password",
    "Lost password"
  ],
  "answer": "To reset your password, go to the login page and click 'Forgot Password'. You will receive an email with instructions.",
  "metadata": [
    { "name": "category", "value": "account" },
    { "name": "difficulty", "value": "easy" }
  ]
}

8.3 Complete Implementation

# Question Answering with Azure AI Language
from azure.ai.language.questionanswering import QuestionAnsweringClient
from azure.ai.language.questionanswering.models import (
    AnswersOptions,
    ShortAnswerOptions,
    QueryType
)
from azure.core.credentials import AzureKeyCredential
import os

class QAChatbot:
    """Chatbot based on Azure AI Language Question Answering."""

    def __init__(self, knowledge_base_name: str, deployment_name: str):
        endpoint = os.environ["AZURE_LANGUAGE_ENDPOINT"]
        key = os.environ["AZURE_LANGUAGE_KEY"]

        self.client = QuestionAnsweringClient(
            endpoint=endpoint,
            credential=AzureKeyCredential(key)
        )
        self.kb_name = knowledge_base_name
        self.deployment = deployment_name
        self.history = []

    def ask_question(self,
                      question: str,
                      top_k: int = 3,
                      confidence_threshold: float = 0.5) -> dict:
        """
        Asks a question to the knowledge base.

        Args:
            question: The user's question
            top_k: Maximum number of answers to return
            confidence_threshold: Minimum score to return an answer

        Returns:
            dict with best answer and alternatives
        """
        options = AnswersOptions(
            question=question,
            top=top_k,
            confidence_threshold=confidence_threshold,
            short_answer_options=ShortAnswerOptions(
                enable=True,  # Extract a precise short answer
                top=1
            )
        )

        result = self.client.get_answers(
            options=options,
            project_name=self.kb_name,
            deployment_name=self.deployment
        )

        # Record in history
        self.history.append({
            "question": question,
            "answer": result.answers[0].answer if result.answers else "No answer"
        })

        if not result.answers:
            return {
                "answer": "I couldn't find an answer to this question.",
                "confidence": 0.0,
                "alternatives": []
            }

        best = result.answers[0]

        return {
            "question": question,
            "answer": best.answer,
            "confidence": round(best.confidence or 0, 3),
            "short_answer": (
                best.short_answer.text
                if best.short_answer else None
            ),
            "source": best.source,
            "metadata": dict(best.metadata) if best.metadata else {},
            "alternatives": [
                {
                    "answer": a.answer[:100] + "...",
                    "confidence": round(a.confidence or 0, 3)
                }
                for a in result.answers[1:]
            ]
        }

    def interactive_dialog(self) -> None:
        """Launches an interactive command-line dialog."""
        print("🤖 Chatbot started. Type 'quit' to exit.")
        print("-" * 50)

        while True:
            question = input("\n👤 You: ").strip()

            if question.lower() in ["quit", "exit", "q"]:
                print("Goodbye!")
                break

            if not question:
                continue

            response = self.ask_question(question)

            print(f"\n🤖 Bot: {response['answer']}")

            if response['short_answer']:
                print(f"   💡 Short answer: {response['short_answer']}")

            print(f"   📊 Confidence: {response['confidence']:.0%}")

            if response['confidence'] < 0.7:
                print("   ⚠️  Low confidence - answer may not be precise")

# Usage
bot = QAChatbot(
    knowledge_base_name="faq-customer-support",
    deployment_name="production"
)

# Simple questions
questions = [
    "How do I reset my password?",
    "What are your opening hours?",
    "How do I cancel my order?",
    "What is the meaning of life?"  # Out of scope
]

print("=== FAQ Chatbot Test ===\n")
for q in questions:
    response = bot.ask_question(q)
    print(f"Q: {q}")
    print(f"A: {response['answer'][:150]}...")
    print(f"   (confidence: {response['confidence']:.0%})")
    print()

9. Practical Implementation with the Python SDK

9.1 Installation

# Azure AI Language packages
pip install azure-ai-textanalytics
pip install azure-ai-language-conversations
pip install azure-ai-language-questionanswering
pip install azure-cognitiveservices-speech
pip install azure-ai-translation-text
pip install azure-identity

9.2 Complete NLP Pipeline

# Complete NLP pipeline: from audio to structured analysis
import os
import json
from dataclasses import dataclass, asdict
from typing import Optional
import azure.cognitiveservices.speech as speechsdk
from azure.ai.textanalytics import TextAnalyticsClient
from azure.core.credentials import AzureKeyCredential

@dataclass
class NLPAnalysisResult:
    """Complete NLP analysis result."""
    original_text: str
    detected_language: Optional[str] = None
    language_confidence: float = 0.0
    sentiment: Optional[str] = None
    positive_score: float = 0.0
    negative_score: float = 0.0
    neutral_score: float = 0.0
    key_phrases: list = None
    entities: list = None

    def __post_init__(self):
        if self.key_phrases is None:
            self.key_phrases = []
        if self.entities is None:
            self.entities = []

class NLPPipeline:
    """Complete NLP pipeline with Azure AI Language."""

    def __init__(self):
        endpoint = os.environ["AZURE_LANGUAGE_ENDPOINT"]
        key = os.environ["AZURE_LANGUAGE_KEY"]

        self.lang_client = TextAnalyticsClient(
            endpoint=endpoint,
            credential=AzureKeyCredential(key)
        )

    def analyze_full(self, text: str) -> NLPAnalysisResult:
        """
        Performs a complete NLP analysis of a text.
        Language detection + Sentiment + Key phrases + NER
        """
        result = NLPAnalysisResult(original_text=text)

        try:
            # Detect language
            lang_result = self.lang_client.detect_language(documents=[text])
            if lang_result and not lang_result[0].is_error:
                language = lang_result[0].primary_language
                result.detected_language = language.iso6391_name
                result.language_confidence = language.confidence_score

            # Sentiment
            sentiment_result = self.lang_client.analyze_sentiment(
                documents=[text],
                language=result.detected_language or "en"
            )
            if sentiment_result and not sentiment_result[0].is_error:
                doc = sentiment_result[0]
                result.sentiment = doc.sentiment
                result.positive_score = doc.confidence_scores.positive
                result.negative_score = doc.confidence_scores.negative
                result.neutral_score = doc.confidence_scores.neutral

            # Key phrases
            kp_result = self.lang_client.extract_key_phrases(
                documents=[text],
                language=result.detected_language or "en"
            )
            if kp_result and not kp_result[0].is_error:
                result.key_phrases = list(kp_result[0].key_phrases)

            # Entities
            ner_result = self.lang_client.recognize_entities(
                documents=[text],
                language=result.detected_language or "en"
            )
            if ner_result and not ner_result[0].is_error:
                result.entities = [
                    {
                        "text": e.text,
                        "category": e.category,
                        "confidence": round(e.confidence_score, 3)
                    }
                    for e in ner_result[0].entities
                ]

        except Exception as e:
            print(f"Error in NLP analysis: {e}")

        return result

    def analyze_batch(self, texts: list[str]) -> list[NLPAnalysisResult]:
        """Analyzes multiple texts."""
        return [self.analyze_full(text) for text in texts]

    def generate_report(self, analyses: list[NLPAnalysisResult]) -> dict:
        """Generates a statistical report on analyses."""
        if not analyses:
            return {}

        sentiments = [a.sentiment for a in analyses if a.sentiment]
        languages = [a.detected_language for a in analyses if a.detected_language]

        return {
            "total_texts": len(analyses),
            "sentiment_distribution": {
                "positive": sentiments.count("positive"),
                "negative": sentiments.count("negative"),
                "neutral": sentiments.count("neutral"),
                "mixed": sentiments.count("mixed")
            },
            "detected_languages": list(set(languages)),
            "average_sentiment_score": {
                "positive": sum(a.positive_score for a in analyses) / len(analyses),
                "negative": sum(a.negative_score for a in analyses) / len(analyses)
            },
            "total_entities": sum(len(a.entities) for a in analyses),
            "total_key_phrases": sum(len(a.key_phrases) for a in analyses)
        }

# Main script
if __name__ == "__main__":
    pipeline = NLPPipeline()

    # Analyze customer reviews
    reviews = [
        "Excellent service! The team was very professional and responsive.",
        "Disappointed with the quality of the products received. The packaging was damaged.",
        "Order received on time. Nothing particular to report.",
        "Chef Alex Turner from Seattle prepared an exceptional meal on January 15th."
    ]

    analyses = pipeline.analyze_batch(reviews)
    report = pipeline.generate_report(analyses)

    print("=== NLP Analysis Report ===")
    print(json.dumps(report, ensure_ascii=False, indent=2))

    print("\n=== Detail per review ===")
    for i, analysis in enumerate(analyses, 1):
        print(f"\n[Review {i}]")
        print(f"  Text: {analysis.original_text[:80]}...")
        print(f"  Language: {analysis.detected_language} ({analysis.language_confidence:.0%})")
        print(f"  Sentiment: {analysis.sentiment} (+{analysis.positive_score:.0%} / -{analysis.negative_score:.0%})")
        print(f"  Key phrases: {', '.join(analysis.key_phrases[:3])}")
        if analysis.entities:
            print(f"  Entities: {[f'{e[\"text\"]}({e[\"category\"]})' for e in analysis.entities[:3]]}")

10. NLP Solution Architecture

10.1 Reference Architecture for an Intelligent Contact Center

flowchart TB
    CLIENT["📞 Customer"] -->|Phone call| GATEWAY["Azure Communication\nServices"]
    GATEWAY -->|Audio stream| STT["Azure AI Speech\nSpeech-to-Text"]
    STT -->|Real-time transcription| LANG["Azure AI Language\nSentiment + NER analysis"]
    LANG -->|Intent + Entities| CLU["Azure AI Language\nCLU (Intent Detection)"]
    CLU -->|Required action| ORCHESTRATOR["Azure Function\nOrchestration"]
    ORCHESTRATOR -->|Appropriate response| TTS["Azure AI Speech\nText-to-Speech"]
    TTS -->|Synthesized speech| CLIENT

    ORCHESTRATOR --> CRM["CRM / Customer\ndatabase"]
    ORCHESTRATOR --> TICKET["Ticketing system"]

    subgraph "Monitoring"
        APPINSIGHTS["Application Insights\nMetrics + Alerts"]
        STORAGE["Azure Storage\nTranscription archiving"]
    end

    STT -.-> STORAGE
    LANG -.-> APPINSIGHTS

10.2 Multi-Region Deployment for Low Latency

# Example Terraform configuration for multi-region deployment
# language-services.tf

variable "regions" {
  type    = list(string)
  default = ["westeurope", "eastus", "eastasia"]
}

resource "azurerm_cognitive_account" "language" {
  for_each = toset(var.regions)

  name                = "lang-service-${each.key}"
  location            = each.key
  resource_group_name = azurerm_resource_group.nlp_rg.name
  kind                = "TextAnalytics"
  sku_name            = "S"

  custom_subdomain_name = "lang-${each.key}-${random_id.suffix.hex}"

  tags = {
    Environment = "Production"
    Region      = each.key
  }
}

resource "azurerm_traffic_manager_profile" "nlp_traffic" {
  name                = "nlp-traffic-manager"
  resource_group_name = azurerm_resource_group.nlp_rg.name

  traffic_routing_method = "Performance"  # Latency-based routing

  dns_config {
    relative_name = "nlp-service"
    ttl           = 30
  }

  monitor_config {
    protocol = "HTTPS"
    port     = 443
    path     = "/cognitiveservices/v1"
  }
}

11. Exam Tips and Common Pitfalls

11.1 Critical Distinctions

flowchart TD
    Q1{"Analyze text\n(sentiment, language, entities, key phrases)"}
    Q1 --> LANG["Azure AI Language\n(Text Analytics)"]

    Q2{"Recognize or\nsynthesize speech"}
    Q2 --> SPEECH["Azure AI Speech\n(Speech-to-Text / Text-to-Speech)"]

    Q3{"Translate written\ntext"}
    Q3 --> TRANS["Azure AI Translator\n(Text → Text)"]

    Q4{"Translate speech\n(audio to audio/text)"}
    Q4 --> SPEECH_TRANS["Azure AI Speech\n(Speech Translation)"]

    Q5{"Understand\nuser intent"}
    Q5 --> CLU_BOX["Azure AI Language\n(CLU - Conversational Language Understanding)"]

    Q6{"Answer questions\nfrom a FAQ"}
    Q6 --> QA["Azure AI Language\n(Custom Question Answering)"]

11.2 Exam Pitfall Table

11.3 Typical Exam Questions

Q1: An application needs to analyze customer comments to determine if they are positive or negative. Which service?

A: Azure AI Language – Sentiment Analysis (Text Analytics)

Q2: A company wants to create a voice assistant that understands commands like “book a room” or “cancel my appointment”. Which service?

A: Azure AI Language – Conversational Language Understanding (CLU) to identify intents and entities.

Q3: An automatic translation service needs to translate Word documents into 5 languages simultaneously. Which service?

A: Azure AI Translator (supports Word, PDF, up to 1000 documents per request, 90+ languages).

Q4: What is the difference between Speech Recognition and Speech Synthesis?

A: Recognition = speech to text (microphone → transcription). Synthesis = text to speech (text → audio).

Q5: What is tokenization in the NLP context?

A: It is the process of breaking text into words, subwords, or combinations of words to which numeric identifiers are assigned. It is the first step in NLP processing.

11.4 Final Checklist Before the Exam

✅ I know the 3 main Azure NLP services (Language, Speech, Translator)
✅ I know that Cognitive Services = Azure AI Services (multi-service)
✅ I understand the difference between Text Analytics, CLU, and Question Answering
✅ I can distinguish Speech Recognition (STT) and Speech Synthesis (TTS)
✅ I know that Translator = text only (not audio)
✅ I know the concepts of Utterance, Intent, and Entity
✅ I understand the difference between Key Phrases and NER (Entities)
✅ I know what tokenization is
✅ I understand Precision, Recall, and F1 metrics
✅ I know that language detection can return NaN for unknown language

12. Practical Exercises and Scenarios

12.1 Scenario: Automatic Analysis of E-commerce Reviews

Context: An online marketplace wants to automatically analyze customer reviews to:

1. Detect urgent negative reviews
2. Extract mentioned products and issues
3. Generate a satisfaction dashboard

# Complete review analysis solution
import os
import json
from azure.ai.textanalytics import TextAnalyticsClient
from azure.core.credentials import AzureKeyCredential
from datetime import datetime

class EcommerceReviewAnalyzer:
    """Automatic e-commerce review analysis."""

    URGENT_NEGATIVE_THRESHOLD = 0.85  # Negative score above which = urgent

    def __init__(self):
        self.client = TextAnalyticsClient(
            endpoint=os.environ["AZURE_LANGUAGE_ENDPOINT"],
            credential=AzureKeyCredential(os.environ["AZURE_LANGUAGE_KEY"])
        )

    def analyze_reviews_batch(self, reviews: list[dict]) -> list[dict]:
        """
        Analyzes a list of reviews (with id, text, review_date).

        Returns:
            List of reviews enriched with NLP analysis
        """
        texts = [r["text"] for r in reviews]

        # Parallel analysis
        sentiments = self.client.analyze_sentiment(
            documents=texts,
            language="en",
            show_opinion_mining=True
        )

        key_phrases = self.client.extract_key_phrases(
            documents=texts,
            language="en"
        )

        entities = self.client.recognize_entities(
            documents=texts,
            language="en"
        )

        results = []
        for i, (orig_review, sent, kp, ent) in enumerate(
            zip(reviews, sentiments, key_phrases, entities)
        ):
            analyzed_review = {
                **orig_review,
                "sentiment": sent.sentiment if not sent.is_error else None,
                "positive_score": round(sent.confidence_scores.positive, 3) if not sent.is_error else 0,
                "negative_score": round(sent.confidence_scores.negative, 3) if not sent.is_error else 0,
                "urgent": (
                    sent.confidence_scores.negative >= self.URGENT_NEGATIVE_THRESHOLD
                    if not sent.is_error else False
                ),
                "key_phrases": list(kp.key_phrases) if not kp.is_error else [],
                "mentioned_products": [
                    e.text for e in ent.entities
                    if e.category == "Product"
                ] if not ent.is_error else [],
                "mentioned_organizations": [
                    e.text for e in ent.entities
                    if e.category == "Organization"
                ] if not ent.is_error else [],
                "analysis_date": datetime.now().isoformat()
            }

            # Extract negative aspects (Opinion Mining)
            negative_aspects = []
            if not sent.is_error:
                for sentence in sent.sentences:
                    for opinion in sentence.mined_opinions:
                        if opinion.target.sentiment == "negative":
                            negative_aspects.append({
                                "aspect": opinion.target.text,
                                "assessments": [a.text for a in opinion.assessments]
                            })

            analyzed_review["negative_aspects"] = negative_aspects
            results.append(analyzed_review)

        return results

    def generate_alerts(self, analyzed_reviews: list[dict]) -> list[dict]:
        """Generates alerts for urgent reviews."""
        return [
            {
                "review_id": r["id"],
                "text": r["text"][:100] + "...",
                "negative_score": r["negative_score"],
                "problematic_aspects": r.get("negative_aspects", []),
                "priority": "HIGH" if r["negative_score"] > 0.95 else "MEDIUM"
            }
            for r in analyzed_reviews
            if r.get("urgent", False)
        ]

    def dashboard(self, analyzed_reviews: list[dict]) -> dict:
        """Generates a satisfaction dashboard."""
        total = len(analyzed_reviews)
        if total == 0:
            return {}

        positives = sum(1 for r in analyzed_reviews if r["sentiment"] == "positive")
        negatives = sum(1 for r in analyzed_reviews if r["sentiment"] == "negative")
        neutrals = sum(1 for r in analyzed_reviews if r["sentiment"] == "neutral")
        mixed = sum(1 for r in analyzed_reviews if r["sentiment"] == "mixed")
        urgent = sum(1 for r in analyzed_reviews if r.get("urgent", False))

        # Simplified NPS score
        nps = (positives / total * 100) - (negatives / total * 100)

        return {
            "period": datetime.now().strftime("%Y-%m"),
            "total_reviews": total,
            "distribution": {
                "positive": f"{positives} ({positives/total:.0%})",
                "negative": f"{negatives} ({negatives/total:.0%})",
                "neutral": f"{neutrals} ({neutrals/total:.0%})",
                "mixed": f"{mixed} ({mixed/total:.0%})"
            },
            "satisfaction_score": round(nps, 1),
            "urgent_reviews": urgent,
            "overall_rating": "✅ Good" if nps > 50 else "⚠️ Needs improvement" if nps > 0 else "❌ Critical"
        }

# Test
analyzer = EcommerceReviewAnalyzer()

test_reviews = [
    {"id": "1", "text": "Excellent product! Ultra-fast delivery. Highly recommend."},
    {"id": "2", "text": "Completely disappointed. Package arrived broken and customer service doesn't respond."},
    {"id": "3", "text": "Order received. Product matches description. Nothing to report."},
    {"id": "4", "text": "Good product quality but delivery was too slow for the price paid."}
]

analyses = analyzer.analyze_reviews_batch(test_reviews)
alerts = analyzer.generate_alerts(analyses)
dash = analyzer.dashboard(analyses)

print("=== Dashboard ===")
print(json.dumps(dash, ensure_ascii=False, indent=2))

print("\n=== Alerts ===")
for alert in alerts:
    print(f"[{alert['priority']}] Review {alert['review_id']}: {alert['text']}")

13. Summary and Key Points

13.1 Complete Service → Capabilities Mapping

13.2 Decision Architecture

flowchart TD
    INPUT{"Input type?"}

    INPUT -->|Written text| TEXT_BRANCH{"What to do\nwith the text?"}
    INPUT -->|Audio/Speech| SPEECH_BRANCH{"What to do\nwith the audio?"}

    TEXT_BRANCH -->|Analyze content| LANG_TEXT["Azure AI Language\n(Text Analytics)"]
    TEXT_BRANCH -->|Understand intent| LANG_CLU["Azure AI Language\n(CLU)"]
    TEXT_BRANCH -->|Answer questions| LANG_QA["Azure AI Language\n(QA)"]
    TEXT_BRANCH -->|Translate| TRANSLATOR["Azure AI Translator"]
    TEXT_BRANCH -->|Read aloud| SPEECH_TTS["Azure AI Speech\n(TTS)"]

    SPEECH_BRANCH -->|Transcribe| SPEECH_STT["Azure AI Speech\n(STT)"]
    SPEECH_BRANCH -->|Translate| SPEECH_TRANS["Azure AI Speech\n(Speech Translation)"]

14. Glossary

Additional resources:

NLP Capabilities – Overview (Condensed Reference)

NLP (Natural Language Processing) = enabling computers to understand and generate human language.

Core NLP capabilities on Azure:

Language Detection (Condensed)

Identify the language of a text:

Input: "Hello, how are you?"
Output:
  language: "English"
  iso6391Name: "en"
  confidenceScore: 0.99

Input: "Hola, ¿cómo estás?"
Output:
  language: "Spanish"
  iso6391Name: "es"
  confidenceScore: 0.98

Use case: Automatically route requests to the correct language server.

Sentiment Analysis (Condensed)

Analyze sentiment (positive/neutral/negative):

Input: "This product is excellent! Fast delivery."
Output:
  sentiment: "positive"
  confidenceScores:
    positive: 0.94
    neutral: 0.05
    negative: 0.01

Input: "The service was terrible. Never again."
Output:
  sentiment: "negative"
  confidenceScores:
    positive: 0.01
    neutral: 0.04
    negative: 0.95

Opinion Mining (aspect analysis):

Input: "The food was delicious but the service was slow."
Output:
  sentences:
  - text: "The food was delicious"
    sentiment: positive
    opinions:
    - target: "food"
      assessments: [{text: "delicious", sentiment: positive}]
  - text: "the service was slow"
    sentiment: negative
    opinions:
    - target: "service"
      assessments: [{text: "slow", sentiment: negative}]

Key Phrase Extraction (Condensed)

Identify main concepts:

Input: "The annual shareholders meeting will be held in London in June to
        discuss the financial results of the fourth quarter."
Output:
  keyPhrases:
  - "annual shareholders meeting"
  - "London"
  - "June"
  - "financial results"
  - "fourth quarter"

Difference with Entity Recognition:

Key Phrases → Important concepts (any type)
Entity Recognition → Categorized entities (person, place, organization...)

Named Entity Recognition (NER) – Condensed

NER = Named Entity Recognition — extract and categorize named entities.

Input: "Marie Curie was born in Warsaw on November 7, 1867."
Output:
  entities:
  - text: "Marie Curie"
    category: "Person"
    confidence: 0.99
  - text: "Warsaw"
    category: "Location"
    confidence: 0.98
  - text: "November 7, 1867"
    category: "DateTime"
    confidence: 0.96

PII (Personally Identifiable Information) Detection:

Input: "My card number is 4111-1111-1111-1111 and my email is john@test.com"
Output:
  entities:
  - text: "4111-1111-1111-1111"
    category: "CreditCardNumber"
    confidenceScore: 0.99
  - text: "john@test.com"
    category: "Email"
    confidenceScore: 0.99
  redactedText: "My card number is **** **** **** **** and my email is ***"

Tokenization – How NLP Understands Text (Condensed)

Tokenization = transform raw text into analyzable units.

The 5 steps of tokenization:

Original text: "The quick brown foxes are running!"

Step 1 - Normalize: "the quick brown foxes are running"
Step 2 - Tokenize: ["the", "quick", "brown", "foxes", "are", "running"]
Step 3 - Remove stop words: ["quick", "brown", "foxes", "running"]
Step 4 - Stem/Lemmatize: ["quick", "brown", "fox", "run"]
Step 5 - Assign IDs: [2341, 891, 4521, 1876]

Document generated for AI-900 Azure AI Fundamentals – NLP Workloads on Azure

Search Terms

ai-900 · nlp · workloads · azure · ai · services · artificial · intelligence · generative · text · condensed · language · speech · analysis · architecture · exam · recognition · capabilities · clu · practical · reference · tokenization · base · detection