Vector Databases and Embeddings for Developers

Technologies: C#, Microsoft Semantic Kernel, OpenAI

Table of Contents

1. Introduction
2. Module 1 — Fundamentals of Embeddings and Vector Databases
   • 1.1 Embeddings Overview
   • 1.2 Differences Between Vector and Traditional Databases
   • 1.3 The Role of Vector Databases in Managing AI System Data
3. Module 2 — The Role of Vector Databases as External Memory for LLMs
   • 2.1 How Vector Databases Serve as External Memory
   • 2.2 Reducing LLM Hallucinations
   • 2.3 The Role of Embeddings and Vector Databases in RAG Systems
4. Module 3 — Using Pre-trained Models and APIs to Create Vector Embeddings
   • 3.1 Demo 1 — Creating Vectors from Text Queries
   • 3.2 Demo 2 — Completing LLM Responses (Full RAG System)
5. Summary and Recap

Introduction

This course covers the foundational technologies powering modern AI applications:

• 🔢 Embeddings — how AI represents meaning as numbers
• 🗄️ Vector databases — how they enable semantic search at scale
• 🔗 How these technologies combine in RAG systems to make applications accurate and reliable

┌─────────────────────────────────────────────────────────────┐
│                   Overall Architecture                       │
│                                                             │
│  Text / Images                                              │
│       │                                                     │
│       ▼                                                     │
│  [Embedding Model]  ──►  [Numeric vector]                   │
│                                     │                       │
│                                     ▼                       │
│                            [Vector Database]                │
│                                     │                       │
│  User query ──► [Query embedding]                           │
│                                     │                       │
│                            [Cosine similarity]              │
│                                     │                       │
│                            [Relevant context]               │
│                                     │                       │
│                                     ▼                       │
│                               [LLM / GPT]                   │
│                                     │                       │
│                            [Accurate response]              │
└─────────────────────────────────────────────────────────────┘

Module 1

Fundamentals of Embeddings and Vector Databases

⏱ Duration: 14 minutes 30 seconds

1.1 Embeddings Overview

What is an Embedding?

An embedding is a numerical representation of data — usually text, but also images — that captures the meaning and relationships between concepts.

Instead of treating words as a simple sequence of letters, embeddings convert them into arrays of numbers representing their semantic meaning.

Concrete example:

Embedding representation for the word “cat”:

"cat": [0.42, -0.18, 0.71, -0.33, 0.56, -0.09, 0.28, 0.61,
        -0.44, 0.15, 0.38, -0.52, 0.19, 0.67, -0.25, 0.41,
        0.73, -0.31, 0.08, 0.54, -0.47, 0.22, 0.59, -0.14,
        0.36, 0.68, -0.29, 0.11, 0.48, -0.38, 0.63, 0.27,
        ...]

Embeddings typically use hundreds or thousands of dimensions to capture the nuance of language.

The GPS Analogy

┌──────────────────────────────────────────────────────────┐
│         GPS (physical space)  ↔  Embeddings (meaning)   │
│                                                          │
│  London  🏙️ ──── Paris  🗼      "doctor" ── "physician" │
│  Nearby coordinates              Nearby vectors          │
│                                                          │
│  → Physical proximity              → Proximity of meaning│
└──────────────────────────────────────────────────────────┘

Just as GPS uses latitude and longitude to locate physical places, embeddings use numeric coordinates to locate words in semantic space.

Embeddings Capture Relationships

The most fascinating property of embeddings is their ability to encode mathematical relationships between concepts:

graph LR
    A("Paris") -- "− France" --> B("concept: capital of")
    B -- "+ Italy" --> C("Rome")

    style A fill:#4a90e2,color:#fff,stroke:#2c6fad
    style B fill:#e67e22,color:#fff,stroke:#ca6f1e
    style C fill:#27ae60,color:#fff,stroke:#1e8449

$$\vec{Paris} - \vec{France} + \vec{Italy} \approx \vec{Rome}$$

$$\vec{king} - \vec{man} + \vec{woman} \approx \vec{queen}$$

This is not magic — these are patterns learned from massive amounts of data.

Traditional Encoding vs. Embeddings

How Embeddings are Created

flowchart LR
    A["📚 Data\n(books, websites, articles)"] --> B["🧠 Neural network\n(massive training)"]
    B --> C["📦 Pre-trained model\n(OpenAI, etc.)"]
    C --> D["🔌 API Call\n(single request)"]
    D --> E["🔢 Embedding\n(numeric vector)"]

The process in practice:

1. OpenAI trains neural networks on billions of words
2. These models learn how words relate by seeing them in context millions of times
3. The result is a pre-trained model accessible via a simple API call
4. You send your text (word, sentence, or paragraph) → you receive an embedding

💡 The heavy lifting is already done. You leverage years of research and cloud computing resources with a single API call.

1.2 Differences Between Vector and Traditional Databases

What is a Vector Database?

A vector database is a database specially designed for one task: finding the most similar vectors among potentially billions of options.

┌─────────────────────────────────────────────────────┐
│  Traditional database  →  EXACT matching            │
│                                                     │
│  "Find the customer with ID 12345"                  │
│                                                     │
│  Vector Database  →  APPROXIMATE matching           │
│                                                     │
│  "Find the 5 documents most SIMILAR                 │
│   to this query"                                    │
└─────────────────────────────────────────────────────┘

Comparison: Traditional vs. Vector Databases

graph TB
    subgraph SQL["🗃️ Traditional Database (SQL)"]
        direction TB
        S1["Exact matching"]
        S2["Rows and columns"]
        S3["Transactions and structured data"]
        S4["Found / Not found"]
        S5["Ex: Find user by ID"]
    end

    subgraph VDB["🔍 Vector Database"]
        direction TB
        V1["Similarity search"]
        V2["Find top 5 closest to [0.3, 0.6, …]"]
        V3["Unstructured data and semantic search"]
        V4["Ranked by similarity score"]
        V5["Ex: Find docs related by concept"]
    end

⚠️ In production, both types are typically used together — each for what it does best.

How Vector Search Works Internally

flowchart TD
    A["🙋 User query\n'affordable laptops for students'"] --> B["🔢 Convert to vector\n(query embedding)"]
    B --> C["🗄️ Vector Database\n(millions of stored vectors)"]
    C --> D["🗺️ KNN Algorithm\nNavigates graph structures"]
    D --> E["🏎️ Progressive approximation\n(like a highway network)"]
    E --> F["🏆 Top K results\nranked by similarity"]

The road network analogy:

Without optimization (naive approach):
  ❌  Compare with EVERY vector → too slow for millions of docs

With KNN algorithm:
  ✅  Take the "highways" → approach quickly
  ✅  Then the "back roads" → refine to destination
  ✅  Approximate result but DRAMATICALLY faster

Vector search steps:

1. The query is converted into a vector (embedding)
2. The vector database holds thousands or millions of vectors
3. The KNN (K-Nearest Neighbors) algorithm navigates graph structures
4. It efficiently identifies the nearest neighbors
5. The Top K most similar results are returned

Popular Vector Databases

┌─────────────────────────────────────────────────────────────────┐
│                    Popular Vector Databases                      │
├──────────────────┬──────────────────────────────────────────────┤
│  Pinecone        │  Fully managed, cloud native, no infra       │
│  Chroma          │  Lightweight, perfect for local development  │
│  Azure AI Search │  Enterprise, Microsoft integration, security │
│  Weaviate        │  Open source, built-in ML support            │
│  Qdrant          │  High performance, built in Rust             │
│  Elasticsearch   │  Vector support added, if already in use     │
└──────────────────┴──────────────────────────────────────────────┘

💡 With Microsoft Semantic Kernel, switching vector databases is often a single line of code. No vendor lock-in.

When to Use Each Type

flowchart TD
    Q{"What type of\nsearch do I need?"}
    Q --> |"EXACT matching"| SQL
    Q --> |"Similarity / Approximation"| VDB

    SQL["🗃️ SQL Database\n• User accounts\n• Transactions\n• Structured data\n• Inventories"]
    VDB["🔍 Vector Database\n• Semantic search\n• Recommendations\n• Intelligent Q&A\n• Image similarity\n• AI features"]

1.3 The Role of Vector Databases in Managing AI System Data

The Problem with Keyword Search

Scenario: A user searches for "Affordable laptops for students".

The database contains:

• "Budget laptops for college"
• "Cheap laptops for education"
• "Student-priced laptop"

These products are exactly what the user is looking for — but keyword search returns zero results.

Why Traditional Search Fails

graph TD
    A["❌ Keyword search"]
    A --> B["1️⃣ Only exact matches\n'reset password' ≠ 'forgot login'"]
    A --> C["2️⃣ Synonyms ignored\n'physician' ≠ 'doctor', 'car' ≠ 'automobile'"]
    A --> D["3️⃣ Context not understood\n'bank' (river) = 'bank' (finance)"]
    A --> E["4️⃣ User must guess\n'invoice' or 'bill'? 'refund' or 'reimbursement'?"]

The Solution: Semantic Search with Vectors

sequenceDiagram
    participant U as User
    participant E as Embedding Service
    participant V as Vector Database
    participant R as Results

    U->>E: "Affordable laptops for students"
    E-->>V: Query vector [0.42, 0.17, ...]
    V->>V: Compute cosine similarity with all vectors
    V-->>R: "Budget computer" → score 0.91 ✅
    V-->>R: "Cheap notebook" → score 0.88 ✅
    V-->>R: "Student laptop" → score 0.87 ✅
    R-->>U: Results ranked by relevance

Results with semantic search:

The Semantic Search Process — Step by Step

flowchart LR
    A["🙋 User asks\na question"] --> B["🔌 OpenAI API\ngenerates an embedding"]
    B --> C["🔢 Query\nvector"]
    C --> D["🗄️ Vector Database\ncompares all vectors"]
    D --> E["📐 Similarity algorithm\ncosine (score 0 to 1)"]
    E --> F["🏆 Top K\nmost relevant documents"]
    F --> G["✅ Response\nto the user"]

Cosine similarity measure:

Score = 1.0  →  Identical meaning
Score = 0.9  →  Very similar
Score = 0.7  →  Related
Score = 0.5  →  Relevance boundary
Score = 0.0  →  Completely unrelated

This entire process runs in milliseconds, even with millions of documents.

Real-World Applications

┌──────────────────────────────────────────────────────────────────┐
│                   Application Domains                             │
├──────────────────┬───────────────────────────────────────────────┤
│  Knowledge Base  │  Semantic search in wikis, docs, FAQs         │
│  E-commerce      │  Product search, recommendations,             │
│                  │  visual similarity                            │
│  Customer service│  Intelligent agents, automatic routing,       │
│                  │  similar issue detection                      │
│  Content mgmt    │  Email categorization, document classification│
│  Media / Entmt.  │  Recommendation engines (films, music)        │
│  Research & Dev  │  Articles by concept, similar patterns,       │
│                  │  code search by description                   │
└──────────────────┴───────────────────────────────────────────────┘

Module 2

The Role of Vector Databases as External Memory for LLMs

⏱ Duration: 14 minutes 46 seconds

2.1 How Vector Databases Serve as External Memory

The LLM Knowledge Gap

Large Language Models like GPT have impressive general knowledge of the world. However, they do not know:

graph TD
    LLM["🧠 LLM (e.g., GPT)"]

    LLM --> K1["✅ Knows:\nGeneral world facts\nLanguage patterns\nLogical reasoning\nCommon sense"]
    LLM --> K2["❌ Does not know:\nYour company-specific data\nInfo after the training cutoff\nPrivate / proprietary data\nReal-time information\nUser-specific content"]

This gap creates a problem: users ask questions specific to your organization, but the LLM can only answer from its general training knowledge.

The Vector Database as External Memory

graph LR
    subgraph LLM["🧠 LLM (base)"]
        direction TB
        L1["Built-in knowledge"]
        L2["General facts"]
        L3["Language patterns"]
        L4["Common sense"]
    end

    subgraph VDB["🗄️ Vector Database (external memory)"]
        direction TB
        V1["Company documentation"]
        V2["Product specifications"]
        V3["Current processes"]
        V4["Real-time data"]
    end

    VDB -- "Just-in-time context" --> LLM
    LLM --> R["💬 Accurate\nand up-to-date response"]

Think of it as giving the LLM a reference book open to exactly the right page.
The LLM doesn’t need to memorize your entire company wiki — it just needs to access the relevant parts at the right time.

How LLMs Use Retrieved Context

Without external memory (Vector DB):

User:     "What is your return policy?"
LLM only: "Most retailers offer returns within 30 days..."
          ❌ Generic response based on training

With external memory (Vector DB):

1. Query: "What is your return policy?"
2. → Find relevant vectors in the Vector DB
3. → Retrieve: "Our policy allows returns within 14 days with a receipt..."
4. → Provide to LLM as context
5. LLM responds: "According to our policy, you can return your purchases 
                  within 14 days with your receipt..."
               ✅ Accurate response, based on company facts

Benefits: Traditional LLM vs. External Memory

2.2 Reducing LLM Hallucinations

What is a Hallucination?

A hallucination occurs when an LLM generates information that seems plausible but is factually incorrect or invented. The response is confident, well-written… and wrong.

Why LLMs Hallucinate

graph TD
    H["🌀 LLM Hallucinations"]
    H --> R1["📊 Learned patterns,\nnot facts\nEx: 'Most offer 30 days'\neven if your policy is different"]
    H --> R2["🕳️ Fills gaps\nwith plausible content\nEx: Invents product features\nor procedures"]
    H --> R3["💪 Overconfidence\nNo natural 'I don't know' response\nAlways generates something,\neven under uncertainty"]

The Business Cost of Hallucinations

┌──────────────────────────────────────────────────────────────────┐
│                    Impact on Customers                            │
│  • Incorrect information (e.g., refund deadline 3d vs 10d)      │
│  • Incorrect expectations created                               │
│  • Eroded trust                                                 │
├──────────────────────────────────────────────────────────────────┤
│                    Impact on the Business                         │
│  • Disputes and support tickets                                 │
│  • Brand reputation damage                                      │
│  • Potential legal liability                                    │
└──────────────────────────────────────────────────────────────────┘

This is why hallucinations make LLMs unusable for customer-facing applications without proper safeguards.

Key Principle: No Evidence = No Response

flowchart TD
    Q["❓ User question"] --> |"Without Vector DB"| A["🧠 LLM alone"]
    Q --> |"With Vector DB"| B["🔍 Search in Vector DB"]

    A --> C["💬 Hallucinated response\n❌ Invented from training patterns"]

    B --> D{"Documents\nfound?"}
    D --> |"✅ Yes"| E["📄 Factual context retrieved"]
    D --> |"❌ No"| F["💬 'I don't have information\non this topic, let me\nconnect you to a human agent.'"]

    E --> G["🧠 LLM with context"]
    G --> H["✅ Accurate response\nbased on facts"]

Retrieval as Fact-Checking

Exam analogy:

Gracefully Handling Unknown Information

❌ Bad pattern:
   User:     "Same-day delivery?"
   AI Agent: "Yes, for orders before 2pm."
   → Invented! Can cause serious problems.

✅ Good pattern:
   User:     "Same-day delivery?"
   Vector DB: No relevant vector found.
   AI Agent: "I don't have information about same-day deliveries.
              Let me connect you with a human agent."
   → Honest, helpful, risk-free.

Source Verification and Citation

Sources used in this response:
  - Return policy v2.4 (updated Nov 2025)
  - Customer service guide, section 4.2

Benefits of source citation:

• Users can verify the information
• Builds trust and credibility
• Allows identifying outdated content
• Improves AI system transparency

2.3 The Role of Embeddings and Vector Databases in RAG Systems

What is RAG?

RAG = Retrieval-Augmented Generation

  R → Retrieval   : Search for relevant information 
                    (via embeddings + vector databases)
  A → Augmented   : Add this information to the LLM prompt 
                    as context
  G → Generation  : The LLM creates a response grounded in facts

In one sentence: RAG gives the LLM the right information at the right time.
That’s why it has become the standard architecture for production AI applications.

The Three Essential Components of RAG

graph TD
    subgraph RAG["🔧 RAG System"]
        E["🔢 Embeddings\n• Converts queries and docs to vectors\n• Enables semantic search\n• Bridges text and math"]
        V["🗄️ Vector Database\n• Stores document embeddings\n• Fast similarity search\n• Acts as LLM external memory"]
        L["🧠 Language Model (LLM)\n• Generates natural language responses\n• Uses retrieved context\n• Maintains conversational quality"]

        E <--> V
        V <--> L
        L <--> E
    end

⚠️ Removing a single component causes the system to fail. Together, they create intelligent and reliable AI applications.

The RAG Workflow in 5 Steps

sequenceDiagram
    participant U as 👤 User
    participant SK as ⚙️ Semantic Kernel
    participant OAI as 🔌 OpenAI API
    participant VDB as 🗄️ Vector Database
    participant LLM as 🧠 LLM (GPT)

    U->>SK: "What is the warranty period?"
    SK->>OAI: Generate query embedding
    OAI-->>SK: Vector [0.42, -0.18, 0.71, ...]
    SK->>VDB: SearchAsync(query, limit=5, minScore=0.45)
    VDB-->>SK: Relevant documents + similarity scores
    SK->>LLM: Augmented prompt with factual context
    LLM-->>SK: Natural language response, grounded in facts
    SK-->>U: ✅ Accurate and verifiable response

RAG with Semantic Kernel

Microsoft Semantic Kernel orchestrates the entire RAG pipeline:

┌────────────────────────────────────────────────────────────┐
│                  Semantic Kernel                            │
│                                                            │
│  ┌─────────────┐   ┌──────────────┐   ┌────────────────┐  │
│  │  Memory     │   │  Embeddings  │   │  Language      │  │
│  │  (vector    │◄──┤  (OpenAI,    │──►│  Model         │  │
│  │  storage)   │   │  Azure, etc.)│   │  (GPT, etc.)   │  │
│  └─────────────┘   └──────────────┘   └────────────────┘  │
│                           │                                │
│                    ┌──────▼──────┐                         │
│                    │ RAG Pipeline│                         │
│                    │ (orchestrated)│                       │
│                    └─────────────┘                         │
└────────────────────────────────────────────────────────────┘

RAG is the Standard Pattern — Why?

RAG vs. Other Approaches

RAG Production Considerations

┌──────────────────────┬──────────────────────────────────────┐
│  Data quality        │  Embeddings are only as good as      │
│                      │  the source documents                │
├──────────────────────┼──────────────────────────────────────┤
│  Performance         │  Embedding latency + vector search   │
│                      │  latency                             │
├──────────────────────┼──────────────────────────────────────┤
│  Reliability         │  Handling cases where no result      │
│                      │  is found                            │
├──────────────────────┼──────────────────────────────────────┤
│  Scalability         │  Number of vectors, throughput,      │
│                      │  continuous indexing                 │
└──────────────────────┴──────────────────────────────────────┘

Module 3

Using Pre-trained Models and APIs to Create Vector Embeddings

⏱ Duration: 14 minutes 15 seconds

3.1 Demo 1 — Creating Vectors from Text Queries

Demo Overview

In this demo, we will:

• Set up Semantic Kernel with memory
• Create embeddings from text documents
• Build a simple knowledge base with support documentation

Required NuGet Packages

<!-- Install via NuGet Package Manager or Package Manager Console -->
<PackageReference Include="Microsoft.SemanticKernel" Version="1.*" />
<PackageReference Include="Microsoft.SemanticKernel.Plugins.Memory" Version="1.*" />
<PackageReference Include="Microsoft.SemanticKernel.Connectors.OpenAI" Version="1.*" />

Or via the Package Manager Console in Visual Studio:

Install-Package Microsoft.SemanticKernel
Install-Package Microsoft.SemanticKernel.Plugins.Memory
Install-Package Microsoft.SemanticKernel.Connectors.OpenAI

Complete Code — Demo 1: Knowledge Base Creation

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.OpenAI;
using Microsoft.SemanticKernel.Memory;

// Suppress warnings for experimental features
// In production, these features should be reviewed carefully
#pragma warning disable SKEXP0001
#pragma warning disable SKEXP0050
#pragma warning disable SKEXP0010

// Configuration constants
// ⚠️ In production, store the key in Azure Key Vault or an environment variable
// NEVER hard-code an API key in source code
const string openAIApiKey    = Environment.GetEnvironmentVariable("OPENAI_API_KEY") 
                               ?? throw new InvalidOperationException("OPENAI_API_KEY not defined");
const string embeddingModel  = "text-embedding-ada-002";
const string chatModel       = "gpt-3.5-turbo";
const string collectionName  = "support";

// 1. Build the main Kernel
var kernel = Kernel.CreateBuilder()
    .AddOpenAIChatCompletion(chatModel, openAIApiKey)
    .Build();

// 2. Embedding service (generates vectors from text)
var embeddingService = new OpenAITextEmbeddingGenerationService(embeddingModel, openAIApiKey);

// 3. Volatile in-memory store (in-memory vector store)
//    In production: replace with Pinecone, Azure AI Search, Qdrant, etc.
var memoryStore = new VolatileMemoryStore();
var memory      = new SemanticTextMemory(memoryStore, embeddingService);

// 4. Ingest documents into the knowledge base
//    Semantic Kernel automatically generates embeddings during SaveInformationAsync
Console.WriteLine("Ingesting documents into the knowledge base...");

await memory.SaveInformationAsync(
    collection : collectionName,
    id         : "return-policy",
    text       : "Our return policy allows customers to return products " +
                 "within 14 days of purchase for a full refund."
);

await memory.SaveInformationAsync(
    collection : collectionName,
    id         : "warranty",
    text       : "All products come with a 12-month warranty covering " +
                 "manufacturing defects. The warranty does not cover accidental damage."
);

await memory.SaveInformationAsync(
    collection : collectionName,
    id         : "shipping",
    text       : "Standard shipping takes 3 to 5 business days. " +
                 "Express shipping is available for next-day delivery."
);

await memory.SaveInformationAsync(
    collection : collectionName,
    id         : "battery-life",
    text       : "The ProX tablet offers 12 hours of video playback " +
                 "and 15 hours of audio playback under normal use."
);

Console.WriteLine("✅ Knowledge base successfully created!");
Console.WriteLine($"   → {4} documents ingested into collection '{collectionName}'");

Document Ingestion Flow

flowchart LR
    A["📄 Text document\n'Our return policy...'"] --> B["🔌 OpenAI API\ntext-embedding-ada-002"]
    B --> C["🔢 Generated vector\n[0.42, -0.18, 0.71, ...]"]
    C --> D["🗄️ VolatileMemoryStore\n(or Pinecone, Qdrant, etc.)"]
    D --> E["✅ Document indexed\nready for search"]

💡 Semantic Kernel automatically handles embedding generation during SaveInformationAsync. You send raw text — the vector is computed behind the scenes.

3.2 Demo 2 — Completing LLM Responses (Full RAG System)

Demo Overview

This demo builds on Demo 1 and completes the RAG pipeline:

• Create an AskQuestion method that implements the RAG pipeline
• Show how Semantic Kernel automatically integrates queries
• Generate accurate responses based on retrieved documents

Complete Code — Demo 2: RAG Pipeline

using System.Text;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Connectors.OpenAI;
using Microsoft.SemanticKernel.Memory;

#pragma warning disable SKEXP0001
#pragma warning disable SKEXP0050
#pragma warning disable SKEXP0010

// ── Configuration (same setup as Demo 1) ──────────────────────────────────────
const string openAIApiKey   = Environment.GetEnvironmentVariable("OPENAI_API_KEY")
                              ?? throw new InvalidOperationException("OPENAI_API_KEY not defined");
const string embeddingModel = "text-embedding-ada-002";
const string chatModel      = "gpt-3.5-turbo";
const string collectionName = "support";

var kernel           = Kernel.CreateBuilder()
                           .AddOpenAIChatCompletion(chatModel, openAIApiKey)
                           .Build();
var embeddingService = new OpenAITextEmbeddingGenerationService(embeddingModel, openAIApiKey);
var memoryStore      = new VolatileMemoryStore();
var memory           = new SemanticTextMemory(memoryStore, embeddingService);

// ── Document Ingestion (identical to Demo 1) ──────────────────────────────────
#region Document Ingestion
await memory.SaveInformationAsync(collectionName, id: "return-policy",
    text: "Our return policy allows customers to return products " +
          "within 14 days of purchase for a full refund.");

await memory.SaveInformationAsync(collectionName, id: "warranty",
    text: "All products come with a 12-month warranty covering " +
          "manufacturing defects.");

await memory.SaveInformationAsync(collectionName, id: "shipping",
    text: "Standard shipping takes 3 to 5 business days. " +
          "Express shipping is available for next-day delivery.");

await memory.SaveInformationAsync(collectionName, id: "battery-life",
    text: "The ProX tablet offers 12 hours of video playback " +
          "and 15 hours of audio playback under normal use.");
#endregion

// ── Questions array + RAG loop ────────────────────────────────────────────────
var questions = new[]
{
    "What is your return policy?",
    "How long is the product warranty?",
    "How long does shipping take?",
    "What is the battery life of the tablet?",
    "Do you offer same-day delivery?"       // Question with no answer in the KB
};

foreach (var question in questions)
{
    Console.WriteLine(new string('─', 60));
    Console.WriteLine($"❓ Question: {question}");
    var answer = await AskQuestion(kernel, memory, question);
    Console.WriteLine($"💬 Answer  : {answer}");
}

// ── RAG Method: AskQuestion ───────────────────────────────────────────────────
/// <summary>
/// Implements the full RAG pipeline:
/// 1. Searches for relevant documents (with automatic embedding conversion)
/// 2. Builds factual context
/// 3. Generates an LLM response based on retrieved facts
/// </summary>
async Task<string> AskQuestion(Kernel kernel, ISemanticTextMemory memory, string question)
{
    // ── STEP 1: Semantic search ────────────────────────────────────────────────
    // SearchAsync AUTOMATICALLY converts the question to an embedding
    // via the same embeddingService configured at startup.
    // We pass the question as a string — Semantic Kernel handles vectorization.
    var searchResults = memory.SearchAsync(
        collection       : collectionName,
        query            : question,          // ← Raw string, no manual vector
        limit            : 1,                 // Return the 1 most relevant result
        minRelevanceScore: 0.45               // Minimum similarity threshold
    );

    // ── STEP 2: Build context ──────────────────────────────────────────────────
    // SearchAsync returns an IAsyncEnumerable — iterate asynchronously
    var contextBuilder = new StringBuilder();
    await foreach (var result in searchResults)
    {
        contextBuilder.AppendLine($"ID:          {result.Metadata.Id}");
        contextBuilder.AppendLine($"Content:     {result.Metadata.Text}");
        contextBuilder.AppendLine($"Relevance:   {result.Relevance:P0}");
        contextBuilder.AppendLine();
    }

    // ── STEP 3: Handle "no info found" case ───────────────────────────────────
    if (contextBuilder.Length == 0)
    {
        return "I don't have information on this topic in my knowledge base. " +
               "Let me connect you with a human agent.";
    }

    // ── STEP 4: Generate augmented response (RAG) ─────────────────────────────
    var prompt = $"""
        You are a customer support assistant. Use ONLY the information
        below to answer the question. Do not invent information
        absent from the context.

        === CONTEXT ===
        {contextBuilder}
        ===============

        Question: {question}

        Answer:
        """;

    var response = await kernel.InvokePromptAsync(prompt);
    return response.ToString();
}

RAG Pipeline — Detailed Code View

flowchart TD
    Q["❓ question = 'What is your return policy?'"]
    Q --> SA["memory.SearchAsync(\n  collection: 'support',\n  query: question,\n  limit: 1,\n  minRelevanceScore: 0.45\n)"]

    SA --> |"Auto-embed"| OAI["🔌 OpenAI\nGenerates query vector"]
    OAI --> VDB["🗄️ Vector Database\nCosine similarity search"]

    VDB --> |"Score ≥ 0.45"| CTX["📄 Retrieved context\n'return-policy' — 87% similarity"]
    VDB --> |"Score < 0.45 or empty"| NONE["💬 'I don't have info\non this topic...'"]

    CTX --> PROMPT["📝 Augmented prompt\n=== CONTEXT ===\n[retrieved document]\n===============\nQuestion: ..."]
    PROMPT --> LLM["🧠 LLM (GPT)\nInvokePromptAsync"]
    LLM --> ANS["✅ Accurate response\nbased on facts"]

Key Implementation Points

┌──────────────────────────────────────────────────────────────────┐
│  💡 Important technical points                                    │
├──────────────────────────────────────────────────────────────────┤
│  1. SearchAsync auto-vectorizes the query                        │
│     → No need to manually convert to embedding                  │
│     → Same embeddingService as during ingestion                 │
│                                                                  │
│  2. IAsyncEnumerable for results                                 │
│     → Asynchronous iteration with await foreach                 │
│     → Efficient for large result sets                           │
│                                                                  │
│  3. minRelevanceScore = 0.45                                     │
│     → Minimum relevance threshold                               │
│     → Below → "I don't know" response                          │
│     → Adjust based on doc quality and use case                  │
│                                                                  │
│  4. Prompt engineering for RAG                                  │
│     → Explicit instruction: use ONLY the context               │
│     → Prevents hallucinations outside the provided context      │
│                                                                  │
│  5. Graceful handling of missing results                         │
│     → No invention → redirect to human support                 │
└──────────────────────────────────────────────────────────────────┘

Expected Console Output

────────────────────────────────────────────────────────────
❓ Question: What is your return policy?
💬 Answer  : According to our policy, you can return your purchases 
              within 14 days of purchase for a full refund.

────────────────────────────────────────────────────────────
❓ Question: How long is the product warranty?
💬 Answer  : All products come with a 12-month warranty covering 
              manufacturing defects.

────────────────────────────────────────────────────────────
❓ Question: Do you offer same-day delivery?
💬 Answer  : I don't have information on this topic in my knowledge 
              base. Let me connect you with a human agent.

Switching Vector Databases — A Single Line

One of the great advantages of Semantic Kernel is portability:

// VolatileMemoryStore (in-memory, for development)
var memoryStore = new VolatileMemoryStore();

// ─── OR ── replace with a single line to change backend ──────────────────────

// Azure AI Search (enterprise production)
var memoryStore = new AzureAISearchMemoryStore(endpoint, apiKey);

// Qdrant (high performance)
var memoryStore = new QdrantMemoryStore(host: "localhost", port: 6333, vectorSize: 1536);

// Chroma (local development)
var memoryStore = new ChromaMemoryStore("http://localhost:8000");

🔄 The rest of the code does not change. That is the power of Semantic Kernel abstraction.

Summary and Recap

What You Learned

mindmap
  root((Vector DB & Embeddings))
    Module 1
      Embeddings
        Numerical representation of meaning
        GPS analogy
        Paris - France + Italy = Rome
        text-embedding-ada-002
      Vector Databases
        Similarity vs exact matching
        KNN algorithm
        Pinecone, Chroma, Azure AI Search
      Semantic Search
        Cosine similarity
        Score 0 to 1
        Milliseconds for millions of docs
    Module 2
      External memory for LLMs
        Knowledge gap
        Just-in-time retrieval
        Instant updates
      Hallucinations
        Causes: patterns, gaps, overconfidence
        Solution: no evidence = no answer
        Open-book vs closed-book exam
      RAG Architecture
        Retrieval + Augmented + Generation
        3 essential components
        Production standard
    Module 3
      Semantic Kernel
        NuGet packages
        SemanticTextMemory
        SaveInformationAsync
      Demo 1
        Knowledge base
        Document ingestion
        VolatileMemoryStore
      Demo 2
        Full RAG pipeline
        SearchAsync auto-embed
        minRelevanceScore = 0.45
        Graceful fallback

Module Recap

RAG Architecture — Complete Final View

graph TB
    subgraph Ingestion["📥 Ingestion Phase (offline)"]
        D1["📄 Documents\n(policies, specs, docs)"] --> EMB1["🔌 Embedding Model\ntext-embedding-ada-002"]
        EMB1 --> VDB1["🗄️ Vector Database\n(Pinecone / Azure AI Search / Qdrant)"]
    end

    subgraph Query["🔍 Query Phase (runtime)"]
        U["👤 User"] --> Q["❓ Question\n(plain text)"]
        Q --> EMB2["🔌 Embedding Model\nSame model as at ingestion"]
        EMB2 --> SEARCH["🔍 SearchAsync\nlimit=K, minScore=0.45"]
        VDB1 --> SEARCH
        SEARCH --> CTX["📄 Retrieved context\n(Top K documents)"]
        CTX --> PROMPT["📝 Augmented prompt\nContext + Question"]
        PROMPT --> LLM["🧠 LLM (GPT)\nvia Semantic Kernel"]
        LLM --> ANS["✅ Accurate and\nverifiable response"]
        ANS --> U
    end

    style Ingestion fill:#e8f4f8,stroke:#4a90e2
    style Query fill:#f8f4e8,stroke:#e6a817

Quick Reference Commands

// ── Configure the Kernel ──────────────────────────────────────────────────────
var kernel = Kernel.CreateBuilder()
    .AddOpenAIChatCompletion("gpt-3.5-turbo", apiKey)
    .Build();

// ── Configure vector memory ───────────────────────────────────────────────────
var memory = new SemanticTextMemory(
    new VolatileMemoryStore(),                                    // backend
    new OpenAITextEmbeddingGenerationService("text-embedding-ada-002", apiKey)
);

// ── Ingest a document ─────────────────────────────────────────────────────────
await memory.SaveInformationAsync("my-collection", id: "doc-1", text: "Content...");

// ── Semantic search (auto-embed) ──────────────────────────────────────────────
var results = memory.SearchAsync("my-collection", "my query", limit: 5, minRelevanceScore: 0.45);
await foreach (var result in results)
{
    Console.WriteLine($"{result.Metadata.Id}: {result.Relevance:P0}");
}

// ── Generate augmented response ───────────────────────────────────────────────
var response = await kernel.InvokePromptAsync($"Context: {ctx}\nQuestion: {q}");

Search Terms

vector · databases · embeddings · developers · rag · search · artificial · intelligence · generative · ai · traditional · llm · external · memory · semantic · database · hallucinations · knowledge · llms · pipeline · role · system · vectors · view