Practical Application of LLMs

GitHub: https://github.com/ttaulli/Practical-Application-of-LLMs

Table of Contents

1. Module 1 — Demystifying Large Language Models (LLMs)
   • 1.1 LLM Architectures and Capabilities
   • 1.2 Key Factors Influencing Performance and Cost
   • 1.3 Deploying LLMs via APIs
   • 1.4 Demo — Sentiment Analysis with Python and OpenAI
   • 1.5 Self-hosting Open-source LLMs
2. Module 2 — Fine-tuning and Retrieval-Augmented Generation (RAG)
   • 2.1 Demo — Text Summarization with LLMs
   • 2.2 Fine-tuning for Specialized Applications
   • 2.3 Understanding RAG
   • 2.4 Demo — RAG System for HR
3. Module 3 — Foundations of AI Agents
   • 3.1 Understanding the Basics of AI Agents
   • 3.2 Frameworks and Systems for Building AI Agents
   • 3.3 Demo — Building an AI Agent for Social Media
   • 3.4 Extractive vs Abstractive Approaches with LLMs

Module 1

Module 1 — Demystifying Large Language Models (LLMs): Architectures, Performance and Deployment

1.1 LLM Architectures and Capabilities

The Decoder-only Model

The underlying architecture of LLMs is the transformer. This module focuses on the decoder-only structure.

The origins of this model trace back to the famous 2017 research paper: “Attention Is All You Need”, which introduced the transformer architecture. This model was initially designed as an encoder/decoder structure:

• The encoder processed the input (e.g., a sentence in French)
• The decoder generated the output (e.g., its English translation)

A simplified version of this architecture — removing the encoder entirely — is known as the decoder-only model. This is the foundation behind models like GPT and ChatGPT.

flowchart LR
    subgraph Full_Transformer["Full Transformer (encoder/decoder)"]
        direction LR
        A["Input\n(e.g., French sentence)"] --> B["Encoder"]
        B --> C["Decoder"]
        C --> D["Output\n(e.g., English translation)"]
    end

    subgraph Decoder_Only["Decoder-only Model (GPT, etc.)"]
        direction LR
        E["Input tokens\n(prompt)"] --> F["Decoder\n(Self-Attention +\nPositional Encoding)"]
        F --> G["Next token\n(prediction)"]
        G -->|"Autoregressive loop"| F
    end

Key Components of the Decoder-only Model

1. Self-Attention

The model uses a mechanism called self-attention. This allows it to look at all previous tokens in the prompt and determine which ones are most relevant when generating the next token.

Example: If the prompt is “a sentence about a funny dog”, the model will give more weight to the words dog and funny when deciding what to write next.

2. Positional Encoding

Positional encoding helps the model understand the order of words. Unlike humans, neural networks have no built-in sense of sequence — positional encoding tells the model that “once” came before “upon” and “a” before “time”.

flowchart TD
    A["Input prompt\n'Once upon a time, a funny dog...'"]
    A --> B["Tokenization\n(text → token IDs)"]
    B --> C["Positional Encoding\n(token order)"]
    C --> D["Self-Attention\n(weighting relevant tokens)"]
    D --> E["Feed-Forward Layer"]
    E --> F["Predicted token\n(next word)"]
    F -->|"Repeat (autoregressive)"| D

3. Tokenization

A token is a unit of text — often a word, part of a word, or even a punctuation mark — that has been mapped to a number the model can understand.

Decoder-only Model Families

Note: After GPT-3.5, OpenAI no longer discloses the number of parameters, likely for competitive reasons. Llama and Gemma are open-source, so the figures are available.

Advantages of the Decoder-only Model

mindmap
  root((Decoder-only Advantages))
    Simpler architecture
      Easy to implement
      Easy to debug
      Easy to scale
    Sequential text generation
      Code generation
      Conversations / chat
    Variable length handling
      Variable inputs
      Variable outputs
    Efficient inference
      Key-Value Caching
      No redundant computation
    Flexible prompting
      Zero-shot prompting
      Few-shot prompting

Disadvantages of the Decoder-only Model

1.2 Key Factors Influencing Performance and Cost

Context Window

The context window of an LLM is the amount of text (in tokens) the model processes in a single pass.

Context Window Comparison

Illustration — Problem with a Too-Small Context Window

Imagine a Python script and an LLM with a small context window:

def is_prime(n):
    if n <= 1:
        return False
    for i in range(2, int(n**0.5) + 1):
        if n % i == 0:
            return False
    return True

num = int(input("Enter a number: "))
print(f"{num} is {'a prime' if is_prime(num) else 'not a prime'} number.")

If the LLM can only see part of this code (due to a too-small context window), its explanation will be incomplete and incorrect.

Problems with Large Context Windows

flowchart TD
    A["Large Context Window"] --> B["'Lost in the Middle' Effect"]
    A --> C["Degraded signal-to-noise ratio"]
    A --> D["Persistent hallucinations"]
    A --> E["High computation costs and latency"]

    B --> B1["The LLM ignores content\nburied in the middle of the context\n(focuses on beginning and end)"]
    C --> C1["Too much irrelevant or\ncontradictory data → incoherent\nor incorrect responses"]
    D --> D1["A large context window does\nnot guarantee the absence\nof hallucinations"]
    E --> E1["More content = more computation\n= higher costs and response time"]

Tokenization

LLMs only process numbers. Text must therefore be tokenized.

Exploration tool: OpenAI Tokenizer

Tokenization Example

Phrase: “I love programming in Python 🐍, it’s incredibly versatile!”

Each model has its own tokenization algorithm. Efficient tokenization can reduce costs and improve response quality.

Temperature

Temperature controls the level of creativity or determinism in the model’s responses.

Experimentation tool: OpenAI Playground

xychart-beta
    title "Effect of Temperature on Responses"
    x-axis ["0\n(Deterministic)", "0.5\n(Balanced)", "1\n(Creative)", "2\n(Random)"]
    y-axis "Creativity / Variability" 0 --> 100
    bar [5, 35, 65, 95]

LLM Training

flowchart LR
    A["Pre-training\n(From scratch)"] -->|"Massive data\n+ compute + expertise"| B["Base Model\n(Foundation Model)"]
    B --> C["Fine-tuning\n(Adjustment on\nspecific data)"]
    B --> D["RAG\n(Retrieval-Augmented\nGeneration)"]
    C --> E["Specialized Model"]
    D --> E

Pre-training: Training a model from scratch requires enormous amounts of data, significant computing power, and data science expertise. Impractical for most organizations.

Model Size

The model size is based on the number of parameters. A larger number of parameters can be impressive, but it is not necessarily the model to use for your use case.

Smaller models are often preferable for specific use cases, especially on edge devices where computing resources are limited.

1.3 Deploying LLMs via APIs

Advantages of APIs

mindmap
  root((LLM APIs))
    Advantages
      Plug-and-play endpoints
        No backend configuration needed
        Standardization facilitating provider switching
      Scalability
        No server or GPU management
        Reliability managed by provider
      Only available option
        OpenAI / Anthropic / Google require the API
      Fast time-to-market
        Accelerated application launch
      Seamless integration
        AWS / Google Cloud / Azure
    Disadvantages
      Cost per token
        Prices dropping but can still be high
      Limited customization
        Mainly via prompt engineering
      Data privacy concerns
        Data sent to the cloud
        Regulated sectors: finance, healthcare
      Latency
        Data traveling over the Internet
      No direct model access

Advantages vs Disadvantages — Summary

1.4 Demo — Sentiment Analysis with Python and OpenAI

OpenAI API link: https://openai.com/api/
GitHub: https://github.com/ttaulli/Practical-Application-of-LLMs/blob/main/Sentiment-Analysis.ipynb

This demo uses Python, the OpenAI API, and the LangChain framework to perform sentiment analysis on customer feedback.

What is LangChain?

LangChain is a popular open-source project that enables building applications using LLMs in Python or JavaScript. Its key advantages:

• Numerous integrations: OpenAI, Hugging Face, Google, AWS
• Creating AI Agents
• RAG (Retrieval-Augmented Generation) systems

Complete Code — Sentiment Analysis

Dependency installation:

pip install langchain-core==0.3.0 langchain-openai==0.3.27 python-dotenv

Python program:

import os
from dotenv import load_dotenv
from langchain_core.prompts import PromptTemplate
from langchain_openai import ChatOpenAI

# Load API key from .env file
load_dotenv()  # loads .env from the current directory

api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
    raise ValueError("Missing OPENAI_API_KEY in environment")

# Instantiate the model
llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)

# Create the prompt template
prompt = PromptTemplate(
    input_variables=["review_text"],
    template='Classify the sentiment of the following text as positive, neutral, or negative:\n\n"{review_text}"'
)

# LangChain chain (LCEL RunnableSequence)
chain = prompt | llm

def analyze_sentiment(text: str) -> str:
    result = chain.invoke({"review_text": text})
    return result.content.strip()

if __name__ == "__main__":
    samples = [
        "I absolutely love this product!",
        "It's okay, could be better.",
        "Worst experience ever."
    ]
    for s in samples:
        print(f"Text: {s}\nSentiment: {analyze_sentiment(s)}\n")

Expected output:

Text: I absolutely love this product!
Sentiment: Positive

Text: It's okay, could be better.
Sentiment: Neutral

Text: Worst experience ever.
Sentiment: Negative

Security: Always keep your OpenAI API key private. Add .env to your .gitignore file to avoid accidentally sharing it.

1.5 Self-hosting Open-source LLMs

Advantages of Self-hosting

Hugging Face — Model Hub

There are over 1.8 million models available on Hugging Face:

├── Multimodal (text, audio, video)
├── Computer Vision
│   ├── Image-to-Video
│   ├── Text-to-Video
│   └── Zero-shot Object Detection
├── NLP (Natural Language Processing)
│   ├── Text Classification
│   ├── Token Classification
│   └── Question Answering
├── Audio
│   ├── Automatic Speech Recognition
│   └── Voice Activity Detection
├── Tabular (Classification, Regression)
└── Reinforcement Learning

Hugging Face CLI

# Installation
pip install -U "huggingface_hub[cli]"

# Help
huggingface-cli --help

# Download a model
huggingface-cli download google-bert/bert-base-uncased

Ollama — Running LLMs Locally

Ollama is a tool that simplifies running LLMs locally. It supports models like Llama, DeepSeek, Mistral, etc.

# Installation (Linux/macOS)
curl -fsSL https://ollama.com/install.sh | sh

# Download models
ollama pull llama3.2
ollama pull deepseek-r1:1.5b

# Run a model
ollama run llama3.2
ollama run deepseek-r1:1.5b

Module 2

Module 2 — Fine-tuning and Retrieval-Augmented Generation (RAG) for Real-World Solutions

2.1 Demo — Text Summarization with LLMs

OpenAI API link: https://openai.com/api/
GitHub: https://github.com/ttaulli/Practical-Application-of-LLMs/blob/main/Text-Summarization.ipynb

This demo uses The Yellow Wallpaper (1892, Charlotte Perkins Gilman, ~6,000 words) as a document to summarize, and evaluates quality with ROUGE, METEOR, and BLEU metrics.

Evaluation Metrics

Complete Code — Text Summarization

Dependency installation:

pip install rouge-score absl-py nltk openai python-dotenv tiktoken evaluate

Python program:

import os
from dotenv import load_dotenv
import tiktoken
from openai import OpenAI
import evaluate

load_dotenv()
api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
    raise ValueError("Missing OPENAI_API_KEY in environment")

client = OpenAI(api_key=api_key)

def chunk_text(text, max_tokens=2000, model="gpt-4"):
    """Splits text into manageable chunks based on token limit."""
    enc = tiktoken.encoding_for_model(model)
    tokens = enc.encode(text)
    for i in range(0, len(tokens), max_tokens):
        yield enc.decode(tokens[i : i + max_tokens])

def summarize_chunk(chunk):
    """Sends a text chunk to the OpenAI model and returns a concise summary."""
    resp = client.chat.completions.create(
        model="gpt-4",
        messages=[
            {"role": "system", "content": "You are a helpful assistant that summarizes text."},
            {"role": "user", "content": f"Summarize this:\n\n{chunk}"}
        ],
        temperature=0.3,
        max_tokens=1024,
    )
    return resp.choices[0].message.content.strip()

def summarize_long_text(text):
    """Summarizes a long text by chunking it if necessary."""
    chunks = list(chunk_text(text))
    summaries = [summarize_chunk(c) for c in chunks]
    if len(summaries) > 1:
        return summarize_chunk("\n\n".join(summaries))
    return summaries[0]

if __name__ == "__main__":
    # Load metrics
    rouge = evaluate.load("rouge")
    meteor = evaluate.load("meteor")
    bleu = evaluate.load("bleu")

    with open("long_doc.txt", "r") as f:
        text = f.read()

    ref_summary = ""
    if os.path.exists("ref_summary.txt"):
        ref_summary = open("ref_summary.txt", "r").read()
    else:
        print("Warning: No ref_summary.txt found — benchmarks will be skipped.")

    print("🔍 Summarizing...")
    summary = summarize_long_text(text)
    print("\n📄 Summary:\n", summary)

    if ref_summary:
        rouge_scores = rouge.compute(predictions=[summary], references=[ref_summary])
        meteor_scores = meteor.compute(predictions=[summary], references=[ref_summary])
        bleu_scores = bleu.compute(predictions=[summary], references=[ref_summary])

        print("\n📊 Evaluation Metrics:")
        print(f"ROUGE-1: {rouge_scores['rouge1']:.3f}, ROUGE-2: {rouge_scores['rouge2']:.3f}, ROUGE-L: {rouge_scores['rougeL']:.3f}")
        print(f"METEOR: {meteor_scores['meteor']:.3f}")
        print(f"BLEU: {bleu_scores['bleu']:.3f}")
    else:
        print("🔹 No reference summary provided — metrics skipped.")

Sample output:

🔍 Summarizing...

📄 Summary:
"The Yellow Wallpaper" by Charlotte Perkins Gilman is a story about a woman
suffering from postpartum depression. Her physician husband, John, prescribes
her rest and isolation in a colonial mansion...

📊 Evaluation Metrics:
ROUGE-1: 0.466, ROUGE-2: 0.147, ROUGE-L: 0.290
METEOR: 0.430
BLEU: 0.060

2.2 Fine-tuning for Specialized Applications

What is Fine-tuning?

Fine-tuning consists of taking a pre-trained LLM and continuing to train it on smaller, more specific datasets. The goal is to adapt a generalist model (like GPT-3 or BERT) into a specialized model.

flowchart LR
    A["Pre-trained Model\n(Generalist)\ne.g., GPT-3, BERT"] -->|"Fine-tuning on\nspecific data"| B["Specialized Model\n(Expert)"]
    
    subgraph Examples["Application Examples"]
        C["Medical"]
        D["Legal"]
        E["Financial"]
        F["Multilingual"]
    end
    
    B --> Examples

Fine-tuning Use Cases

Advantages of Fine-tuning

mindmap
  root((Fine-tuning Advantages))
    Improved accuracy
      Learning precise patterns and terminology
      Benchmarks showing significant gains
    Reduced hallucinations
      More reliable reasoning
      More specific dataset
    Improved efficiency
      Less compute and memory
      Reduced training time
      Lower costs
    Data privacy
      Secure use of internal content

Disadvantages and Risks of Fine-tuning

flowchart TD
    A["Fine-tuning Risks"] --> B["Multidisciplinary expertise required\n(Data Science + domain knowledge)"]
    A --> C["Risk of over-specialization\n(Catastrophic Forgetting)\nModel forgets its general knowledge"]
    A --> D["Possible loss of built-in\nsafety mechanisms"]
    A --> E["Configuration and infrastructure complexity"]
    A --> F["Trial-and-error complexity\n(hyperparameters, data, etc.)"]

Fine-tuning Process

flowchart LR
    S1["1. Select\nbase model"] --> S2["2. Prepare\ndataset"]
    S2 --> S3["3. Choose fine-tuning\nmethod\n(e.g., LoRA, QLoRA)"]
    S3 --> S4["4. Training\nand testing"]
    S4 --> S5["5. Evaluation\nand iteration"]
    S5 --> S6["6. Deployment\nand monitoring"]
    S5 -->|"Insufficient results"| S2

2.3 Understanding RAG

What is RAG?

Retrieval-Augmented Generation (RAG) is a technique where an LLM retrieves relevant information from an external data source to generate more accurate and up-to-date responses.

Unlike fine-tuning which modifies the model’s internal weights, RAG offers more flexibility, easier updates, and requires fewer resources.

RAG Pipeline Architecture

flowchart TD
    subgraph Ingestion["1. Data Ingestion"]
        A["Source documents\n(PDFs, articles, manuals, web pages)"]
        A --> B["Chunking\n(Splitting into small pieces)"]
    end

    subgraph Indexing["2. Indexing"]
        B --> C["Embedding Model\n(Text → numeric vectors)"]
        C --> D["Vector Database\n(Pinecone, Chroma, etc.)"]
    end

    subgraph Retrieval["3. Retriever"]
        E["User query\n'How many days of paid parental leave?'"]
        E --> F["Query embedding"]
        F --> G["Semantic search\n(Similarity Search in Vector DB)"]
        G --> H["Relevant chunks\nretrieved"]
    end

    subgraph Generation["4. Generator"]
        H --> I["LLM receives:\n- Retrieved context\n- Original question"]
        I --> J["Generated response\n'According to our HR policy,\nyou are eligible for 12 weeks...'"]
    end

    D --> G

Concrete Example — RAG for HR

Question: “How many days of paid parental leave do we get?”

Data retrieved from the vector database:
"Employees are eligible for 12 weeks of paid parental leave following
the birth or adoption of a child. The leave must be taken within the
first year of the event."

Response generated by the LLM:
"According to our HR policy, you're eligible for 12 weeks of paid
parental leave, which you can take anytime within the first year
after your child is born or adopted."

Agentic RAG

Agentic RAG goes further by adding reasoning and planning capabilities:

flowchart LR
    Q["Complex question\n'Evaluate if we should renew XYZ contract'"] --> R["Retriever"]
    R --> P["Reasoning & Planning\n(Multi-source analysis)"]
    P --> G["Generator"]
    G --> A["Actions\n(Recommendations, reports)"]
    
    subgraph Sources
        S1["SLA data"]
        S2["Cost history"]
        S3["Customer satisfaction"]
    end
    
    Sources --> R

Generated response:

“Vendor XYZ met 98% of SLA targets this year, with a slight cost increase of 3%. Customer satisfaction remains high. Based on this, I recommend renewing the contract for another year.”

Disadvantages of RAG

2.4 Demo — RAG System for HR

GitHub: https://github.com/ttaulli/Practical-Application-of-LLMs/blob/main/rag-hr.ipynb

This program allows interaction with a fictional company’s (Acme Corporation) HR manual via a RAG system using LangChain, OpenAI, and Chroma (open-source vector database).

System Architecture

flowchart LR
    DOC["hr_policy_long.txt\n(Acme Corp HR Manual)"] --> LOADER["TextLoader"]
    LOADER --> SPLITTER["RecursiveCharacterTextSplitter\nchunk_size=500, overlap=50"]
    SPLITTER --> EMBED["OpenAIEmbeddings\n(Vectorization)"]
    EMBED --> CHROMA["Chroma Vector DB\n(Local storage)"]
    
    USER["User question"] --> RETRIEVER["Retriever\n(vectordb.as_retriever())"]
    CHROMA --> RETRIEVER
    RETRIEVER --> QA["RetrievalQA Chain\n(gpt-3.5-turbo)"]
    QA --> ANSWER["Response"]

Complete Code — RAG for HR

Dependency installation:

pip install langchain langchain-openai langchain-community chromadb python-dotenv

Python program:

import os
from dotenv import load_dotenv
from langchain_community.document_loaders import TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_community.vectorstores import Chroma
from langchain_core.prompts import PromptTemplate
from langchain.chains import RetrievalQA

# Load API key
load_dotenv()
api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
    raise ValueError("Missing OPENAI_API_KEY in environment")

# Connect to GPT-3.5
llm = ChatOpenAI(api_key=api_key, model="gpt-3.5-turbo", temperature=0)

# Load HR document
loader = TextLoader("hr_policy_long.txt")
docs = loader.load()

# Split into chunks (500 chars, 50 overlap for context continuity)
splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
split_docs = splitter.split_documents(docs)
print(f"Number of chunks created: {len(split_docs)}")  # → 10 chunks

# Create embeddings and store in Chroma (local)
embeddings = OpenAIEmbeddings(api_key=api_key)
vectordb = Chroma.from_documents(split_docs, embeddings, persist_directory="./hr_chroma_db")

# Configure the retriever
retriever = vectordb.as_retriever()

# Create the custom prompt
prompt = PromptTemplate(
    input_variables=["context", "question"],
    template="""
You are an HR assistant. Use the following HR policy context to answer the question at the end.
If the answer is not in the document, respond with 'I don't know.'

Context:
{context}

Question:
{question}

Answer:
"""
)

# Create the RAG chain
qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    retriever=retriever,
    chain_type="stuff",
    chain_type_kwargs={"prompt": prompt}
)

# Test a query
query = "What is the company policy on overtime pay?"
response = qa_chain.invoke(query)
print("Q:", query)
print("A:", response)

Sample output:

Number of chunks created: 10

Q: What is the company policy on overtime pay?
A: Non-exempt employees will be compensated for overtime in accordance with
federal and state laws. All overtime must be pre-approved by a supervisor.

Module 3

Module 3 — Foundations of AI Agents

3.1 Understanding the Basics of AI Agents

What an AI Agent is NOT

An AI Agent is not a traditional chatbot where the user provides a prompt and the AI generates a response. That is useful, but it is not an agent.

Definitions by Major Companies

AI Agent Workflow (SaaS Model)

flowchart LR
    P["1. Perception\n(Agent's eyes and ears)\n• Text, microphone, camera, sensors\n• Transforms into actionable data"] --> C
    C["2. Cognition\n(Agent's brain)\n• Reasoning\n• Planning\n• Decision making"] --> D
    D["3. Decision\n(Action selection)\n• Evaluating options\n• Selecting best action"] --> A
    A["4. Action\n(Execution)\n• Tool calls\n• Environment interaction\n• Content generation"] --> L
    L["5. Learning\n• Memorizing results\n• Continuous improvement\n• Adaptation"]
    L -->|"Feedback loop"| P

Advantages of AI Agents

mindmap
  root((AI Agents Advantages))
    Improved productivity
      Automation of complex tasks
      Workflow delegation
    24/7 operation
      No time limitations
      Continuous availability
    Data-driven decision making
      Analysis of large amounts of information
      Objective recommendations
    Scalability and flexibility
      Easy addition of new agents
      Adaptation to different contexts

Disadvantages and Risks

3.2 Frameworks and Systems for Building AI Agents

Overview of Approaches

flowchart TD
    A["Build an AI Agent"] --> B["Native development\n(Python/other language)"]
    A --> C["Proprietary SaaS platforms"]
    A --> D["Low-code / No-code platforms"]
    A --> E["Cloud platforms"]
    A --> F["Open-source systems"]

    B --> B1["✓ Full control\n✓ Maximum customization\n✗ Time intensive\n✗ Data science expertise required"]

    C --> C1["Salesforce AgentForce\nServiceNow AI Agent\nSAP Joule Agent"]
    D --> D1["Zapier Agents\nn8n"]
    E --> E1["Google Vertex AI\nAWS Bedrock\nAzure OpenAI"]
    F --> F1["LangGraph\nAutoGen\nCrewAI"]

Proprietary SaaS Platforms

SaaS drawbacks: High costs, limited customization, vendor lock-in risk.

Low-code / No-code Platforms

Cloud Platforms

Open-source Systems

3.3 Demo — Building an AI Agent for Social Media

This demo uses CrewAI to create a multi-agent system that generates social media posts.

CrewAI Multi-agent Architecture

flowchart TD
    subgraph Crew["SocialMediaCrew"]
        direction LR
        A["Agent 1: Topic Researcher\nRole: Topic Researcher\nTool: SerperDevTool (web search)\nGoal: Find trending topics and hashtags"] --> B
        B["Agent 2: Post Writer\nRole: Copywriter\nGoal: Write engaging posts"] --> C
        C["Agent 3: Critic\nRole: Critic / Editor\nGoal: Improve tone,\nclarity and impact"]
    end
    
    INPUT["Input: {platform}"] --> A
    C --> OUTPUT["Final post\nready to publish"]

Key Principles of CrewAI

• Specialization: Each agent focuses on what it does best
• Extensibility: Easy to add a new agent to the chain
• Robustness: Division of responsibilities reduces errors

Installing CrewAI

# Windows — Install UV (recommended package manager)
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

# Linux/macOS
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install CrewAI via UV
uv tool install crewai

# Update shell (PATH)
uv tool update-shell

# Verify installation
crewai --version

# Alternative via pip
pip install crewai crewai-tools

Creating the CrewAI Project

# Initialize the scaffolding
crewai create crew social_media_agent

# Generated structure:
# social_media_agent/
# ├── config/
# │   ├── agents.yaml
# │   └── tasks.yaml
# ├── src/social_media_agent/
# │   ├── crew.py
# │   └── main.py
# ├── .env
# └── pyproject.toml

Agent Configuration (config/agents.yaml)

topic_researcher:
  role: "Topic Researcher"
  goal: "Find trending topics and hashtags for {platform}"
  backstory: "Analyst scouring the web for insights and keywords"

post_writer:
  role: "Post Writer"
  goal: "Write an engaging social media post about {topic}"
  backstory: "Creative copywriter focused on brand tone and clarity"

critic:
  role: "Critic"
  goal: "Improve the draft for tone, grammar, and effectiveness"
  backstory: "Seasoned editor dedicated to polish and impact"

Task Configuration (config/tasks.yaml)

research_task:
  description: "Research trends and hashtags for {platform}"
  agent: topic_researcher
  expected_output: "List of 3–5 trending topics with hashtags"

write_task:
  description: "Write a post using one topic"
  agent: post_writer
  context: [research_task]
  expected_output: "Draft social media post and include emojis (max 280 characters)"

critique_task:
  description: "Revise the draft"
  agent: critic
  context: [write_task]
  expected_output: "Improved post ready for publishing"

Main Code (crew.py)

from crewai import Agent, Crew, Process, Task
from crewai.project import CrewBase, agent, task, crew
from crewai_tools import SerperDevTool

@CrewBase
class SocialMediaCrew:
    agents_config = "config/agents.yaml"
    tasks_config = "config/tasks.yaml"

    @agent
    def topic_researcher(self):
        return Agent(
            config=self.agents_config["topic_researcher"],
            tools=[SerperDevTool()]  # Web search tool
        )

    @agent
    def post_writer(self):
        return Agent(config=self.agents_config["post_writer"])

    @agent
    def critic(self):
        return Agent(config=self.agents_config["critic"])

    @task
    def research_task(self):
        return Task(
            config=self.tasks_config["research_task"],
            agent=self.topic_researcher()
        )

    @task
    def write_task(self):
        return Task(
            config=self.tasks_config["write_task"],
            agent=self.post_writer()
        )

    @task
    def critique_task(self):
        return Task(
            config=self.tasks_config["critique_task"],
            agent=self.critic()
        )

    @crew
    def crew(self):
        return Crew(
            agents=self.agents,
            tasks=self.tasks,
            process=Process.sequential  # Sequential execution
        )

Running the Project

# Launch the crew
crewai run

# Configure API keys in .env
OPENAI_API_KEY=your_key_here
SERPER_API_KEY=your_serper_key_here

3.4 Extractive vs Abstractive Approaches with LLMs

Comparison of the Two Approaches

flowchart TD
    A["Original text\n(source document)"] --> B{Approach}
    
    B -->|"Extractive"| C["Extractive Summarization\n• Identifies the most important sentences\n• Assembles them word for word\n• Like a highlighter on a document"]
    B -->|"Abstractive"| D["Abstractive Summarization\n• Generates new sentences\n• Captures the essence of the text\n• Reformulation + paraphrase"]
    
    C --> E["✓ Accuracy — no reformulation\n✗ Result sometimes disjointed"]
    D --> F["✓ Better flow and coherence\n✓ More natural and human\n✗ Risk of meaning loss\n✗ Risk of hallucinations"]

Comparative Summary

LLMs and Abstractive Summarization

LLMs improve abstractive summarization through:

• Attention mechanisms: Capturing long-range dependencies and nuances
• Contextual understanding: Better coherence
• Scalability: Handling documents with hundreds of thousands of tokens

Hybrid Approach (recommended)

flowchart LR
    A["Original document"] --> B["Step 1 — Extractive\n(Key sentence extraction)"]
    B --> C["Selected key sentences"]
    C --> D["Step 2 — Abstractive via LLM\n(Coherent rewriting)"]
    D --> E["Final summary\n✓ Factual grounding\n✓ Improved readability"]
    
    style E fill:#d4edda,stroke:#28a745

General Summary

mindmap
  root((Practical Application of LLMs))
    Module 1 — Architectures
      Decoder-only Model
        Self-Attention
        Positional Encoding
        Tokenization
      Model Families
        GPT OpenAI
        LLaMA Meta
        Gemini/Gemma Google
      Context Window
      Temperature
      APIs vs Self-hosting
    Module 2 — Fine-tuning and RAG
      Fine-tuning
        Domain adaptation
        Reduced hallucinations
        Catastrophic Forgetting
      RAG Pipeline
        Data Ingestion
        Indexing + Embeddings
        Vector Database
        Retriever + Generator
      Evaluation Metrics
        ROUGE
        METEOR
        BLEU
    Module 3 — AI Agents
      Agent Workflow
        Perception
        Cognition
        Decision
        Action
        Learning
      Frameworks
        LangGraph
        AutoGen
        CrewAI
      Platforms
        SaaS Salesforce ServiceNow SAP
        Cloud AWS Google Azure
        Low-code Zapier n8n
      Summarization
        Extractive
        Abstractive
        Hybrid

Resources and Useful Links

Search Terms

practical · application · llms · llm · development · artificial · intelligence · generative · ai · model · rag · advantages · fine-tuning · decoder-only · disadvantages · agent · context · crewai · agents · approaches · architecture · platforms · summarization · tokenization