GenAI Orchestration and Agent Patterns

Comprehensive guide on orchestration and generative AI agent patterns. Key IT and AI terms are kept in English.

Table of Contents

1. Module 1 — Multi-step Reasoning and Planning for LLM Agents

   • Introduction to Multi-step Reasoning
   • Cognitive Architectures
     • ReAct
     • Chain-of-Thought
     • Tree-of-Thought
   • Planning Systems
   • Reasoning Loops and Reflection
   • Hard-coded vs. Emergent Reasoning
   • Demo: Plan–Execute–Reflect Pattern

2. Module 2 — Multi-agent Architectures and Collaboration

   • Introduction to Multi-agent Systems
   • Multi-agent Collaboration Patterns
   • Agent Roles and Specialization
   • Communication Protocols
   • Demo: Multi-agent Interaction

3. Module 3 — Tool Integration and Chaining

   • Introduction to Tool-augmented Agents
   • Tool Discovery and Selection
   • Tool Invocation and Result Interpretation
   • Designing Tool Registries and Interfaces
   • Demo: Tool Chaining

4. Module 4 — Error Handling and Recovery in Agent Systems

   • Common LLM Agent Failure Modes
   • Context Management and Overflow Issues
   • Safety Constraints and Guardrails
   • Logging and Debugging Strategies
   • Demo: Retry and Fallback

5. Module 5 — Agent Orchestration and Best Practices

   • Agent Orchestration Overview
   • Popular Agent Frameworks
   • Custom Agent Implementations
   • Persistent State Management
   • Demo: Observability, Debugging and Performance

Module 1 — Multi-step Reasoning and Planning for LLM Agents

Introduction to Multi-step Reasoning

Single-step reasoning works well for simple, well-defined questions, but fails quickly when tasks become complex or ambiguous. Without planning or reflection capability, an agent has no mechanism to detect errors, adapt its approach or correct incorrect assumptions during execution.

The fundamental loop of an LLM (Large Language Model) agent rests on three distinct phases:

flowchart LR
    A([🎯 Goal]) --> B[Planning\nWhat to do and in\nwhat order?]
    B --> C[Execution\nExecute one step\nat a time]
    C --> D[Reflection\nEvaluate the\nobtained result]
    D -->|Error detected| B
    D -->|Satisfactory result| E([✅ Done])

    style A fill:#4A90D9,color:#fff
    style B fill:#7B68EE,color:#fff
    style C fill:#5BA85A,color:#fff
    style D fill:#E8A838,color:#fff
    style E fill:#4A90D9,color:#fff

Cognitive Architectures

ReAct

ReAct (Reasoning + Acting) goes beyond producing a single response by combining reflection with action. In a ReAct loop, the agent reasons about the current goal, decides what to do next, then takes a concrete action in the environment.

flowchart TD
    A([Start]) --> B["🤔 Thought\n(Reasoning)"]
    B --> C["⚡ Action\n(Tool / API call)"]
    C --> D["👁️ Observation\n(Action result)"]
    D --> E{Goal\nreached?}
    E -->|No| B
    E -->|Yes| F([✅ Final response])

    style A fill:#4A90D9,color:#fff
    style B fill:#7B68EE,color:#fff
    style C fill:#5BA85A,color:#fff
    style D fill:#E8A838,color:#fff
    style F fill:#4A90D9,color:#fff

When to use ReAct?

• When the agent must call APIs, query databases or interact with external systems repeatedly
• In dynamic situations where the agent needs new information before continuing to reason
• For research, data collection and workflow orchestration — each action reveals new context

Chain-of-Thought

Chain-of-Thought (CoT) is a prompting technique that encourages the model to show its intermediate reasoning steps before producing a final response. Instead of jumping directly to a conclusion, the model works through the problem linearly, step by step.

flowchart LR
    P([Problem]) --> S1[Step 1\nReasoning] --> S2[Step 2\nReasoning] --> S3[Step 3\nReasoning] --> R([Final response])

    style P fill:#4A90D9,color:#fff
    style S1 fill:#7B68EE,color:#fff
    style S2 fill:#7B68EE,color:#fff
    style S3 fill:#7B68EE,color:#fff
    style R fill:#5BA85A,color:#fff

When to use Chain-of-Thought?

Note: CoT increases token usage and response time. For simple tasks, simpler prompting strategies are more effective.

Tree-of-Thought

Tree-of-Thought (ToT) allows an agent to explore multiple possible reasoning paths instead of committing to a single linear chain. The model generates several candidate steps at each decision point, evaluates their quality and continues reasoning along the most promising branches.

flowchart TD
    P([Problem]) --> B1[Branch A]
    P --> B2[Branch B]
    P --> B3[Branch C]
    B1 --> B1a[A.1 ✓]
    B1 --> B1b[A.2 ✗]
    B2 --> B2a[B.1 ✓]
    B2 --> B2b[B.2 ✓]
    B3 --> B3a[C.1 ✗]
    B2a --> R([✅ Best answer])
    B2b --> R

    style P fill:#4A90D9,color:#fff
    style B1 fill:#7B68EE,color:#fff
    style B2 fill:#5BA85A,color:#fff
    style B3 fill:#E8A838,color:#fff
    style R fill:#4A90D9,color:#fff

When to use Tree-of-Thought?

• When a problem has multiple reasonable ways forward and choosing too early can lead to poor results
• For open-ended tasks: design, analysis, complex decision-making
• Planning, strategy and diagnostic scenarios

Note: ToT introduces additional cost, latency and complexity. For simple tasks or time-sensitive applications, Chain-of-Thought is often more effective and practical.

Comparing Cognitive Architectures

Planning Systems

Planning Systems define how an LLM agent transforms a high-level goal into a sequence of actionable steps. By tracking dependencies and evaluating intermediate results, planning systems allow agents to adapt during work.

Why are planning systems necessary?

Key steps in implementing a planning system

flowchart TD
    A[1. Define goal\nand success criteria] --> B[2. Decompose\ninto subtasks]
    B --> C[3. Identify\ndependencies]
    C --> D[4. Assign\nresources/tools]
    D --> E[5. Execute\nstep by step]
    E --> F[6. Evaluate\nintermediate results]
    F --> G{Plan still\nvalid?}
    G -->|Yes| E
    G -->|No| H[7. Revise the plan]
    H --> E
    E --> I{Success\ncriteria met?}
    I -->|Yes| J([✅ Goal accomplished])

    style A fill:#4A90D9,color:#fff
    style J fill:#5BA85A,color:#fff

Step details:

1. Define the goal — Without explicit success criteria, agents cannot evaluate their progress or know when to stop.
2. Decompose into subtasks — Each subtask must be small enough to be solved reliably.
3. Identify dependencies — Some steps cannot begin before others are complete.
4. Assign resources — Select the appropriate tools, APIs or agents for each step.
5. Execute — Carry out the steps in the defined order.
6. Evaluate — Validate intermediate results to detect errors early.
7. Revise — If necessary, update the plan based on new information.

Reasoning Loops and Reflection

Reasoning Loops describe how an agent thinks repeatedly about a problem instead of making a single decision. The agent reasons about the current state, takes an action, then evaluates whether that action brought it closer to the goal.

Key elements of an effective reasoning loop

flowchart TD
    A([Current state]) --> B[State awareness\n- Goal\n- What has been done\n- What remains]
    B --> C[Verification\nAre results\nconsistent?]
    C --> D[Decision\nContinue / Tool / Revise]
    D --> E{Termination\ncriteria met?}
    E -->|No| A
    E -->|Yes| F([✅ Final response])

    style A fill:#4A90D9,color:#fff
    style F fill:#5BA85A,color:#fff

Intermediate step verification techniques

Hard-coded vs. Emergent Reasoning

These two approaches represent the two extremes of the LLM agent design spectrum.

quadrantChart
    title Hard-coded vs. Emergent Reasoning
    x-axis Low control --> High control
    y-axis Low adaptability --> High adaptability
    quadrant-1 Optimal hybrid
    quadrant-2 Pure emergent
    quadrant-3 Minimal viable
    quadrant-4 Pure hard-coded
    Hard-coded: [0.85, 0.2]
    Basic ReAct: [0.6, 0.5]
    Chain-of-Thought: [0.7, 0.4]
    Tree-of-Thought: [0.4, 0.75]
    Emergent LLM: [0.15, 0.85]
    Guided hybrid: [0.65, 0.65]

Comparing the approaches

Decision guide

flowchart TD
    A([Design an agent]) --> B{Are errors\ncritical or costly?}
    B -->|Yes| C[Use Hard-coded\nvalidations + checkpoints]
    B -->|No| D{Regulated domain\nor safety-critical?}
    D -->|Yes| C
    D -->|No| E{Is the task\nnew / ambiguous?}
    E -->|Yes| F[Use Emergent LLM\nreflection + guardrails]
    E -->|No| G{Performance\noptimization?}
    G -->|Yes| C
    G -->|No| H[Guided hybrid\napproach]

    style C fill:#5BA85A,color:#fff
    style F fill:#7B68EE,color:#fff
    style H fill:#E8A838,color:#fff

Demo: Plan–Execute–Reflect Pattern

Notebook: 01/demos.ipynb

This demonstration implements the Plan–Execute–Reflect pattern: the agent plans first, then executes, then evaluates its own result.

Setup — OpenAI client initialization

from openai import OpenAI

with open("../api_key.txt", "r") as f:
    API_KEY = f.read().strip()

client = OpenAI(api_key=API_KEY)

Task definition

task = "Calculate the average of the numbers 10, 20, and 40, and explain the result."
print(task)

PLAN phase — Generating a plan without executing

plan_prompt = f"""
You are an AI agent.

Task: {task}

First, create a simple step-by-step plan.
Do NOT execute the steps yet.
"""

plan_response = client.responses.create(
    model="gpt-4.1-mini",
    input=plan_prompt
)

plan = plan_response.output_text
print("PLAN:\n", plan)

The agent produces a step-by-step plan without executing yet. This separation of planning and execution is critical for robust multi-step reasoning.

EXECUTE phase — Sequential plan execution

execute_prompt = f"""
You previously created this plan:

{plan}

Now execute the plan step by step and produce the result.
"""

execute_response = client.responses.create(
    model="gpt-4.1-mini",
    input=execute_prompt
)

execution_result = execute_response.output_text
print("EXECUTION RESULT:\n", execution_result)

The previously generated plan is re-injected into the model. The agent performs the calculations and produces a concrete result, but has not yet evaluated whether this result is correct.

REFLECT phase — Self-evaluation of the result

numbers = [10, 20, 30, 40]
threshold = 30

avg = sum(numbers) / len(numbers)
result = avg > threshold

reflect_prompt = f"""
Here is the task:
{task}

Here is the result produced:
{execution_result}

Reflect on the result:
- Is it correct?
- If there is a mistake, explain it
- If needed, provide a corrected result
"""

reflect_response = client.responses.create(
    model="gpt-4.1-mini",
    input=reflect_prompt
)

reflection = reflect_response.output_text
print("REFLECTION:\n", reflection)

execution_result = {
    "average": avg,
    "above_threshold": result
}

Reflection enables self-correction. If errors are detected, the agent explains them and provides a corrected result. This is what makes agent workflows far more robust.

Plan–Execute–Reflect flow

sequenceDiagram
    participant U as User
    participant A as Agent
    participant LLM as LLM (gpt-4.1-mini)

    U->>A: Task: calculate the average
    A->>LLM: Prompt: create a plan (without executing)
    LLM-->>A: Structured plan
    A->>LLM: Prompt: execute the plan
    LLM-->>A: Execution result
    A->>LLM: Prompt: reflect on the result
    LLM-->>A: Reflection + corrections
    A-->>U: Validated final result

Module 2 — Multi-agent Architectures and Collaboration

Introduction to Multi-agent Systems

Multi-agent systems are composed of multiple autonomous agents that can think, make decisions and act on their own. Instead of relying on a single model, these agents interact and coordinate, enabling the system to handle complex tasks more effectively.

Types of multi-agent systems

flowchart TD
    subgraph Hierarchical
        C1[Coordinator Agent] --> S1[Agent A]
        C1 --> S2[Agent B]
        C1 --> S3[Agent C]
    end

    subgraph Peer-based
        P1[Agent 1] <--> P2[Agent 2]
        P2 <--> P3[Agent 3]
        P1 <--> P3[Agent 3]
    end

    subgraph Specialized
        E1[Medical Expert] --> AG[Aggregator]
        E2[Finance Expert] --> AG
        E3[Legal Expert] --> AG
    end

Challenges of multi-agent systems

Multi-agent Collaboration Patterns

Collaboration patterns provide the necessary structure for agents to work together effectively.

Why are collaboration patterns important?

• Allow handling complex tasks that a single agent cannot
• Define clear interaction structures, reducing duplicated work
• Facilitate conflict management as the number of agents grows
• Improve reliability by making agent behavior more predictable

Pattern 1: Hierarchical Task Delegation

flowchart TD
    T([Complex task]) --> C[Top-level Agent\nCoordinator]
    C -->|Delegate subtask A| A1[Specialized Agent A]
    C -->|Delegate subtask B| A2[Specialized Agent B]
    C -->|Delegate subtask C| A3[Specialized Agent C]
    A1 -->|Result A| C
    A2 -->|Result B| C
    A3 -->|Result C| C
    C --> R([Integrated final result])

    style T fill:#4A90D9,color:#fff
    style C fill:#7B68EE,color:#fff
    style R fill:#5BA85A,color:#fff

How it works:

1. The top-level agent takes a complex goal and divides it into smaller, well-defined tasks
2. Each sub-agent is selected based on its strengths — efficiency and accuracy
3. Instructions flow from parent to child agents, updates and results flow back up
4. By distributing work, the system scales more easily and avoids overloading a single agent

Pattern 2: Debate and Consensus

Debate and consensus mechanisms are collaboration strategies that help multiple agents reach a shared decision when they produce different opinions, plans or outputs.

sequenceDiagram
    participant C as Coordinator
    participant A1 as Agent 1
    participant A2 as Agent 2
    participant A3 as Agent 3

    C->>A1: What is the best solution?
    C->>A2: What is the best solution?
    C->>A3: What is the best solution?
    A1-->>C: Proposal A + reasoning
    A2-->>C: Proposal B + reasoning
    A3-->>C: Proposal C + reasoning
    C->>A1: Critique of B and C?
    C->>A2: Critique of A and C?
    A1-->>C: Critical analysis
    A2-->>C: Critical analysis
    C-->>C: Synthesize reasoning
    C->>A1: Final vote
    C->>A2: Final vote
    C->>A3: Final vote
    C-->C: Consensus decision

How debate works:

• Each agent explains not only its decision, but why it made it
• Each agent can critique others’ reasoning, point out weaknesses or challenge assumptions
• This exchange helps detect hidden errors and leads to a more thoughtful and justified outcome

Agent Roles and Specialization

Each agent is designed with specific responsibilities, capabilities and decision boundaries.

Task assignment by capability

Specialized vs. flexible roles

flowchart LR
    subgraph Specialized agents
        AS1[Medical Agent\n─ Clinical logic]
        AS2[Finance Agent\n─ Risk/Cost]
        AS3[Legal Agent\n─ Compliance]
    end
    subgraph Flexible agents
        AF1[Generalist Agent\n─ Coordination]
        AF2[Adaptive Agent\n─ New tasks]
    end
    AS1 & AS2 & AS3 --> AF1
    AF1 <--> AF2

Communication Protocols

Agent-to-agent communication is how multiple agents coordinate their actions and work toward a common goal.

Message types

flowchart LR
    A[Sender Agent] -->|Send\none-way| B[Receiver Agent]
    A -->|Request\nawaits response| C[Expert Agent]
    A -->|Broadcast\nmulti-agent| D[All agents]
    C -->|Response| A

    style A fill:#4A90D9,color:#fff
    style B fill:#7B68EE,color:#fff
    style C fill:#5BA85A,color:#fff
    style D fill:#E8A838,color:#fff

Coordination and delegation

Coordination describes how multiple agents align their actions to achieve a common goal. In a multi-agent system, agents often work in parallel, creating risks of duplication, delays or conflicts.

Task delegation is the process by which an agent assigns work to other agents based on their capabilities and availability.

Demo: Multi-agent Interaction

Notebook: 02/demos.ipynb

This demonstration shows how multiple agents collaborate on the same task, each with a different role, then a coordinator agent synthesizes the results.

Setup

from openai import OpenAI

with open("../api_key.txt", "r") as f:
    API_KEY = f.read().strip()

client = OpenAI(api_key=API_KEY)

Defining a generic agent function

def agent(agent_name, role, task):
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": f"You are {agent_name}. Your role is: {role}."},
            {"role": "user", "content": task}
        ],
        temperature=0.7
    )
    return response.choices[0].message.content

Each agent receives a name, a role and a task. The role guides its way of thinking — this is the core idea of task delegation in multi-agent systems.

Shared task

task = "Suggest a short, catchy slogan for an AI healthcare startup."

Multi-agent collaboration (parallel resolution)

responses = {
    "Creative Agent": agent("Creative Agent", "Come up with creative slogans", task),
    "Analytical Agent": agent("Analytical Agent", "Focus on clarity and trust", task),
    "Marketing Agent": agent("Marketing Agent", "Make it appealing to customers", task),
}

Each agent approaches the same task differently based on its assigned role. This demonstrates parallel problem solving — multiple agents working independently toward the same goal.

Coordinator agent — Consensus synthesis

def coordinator(responses):
    summary = "The following agents proposed slogans:\n\n"
    for agent, response in responses.items():
        summary += f"{agent}: {response}\n\n"
    
    summary += "Based on consensus, choose the best single slogan."

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are a coordinator agent."},
            {"role": "user", "content": summary}
        ],
        temperature=0.5
    )
    
    return response.choices[0].message.content

final_decision = coordinator(responses)

The coordinator agent collects all responses, examines them together and makes a decision based on consensus. This is how multi-agent systems synthesize their outputs into a final response.

Demo architecture

flowchart TD
    U([User]) --> T[Shared task\nAI health startup slogan]
    T --> CA[Creative Agent\nCreative slogans]
    T --> AA[Analytical Agent\nClarity and trust]
    T --> MA[Marketing Agent\nCustomer appeal]
    CA -->|Creative proposal| COORD[Coordinator Agent\nConsensus agent]
    AA -->|Analytical proposal| COORD
    MA -->|Marketing proposal| COORD
    COORD --> F([✅ Final slogan\nby consensus])

    style U fill:#4A90D9,color:#fff
    style COORD fill:#7B68EE,color:#fff
    style F fill:#5BA85A,color:#fff

Module 3 — Tool Integration and Chaining

Introduction to Tool-augmented Agents

Tool-augmented agents go beyond pure linguistic reasoning by giving AI systems the ability to interact with external tools and APIs. Instead of relying solely on what the model already knows, the agent can actively retrieve information, trigger actions or perform calculations.

Key advantages of tool integration

mindmap
  root((Tool-augmented Agents))
    Extended access
      External APIs
      Databases
      Real-world actions
    Real-time data
      Current context
      Operational data
      System metrics
    Modularity
      Add capabilities without retraining
      Logic decoupling
      Independent evolution
    Automation
      Multi-system workflows
      Intelligent orchestration
      Without manual intervention
    Source of truth
      Data validation
      Fact-based decisions
      Hallucination reduction

Tool Discovery and Selection

Discovery ensures the agent has visibility into all available tools and understands their capabilities. Selection allows the agent to reason about which tool best fits the current task.

How agents identify available tools

flowchart LR
    TR[(Tool Registry\nCentralized catalog)] -->|Standardized schema| A[Agent]
    A -->|Query at runtime| TR
    TR -->|Metadata, descriptions,\ninput/output schemas| A
    A -->|Filter by context| ST[Selected tools]
    ST -->|Invocation| T1[Tool A]
    ST -->|Invocation| T2[Tool B]

Tool selection criteria

Tool Invocation and Result Interpretation

Instead of hardcoding function calls, agents generate structured requests that translate their reasoning into well-defined inputs.

Tool invocation flow

sequenceDiagram
    participant A as Agent
    participant V as Validator
    participant T as External Tool

    A->>A: Reasoning about the task
    A->>A: Formulating structured request (JSON)
    A->>V: Input validation (schema + constraints)
    V-->>A: Validated input + enriched context
    A->>T: Invocation with structured parameters
    T-->>A: Response (JSON / structured data)
    A->>A: Parsing + response validation
    A->>A: Integration into next reasoning step

Key principles:

• Tools are not directly embedded in the agent code — they use standardized interfaces that decouple reasoning logic from specific implementations
• When an agent decides to use a tool, it formulates a structured request (not an ad hoc function call), enabling logging, inspection and validation
• Tool inputs are produced as a direct result of the agent’s reasoning process and must conform to the expected schema
• Agents typically use JSON to serialize inputs when communicating with external systems

Designing Tool Registries and Interfaces

Standardized tool registries and interfaces allow agents to integrate new tools without changing their reasoning logic.

Key design principles

flowchart TD
    subgraph Design principles
        LP[Loose Coupling] --> SC[Standardized\ninterfaces]
        SC --> MD[Clear tool\nmetadata]
        MD --> DM[Dynamic\nmanagement]
        DM --> MOD[Modularity\nAgent ≠ Tool]
    end

Demo: Tool Chaining

Notebook: 03/demos.ipynb

This demonstration implements tool chaining: the output of the first tool becomes the input of the second, creating a multi-step pipeline.

Setup

from openai import OpenAI
import json

with open("../api_key.txt", "r") as f:
    API_KEY = f.read().strip()

client = OpenAI(api_key=API_KEY)

Defining tools (callable functions)

# Tool 1: Text summarization
def summarize_text(text: str) -> str:
    return f"SUMMARY: {text[:250]}..."

# Tool 2: Quiz generation from a summary
def generate_quiz(summary: str) -> str:
    return (
        "1. What is Agentic AI?\n"
        "2. Why is tool usage important for agents?\n"
        "3. Name one real-world use case of Agentic AI."
    )

Registering tools with OpenAI (Function Calling Schema)

tools = [
    {
        "type": "function",
        "function": {
            "name": "summarize_text",
            "description": "Summarize a given text",
            "parameters": {
                "type": "object",
                "properties": {
                    "text": {"type": "string"}
                },
                "required": ["text"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "generate_quiz",
            "description": "Generate quiz questions from a summary",
            "parameters": {
                "type": "object",
                "properties": {
                    "summary": {"type": "string"}
                },
                "required": ["summary"]
            }
        }
    }
]

This schema tells the model which tools exist, what they do and what inputs they expect. Once registered, the model can decide when to call a tool based on the user’s request.

User prompt (natural language)

user_prompt = """
Summarize the following text and then create quiz questions from it.

Text:
Agentic AI systems can autonomously plan, reason, and act by using tools.
They are commonly used in workflows that require multi-step decision making.
"""

The user does not mention the tools. The model determines the required steps itself.

First call — The model chooses which tool to use

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": user_prompt}],
    tools=tools,
    tool_choice="auto"
)

message = response.choices[0].message

Executing tool 1 (Summary)

tool_call = message.tool_calls[0]
args = json.loads(tool_call.function.arguments)

# LLM call to actually summarize
summary_response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "Summarize the following text concisely."},
        {"role": "user", "content": args["text"]}
    ]
)

summary = summary_response.choices[0].message.content
print("🔹 Tool 1 Output (Summary):")
print(summary)

Chaining — Feeding the output to tool 2

quiz = generate_quiz(summary=summary)

print("\n🔹 Tool 2 Output (Final Result):")
print(quiz)

Tool Chaining flow

flowchart LR
    U([User\nnatural language prompt]) --> LLM1[LLM\nDecides which tool\nto call first]
    LLM1 -->|Structured call| T1[Tool 1\nSummarization]
    T1 -->|Summary produced| T2[Tool 2\nQuiz generation]
    T2 --> R([Final result\nquiz questions])

    style U fill:#4A90D9,color:#fff
    style LLM1 fill:#7B68EE,color:#fff
    style T1 fill:#5BA85A,color:#fff
    style T2 fill:#E8A838,color:#fff
    style R fill:#4A90D9,color:#fff

Module 4 — Error Handling and Recovery in Agent Systems

Common LLM Agent Failure Modes

Tool execution failures are inevitable when agents interact with external systems. A well-designed agent detects these failures and handles them gracefully.

Typical failure types

flowchart TD
    F([Tool failures]) --> N[🌐 Network /\nConnectivity]
    F --> A[🔑 Auth /\nAuthorization]
    F --> I[📋 Invalid or\nmalformed input]
    F --> T[⏱️ Timeout /\nRate limits]
    F --> R[📄 Unexpected\nresponse format]
    F --> P[⚠️ Partial or\ninconsistent results]

    N -->|Retry / alternate tool| S[✅ Recovery]
    A -->|Credential refresh| S
    I -->|Pre-validation| S
    T -->|Backoff + scheduled retry| S
    R -->|Robust parsing + validation| S
    P -->|Completeness check| S

    style F fill:#E74C3C,color:#fff
    style S fill:#5BA85A,color:#fff

Context Management and Overflow Issues

The context is everything the agent sends to the LLM: the current prompt, conversation history, stored memory, tool outputs and system instructions.

Context overflow problems

flowchart TD
    CW[Context window\ntoken-limited] --> P1[Current prompt]
    CW --> P2[Conversation\nhistory]
    CW --> P3[Agent\nmemory]
    CW --> P4[Tool\noutputs]
    CW --> P5[System\ninstructions]

    P1 & P2 & P3 & P4 & P5 -->|If total > limit| OV[⚠️ Overflow!]
    OV --> C1[Silent truncation\nof history]
    OV --> C2[Loss of important\ncontext]
    OV --> C3[Incoherent\nreasoning]
    OV --> C4[Hallucinations]

Context management strategies

Safety Constraints and Guardrails

Safety constraints and guardrails are rules that limit what an agent is allowed to do. They help prevent unsafe actions, tool misuse and unintended behaviors.

Types of guardrails

mindmap
  root((Guardrails))
    Safety
      Harmful outputs blocked
      Usage policies respected
    Input validation
      Input sanitization
      Prompt injection prevention
    Output validation
      Accuracy verified
      Correct format
      Policy compliance
    Tool usage
      Authorized parameters
      Rate limits enforced
      Execution boundaries
    Policy and Compliance
      Regulatory compliance
      Organizational values
    Resource and Budget
      Cost limits
      Long executions blocked
      APIs not overloaded

Logging and Debugging Strategies

In agent systems, effective logging and debugging are essential for understanding why an agent failed or behaved unexpectedly.

What to log

flowchart LR
    subgraph Reasoning traces
        RT1[Intermediate\nreasoning steps]
        RT2[Model prompts\nand responses]
        RT3[State transitions]
    end
    subgraph Tool calls
        TC1[Inputs + parameters]
        TC2[Outputs + errors]
        TC3[Latency per call]
    end
    subgraph Performance metrics
        PM1[Success rate]
        PM2[Total latency]
        PM3[Estimated cost]
        PM4[Number of steps]
    end

    RT1 & RT2 & RT3 --> LOG[(Log Store)]
    TC1 & TC2 & TC3 --> LOG
    PM1 & PM2 & PM3 & PM4 --> LOG
    LOG --> DBG[🔍 Debugging\nand optimization]

Demo: Retry and Fallback

Notebook: 04/demos.ipynb

This demonstration shows how an agent can retry failed actions and automatically switch to more reliable alternatives.

Setup

from openai import OpenAI
import json

with open("../api_key.txt", "r") as f:
    API_KEY = f.read().strip()

client = OpenAI(api_key=API_KEY)

Defining tools (primary tool + fallback)

# Primary tool (intentionally fails)
def structured_json_summarizer(text: str) -> dict:
    raise RuntimeError("Schema validation failed")

# Fallback tool (always functional)
def simple_text_summarizer(text: str) -> str:
    return {
        "summary": text[:100],
        "confidence": "low"
    }

Registering tools

tools = [
    {
        "type": "function",
        "function": {
            "name": "structured_json_summarizer",
            "description": "Generate a structured JSON summary",
            "parameters": {
                "type": "object",
                "properties": {"text": {"type": "string"}},
                "required": ["text"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "simple_text_summarizer",
            "description": "Generate a simple fallback summary",
            "parameters": {
                "type": "object",
                "properties": {"text": {"type": "string"}},
                "required": ["text"]
            }
        }
    }
]

First attempt — The model chooses the primary tool

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": user_prompt}],
    tools=tools,
    tool_choice="auto"
)

message = response.choices[0].message
tool_call = message.tool_calls[0]
args = json.loads(tool_call.function.arguments)

Primary tool execution (simulated failure)

try:
    result = structured_json_summarizer(**args)
except Exception as e:
    print("❌ Primary tool failed:", e)

Retry with clarified instructions

retry_prompt = """
The previous attempt failed.
Please generate a simpler summary in JSON format.
"""

retry_response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "user", "content": retry_prompt},
        {"role": "user", "content": user_prompt}
    ],
    tools=tools,
    tool_choice="auto"
)

retry_message = retry_response.choices[0].message
retry_tool_call = retry_message.tool_calls[0]
retry_args = json.loads(retry_tool_call.function.arguments)

Fallback execution

fallback_result = simple_text_summarizer(**retry_args)

print("✅ Fallback Tool Output:")
print(json.dumps(fallback_result, indent=2))

Retry & Fallback flow

flowchart TD
    U([User]) --> LLM1[LLM chooses\nprimary tool]
    LLM1 --> T1[Primary tool\nstructured_json_summarizer]
    T1 -->|RuntimeError| ERR[❌ Failure detected]
    ERR --> RETRY[Retry with\nclarified prompt]
    RETRY --> LLM2[LLM chooses\nfallback tool]
    LLM2 --> T2[Fallback tool\nsimple_text_summarizer]
    T2 -->|Success| R([✅ Fallback\nresult])

    style ERR fill:#E74C3C,color:#fff
    style R fill:#5BA85A,color:#fff
    style RETRY fill:#E8A838,color:#fff

Module 5 — Agent Orchestration and Best Practices

Agent Orchestration Overview

Agent orchestration refers to how multiple AI agents, tools and steps are coordinated to solve complex problems that cannot be handled in a single model call.

Instead of treating an agent as an isolated component, orchestration defines how agents plan tasks, pass context, manage shared state and invoke tools in the right order.

Role of orchestration

flowchart TD
    subgraph Orchestration layer
        O[Orchestrator] --> AGT1[Agent 1]
        O --> AGT2[Agent 2]
        O --> AGT3[Agent 3]
        O --> T1[Tool A]
        O --> T2[Tool B]
        O --> DS[(Data sources)]
    end
    
    U([User /\nSystem]) --> O
    AGT1 & AGT2 & AGT3 --> O
    O --> R([Final\nresult])

    style O fill:#7B68EE,color:#fff
    style R fill:#5BA85A,color:#fff

Key principles of agent orchestration

Popular Agent Frameworks

Agent frameworks are software libraries designed to simplify the development of autonomous agents. They provide standardized components for reasoning, tool use, memory management and task orchestration.

Comparative overview

quadrantChart
    title Frameworks by Flexibility vs. Specialization
    x-axis Specialized --> Generalist
    y-axis Low control --> High control
    quadrant-1 Flexible + Controlled
    quadrant-2 Generalist + Less controlled
    quadrant-3 Specialized + Less controlled
    quadrant-4 Specialized + Controlled
    LangChain: [0.75, 0.65]
    LlamaIndex: [0.35, 0.6]
    AutoGPT: [0.6, 0.2]
    CrewAI: [0.55, 0.75]

LangChain

LangChain is a simple and flexible framework for building agent-based applications using chains, tools and memory.

LlamaIndex

LlamaIndex focuses on data-centric agent workflows, especially Retrieval-Augmented Generation (RAG).

AutoGPT

AutoGPT popularized fully autonomous agents that plan and execute tasks with minimal human input.

CrewAI

CrewAI is designed for role-based multi-agent collaboration.

Framework selection table

Custom Agent Implementations

There are key scenarios where building a custom agent is a better choice than relying on an existing framework.

When to build a custom agent?

State and execution components of a custom agent

classDiagram
    class AgentState {
        +conversation_history: List
        +working_memory: Dict
        +task_queue: Queue
        +results_store: Dict
        +add_to_memory(data)
        +evict_old_context()
        +get_relevant_context(query)
    }

    class ExecutionEngine {
        +plan(goal) Plan
        +execute_step(step) Result
        +reflect(result) bool
        +route_to_tool(tool_name, args)
    }

    class ToolRegistry {
        +tools: Dict
        +register(tool)
        +discover(context) List
        +invoke(name, args) Result
    }

    AgentState <--> ExecutionEngine
    ExecutionEngine --> ToolRegistry

Persistent State Management

Persistent state management is essential for agents to preserve context and reasoning over time.

Conversational memory and working memory

Task queues and result storage

flowchart LR
    subgraph Planning
        P[Agent\nPlanning] -->|Enqueue| Q[(Task Queue)]
    end
    subgraph Execution
        Q -->|Dequeue| W[Workers]
        W -->|Async| T1[Tool A]
        W -->|Async| T2[Tool B]
        W -->|Async| T3[Human]
    end
    subgraph Storage
        T1 & T2 & T3 -->|Results| RS[(Result Store)]
        RS --> A[Next\nAgent]
    end

Demo: Observability, Debugging and Performance

Notebook: 05/demos.ipynb

This demonstration implements a LangChain pipeline with a complete observability layer to measure and monitor agent performance.

Setup — Imports and LangChain initialization

import time
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnableLambda

Secure API key loading (best practice)

with open("../api_key.txt", "r") as f:
    API_KEY = f.read().strip()

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-4o-mini",
    temperature=0,
    openai_api_key=API_KEY
)

The API key is loaded from a file rather than being hardcoded in the notebook — standard security practice.

Observability layer — Metrics dictionary

metrics = {
    "steps": 0,
    "start_time": None,
    "end_time": None,
    "success": False
}

This simple dictionary represents a minimal observability layer. It tracks step count, execution timing and whether the task completed successfully — directly mapping to common agent KPIs.

Step 1: Summary chain (logged)

summary_prompt = ChatPromptTemplate.from_messages([
    ("system", "Summarize the text in one sentence."),
    ("user", "{text}")
])

summarize_chain = (
    summary_prompt
    | llm
    | RunnableLambda(lambda x: x.content)
)

This LangChain runnable represents a single observable step in the workflow. Decomposing workflows into small chains like this makes debugging and optimization easier.

Step 2: Quiz generation chain (logged)

quiz_prompt = ChatPromptTemplate.from_messages([
    ("system", "Generate 3 quiz questions from the summary."),
    ("user", "{summary}")
])

quiz_chain = (
    quiz_prompt
    | llm
    | RunnableLambda(lambda x: x.content)
)

This chain depends on the previous step’s output — this is where chaining comes in. Each runnable is independent, but together they form a multi-step pipeline.

Manual agent loop with full observability

metrics["start_time"] = time.time()

# Step 1
summary = summarize_chain.invoke({
    "text": "Agentic AI systems can reason and act using tools."
})
metrics["steps"] += 1
print("🔍 Summary:\n", summary)

# Step 2
quiz = quiz_chain.invoke({
    "summary": summary
})
metrics["steps"] += 1
print("\n📝 Quiz:\n", quiz)

metrics["end_time"] = time.time()
metrics["success"] = True

Each step is executed in sequence with timing capture. This gives full control over execution and makes each step explicit and observable — ideal for debugging and performance analysis.

KPI calculation

latency = metrics["end_time"] - metrics["start_time"]

print("\n📊 KPIs")
print("Task Success:", metrics["success"])
print("Steps to Completion:", metrics["steps"])
print("Latency (seconds):", round(latency, 2))
print("Estimated Cost: Low (2 short LLM calls)")
print("Output Quality: Human-verified")

Common agent KPI table

Observability pipeline architecture

flowchart TD
    START([Start]) --> M1[metrics.start_time = now]
    M1 --> S1[Chain 1: Summary]
    S1 --> M2[metrics.steps += 1]
    M2 --> S2[Chain 2: Quiz]
    S2 --> M3[metrics.steps += 1]
    M3 --> M4[metrics.end_time = now]
    M4 --> M5[metrics.success = True]
    M5 --> KPI[KPI calculation\nLatency / Steps / Success]
    KPI --> R([Performance report])

    style START fill:#4A90D9,color:#fff
    style KPI fill:#7B68EE,color:#fff
    style R fill:#5BA85A,color:#fff

General Summary

Overview of Orchestration Patterns

flowchart TD
    subgraph Reasoning
        COT[Chain-of-Thought\nLinear, step-by-step]
        TOT[Tree-of-Thought\nParallel exploration]
        REACT[ReAct\nThink + Act + Observe]
        PER[Plan-Execute-Reflect\nPlanning → Execution → Reflection]
    end

    subgraph Multi-agent
        HIER[Hierarchical\nCoordinator → Sub-agents]
        PEER[Peer-based\nDirect collaboration]
        DEBATE[Debate and Consensus\nVoting between agents]
    end

    subgraph Tools
        TOOL[Tool Augmentation\nAPIs + Real-time data]
        CHAIN[Tool Chaining\nOutput → Next input]
        REG[Tool Registry\nDynamic discovery]
    end

    subgraph Reliability
        ERR[Error Handling\nRetry + Fallback]
        GUARD[Guardrails\nSafety + Compliance]
        OBS[Observability\nLogs + KPIs + Traces]
    end

    subgraph Orchestration
        ORCH[Frameworks\nLangChain / LlamaIndex / CrewAI]
        STATE[Persistent state\nMemory + Task queues]
        CUSTOM[Custom agents\nFull control]
    end

Pattern Selection Matrix

Document based on the “GenAI Orchestration and Agent Patterns” course — including voice transcriptions and demo notebooks (01–05).

Search Terms

genai · orchestration · agent · patterns · ai · agents · artificial · intelligence · generative · tool · multi-agent · systems · reasoning · setup · task · tools · chaining · collaboration · execution · fallback · flow · observability · pattern · plan