Filter and Manipulate Data with Aggregations in MongoDB

Demo database: sample_mflix (movies collection) — MongoDB Atlas

Table of Contents

1. Understanding and exploring Aggregation Pipelines

• 1.1 Aggregation Pipeline Definition and Environment Configuration
• 1.2 Demo: Compare an Aggregation Pipeline with find()
• 1.3 Demo: Track the flow of documents through stages
• 1.4 Demo: Identify key stages at a glance
• 1.5 Demo: Write and run your first Aggregation Pipeline

2. Filter and reshape with $match and $project

• 2.1 Demo: Filter documents with $match
• 2.2 Demo: Reshape the output with $project
• 2.3 Demo: Rename and calculate fields with $project
• 2.4 Demo: Combine $match and $project
• 2.5 Demo: Observe the evolution of the shape of a document

3. Summarize data with $group

• 3.1 Demo: Group documents with $group
• 3.2 Demo: Apply accumulators to obtain insights
• 3.3 Demo: Format grouped results for readability
• 3.4 Demo: Combine $match and $group for targeted results
• 3.5 Demo: Sort grouped results with $sort

4. Build and run multi stage pipelines

• 4.1 Demo: Chaining internships into a pipeline
• 4.2 Demo: Answering real questions with pipelines
• 4.3 Demo: Saving and reusing pipelines
• 4.4 When to use Aggregation rather than find()
• 4.5 Demo: Explore AI with Aggregations

5. Quick reference of operators and stages

1. Understand and explore Aggregation Pipelines

Module duration: 24m 20s

1.1 Aggregation Pipeline Definition and Environment Configuration

What is an Aggregation Pipeline?

Aggregations are one of the most powerful features of MongoDB. They allow you to filter, reshape and manipulate data in ways that go far beyond simple lookups. An aggregation pipeline is MongoDB’s native framework for moving beyond simple queries and into richer data analysis.

This framework actively processes documents: it takes raw records and prepares them step by step through a sequence of stages. Each stage transforms the documents it receives and passes the result to the next stage. Imagine a conveyor belt: documents enter one side, each station does its job, and the output is gradually shaped into something useful.

The three big ideas of a pipeline

1. Multi-stage data processing — You are not limited to a single query. You can chain several operations in sequence.
2. Assembly line of stages — Each stage passes its output to the next, building on the work that came before.
3. Filter, reshape, summarize, prepare — Pipelines are designed to filter, reshape, summarize and prepare data, delivering you polished, ready-to-use results.

Why pipelines rather than find()?

find() is excellent for simple lookups. It excels at quickly returning matching documents. But pipelines go further:

The real advantage of pipelines: they can filter, reshape and summarize in a single pass, without ever leaving MongoDB. No need to export to Excel or external tools.

The essential stages of a pipeline

Documents bruts
      ↓
   $match     →  filtre les documents selon des conditions
      ↓
  $project    →  remodèle les documents (inclure, exclure, calculer)
      ↓
   $group     →  regroupe plusieurs documents en résumés
      ↓
   $sort      →  organise les résultats dans un ordre logique
      ↓
 Résultat propre, prêt pour un rapport

Environment Configuration

For demonstrations, we use:

• MongoDB Atlas — a cloud cluster with the sample_mflix dataset loaded
• The dataset notably contains the movies collection with fields like title, year, runtime, cast, genres, rated, imdb.rating, released, etc.
• MongoDB Shell — for online commands
• MongoDB Compass — for pipeline building GUI

Connection via shell:

// Connexion via le string de connexion Atlas, puis :
use sample_mflix

1.2 Demo: Compare an Aggregation Pipeline with find()

The find() function is a single-step query. It’s fast and simple, but its main role is to filter documents. Its ability to reshape results is quite limited.

Aggregation pipelines, on the other hand, offer a complete multi-stage process. Instead of stopping at filtering, you can reshape and summarize documents as they move through the different stages.

Example 1: Same result via find() and aggregate()

Task: Return films released after 2010, display the title and a titleLength field (number of characters in the title).

use sample_mflix

// A) find() : calcul du titleLength dans la projection
db.movies.find(
  { year: { $gt: 2010 } },
  { title: 1, year: 1, titleLength: { $strLenCP: "$title" }, _id: 0 }
).limit(5)

// B) aggregate() : même filtre et même champ calculé
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $project: { title: 1, year: 1, titleLength: { $strLenCP: "$title" }, _id: 0 } },
  { $limit: 5 }
])

Both produce the same result: films like ‘On the Road’ (titleLength: 11) and ‘The Secret Life of Walter Mitty’ (titleLength: 31).

Teaching: aggregate() can do the same job as find(), but in a more structured and extensible way.

Example 2: Where aggregation is cleaner

Task: Keep films with titles longer than 20 characters, return the title and its length.

// A) find() : l'expression est dupliquée dans le filtre $expr ET dans la projection
db.movies.find(
  { $expr: { $gt: [ { $strLenCP: "$title" }, 20 ] } },
  { title: 1, titleLength: { $strLenCP: "$title" }, _id: 0 }
).limit(5)

// B) aggregate() : calculer une fois, puis filtrer et formater
db.movies.aggregate([
  { $project: { title: 1, titleLength: { $strLenCP: "$title" }, _id: 0 } },
  { $match: { titleLength: { $gt: 20 } } },
  { $limit: 5 }
])

⚠️ Common pitfall: Filtering on a calculated field BEFORE calculating it returns an empty set without errors.

// ⚠️ Returns nothing because titleLength is not yet defined at the $match stage
db.movies.aggregate([
{ $match: { titleLength: { $gt: 20 } } },
{ $project: { title: 1, titleLength: { $strLenCP: "$title" }, _id: 0 } },
{ $limit: 5 }
])

Example 3: Sort on a calculated field

Task: Sort movies by title length.

// ❌ find() : impossible de trier sur une expression calculée
db.movies.find(
  {},
  { title: 1, titleLength: { $strLenCP: "$title" }, _id: 0 }
).sort({ $strLenCP: { $title: -1 } }).limit(5)
// ❌ Erreur : find() ne peut pas trier sur des expressions

// ✅ aggregate() : calculer, puis trier sur le champ calculé
db.movies.aggregate([
  { $project: { title: 1, titleLength: { $strLenCP: "$title" }, _id: 0 } },
  { $sort: { titleLength: -1 } },
  { $limit: 5 }
])

1.3 Demo: Track document flow across stages

An aggregation pipeline can be viewed as documents flowing through a series of stations, like an assembly line. Each internship has a very specific responsibility:

• $match: decides which documents continue
• $project: reshapes the documents into the desired form
• $group: groups documents to create summaries
• $sort: arranges results in a useful order

True flexibility comes from being able to use as many or as few stages as necessary.

Example 1: Simple filter

use sample_mflix

// Filtre uniquement : films sortis après 2010
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } }
])

Lesson: $match is the foundation of any pipeline. It acts as the gatekeeper, deciding which documents are worth keeping before doing anything else.

Example 2: Filter + reshaping

// Filtre + projection : garder seulement title et year
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $project: { title: 1, year: 1, _id: 0 } }
])

Teaching: $project transforms a noisy slice into a clean list without changing the underlying document set.

Example 3: Filter + reshaping + calculated field

// Ajouter un champ calculé : label concaténé
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $project: { 
      title: 1,
      year: 1,
      label: { $concat: ["Title: ", "$title", " (", { $toString: "$year" }, ")"] },
      _id: 0
    } 
  }
])

Example 4: Multi-stage pipeline with filter on calculated field

// Filtrer, calculer titleLength, puis filtrer à nouveau sur ce champ calculé
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $project: { 
      title: 1,
      year: 1,
      titleLength: { $strLenCP: "$title" },
      _id: 0 
    } 
  },
  { $match: { titleLength: { $gt: 30 } } }
])

1.4 Demo: Identify key stages at a glance

The four essential stages form the fundamental building blocks:

Progressive demonstration with sample_mflix

Step 1 — Single match: $match gives a targeted slice of the collection. But the output may contain a lot of unnecessary data.

Step 2 — Match + Project: We keep the same condition and add $project. This eliminates the noise and only displays the title and year. Examples: ‘On the Road’ (2012), ‘The Secret Life of Walter Mitty’ (2013).

Step 3 — Match + Project + Group: We filter the films since 2010, group them by year and count how many films were released each year. The results show trends: 893 films in 2011, more than 1100 in 2013.

Step 4 — Match + Project + Group + Sort: We sort by number of films in descending order. 2013 comes first with 1105 films, followed by 2014 with 1073.

use sample_mflix

// Étape 1 : $match seul
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } }
])

// Étape 2 : $match + $project
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $project: { title: 1, year: 1, _id: 0 } }
])

// Étape 3 : $match + $group
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $group: { _id: "$year", movieCount: { $sum: 1 } } }
])

// Étape 4 : $match + $group + $sort
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $group: { _id: "$year", movieCount: { $sum: 1 } } },
  { $sort: { movieCount: -1 } }
])

Teaching: $group transforms a list of documents into a summary, giving the big picture instead of the details. $sort adds narrative direction to the results.

1.5 Demo: Write and run your first Aggregation Pipeline

Full Pipeline — Shell Version

This comprehensive pipeline ties the four essential stages together. The data journey:

1. $match (filter) — Only films after 2010 are accepted.
2. $project (remodel 1) — We keep only title and year, removing cast, plot, poster, etc.
3. $project (remodel 2) — Add titleLength (number of characters in the title).
4. $group (summary) — We group by year, counting the films and calculating the average length of the titles.
5. $project (final remodel) — We rename _id to year, and the internal fields to readable names.
6. $sort (orders) — The most prolific years appear first.

use sample_mflix

// Pipeline complet — MongoDB Shell
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $project: { title: 1, year: 1, _id: 0 } },
  { $project: { title: 1, year: 1, titleLength: { $strLenCP: "$title" } } },
  { $group: { 
      _id: "$year", 
      totalMovies: { $sum: 1 },
      avgTitleLength: { $avg: "$titleLength" }
    } 
  },
  { $project: { 
      year: "$_id", 
      totalMovies: 1, 
      avgTitleLength: 1, 
      _id: 0 
    } 
  },
  { $sort: { totalMovies: -1 } }
])

Expected result: 2013 leads with over 1100 films and an average title length of around 15 characters, followed by 2014 and 2012.

Full pipeline — Compass version (strict JSON format)

The same pipeline can be used directly in MongoDB Compass by passing the operators as strings:

[
  { "$match": { "year": { "$gt": 2010 } } },
  { "$project": { "title": 1, "year": 1, "_id": 0 } },
  { "$project": { "title": 1, "year": 1, "titleLength": { "$strLenCP": "$title" } } },
  { "$group": { 
      "_id": "$year", 
      "totalMovies": { "$sum": 1 }, 
      "avgTitleLength": { "$avg": "$titleLength" } 
    } 
  },
  { "$project": { 
      "year": "$_id", 
      "totalMovies": 1, 
      "avgTitleLength": 1, 
      "_id": 0 
    } 
  },
  { "$sort": { "totalMovies": -1 } }
]

Note: MongoDB Compass allows you to visualize the results stage by stage, which is very useful for understanding how the data evolves at each stage.

2. Filter and reshape with $match and $project

Module duration: 17m 17s

2.1 Demo: Filter documents with $match

Key concept

$match works like a funnel: the whole collection tries to enter it, but only documents satisfying your conditions pass. It supports the same operators as find(): comparisons, existence checks, expression-based logic.

You can place $match anywhere in the pipeline and chain it with other stages. If you know how to filter with find(), you already know how to use $match.

Example 1: Simple digital filter

use sample_mflix

// Films sortis après 2010 — filtre de base
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $project: { title: 1, year: 1, _id: 0 } },
  { $limit: 5 }
])

Result: Movies like ‘On the Road’ and ‘The Secret Life of Walter Mitty’. $match acts as a direct gatekeeper — it passes only the necessary data.

Example 2: Compound filter (multiple conditions)

// Dramas notés PG-13, sortis entre 2012 et 2018
db.movies.aggregate([
  { $match: { 
      year: { $gte: 2012, $lte: 2018 },
      genres: { $in: ["Drama"] },
      rated: "PG-13"
    } 
  },
  { $project: { title: 1, year: 1, genres: 1, rated: 1, _id: 0 } },
  { $limit: 5 }
])

Result: Movies like ‘The Giver’ and ‘Lincoln’. $match can handle multiple conditions simultaneously and produce exactly the desired slice.

Example 3: Filter calculated with $expr

// Films avec un runtime d'au moins 120 minutes (filtre basé sur une expression)
db.movies.aggregate([
  { $match: { $expr: { $gte: ["$runtime", 120] } } },
  { $project: { title: 1, runtime: 1, _id: 0 } },
  { $limit: 5 }
])

Teaching: Instead of filtering directly on a field, we use an expression. $expr allows comparisons between fields or calculations in the filter.

Example 4: Filter on a date

// Films sortis à partir du 1er janvier 2014
db.movies.aggregate([
  { $match: { released: { $gte: ISODate("2014-01-01T00:00:00Z") } } },
  { $project: { title: 1, released: 1, _id: 0 } },
  { $limit: 5 }
])

2.2 Demo: Reshape output with $project

Key concept

If $match is a funnel that decides what comes in, $project is the editor’s red pencil. It doesn’t change what documents you have, but it changes how they look by including, excluding or adding new fields.

$project can:

• Include the desired fields (value 1)
• Exclude unwanted fields (value 0)
• Calculate new fields on the fly

Example 1: Inclusion, exclusion, mixed projection

use sample_mflix

// Sans projection : sortie très volumineuse
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $limit: 5 }
])

// Inclusion : garder seulement title et year
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $project: { title: 1, year: 1, _id: 0 } },
  { $limit: 5 }
])

// Exclusion : cacher plot, fullplot et tomatoes.consensus
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $project: { plot: 0, fullplot: 0, "tomatoes.consensus": 0 } },
  { $limit: 5 }
])

Teaching: The projection acts as a cutting tool. It removes the superfluous and makes the output clean. Both modes (inclusion and exclusion) are available, whichever is simpler.

Example 2: Resize an array with $slice

// Retourner seulement les 2 premiers genres de chaque film
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $project: { 
      title: 1,
      year: 1,
      genres: { $slice: ["$genres", 2] },
      _id: 0
    }
  },
  { $limit: 5 }
])

Result: Instead of a long list of categories, a quick overview — for example Adventure and Drama for ‘On the Road’. $project is not limited to pruning fields; he can actively reshape the paintings.

Example 3: Create a calculated field — $toUpper

// Créer shortTitle : le titre en majuscules
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $project: { 
      title: 1,
      year: 1,
      shortTitle: { $toUpper: "$title" },
      _id: 0
    }
  },
  { $limit: 5 }
])

2.3 Demo: Rename and calculate fields with $project

$project decides how each document looks: which fields stay, which disappear, which get new names, and which are calculated on the fly.

Example 1: Rename a field

use sample_mflix

// Renommer year en releaseYear
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $project: { title: 1, releaseYear: "$year", _id: 0 } },
  { $limit: 5 }
])

Result: ‘On the Road’ appears with releaseYear: 2012, ‘The Secret Life of Walter Mitty’ with 2013. A simple rename offers a considerable gain in readability.

Example 2: Calculate a decade

// Ajouter un champ decade : arrondir l'année à la décennie
db.movies.aggregate([
  { $match: { year: { $gte: 1998 } } },
  { $project: { 
      title: 1,
      year: 1,
      decade: { $multiply: [ { $floor: { $divide: ["$year", 10] } }, 10 ] },
      _id: 0
    }
  },
  { $limit: 20 }
])

Result: ‘Xiu Xiu: The Sent-Down Girl’ (year: 1998, decade: 1990), ‘Crime and Punishment’ (year: 2002, decade: 2000). A little math transforms raw numbers into categories on which we can sort, group or create graphs.

Example 3: Combine renaming + calculated fields

// Titre en majuscules, releaseYear, et decade en un seul $project
db.movies.aggregate([
  { $match: { year: { $gt: 1998 } } },
  { $project: { 
      movieTitle: { $toUpper: "$title" },
      releaseYear: "$year",
      decade: { $multiply: [ { $floor: { $divide: ["$year", 10] } }, 10 ] },
      _id: 0
    }
  },
  { $limit: 10 }
])

Result: ‘KATE AND LEOPOLD’ (releaseYear: 2001, decade: 2000), ‘ONEGIN’ (1999, decade: 1990). A single stage provides three improvements — the output feels like something deliberately designed.

Example 4: Conditional field with $cond

// Ajouter un flag isModern : "Yes" si year > 2000, sinon "No"
db.movies.aggregate([
  { $match: { year: { $gte: 1990 } } },
  { $project: { 
      title: 1,
      year: 1,
      isModern: { $cond: [ { $gt: ["$year", 2000] }, "Yes", "No" ] },
      _id: 0
    }
  },
  { $limit: 10 }
])

Teaching: $cond allows you to create conditional fields in the projection, adding Boolean logic directly into the document shape.

2.4 Demo: Combine $match and $project

Used together, $match reduces the set of documents and $project makes what remains easy to read. This combination keeps the results small, focused and ready for what’s next.

Example 1: Action films after 2010

use sample_mflix

// Filtrer les films d'action après 2010, projeter title, year, genres
db.movies.aggregate([
  { $match: { 
      year: { $gt: 2010 },
      genres: { $in: ["Action"] }
    }
  },
  { $project: { title: 1, year: 1, genres: 1, _id: 0 } },
  { $limit: 5 }
])

Result: ‘Jurassic World’ (2015, genres: Action, Adventure, Sci-Fi), ‘John Carter’ (2012, genres: Action, Adventure, Fantasy). The $match + $project combination: let $match choose the right set and let $project make it readable.

Example 2: PG films after 2005

// Films PG sortis après 2005
db.movies.aggregate([
  { $match: { 
      year: { $gt: 2005 },
      rated: "PG"
    }
  },
  { $project: { title: 1, year: 1, rated: 1, _id: 0 } },
  { $limit: 5 }
])

Result: ‘Over the Hedge’ (2006, PG), ‘Coraline’ (2009, PG). Reusability: we can vary the conditions in $match while keeping a stable and predictable output for the reports.

Example 3: Dramas with calculated field

// Dramas depuis 2000 avec longueur de titre
db.movies.aggregate([
  { $match: { 
      year: { $gte: 2000 },
      genres: { $in: ["Drama"] }
    }
  },
  { $project: { 
      title: 1,
      year: 1,
      titleLength: { $strLenCP: "$title" },
      _id: 0
    }
  },
  { $limit: 5 }
])

Result: ‘Crime and Punishment’ (2002, titleLength: 20), ‘Glitter’ (2001, titleLength: 7). $project can both reduce and calculate at the same time without leaving the pipeline.

2.5 Demo: Observe the evolution of the shape of a document

This demo keeps the criteria constant and observes how the shape of the document changes at each stage — the opposite of previous demos where we changed the criteria while keeping the output simple.

Constant criterion: Drama films released after 2010.

Step 1: Raw Filter

use sample_mflix

// Sortie brute : beaucoup de champs, tableaux imbriqués
db.movies.aggregate([
  { $match: { year: { $gt: 2010 }, genres: { $in: ["Drama"] } } },
  { $limit: 2 }
])

The output shows ‘On the Road’ (2012) and ‘The Secret Life of Walter Mitty’ (2013) with long tables and nested fields. Correct, but difficult to read.

Step 2: Filter + simple projection

// Garder seulement title, year, genres
db.movies.aggregate([
  { $match: { year: { $gt: 2010 }, genres: { $in: ["Drama"] } } },
  { $project: { title: 1, year: 1, genres: 1, _id: 0 } },
  { $limit: 2 }
])

The release is lightened: ‘On the Road’ (2012, genres: Adventure, Drama), ‘The Secret Life of Walter Mitty’ (2013, genres: Adventure, Comedy, Drama). Teaching: $project transforms a noisy slice into a clean list without modifying the underlying set.

Step 3: Filter + projection + calculated field

// Ajouter titleLength
db.movies.aggregate([
  { $match: { year: { $gt: 2010 }, genres: { $in: ["Drama"] } } },
  { $project: { title: 1, year: 1, genres: 1, _id: 0 } },
  { $project: { title: 1, year: 1, genres: 1, titleLength: { $strLenCP: "$title" }, _id: 0 } },
  { $limit: 2 }
])

‘On the Road’ returns with titleLength: 11, ‘The Secret Life of Walter Mitty’ with titleLength: 31. You now have a small metric that you can sort or group by in the next stage.

Step 4: Secondary filter on the calculated field

// Garder uniquement les titres de plus de 20 caractères
db.movies.aggregate([
  { $match: { year: { $gt: 2010 }, genres: { $in: ["Drama"] } } },
  { $project: { title: 1, year: 1, genres: 1, _id: 0 } },
  { $project: { title: 1, year: 1, genres: 1, titleLength: { $strLenCP: "$title" }, _id: 0 } },
  { $match: { titleLength: { $gt: 20 } } },
  { $limit: 2 }
])

‘The Secret Life of Walter Mitty’ (titleLength: 31) remains and is joined by ‘The Crimson Petal and the White’ (2011, titleLength: 31).

The fundamental rhythm: Filter first → project to make readable → calculate a useful field → filter again on this new context. This is how we move documents from raw to refined without ever leaving the pipeline.

3. Summarize data with $group

Module duration: 19m 11s

3.1 Demo: Group documents with $group

Key concept

Summarizing is one of the most powerful things you can do in an aggregation pipeline, and $group is the stage designed for it. Here are the three basic things that $group does:

1. Group by field — It takes a value (like a year or a genre) and brings together all the documents that share that value in the same “bucket”.
2. Count or total in each bucket — You can add documents, calculate sums, or measure averages.
3. Summarize for Insights — Instead of scanning thousands of raw records, you see condensed results that highlight trends or comparisons.

Basic syntax:

{ $group: { _id: "<expression de regroupement>", <champ>: { <accumulateur>: <expression> } } }

• _id: defines what we group on (the group key)
• Other fields use accumulators like $sum, $avg, $min, $max, $push

Example 1: Group by year

use sample_mflix

// Compter combien de films ont été sortis par année
db.movies.aggregate([
  { $group: { _id: "$year", movieCount: { $sum: 1 } } }
])

Result: 1952 → 45 films, 1994 → 305 films. $group instantly turns thousands of records into a timeline of totals, allowing you to spot trends over decades rather than scrolling through individual entries.

Example 2: Group by genre with $unwind

// Compter combien de films appartiennent à chaque genre
// Note : $unwind est nécessaire car chaque film peut avoir plusieurs genres
db.movies.aggregate([
  { $unwind: "$genres" },
  { $group: { _id: "$genres", movieCount: { $sum: 1 } } }
])

Why $unwind? A film can have several genres (e.g. Adventure, Drama). Without $unwind, each film would only contribute its combination of genres. $unwind breaks the genres table into separate documents — one per genre — so that each genre is counted independently.

Example 3: Group with title collection via $push

// Collecter un échantillon de titres par année
db.movies.aggregate([
  { $group: { _id: "$year", titles: { $push: "$title" } } },
  { $limit: 3 }
])

Teaching: $push collects values ​​into an array inside the group. Useful for maintaining context or an example from each group.

3.2 Demo: Apply accumulators to get insights

Accumulation is at the heart of data summarization. MongoDB provides several accumulator operators:

Example 1: $sum — Count by year

use sample_mflix

// Nombre de films par année
db.movies.aggregate([
  { $group: { _id: "$year", totalMovies: { $sum: 1 } } },
  { $limit: 5 }
])

Result: 1922 → 3 films, 2010 → 866 films. $sum is the total/count operator — it quickly shows the scale of each group. Analysts can use it to spot industrial booms or quieter periods.

Example 2: $avg — Average IMDb rating by genre

// Note IMDb moyenne par genre (après unwind pour les genres multiples)
db.movies.aggregate([
  { $unwind: "$genres" },
  { $group: { _id: "$genres", avgRating: { $avg: "$imdb.rating" } } },
  { $limit: 5 }
])

Result: Romance ≈ 6.65, Documentary ≈ 7.36. $avg reveals the typical value in each group — useful for quality measurements. We compare the categories not only by number, but by public perception.

Example 3: $min and $max — Shortest and longest duration per year

// Durée min et max par année
db.movies.aggregate([
  { $group: { _id: "$year", shortestRuntime: { $min: "$runtime" }, longestRuntime: { $max: "$runtime" } } },
  { $limit: 5 }
])

Result: For example, in 2005, the durations range from X minutes to Y minutes. $min and $max give the limits of your dataset — showing not only how many films were released, but how much they varied in length.

Example 4: $count — Number of films per rating category

// Nombre de films par rating (G, PG, R, etc.)
db.movies.aggregate([
  { $group: { _id: "$rated", movieCount: { $count: {} } } },
  { $limit: 5 }
])

Teaching: $count is the quickest way to see how many documents fall into a category.

3.3 Demo: Format grouped results for readability

The raw output of $group often looks like system data rather than something a human can read. For example, the group field appears as _id, and the count as movieCount. With a little cleanup, the same results can show year:2010 and moviesReleased:142.

Operators used: $project, $round, $concat, $slice.

Example 1: Rename fields after grouping

use sample_mflix

// _id → year, movieCount → totalMovies
db.movies.aggregate([
  { $group: { _id: "$year", movieCount: { $sum: 1 } } },
  { $project: { year: "$_id", totalMovies: "$movieCount", _id: 0 } },
  { $sort: { totalMovies: -1 } },
  { $limit: 5 }
])

Result: year: 2013, totalMovies: 1105. Renaming transforms developer output into business-readable summaries.

Example 2: Round results and exclude nulls

// Note IMDb moyenne par genre, arrondie à 1 décimale, sans genres null/vides
db.movies.aggregate([
  { $unwind: "$genres" },
  { $match: { genres: { $ne: null, $ne: "" } } },
  { $group: { _id: "$genres", avgRating: { $avg: "$imdb.rating" } } },
  { $project: { genre: "$_id", avgRating: { $round: ["$avgRating", 1] }, _id: 0 } },
  { $sort: { avgRating: -1, genre: 1 } },
  { $limit: 10 }
])

Result: Documentary at 7.4, Film-Noir at 7.4 instead of long floating point values. $round allows you to obtain precise and easy-to-interpret summaries.

Example 3: Create a readable label with $concat

// Créer un label comme "2016 — 120 movies"
db.movies.aggregate([
  { $group: { _id: "$year", movieCount: { $sum: 1 } } },
  { $project: { _id: 0, label: { $concat: [{ $toString: "$_id" }, " — ", { $toString: "$movieCount" }, " movies"] } } },
  { $sort: { label: 1 } },
  { $limit: 5 }
])

Result: label: "1896 — 2 movies". This type of output can be inserted directly into a chart or dashboard without additional steps outside the database.

Example 4: Combine totals and examples for context

// Pour chaque année : totalMovies et les 3 premiers titres
db.movies.aggregate([
  { $group: { _id: "$year", titles: { $push: "$title" }, totalMovies: { $sum: 1 } } },
  { $project: { year: "$_id", totalMovies: 1, sampleTitles: { $slice: ["$titles", 3] }, _id: 0 } },
  { $sort: { totalMovies: -1 } },
  { $limit: 5 }
])

Result: Each row shows the year, number of films, and a sample of 3 representative titles. $slice limits the array to a manageable subset while preserving context.

3.4 Demo: Combine $match and $group for targeted results

When we start combining $match and $group, we unlock one of the most powerful patterns of MongoDB aggregation: first filter documents, then summarize them into significant digits.

Example 1: Comedies by year after 2000

use sample_mflix

// Combien de comédies ont été sorties chaque année après 2000
db.movies.aggregate([
  { $match: { year: { $gt: 2000 }, genres: { $in: ["Comedy"] } } },
  { $group: { _id: "$year", totalComedies: { $sum: 1 } } },
  { $project: { year: "$_id", totalComedies: 1, _id: 0 } },
  { $sort: { year: 1 } },
  { $limit: 5 }
])

Result: 2004 → 214 comedies, 2005 → 214, 2001 → 193. $match + $group can track production trends for a single genre over time, giving a chronological view.

Example 2: Average length by genre for feature films after 2005

// Durée moyenne par genre pour les films > 120 minutes, sortis après 2005
db.movies.aggregate([
  { $match: { year: { $gt: 2005 }, runtime: { $gte: 120 } } },
  { $unwind: "$genres" },
  { $group: { _id: "$genres", avgRuntime: { $avg: "$runtime" } } },
  { $project: { genre: "$_id", avgRuntime: { $round: ["$avgRuntime", 1] }, _id: 0 } },
  { $sort: { avgRuntime: -1 } },
  { $limit: 5 }
])

Result: Western ≈ 169 min, Documentary ≈ 150 min. This pattern shows not only totals but also differences between categories — how genres manage duration and pace.

Example 3: Highly rated films after 2010

// Films avec IMDb rating >= 8 par année, après 2010
db.movies.aggregate([
  { $match: { year: { $gt: 2010 }, "imdb.rating": { $gte: 8 } } },
  { $group: { _id: "$year", highlyRatedCount: { $sum: 1 } } },
  { $project: { year: "$_id", highlyRatedCount: 1, _id: 0 } },
  { $sort: { highlyRatedCount: -1 } },
  { $limit: 5 }
])

Result: 2014 stands out with 83 highly rated films, followed by 2015 with 63. This pattern reveals the years that produced the most high-quality films.

3.5 Demo: Sort grouped results with $sort

Sorting transforms grouped results into something readable. Without sorting, your summaries appear in random order and the story gets lost. With $sort you decide whether the highlights should be the largest, smallest, or a combination of fields.

Example 1: Sort descending (the most prolific years)

use sample_mflix

// Années les plus prolifiques en premier
db.movies.aggregate([
  { $group: { _id: "$year", totalMovies: { $sum: 1 } } },
  { $project: { year: "$_id", totalMovies: 1, _id: 0 } },
  { $sort: { totalMovies: -1 } },
  { $limit: 5 }
])

Result: 2013 → 1105 films, 2014 → 1073 films. The sorting highlights the most productive years.

Example 2: Sort ascending (least prolific years)

// Les années les moins prolifiques en premier
db.movies.aggregate([
  { $group: { _id: "$year", totalMovies: { $sum: 1 } } },
  { $project: { year: "$_id", totalMovies: 1, _id: 0 } },
  { $sort: { totalMovies: 1 } },
  { $limit: 5 }
])

Result: Years like 1909 and 1919 with only one film each. Important: Ascending sorting not only reveals rare cases, but also acts as a data quality check — it may reveal strange or inconsistent values ​​(e.g. years stored as strings).

Example 3: Multi-field sorting (year + gender)

// Paires année/genre uniques : année croissante, genre décroissant
db.movies.aggregate([
  { $unwind: "$genres" },
  { $group: { _id: { year: "$year", genre: "$genres" } } },
  { $project: { year: "$_id.year", genre: "$_id.genre", _id: 0 } },
  { $sort: { year: 1, genre: -1 } }
])

Result: The years are ordered ascending, and within each year, the genres are ordered descending (eg. 1903 with Western then Romance). This multi-field sorting is particularly powerful when you want chronological order while having a clear structure within each group.

Final lesson on $sort: The real power of sorting is not just in the ordering of data, but in how it guides the storytelling of your summaries. By combining it with grouping, projection and unwinding, we can choose to highlight peaks, troughs, or important combinations.

4. Build and run multi stage pipelines

Module duration: 19m 46s

4.1 Demo: Chaining internships into a pipeline

Building multi-stage pipelines is where MongoDB really feels like a data workflow engine. Each stage does focused work, but when you connect them together, the real magic begins.

The big ideas of chaining and reuse

1. Stages can repeat — You can filter, project or group multiple times as the data evolves.
2. Chain many stages for workflows — We create a logical story that goes from raw recordings to illuminating results.
3. Output from one pipeline can feed into another pipeline — Analysis can continue indefinitely as an infinite loop of discovery and refinement.
4. Multiple pipelines in parallel — Designed for specific analyzes (annual performance, gender distribution), this modular approach keeps the logic ordered and reusable.

Example 1: Simple chaining (filter → group → sort → limit)

use sample_mflix

// Étape par étape : construire le pipeline
// Étape 1 : filtre simple
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $limit: 2 }
])

// Étape 2 : filtre + groupe
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $group: { _id: "$year", totalMovies: { $sum: 1 } } }
])

// Étape 3 : filtre + groupe + tri
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $group: { _id: "$year", totalMovies: { $sum: 1 } } },
  { $sort: { totalMovies: -1 } }
])

// Étape 4 : filtre + groupe + tri + limite
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $group: { _id: "$year", totalMovies: { $sum: 1 } } },
  { $sort: { totalMovies: -1 } },
  { $limit: 2 }
])

Example 2: Multi-level grouping (roll-up)

An advanced pattern: group by a fine value, then group again to obtain an overall total.

// Compter les comédies après 2010, par année, puis roll-up en total global
db.movies.aggregate([
  { $match: { year: { $gt: 2010 }, genres: { $in: ["Comedy"] } } },
  { $group: { _id: "$year", yearlyComedies: { $sum: 1 } } },
  { $group: { _id: null, totalComedies: { $sum: "$yearlyComedies" } } },
  { $project: { _id: 0, totalComedies: 1 } }
])

Note: _id: null in the second $group creates a single group that encompasses all documents — useful for calculating overall totals.

4.2 Demo: Answering real-world questions with pipelines

The goal is not to memorize syntax, but to think like an analyst who can turn a question into an answer. We now orchestrate the courses like a complete symphony.

Question 1: Which genres have the longest films?

Background: A streaming company wants to know if modern movies are longer and which genres keep viewers in their seats the longest.

use sample_mflix

// Top 5 genres par durée moyenne, pour les films après 2010
db.movies.aggregate([
  { $match: { year: { $gt: 2010 } } },
  { $unwind: "$genres" },
  { $group: { _id: "$genres", avgRuntime: { $avg: "$runtime" } } },
  { $project: { genre: "$_id", avgRuntime: { $round: ["$avgRuntime", 1] }, _id: 0 } },
  { $sort: { avgRuntime: -1 } },
  { $limit: 5 }
])

Result: War and Crime come out on top at around 109 minutes each, followed by Action, Musical and History. This pipeline, started as a simple question, produces business insight: which genres dominate screentime.

Pipeline breakdown:

• $match → filters modern movies only
• $unwind → opens each genre separately (like opening a box of chocolates)
• $group → measures average duration by genre
• $project → rounds numbers for clean rendering
• $sort + $limit → only keep the top 5

Question 2: Who are the most consistent drama actors of this century?

Context: We want actors who not only appear often, but also star in good films.

// Top 5 acteurs dans les dramas après 2000 avec IMDb rating >= 7
db.movies.aggregate([
  { $match: { 
      year: { $gt: 2000 },
      genres: { $in: ["Drama"] },
      "imdb.rating": { $gte: 7 }
    }
  },
  { $unwind: "$cast" },
  { $group: {
      _id: "$cast",
      appearances: { $sum: 1 },
      avgRating: { $avg: "$imdb.rating" },
      movies: { $push: "$title" }
    }
  },
  { $sort: { appearances: -1, avgRating: -1 } },
  { $limit: 5 },
  { $project: {
      actor: "$_id",
      appearances: 1,
      avgRating: { $round: ["$avgRating", 1] },
      sampleMovies: { $slice: ["$movies", 3] },
      _id: 0
    }
  }
])

Decomposition:

• $match → high quality modern drama filter (IMDb >= 7)
• $unwind "$cast" → each actor becomes its own document
• $group → count appearances, calculate average rating, collect titles
• $sort → prioritize by number of appearances, then by rating
• $limit 5 → keep only the top 5
• $project → polish the output with readable names and $slice for an extract of 3 films

4.3 Demo: Saving and Reusing Pipelines

Saving and reusing pipelines is the key to working smarter instead of starting from scratch every time.

Best practices:

1. Save queries to scripts — each pipeline tested becomes a command ready to run again.
2. Use views for repeated queries (monthly reports, top genres).
3. Share pipelines with teammates — building on each other’s ideas.
4. Adapt saved pipelines for new tasks by adding filters or changing fields.
5. Avoid errors by reusing tested logic.

Example 1: Save a pipeline to a variable

use sample_mflix

// Définir un pipeline réutilisable
const topGenresPipeline = [
  { $unwind: "$genres" },
  { $group: { _id: "$genres", avgRating: { $avg: "$imdb.rating" } } },
  { $project: { genre: "$_id", avgRating: { $round: ["$avgRating", 1] }, _id: 0 } },
  { $sort: { avgRating: -1 } },
  { $limit: 5 }
]

// Réutiliser le pipeline
db.movies.aggregate(topGenresPipeline)

Result: Clean list of top-rated genres: Documentary, Short, Film-Noir. This pipeline becomes a reusable template at any time.

Example 2: Adapt the pipeline for a slightly different question

// Top genres pour les films après 2010 (ajouter un filtre, réutiliser l'existant)
const recentTopGenresPipeline = [
  { $match: { year: { $gt: 2010 } } },  // filtre ajouté
  ...topGenresPipeline                  // réutilisation du pipeline précédent
]

db.movies.aggregate(recentTopGenresPipeline)

Result: Film-Noir and other genres with the highest ratings, but focused on recent data. We adapt without rewriting — minimal modification for a new question.

Example 3: Create a reusable view

// Sauvegarder le pipeline comme une view pour des requêtes continues
db.createView(
  "TopGenresView",   // nom de la view
  "movies",          // collection source
  topGenresPipeline  // définition du pipeline
)

// Interroger la view comme une collection normale
db.TopGenresView.find()

Advantages of views:

• Guaranteed consistency — everyone on the team uses the same trust logic
• Readability — querying db.TopGenresView.find() is much clearer than a long inline pipeline
• Centralized maintenance — changing logic in one place affects all users of the view

4.4 When to use Aggregation rather than find()

The “find() or aggregate()” game

Here is the simple rule:

Mnemonic rule: If it looks like a quick search → think find(). If this sounds like analysis homework → it’s aggregate().

Practical examples

aggregate() is like the more analytical cousin of find(). All the capabilities of find() are available in $match, plus much more.

4.5 Demo: Exploring AI with Aggregations

MongoDB Atlas integrates an AI-powered experience (via Generative AI) in Data Explorer that allows you to build aggregation pipelines in natural language.

How it works

In the Atlas Data Explorer, on a collection (e.g. movies), a “Generate query” link allows you to enter a query in natural language. MongoDB recognizes if the question requires an aggregation pipeline (not a simple find() query) and automatically generates the corresponding stages.

Workflow demonstrated

Step 1 — Initial request:

Demande : "Show me the top genres by average rating"

MongoDB detects that this is an aggregation task and automatically generates:

// Pipeline généré par l'IA
[
  { $unwind: "$genres" },
  { $group: { _id: "$genres", avgRating: { $avg: "$imdb.rating" } } },
  { $sort: { avgRating: -1 } }
]

The preview window immediately displays the results: Short, Documentary, Film-Noir at the top.

Step 2 — Refinement:

Demande : "Show me the top genres by average rating for movies released after 2010"

The AI ​​understands and adds a $match stage for the release date while retaining the rest of the structure.

What it represents: We go from a natural language question to a complete and functional pipeline, ready to be executed or modified. This is particularly useful for beginners in aggregation or to speed up analysis prototyping.

5. Quick reference of operators and courses

Main stages

Comparison operators in $match

Accumulators for $group

Expression operators for $project

Essential rules

1. Stage order matters — The output of one stage becomes the entry of the next.
2. Calculate before filtering — If you filter on a calculated field, this field must exist in a previous $project.
3. $match early — Placing $match first reduces the number of documents upfront, which improves performance.
4. $project final — A $project at the end cleans up and renames fields for presentation.
5. $unwind before $group on arrays — To group on values ​​that are in an array, you must first unwind it with $unwind.
6. _id in $group is the grouping key — _id: null creates a single global group for grand totals.

Search Terms

filter · manipulate · data · aggregations · mongodb · nosql · databases · sql · pipeline · calculated · field · group · aggregation · films · find · match · pipelines · sort · year · combine · stages · chaining · concept · fields