Query Data in MongoDB

Database used: sample_mflix (collection movies) Tools covered: MongoDB Compass, MongoDB Shell (mongosh), MongoDB Atlas, VS Code + PyMongo

Table of Contents

1. Introduction and environment setup
2. Writing and refining filters in MongoDB queries

• 2.1 Equality and comparison operators in find()
• 2.2 Combination of conditions with logical operators
• 2.3 Filtering on nested fields and tables
• 2.4 Debugging filters that return unexpected results

3. Module 2 — Application of projection to return targeted fields

• 3.1 Understanding and defining projections
• 3.2 Remove _id field from results
• 3.3 Include or exclude fields in results
• 3.4 Project nested fields and subdocuments

4. Module 3 — Sorting and paging of results for a controlled output

• 4.1 Sort results by one or more fields
• 4.2 Sort by numeric fields, strings and subdocuments
• 4.3 Limit and skip results for pagination
• 4.4 Combine filters, sort and limit in one query

5. Solving Real World Query Challenges as an Analyst

• 5.1 Improve query performance with filters, sorting and pagination
• 5.2 Detect nulls and missing fields in real data
• 5.3 Retrieve Top-N results with ranking logic
• 5.4 Visualize data with MongoDB Charts
• 5.5 Save, reuse and automate MongoDB queries

6. MongoDB Operators Quick Reference

1. Introduction and environment setup

Working with data is no longer just about storing and retrieving it: it’s about understanding it, filtering it and transforming it into relevant insights. MongoDB provides the necessary flexibility, but with flexibility comes complexity.

This course covers the most common challenges data analysts face:

• MongoDB’s flexible schemas may seem chaotic, but applying a little structure gives complete control.
• The dot notation for navigating nested fields and tables.
• logical operators which do not always behave as in SQL.
• Handling missing or null values which can silently distort results.
• pagination with sort, skip and limit combined into a coherent strategy.

Typical Analyst Configuration

Most analysts use local tools like MongoDB Compass or the Shell, connecting directly to cloud-hosted clusters. This combines the performance of the cloud with the familiarity of local tools.

Recommended prerequisite: The Up and Running with MongoDB for Data Analyst course to master the basics of installation and configuration.

Connecting via MongoDB Compass

1. From the home screen, click CONNECT next to an Atlas cluster already backed up.
2. Compass connects and presents the collections in the left sidebar.
3. Select the sample_mflix database, then the movies collection.
4. Compass retrieves the first documents and displays them in readable JSON view — by default, only 25 of the 21,000+ movies are loaded for performance.

Connection via integrated shell in Compass

use sample_mflix
show collections
db.movies.findOne()

Connection via MongoDB Atlas (browser)

1. In Atlas, click Browse Collections to open the Atlas Data Explorer.
2. Same sample_mflix database, same movies collection, same JSON format.

Connection via mongosh (terminal)

1. In Atlas, go to Connect → choose Shell.
2. Copy the provided mongosh URI, paste it into the terminal, press Enter.
3. Enter the password when prompted.

use sample_mflix
show collections
db.movies.findOne()

Consistency between tools: Whether in a browser, a desktop interface or the command line, the data behaves the same way. MongoDB provides a consistent experience across all of its tools.

2. Writing and refining filters in MongoDB queries

2.1 Equality and comparison operators in find()

Filters tell MongoDB exactly which documents to retrieve. The schema of a filter remains the same regardless of the nature of the field or data type.

Simple equality

Find all the films released in 2010:

// Compass — champ Filter :
{ "year": 2010 }

// Shell :
db.movies.find({ "year": 2010 })

Find all movies rated “R” (string type value):

{ "rated": "R" }

Comparison operators

Practical examples with comparisons

Films released before 1970:

{ "year": { "$lt": 1970 } }

Films released before 1960 with runtime ≥ 100 minutes (implicit multi-condition filter $and):

{ "year": { "$lt": 1960 }, "runtime": { "$gte": 100 } }

Films released between 1990 and 2000 included:

{ "year": { "$gte": 1990, "$lte": 2000 } }

Films rated PG or PG-13 with $in:

{ "rated": { "$in": ["PG", "PG-13"] } }

Films not rated “R” with $ne:

{ "rated": { "$ne": "R" } }

Movies neither in English nor in Spanish with $nin:

{ "languages": { "$nin": ["English", "Spanish"] } }

French films released in 2015 (scalar + table):

{ "year": 2015, "languages": { "$in": ["French"] } }

Existence operator $exists

Check if a field is present or absent in a document — ideal for data validation or cleanup:

// Films où le champ 'runtime' est ABSENT
{ "runtime": { "$exists": false } }

// Films où le champ 'plot' EXISTE
{ "plot": { "$exists": true } }

Complete example in Shell

Long films (runtime > 140) from the 1990s:

db.movies.find({
  "year": { "$gte": 1990, "$lte": 1999 },
  "runtime": { "$gt": 140 }
})

Note: Projection and export are available directly from Compass once the results are filtered. The interface also displays the corresponding Shell command, which is excellent for learning syntax.

2.2 Combining conditions with logical operators

When a single filter is not enough, logical operators allow you to combine several conditions precisely.

Implicit and explicit $and

Films released in 2000 and rated PG-13 — implicit form (recommended for readability):

{ "year": 2000, "rated": "PG-13" }

Same result with explicit $and — useful for more complex combinations:

{ "$and": [ { "year": 2000 }, { "rated": "PG-13" } ] }

$or — at least one condition true

Films rated G or PG:

{ "$or": [ { "rated": "G" }, { "rated": "PG" } ] }

Nested $and + $or combination

Films rated PG or PG-13, released in 2000 or later:

{
  "$and": [
    { "year": { "$gte": 2000 } },
    { "$or": [ { "rated": "PG" }, { "rated": "PG-13" } ] }
  ]
}

$not — negation of a condition

$not cannot be used alone: ​​it must wrap a comparison operator.

// Films non classés "R"
{ "rated": { "$not": { "$eq": "R" } } }

// Films avec runtime NON supérieur à 90 minutes (= runtime ≤ 90)
{ "runtime": { "$not": { "$gt": 90 } } }

$nor — none of the conditions must be true

Films that are NOT rated “R” AND were NOT released after 2010:

{
  "$nor": [
    { "rated": "R" },
    { "year": { "$gt": 2010 } }
  ]
}

$or with $exists — missing fields

Movies where the runtime field or the plot field is missing:

{
  "$or": [
    { "runtime": { "$exists": false } },
    { "plot": { "$exists": false } }
  ]
}

$or with $regex — partial matches

Movies with “Star” or “Galaxy” in the title (case insensitive):

{
  "$or": [
    { "title": { "$regex": "Star", "$options": "i" } },
    { "title": { "$regex": "Galaxy", "$options": "i" } }
  ]
}

Same request from the Shell:

db.movies.find({
  $or: [
    { title: { $regex: "Star", $options: "i" } },
    { title: { $regex: "Galaxy", $options: "i" } }
  ]
})

2.3 Filtering on nested fields and tables

The dot notation allows navigating nested documents and tables without additional complexity.

Nested fields (subdocuments)

Films having won at least 5 awards (wins field inside the awards object):

{ "awards.wins": { "$gte": 5 } }

Movies with a viewer.rating of at least 3 (multiple levels of nesting):

{ "tomatoes.viewer.rating": { "$gte": 3 } }

Check the existence of a nested field:

{ "awards.wins": { "$exists": true } }

Arrays

MongoDB checks if a value is present anywhere in an array — no need to know the position:

// Films disponibles en anglais
{ "languages": "English" }

// Films dont les genres incluent Action ou Thriller
{ "genres": { "$in": ["Action", "Thriller"] } }

In Atlas UI: The Atlas Data Explorer in the browser offers exactly the same query capabilities as Compass, ideal for demonstrating filters without local installation.

2.4 Debugging filters that return unexpected results

Silent errors are the most difficult to debug. MongoDB does not return errors for incorrect field names or bad types — it simply returns zero results.

Common pitfalls and fixes

1. Case sensitivity

// ❌ Aucun résultat — valeurs sensibles à la casse par défaut
{ "rated": "pg-13" }

// ✔ Casse correcte
{ "rated": "PG-13" }

2. Typo in field name

// ❌ Échoue silencieusement — MongoDB ne retourne aucune erreur
{ "ratd": "PG-13" }

// ✔ Nom de champ corrigé
{ "rated": "PG-13" }

3. Wrong data type

// ❌ "2000" est une chaîne, pas un nombre
{ "year": "2000" }

// ✔ Type numérique correct
{ "year": 2000 }

4. $in must receive an array

// ❌ $in attend un tableau — erreur explicite
{ "year": { "$in": 2000 } }

// ✔ Valeur enveloppée dans un tableau
{ "year": { "$in": [2000] } }

5. $not must wrap an operator

// ❌ $not ne peut pas agir directement sur une chaîne
{ "rated": { "$not": "R" } }

// ✔ $not enveloppe $eq
{ "rated": { "$not": { "$eq": "R" } } }

6. Partial match with $regex

// ❌ Correspondance exacte — échoue si le titre contient d'autres mots ou une casse différente
{ "title": "godfather" }

// ✔ Regex insensible à la casse pour les correspondances partielles
{ "title": { "$regex": "godfather", "$options": "i" } }

Natural language queries (AI-assisted, Compass + Atlas)

This feature requires you to be connected to your MongoDB Atlas account in Compass. It translates natural language sentences into MongoDB filters:

3. Applying projection to return targeted fields

The projection allows you to define exactly which fields MongoDB should return in the results. It does not modify the documents stored in the collection — it only controls what is sent in response.

3.1 Understanding and defining projections

Without projection, MongoDB returns all fields in a document — cast, plot, ratings, reviews, Tomato scores, etc. The projection reduces this volume to the essentials.

Syntax: db.collection.find(filter, projection)

• Value 1 → include this field in the results
• Value 0 → exclude this field from results

In Compass

// N'inclure que l'année
Filter:  { "year": 1994 }
Project: { "year": 1 }

// N'inclure que le titre
Filter:  { "year": 1994 }
Project: { "title": 1 }

// Inclure titre et année
Filter:  { "year": 1994 }
Project: { "title": 1, "year": 1 }

In the Shell (mongosh)

use sample_mflix

// N'inclure que l'année (+ _id par défaut)
db.movies.find({ year: 1994 }, { year: 1 })

// N'inclure que le titre (+ _id par défaut)
db.movies.find({ year: 1994 }, { title: 1 })

// Inclure titre et année (+ _id par défaut)
db.movies.find({ year: 1994 }, { title: 1, year: 1 })

Note: The _id field is always included by default unless explicitly excluded.

3.2 Remove _id field from results

_id is MongoDB’s built-in unique identifier. It appears in all results by default. To hide it, assign it the value 0 in the projection.

Exception to the consistency rule: _id: 0 can be combined with includes (field: 1) in the same projection. This is the only exception to the principle “you cannot mix inclusions and exclusions”.

In Compass

// Masquer uniquement _id
Filter:  { "year": 1994 }
Project: { "_id": 0 }

// Masquer _id et inclure titre + directeur
Filter:  { "year": 1994 }
Project: { "_id": 0, "title": 1, "director": 1 }
// ⚠ Note : le champ s'appelle "directors" (pluriel) dans sample_mflix

// Masquer _id, inclure titre + année + runtime
Filter:  { "year": 1994 }
Project: { "_id": 0, "title": 1, "year": 1, "runtime": 1 }

In the Shell (mongosh)

// Masquer _id uniquement
db.movies.find({ year: 1994 }, { _id: 0 });

// Masquer _id, inclure titre + directeur
db.movies.find(
  { year: 1994 },
  { _id: 0, title: 1, director: 1 }
);

// Masquer _id, inclure titre + année + runtime
db.movies.find(
  { year: 1994 },
  { _id: 0, title: 1, year: 1, runtime: 1 }
);

Advanced example — Aggregation with final projection

Top 5 most prolific directors in 1994:

db.movies.aggregate([
  { $match: { year: 1994 } },
  { $unwind: "$directors" },
  { $group: {
      _id: "$directors",
      count: { $sum: 1 }
  }},
  { $sort: { count: -1 } },
  { $limit: 5 },
  { $project: {
      _id:      0,
      director: "$_id",
      count:    1
  }}
]);

3.3 Include or exclude fields in results

Fundamental rule: You cannot mix inclusions (1) and exclusions (0) in the same projection — except for _id.

Inclusion — specify which fields to display

// Inclure titre, année et genres seulement
db.movies.find({}, { title: 1, year: 1, genres: 1 })

Exclusion — hide specific fields

// Cacher plot et awards, retourner tout le reste
db.movies.find({}, { plot: 0, awards: 0 })

$slice — limit the number of elements in an array

// Retourner seulement les 3 premiers membres du cast
db.movies.find({}, { cast: { $slice: 3 }, _id: 0 })

// Retourner les 2 derniers membres du cast (nombre négatif = depuis la fin)
db.movies.aggregate([
  {
    $project: {
      _id:      0,
      title:    1,
      cast:     1,
      lastCast: { $slice: ["$cast", -2] }
    }
  }
]);

$elemMatch — return a single element of an array based on a condition

// Inclure seulement les notes avec score > 8.5
db.movies.find(
  {},
  {
    title:   1,
    ratings: { $elemMatch: { score: { $gt: 8.5 } } },
    _id:     0
  }
);

Examples in Compass

// Vue par défaut
Filter:   {}
Project:  (vide)

// Inclusion : titre, année et genres
Filter:   {}
Project:  { "title": 1, "year": 1, "genres": 1 }

// Exclusion : cacher plot et awards
Filter:   {}
Project:  { "plot": 0, "awards": 0 }

// Slice : 3 premiers membres du cast, cacher _id
Filter:   {}
Project:  { "cast": { "$slice": 3 }, "_id": 0 }

// elemMatch : titre + premier rating > 8.5
Filter:   {}
Project:  {
  "title":   1,
  "ratings": { "$elemMatch": { "score": { "$gt": 8.5 } } },
  "_id":     0
}

// Aggregation : cast complet + 2 derniers
[
  {
    $project: {
      title:    1,
      cast:     1,
      lastCast: { $slice: ["$cast", -2] },
      _id:      0
    }
  }
]

3.4 Project nested fields and subdocuments

Dot notation is also used in projections to target deeply nested fields.

In Compass

// Note IMDb uniquement
Filter:   {}
Project:  { "imdb.rating": 1, "_id": 0 }

// Titre + note IMDb
Filter:   {}
Project:  { "title": 1, "imdb.rating": 1, "_id": 0 }

// Note IMDb + nombre d'avis Tomato viewer
Filter:   {}
Project:  { 
  "imdb.rating": 1, 
  "tomatoes.viewer.numReviews": 1, 
  "_id": 0 
}

// Sous-document complet Tomato viewer + titre
Filter:   {}
Project:  { 
  "tomatoes.viewer": 1, 
  "title":           1, 
  "_id":             0 
}

In the Shell (mongosh)

// Note IMDb uniquement
db.movies.find({}, { "imdb.rating": 1, _id: 0 });

// Titre + note IMDb
db.movies.find({}, { title: 1, "imdb.rating": 1, _id: 0 });

// Note IMDb + nombre d'avis Tomato viewer
db.movies.find(
  {},
  { "imdb.rating": 1, "tomatoes.viewer.numReviews": 1, _id: 0 }
);

// Sous-document complet Tomato viewer + titre
db.movies.find(
  {},
  { "tomatoes.viewer": 1, title: 1, _id: 0 }
);

Advanced example — Conditional calculation in an aggregation

Classify each film according to its IMDb rating with a $switch:

db.movies.aggregate([
  {
    $project: {
      _id: 0,
      title: 1,
      verdict: {
        $switch: {
          branches: [
            { case: { $gte: ["$imdb.rating", 8] }, then: "Blockbuster" },
            { case: { $gte: ["$imdb.rating", 6] }, then: "Solid Watch" }
          ],
          default: "It's a pass"
        }
      }
    }
  }
]);

Natural language and SQL queries in Compass

Compass (connected to Atlas) accepts queries in natural language and in SQL syntax:

// Langage naturel
Find movie titles and runtime for films released in 1922

// Syntaxe SQL (convertie automatiquement)
SELECT title, runtime
FROM movies
WHERE year = 1922;

Compass automatically generates the corresponding MongoDB filter and projection, with _id:0 automatically excluded.

4. Sorting and paging results for controlled output

4.1 Sort results by one or more fields

The .sort() method follows after find(). The order isn’t just cosmetic — it changes the way the data tells a story.

Sort values:

• 1 → Ascending order
• -1 → Descending order

In Compass

// Vue non triée
Filter:  { "year": { "$gte": 1990 } }
Project: { "title": 1, "year": 1 }
Sort:    (vide)

// Tri par année — croissant
Filter:  { "year": { "$gte": 1990 } }
Project: { "title": 1, "year": 1 }
Sort:    { "year": 1 }

// Tri par année — décroissant (films les plus récents en premier)
Filter:  { "year": { "$gte": 1990 } }
Project: { "title": 1, "year": 1 }
Sort:    { "year": -1 }

// Tri par titre — alphabétique
Filter:  { "year": { "$gte": 1990 } }
Project: { "title": 1, "year": 1 }
Sort:    { "title": 1 }

// Tri combiné : année décroissante, puis titre croissant
Filter:  { "year": { "$gte": 1990 } }
Project: { "title": 1, "year": 1 }
Sort:    { "year": -1, "title": 1 }

In the Shell (mongosh)

use sample_mflix

// Tri par année — croissant
db.movies.find(
  { year: { $gte: 1990 } },
  { title: 1, year: 1 }
).sort({ year: 1 })

// Tri par année — décroissant
db.movies.find(
  { year: { $gte: 1990 } },
  { title: 1, year: 1 }
).sort({ year: -1 })

// Tri par titre — alphabétique
db.movies.find(
  { year: { $gte: 1990 } },
  { title: 1, year: 1 }
).sort({ title: 1 })

// Tri combiné : année décroissante, puis titre croissant
db.movies.find(
  { year: { $gte: 1990 } },
  { title: 1, year: 1 }
).sort({ year: -1, title: 1 })

4.2 Sort by numeric fields, strings and subdocuments

Numeric sort — by runtime

// Tri croissant (films les plus courts en premier)
db.movies.find(
  { year: { $gte: 2007 } },
  { title: 1, runtime: 1 }
).sort({ runtime: 1 })

// Tri décroissant (films les plus longs en premier)
db.movies.find(
  { year: { $gte: 2007 } },
  { title: 1, runtime: 1 }
).sort({ runtime: -1 })

Common problem: Documents without a runtime field appear first by default. Solution: add a $exists: true filter.

// Tri uniquement sur les documents où runtime existe
db.movies.find(
  { year: { $gte: 2007 }, runtime: { $exists: true } },
  { title: 1, runtime: 1 }
).sort({ runtime: 1 })

Sorting strings with collation

By default, MongoDB sorts in “lexicographic” (ASCII) mode, which can produce unexpected results. For example, “Movie 10” comes before “Movie 2”.

Solution: use collation with numericOrdering: true.

// Tri avec ordre numérique naturel dans les chaînes
db.movies.find(
  { title: { $exists: true } },
  { title: 1 }
).sort({ title: 1 }).collation({ locale: "en", numericOrdering: true })

In Compass:

Sort:      { "title": 1 }
Collation: { locale: "en", numericOrdering: true }

Sorting by field of a subdocument

// Trier par nombre de prix remportés (champ imbriqué awards.wins)
db.movies.find(
  { year: { $gte: 2007 }, "awards.wins": { $exists: true } },
  { title: 1, "awards.wins": 1 }
).sort({ "awards.wins": -1 })

Sorting by a specific element of an array — aggregation pipeline

To sort by the second element of the languages array, you cannot use find() directly. You must use the Aggregation Pipeline:

db.movies.aggregate([
  {
    $match: {
      year: { $gte: 2007 },
      "languages.1": { $exists: true }
    }
  },
  {
    $addFields: {
      second_lang: { $arrayElemAt: ["$languages", 1] }
    }
  },
  {
    $sort: { second_lang: -1 }
  },
  {
    $project: {
      title: 1,
      languages: 1,
      second_lang: 1
    }
  }
])

In Compass (Aggregation tab):

[
  {
    "$match": {
      "year": { "$gte": 2007 },
      "languages.1": { "$exists": true } 
    }
  },
  {
    "$addFields": {
      "second_lang": { "$arrayElemAt": ["$languages", 1] }
    }
  },
  {
    "$sort": { "second_lang": -1 }
  },
  {
    "$project": {
      "title": 1,
      "languages": 1,
      "second_lang": 1
    }
  }
]

Natural language (AI) query: “Find movies released in 2007 or later, extract their second language, and sort by that second language.” — Compass generates the above pipeline automatically, but always check the results, as the AI ​​may omit the $exists filter for the second language.

4.3 Limit and skip results for pagination

Loading thousands of documents at once is neither efficient nor readable. limit() and skip() allow results to be paginated.

Paging formula:

• Page 1: skip(0).limit(pageSize)
• Page N: skip((N-1) * pageSize).limit(pageSize)

In Compass

// Page 2 de tous les films triés par titre (5 résultats par page)
Filter:  {}
Project: { "title": 1, "_id": 0 }
Sort:    { "title": 1 }
Skip:    5
Limit:   5

// Page 1 des films d'action
Filter:  { "genres": "Action" }
Project: { "title": 1, "genres": 1, "_id": 0 }
Sort:    { "title": 1 }
Limit:   5
Skip:    (vide)

// Page 2 des films d'action
Filter:  { "genres": "Action" }
Project: { "title": 1, "genres": 1, "_id": 0 }
Sort:    { "title": 1 }
Limit:   5
Skip:    5

In the Shell (mongosh)

use sample_mflix

// Page 2 — skip 5, afficher 5
db.movies.find(
  {},
  { title: 1, _id: 0 }
).sort({ title: 1 }).skip(5).limit(5)

// Page 1 des films d'action
db.movies.find(
  { genres: "Action" },
  { title: 1, genres: 1, _id: 0 }
).sort({ title: 1 }).limit(5)

// Page 2 des films d'action
db.movies.find(
  { genres: "Action" },
  { title: 1, genres: 1, _id: 0 }
).sort({ title: 1 }).skip(5).limit(5)

// 3 premiers titres en ordre décroissant
db.movies.find(
  {},
  { title: 1, _id: 0 }
).sort({ title: -1 }).limit(3)

Behaviors to avoid

// ⚠️ Limite négative — comportement hérité des anciennes versions, à éviter
db.movies.find({}).sort({ title: -1 }).limit(-3)

// ❌ Skip négatif — erreur : skip doit être >= 0
db.movies.find({}).skip(-3)

Rule: skip() only accepts non-negative integers. Paging only goes one way: forward.

4.4 Combine filters, sort and limit in a single query

The filter + projection + sort + skip + limit combination is the perfect “sandwich query” for any analyst.

In Compass (or Atlas Data Explorer)

// Films d'action depuis 2007 — page 2 (résultats 6 à 15), triés du plus récent
Filter:  { "genres": "Action", "year": { "$gte": 2007 } }
Project: { "title": 1, "year": 1, "genres": 1, "_id": 0 }
Sort:    { "year": -1, "title": 1 }
Limit:   10
Skip:    5

In the Shell (mongosh)

use sample_mflix

db.movies.find(
  { genres: "Action", year: { $gte: 2007 } },
  { title: 1, year: 1, genres: 1, _id: 0 }
).sort({ year: -1 }).skip(5).limit(10)

Explanation of the execution plan (Explain Plan)

Click Explain Plan in Atlas to see what MongoDB did:

1. Scanned 21,000+ documents (entire collection)
2. Filtered → ~940 matching documents
3. Sorted these 940 documents
4. Applied skip(5) → 935 documents remaining
5. Returned 10 final documents

MongoDB suggests adding an index if this query is run frequently — full collection scan is expensive without indexes.

SQL and natural language queries in Atlas

-- Style SQL traduit automatiquement en MongoDB
SELECT title, year, genres
FROM movies
WHERE genres = 'Action' AND year >= 2007
ORDER BY year DESC
OFFSET 5
FETCH NEXT 10 ROWS ONLY;

Natural language:

“Find Action movies released in 2007 or later. Sort them by year in descending order. Skip the first 5 results and return the next 10. Only show the title, year, and genres.”

5. Solving Real World Query Challenges as an Analyst

5.1 Improve query performance with filters, sorting and pagination

In a real-world context, queries are rarely executed manually just once — they are automated in scripts, dashboards, or APIs.

Screenplay: Top 10 action films after 2007, skipping the first 5

In Compass:

Filter:  { "genres": "Action", "year": { "$gte": 2007 } }
Project: { "title": 1, "year": 1, "genres": 1, "_id": 0 }
Sort:    { "year": -1 }
Limit:   10
Skip:    5

In the Shell:

db.movies.find(
  { genres: "Action", year: { $gte: 2007 } },
  { title: 1, year: 1, genres: 1, _id: 0 }
).sort({ year: -1 }).skip(5).limit(10)

In VS Code with PyMongo (find.py):

This Python script connects to the MongoDB cluster, runs the same filter/projection/sorting/pagination at the database level and measures the execution time.

Result: ~3 seconds.

Comparison with the “load everything into memory” approach (findall.py)

This Python script retrieves all documents from the collection in memory (find({})), then filters, sorts and paginates in Python.

Result: ~20-23 seconds — about 7x slower.

Why?

• MongoDB is optimized for filtering and sorting at the storage engine level.
• Filtering in Python requires transferring millions of bytes over the network.
• Each additional sorting layer in Python consumes CPU and memory.

Golden rule: Always push logic (filters, projections, sorting, pagination) to the database. MongoDB is optimized for this type of work. Never recover more data than necessary.

Importance of multi-field sorting for reproducibility

Sorting only by year does not guarantee a stable order if several films have the same year. Always add a secondary field:

// Tri stable : année décroissante, puis titre croissant
.sort([("year", -1), ("title", 1)])

In Python with pymongo:

.sort([("year", -1), ("title", 1)])

5.2 Detect null values ​​and missing fields in real data

Real data is rarely perfect. Null values ​​and missing fields can silently skew analytics.

Triple check on a numeric field

Show only movies whose runtime exists, is not null and is a number:

In Compass:

Filter:  { "runtime": { "$exists": true, "$ne": null, "$type": "number" } }
Project: { title: 1, runtime: 1, _id: 0 }
Limit:   10

In the Shell:

db.movies.find(
  { "runtime": { "$exists": true, "$ne": null, "$type": "number" } },
  { title: 1, runtime: 1, _id: 0 }
).limit(10)

Schema analysis with the Schema tab in Compass

1. Keep the runtime filter in place.
2. Click on the Schema tab → Analyze.
3. Compass samples the filtered documents and displays:

• The frequency of appearance of each field
• The data type of each field
• The distribution of values (histogram for numbers)

For larger sampling, remove the limit and analyze 1000 documents. An undefined or null bar in the histogram is a red flag for data quality.

Analyze frequency of all fields — aggregation pipeline

Useful technique when you do not know the exact structure of the collection (MongoDB is schema-less):

In Compass (Aggregation tab):

[
  { "$project": { "fields": { "$objectToArray": "$$ROOT" } } },
  { "$unwind": "$fields" },
  { "$group": { "_id": "$fields.k", "count": { "$sum": 1 } } },
  { "$sort": { "count": -1 } }
]

In the Shell:

db.movies.aggregate([
  { $project: { fields: { $objectToArray: "$$ROOT" } } },
  { $unwind: "$fields" },
  { $group: { _id: "$fields.k", count: { $sum: 1 } } },
  { $sort: { count: -1 } }
])

This pipeline returns all field names in the collection with their frequency of occurrence — for example, title, year and runtime appear in almost all documents, while poster or comments are less frequent.

5.3 Retrieve Top-N results with ranking logic

One of the most common requests in data analysis: “Give me the top 5 X’s for each Y.”

Scenario: Top 5 highest rated films on IMDb, for each genre.

Aggregation pipeline — Compass / Atlas

[
  {
    "$unwind": "$genres"
  },
  {
    "$match": {
      "imdb.rating": {
        "$ne": null,
        "$type": "number"
      }
    }
  },
  {
    "$setWindowFields": {
      "partitionBy": "$genres",
      "sortBy": { "imdb.rating": -1 },
      "output": {
        "rank": { "$documentNumber": {} }
      }
    }
  },
  {
    "$match": { "rank": { "$lte": 5 } }
  },
  {
    "$project": {
      "_id": 0,
      "title": 1,
      "genre": "$genres",
      "rating": "$imdb.rating",
      "rank": 1
    }
  },
  {
    "$sort": { "genre": 1, "rank": 1 }
  }
]

Pipeline steps explained

In the Shell (mongosh)

use sample_mflix

db.movies.aggregate([
  { $unwind: "$genres" },
  {
    $match: {
      "imdb.rating": { $ne: null, $type: "number" }
    }
  },
  {
    $setWindowFields: {
      partitionBy: "$genres",
      sortBy: { "imdb.rating": -1 },
      output: {
        rank: { $documentNumber: {} }
      }
    }
  },
  { $match: { rank: { $lte: 5 } } },
  {
    $project: {
      _id: 0,
      title: 1,
      genre: "$genres",
      rating: "$imdb.rating",
      rank: 1
    }
  },
  { $sort: { genre: 1, rank: 1 } }
])

Python script with PyMongo (exported from Compass)

from pymongo import MongoClient

client = MongoClient('mongodb+srv://<user>:<password>@cluster0.xxxxx.mongodb.net/')
result = client['sample_mflix']['movies'].aggregate([
    { '$unwind': '$genres' },
    {
        '$match': {
            'imdb.rating': { '$ne': None, '$type': 'number' }
        }
    },
    {
        '$setWindowFields': {
            'partitionBy': '$genres',
            'sortBy': { 'imdb.rating': -1 },
            'output': {
                'rank': { '$documentNumber': {} }
            }
        }
    },
    { '$match': { 'rank': { '$lte': 5 } } },
    {
        '$project': {
            '_id': 0,
            'title': 1,
            'genre': '$genres',
            'rating': '$imdb.rating',
            'rank': 1
        }
    },
    { '$sort': { 'genre': 1, 'rank': 1 } }
])

# Afficher les résultats
for doc in result:
    print(doc)

Note: Compass can export the pipeline in Python code (or JavaScript, Java, Go…) from the Export button. The generated code is complete, but it does not contain a display loop — you have to add the for doc in result: print(doc) manually.

Create a view from a pipeline

In Compass, after running the pipeline:

1. Click on Create View → give a name.
2. The view appears in the sidebar as a normal collection.
3. It can be queried with simple filters: { "rank": 1 } for the first of each genre, { "rank": 2 } for the second, etc.

Save a pipeline

In Compass (Aggregation tab):

1. Click on Save → give a name.
2. The pipeline is saved under the database, accessible from the “saved pipelines” drop-down menu.
3. To reload: clear current steps → select the saved pipeline.

5.4 Visualize data with MongoDB Charts

MongoDB Charts is the visualization tool built into MongoDB Atlas — it allows you to create charts directly connected to data, without exporting to Excel or another tool.

Access MongoDB Charts

From MongoDB Atlas → Charts tab → select data source sample_mflix.movies.

Building a chart from scratch

1. Select the type of chart (eg: Grouped Column Chart).
2. Drag year to X axis → default “binned” type (automatic grouping).
3. Drag _id to Y axis → choose Count to count documents.
4. Disable binning to see each year individually.

Add data quality filter

{ "year": { "$exists": true, "$type": "number" } }

This filter ensures that only valid, numeric years are included in the chart.

Filter by year range with UI sliders

Use the min/max sliders to limit to a range of years (eg: 1990-2000). The range is inclusive of both terminals.

Break down by genre ($unwind on series)

Drag genres to Series → choose Unwind array. Films with multiple genres are counted in each genre. The graph becomes a colored stack showing the evolution of genres over time.

See data behind a chart

Three-dot menu (⋮) → View chart data → table with raw values, exportable in CSV or JSON.

View underlying aggregation pipeline

Click on Aggregation pipeline → MongoDB reveals all the steps (projections, groupings) that generate the graph.

Save to a dashboard

1. Click on Save.
2. Give the chart a name and attach it to a dashboard.
3. The dashboard allows you to resize and reorganize the graphs.

Natural language query in Charts

“What is the average number of awards won by movies with runtime greater than 120 minutes?”

Charts generates an aggregation pipeline in the background and returns a number. This result can be added directly to the dashboard by drag and drop.

5.5 Save, reuse and automate MongoDB queries

Save a query in Compass

1. Run the desired query in Compass.
2. In the Queries panel (next to the Find button), click on the save icon.
3. Give a name → Save.
4. The query appears in the “Recents and Favorites”.
5. To reuse it: click on the saved query → Find.

Reuse the same query in the Shell

use sample_mflix

db.movies.find(
  { genres: "Action", year: { $gte: 2007 } },
  { title: 1, year: 1, genres: 1, _id: 0 }
).sort({ year: -1 }).limit(10)

Automate with a JavaScript script (mongosh)

Create a .js file with query logic:

// query.js
db = connect("mongodb+srv://<user>:<password>@cluster0.xxxxx.mongodb.net/sample_mflix")

db.movies.find(
  { genres: "Action", year: { $gte: 2007 } },
  { title: 1, year: 1, genres: 1, _id: 0 }
).sort({ year: -1 }).limit(10).forEach(doc => printjson(doc))

Run from terminal:

mongosh --file query.js

The same logic runs automatically and displays the results in the terminal.

6. MongoDB Operators Quick Reference

Comparison operators

Logical operators

Element operators

Array operators

Projection operators

Aggregation operators (pipeline stages)

Search Terms

query · data · mongodb · nosql · databases · sql · compass · fields · operators · shell · mongosh · pipeline · aggregation · sorting · atlas · queries · array · comparison · filters · language · natural · nested · save · sort