Development Environments and Package Management in Python 3

Table of Contents

1. Course Overview
2. Managing Python Packages with pip

• 2.1 Module Introduction and Overview
• 2.2 Summary: pip basics
• 2.3 Requirements Specifiers: install specific versions
• 2.4 Packages with dependencies
• 2.5 Requirements Specifiers in depth (PEP 508 / PEP 440)
• 2.6 Install from a GitHub URL
• 2.7 Install from local filesystem
• 2.8 Editable installs
• 2.9 pipx: install non-project packages
• 2.10 Configure pip
• 2.11 Module 2 revision

3. Virtual Environments and Project Dependencies

• 3.1 Summary: the basics of virtual environments
• 3.2 PyCharm and virtual environments
• 3.3 Visual Studio Code and virtual environments
• 3.4 Project dependencies for applications
• 3.5 Requirements for different environments
• 3.6 Project dependencies for libraries
• 3.7 Resolving dependencies
• 3.8 Conflict Management

4. Module 4 — Beyond pip: pipenv and poetry

• 4.1 Module Introduction
• 4.2 Introduction to pipenv
• 4.3 pipenv workflow
• 4.4 pipenv: keep packages up to date
• 4.5 Revision: pipenv
• 4.6 Introduction to poetry
• 4.7 Poetry workflow
• 4.8 poetry: execute commands
• 4.9 poetry: groups and extras
• 4.10 Revision: poetry
• 4.11 Final course review

1. Course Overview

Welcome to this course on managing development environments and packages in Python 3. The instructor, Reindert-Jan Ekker, is a Python developer and trainer at Code Sensei.

As projects grow and become more complex, they become increasingly dependent on other Python packages. Managing these dependencies can sometimes be difficult. This course offers an in-depth analysis of the different tools and best practices for dealing with them.

Main topics covered:

• Specify project dependencies
• Manage dependency conflicts
• Two alternatives to pip: pipenv and poetry

By the end of this course, you will know everything you need to manage your project dependencies like a professional.

Prerequisites: Be familiar with the basics of Python, pip, and virtual environments.

2. Managing Python Packages with pip

2.1 Module Introduction and Overview

This module begins with an in-depth look at how to use pip to manage Python packages. The course was created using specific versions of Python, pip, poetry, pipenv and pipx.

What you will learn:

• Revisiting the essentials of pip — the basic commands
• Take a detailed look at the pip install command and its versatility
• Use different version operators
• Understand how pip handles packages with dependencies
• Install packages not hosted on PyPI (GitHub, VCS, local file system)
• Perform an editable install
• Using pip outside the context of a Python project
• Create pip configuration file

2.2 Summary: pip basics

PyPI — Python Package Index

The site pypi.org is the software package repository for Python. When you install a package with pip, this is where it downloads the package, unless you tell it somewhere else.

Basic commands

Install a package:

python -m pip install arrow

Best practice: use python -m pip rather than pip alone. There are cases where the pip command does not correspond to the correct version of Python. python -m pip avoids any problems.

This command installs the arrow package itself and all its dependencies. For example, it installs arrow, python-dateutil and six.

Inspect an installed package:

python -m pip show arrow

This displays several information about the package: installed version, file location, dependencies, etc. We can see that the package was installed inside the project’s virtual environment.

Uninstall a package:

python -m pip uninstall arrow

Important: pip uninstall removes the package itself, but does not automatically remove its dependencies. You have to remove them by hand.

List installed packages:

python -m pip list

After uninstalling arrow, we see that python-dateutil and six are still present — they were not automatically removed.

2.3 Requirements Specifiers: install specific versions

What is a requirements specifier?

A requirements specifier is composed of a project name followed by optional version specifiers (official documentation at pip.pypa.io).

Syntax:

# Version exacte
python -m pip install "arrow == 1.0.0"

# Dernière version disponible
python -m pip install arrow

# Mettre à jour vers la dernière version
python -m pip install --upgrade arrow
# ou avec le flag court
python -m pip install -U arrow

# Mettre à jour pip lui-même
python -m pip install -U pip

Tip: Always surround the requirements specifier with quotation marks. This allows you to add spaces around operators and avoids shell errors, especially with > and <.

Version comparisons

# Version inférieure à 1.2
python -m pip install "arrow < 1.2"

# Version 1.2 ou supérieure
python -m pip install "arrow >= 1.2"

# Exclure une version spécifique (utile si une version connue est bugguée)
python -m pip install "arrow != 1.2.1"

# Plage de versions (ET logique avec la virgule)
python -m pip install "arrow > 1.0.0, < 3.0"

# Compatible release (~=)
python -m pip install "requests ~= 2.24"

The compatible release (~=): ~= 2.24 is equivalent to >= 2.24, == 2.*. This means: same major version (2), minor version equal to or greater than 24. We will never move to a major version 3.

# Équivalences du ~=
"requests ~= 2.24"  ==  "requests >= 2.24, == 2.*"

Environment markers

After a semicolon in the specifier, you can add an environment marker to specify in which cases the dependency must be installed:

asyncio; python_version == "3.4"
package; sys_platform == "linux"

This is particularly useful in dependency files for libraries (see section 3.6).

Note on pip behavior

When we simply say pip install requests, pip installs the latest compatible version with all other dependency constraints — it is not necessarily the very latest absolute version.

2.4 Packages with dependencies

How pip knows dependencies

When pip installs a package, the package contains a file describing its dependencies. There are two main formats:

Old format — setup.py:

# Exemple simplifié de setup.py pour arrow
install_requires=[
    "python-dateutil >= 2.7.0",
    "typing_extensions; python_version < '3.8'",
]

Note: pip only knows the dependencies after downloading the package, because it needs to read the setup.py or pyproject.toml file.

Modern format — pyproject.toml:

# Exemple de dépendances dans pyproject.toml (FastAPI)
[project]
dependencies = [
    "starlette >= 0.27.0, < 0.28.0",
    "pydantic >= 1.7.4, != 1.7, != 1.7.1, != 1.7.2, != 1.7.3, != 1.8, != 1.8.1, < 2.0.0",
]

The pyproject.toml format is more modern and should completely replace setup.py in the future.

Dependency tree

Installing a single package can trigger the download and installation of an entire tree of packages:

python -m pip install fastapi
# Installe: starlette, pydantic, idna, anyio, sniffio, ...

# Voir qui a requis un package
python -m pip show anyio
# Required-by: starlette
# Requires: idna, sniffio

Extras (optional dependencies)

Packages can define groups of optional dependencies, called extras. You can choose to install them or not:

# Installer fastapi avec les extras "doc" et "dev"
python -m pip install "fastapi[doc,dev]"

# Cette commande installe également fastapi s'il n'est pas encore présent

Common extras groups for FastAPI:

• test — to run unit tests
• doc — to generate HTML documentation
• dev — development dependencies
• all — include everything (HTTP server, Jinja templates, etc.)

2.5 Requirements Specifiers in depth (PEP 508 / PEP 440)

PEP 508 — Full format of a requirements specifier

The official specification PEP 508 describes the requirements specifiers in detail. Full format:

nom_package [extras] version_specifiers ; environment_markers

Examples:

requests >= 2.0
arrow[dev] >= 1.0, < 2.0
asyncio; python_version == "3.4"
package_linux; sys_platform == "linux"

Table of environment markers:

PEP 440 — Version operators

PEP 440 defines version operators:

Compatible release (~=) in detail:

~= 2.2    est équivalent à    >= 2.2, == 2.*
~= 1.4.2  est équivalent à    >= 1.4.2, == 1.4.*

This guarantees compatibility within the same major version (or minor for 3-digit specifications).

Arbitrary equality (===): For software that does not follow standard versioning, === does a simple character string comparison:

foobar === "1.0-special-build"

2.6 Install from GitHub URL

Why install from GitHub?

• Package is not available on PyPI
• A bug-fixed version exists on your fork, but not yet in the official release
• You need a specific version (commit, branch) that is not released

Syntax

# Installer depuis GitHub (HTTPS)
python -m pip install "demo_pkg @ git+https://github.com/user/demo_pkg.git"

# Installer depuis un commit spécifique
python -m pip install "demo_pkg @ git+https://github.com/user/demo_pkg.git@ce44833"

# Installer depuis une branche
python -m pip install "demo_pkg @ git+https://github.com/user/demo_pkg.git@main"

# SSH
python -m pip install "demo_pkg @ git+ssh://git@github.com/user/demo_pkg.git"

Important: Quotation marks are necessary because the argument contains spaces and special characters.

The pip documentation under Topic Guides > VCS Support (pip.pypa.io) describes this syntax in detail. The git+ prefix is ​​required before the URL.

This syntax also works for other version control systems:

• Mercurial: hg+https://...
• Subversion: svn+https://...
• Bazaar: bzr+https://...

2.7 Install from local file system

Install from a local directory

# Installer un projet local (pip lit pyproject.toml)
python -m pip install ~/projects/demo_pkg

# pip traite le fichier pyproject.toml, construit un wheel, et l'installe

What makes a project installable is the presence of a pyproject.toml (or setup.py) file with the appropriate directory structure.

Download without installing

# Télécharger les wheels sans les installer
pip download arrow
# Crée des fichiers .whl dans le répertoire courant

# Installer depuis un fichier wheel local
python -m pip install arrow-1.2.3-py3-none-any.whl

Installation Source Summary

The pip install documentation page lists all installation modes:

2.8 Editable installs

The problem

When developing a package, each code change requires a reinstallation to take effect. It’s tedious.

The solution: installation in editable mode

# Installer en mode éditable
pip install -e path/to/demo_pkg

# ou depuis le répertoire courant
pip install -e .

In editable mode, pip does not copy files. Instead, files in the development directory are added directly to the Python import path (sys.path).

Advantage: Any changes to the source code are immediately available without reinstallation.

Exception: A reinstallation is still necessary if you modify the project metadata in pyproject.toml (for example, declared scripts).

Example: declare a script in pyproject.toml

[project.scripts]
demo = "demo_pkg.demo:main"

This tells pip: when you install this package, create a script called demo, and when this script is executed, call the main function in the demo_pkg.demo module.

Typical workflow during development:

# Une seule fois au début
pip install -e .

# Ensuite, modifier demo.py et tester immédiatement
python -c "import demo_pkg.demo; demo_pkg.demo.main()"
# ou
demo  # si le script est déclaré dans pyproject.toml

2.9 pipx: install non-project packages

The problem

When you want to install a utility tool (Black, Pylint, Ansible) available globally on your machine, several approaches exist, but they all have drawbacks.

Available options

Option 1: pip install --user

python -m pip install --user black

Installs for the current user (not system-wide). But the package is not isolated in its own virtual environment — dependencies can conflict.

Option 2: sudo pip install (Linux)

sudo pip install black

Not recommended! May affect the system Python installation, which is used by critical operating system components. On Ubuntu, pip is often not installed by default to discourage this practice.

Option 3: pipx (recommended)

# Installer pipx (via pip --user ou via le gestionnaire de paquets système)
python -m pip install --user pipx

# Configurer le PATH automatiquement
pipx ensurepath

# Installer un outil avec pipx
pipx install black
pipx install pylint
pipx install ansible

pipx installs each tool in its own isolated virtual environment, which avoids any dependency conflicts. The tool remains globally available thanks to PATH management.

Comparison

2.10 Configure pip

Why a pip configuration file?

In some cases, you want to add options to each pip command:

• Company hosts its own package repository
• Using a proxy to access the Internet
• Custom SSL certificates

Rather than adding these options to each command, we create a pip configuration file.

Configuration file location

Note: On Windows, the file is called pip.ini; on macOS/Linux, pip.conf.

Configuration file format

[global]
index-url = https://my.company.repo/simple/
trusted-host = my.company.repo
proxy = http://proxy.company.com:8080

[install]
timeout = 60

• Section [global] applies to all commands
• Specific sections (e.g. [install]) only apply to the corresponding command

Example: use an internal repository

# Sans configuration : toujours spécifier --index-url
python -m pip install --index-url https://my.company.repo/simple/ my_package

# Avec configuration (pip.conf) : aucune option nécessaire
python -m pip install my_package

2.11 Module 2 Review

Summary of all the ways to install with pip:

# Requirements specifier (le plus courant)
python -m pip install arrow
python -m pip install "arrow == 1.0.0"
python -m pip install "arrow >= 1.2, < 2.0"
python -m pip install "arrow != 1.2.1"
python -m pip install "requests ~= 2.24"

# Avec environment marker
python -m pip install "asyncio; python_version == '3.4'"

# Depuis GitHub
python -m pip install "pkg @ git+https://github.com/user/repo.git"

# Depuis un chemin local
python -m pip install ~/projects/my_package

# Depuis une archive wheel
python -m pip install package.whl

# Installation modifiable
python -m pip install -e .

# Pour l'utilisateur courant
python -m pip install --user black

# Utiliser pipx (utilitaires globaux)
pipx install black
pipx ensurepath

• pip documentation: pip.pypa.io
• PEP 508 (requirements specifiers): peps.python.org/pep-0508
• PEP 440 (version specifiers): peps.python.org/pep-0440
• pipx: pypa.github.io/pipx
• pyenv: handling multiple versions of Python
• pyproject.toml format: to make a project installable

3. Virtual Environments and Project Dependencies

3.1 Summary: The Basics of Virtual Environments

Create a virtual environment

# Créer un environnement virtuel nommé "venv" dans le projet
python -m venv venv

# Créer avec une version spécifique de Python
python3.11 -m venv venv

The most common convention is to create the venv directory inside the project. It could be given another name or placed outside the project, but this approach is the most common.

For a specific version of Python, you must of course have this version installed on the machine.

Enable and disable environment

Linux / macOS:

# Activer
source venv/bin/activate

# Le prompt affiche le nom de l'environnement : (venv) user@machine:~$

# Désactiver
deactivate

Windows:

# Activer
venv\Scripts\activate

# Désactiver
deactivate

Anatomy of a virtual environment

venv/
├── bin/         (Linux/macOS) ou Scripts/ (Windows)
│   ├── python   → copie ou lien symbolique vers Python
│   ├── pip
│   └── activate
├── lib/
│   └── pythonX.Y/
│       └── site-packages/   ← les packages installés vont ici
└── pyvenv.cfg

The pyvenv.cfg file contains the Python version and the path to the base Python interpreter.

Basic requirements file

To list the packages installed in the environment:

pip freeze > requirements.txt

To install from a requirements file:

pip install -r requirements.txt

3.2 PyCharm and virtual environments

Create a new project with PyCharm

PyCharm offers a New Project dialog box which offers to automatically create a virtual environment. Default:

• Create venv in project with name venv
• Allows you to choose another Python if several versions are installed

After creation, PyCharm has a Packages view allowing you to search for and install packages directly from the interface.

Working with a project cloned from GitHub

When cloning a project from GitHub (via Get from VCS), PyCharm does not yet have a virtual environment configured. It’s necessary :

1. Go to project preferences
2. Python Interpreter → Add interpreter (top right)
3. Create a new environment or select an existing one

Warning: In some cases, PyCharm can automatically select the environment of another project, which is generally not desired.

Embedded terminal

In PyCharm, the terminal opens with the virtual environment active — this is a handy feature.

3.3 Visual Studio Code and virtual environments

Python extension required

VS Code is a general editor and is not focused on Python. To work with Python, you must install the Python extension. It displays the active Python environment in the lower right corner of the screen.

Create a virtual environment

VS Code does not create a virtual environment for you — you have to do it manually in the terminal:

python -m venv venv

VS Code automatically detects the creation of the new environment and asks if you want to select it. By answering “Yes”, it automatically configures the active Python interpreter. New terminals opened subsequently start with the active environment.

Select an existing environment

1. Click on the environment indicator at the bottom right
2. Choose an environment from the list, or
3. Click Enter interpreter path to browse the file system

Note: The Python interpreter is in the bin/ (Linux/macOS) or Scripts/ (Windows) folder of the environment.

Key difference with PyCharm

VS Code is much simpler in this regard: it can neither create virtual environments for you nor manage your packages through a GUI. Management is done entirely on the command line.

3.4 Project dependencies for applications

Application vs Library

In Python packaging, there are two types of projects:

• Application: Code that you (or your organization) deploy. You control the execution environment. Example: a web application, an automation script.
• Library: code that you share with others. They use it on their machine, in their own environment. You do not control the execution environment.

Managing dependencies for an application

For an application, we use a requirements file with pinned versions (fixed to an exact version).

# requirements.txt
Flask == 2.3.2
Werkzeug == 2.3.6
Jinja2 == 3.1.2

The advantage of pinning:

• No ambiguity on exact versions in production
• We only test against a single set of packages
• Full reproducibility between environments

Requirements file format

The requirements file supports the same syntax as pip requirements specifiers:

# requirements.txt — exemples de syntaxes supportées
Flask == 2.3.2
requests >= 2.28, < 3.0
"demo_pkg @ git+https://github.com/user/demo_pkg.git"
-e ./local_package   # installation modifiable locale
./path/to/local.whl  # fichier wheel local

Install from a requirements file:

pip install -r requirements.txt

3.5 Requirements for different environments

In practice, it is common to have multiple requirements files for different contexts.

Typical organization

requirements-prod.txt      ← dépendances de production (pinned)
requirements-test.txt      ← test + production
requirements-dev.txt       ← développement + test + production

requirements-prod.txt:

Flask == 2.3.2
SQLAlchemy == 2.0.1

requirements-test.txt:

# Inclure les requirements de production
-r requirements-prod.txt

# Dépendances pour les tests
pytest == 7.4.0
pytest-cov == 4.1.0

requirements-dev.txt:

# Inclure les requirements de test (qui incluent prod)
-r requirements-test.txt

# Outils de développement
flask-debugtoolbar == 0.13.1
black == 23.7.0

Usage:

# En production
pip install -r requirements-prod.txt

# Pour les tests (CI/CD)
pip install -r requirements-test.txt

# En développement
pip install -r requirements-dev.txt

When installing requirements-test.txt, pip first installs the production requirements (via -r requirements-prod.txt), then installs pytest.

3.6 Project dependencies for libraries

Why not pin for a library?

When you distribute a library, users integrate it into their own projects. They may already have other dependencies on the same packages. If we cheat the versions, we make their lives more difficult.

Purpose: to define a range of supported versions wide enough that there is overlap with the versions used by other dependencies in the user’s project.

Using pyproject.toml

[project]
name = "my-library"
version = "1.0.0"
dependencies = [
    "pydantic >= 1.6.2, != 1.7, != 1.7.1, != 1.7.2, != 1.7.3, != 1.8, != 1.8.1, < 2.0.0",
    "httpx >= 0.23.0",
    "typing-extensions; python_version < '3.10'",
]

Version exclusions (!= 1.7, != 1.7.1, etc.) are usually due to known bugs or incompatible changes in those specific versions.

pyproject.toml format: This is the modern format, replacing setup.py. It is recommended to adopt it for all new projects.

Difference application vs library

3.7 Resolving dependencies

How pip resolves dependencies

Consider this requirements file:

requests
idna == 2.4

When pip tries to install the latest version of requests, it discovers that recent requests requires idna > 2.5. But we specified idna == 2.4.

Pip then performs backtracking: it downloads older and older versions of requests until it finds one that accepts idna == 2.4.

Essai → requests 2.31.0 → requiert idna > 2.5 → incompatible ❌
Essai → requests 2.30.0 → requiert idna > 2.5 → incompatible ❌
...
Essai → requests 2.10.0 → accepte idna 2.4 → compatible ✓

What this entails

• pip install requests does not necessarily mean “the latest version of requests” but “the latest version of requests compatible with all other constraints”
• In complex projects, backtracking can explore an entire tree of combinations and take considerable time
• This is one of the motivations for using tools like pipenv and poetry (see Module 4)

3.8 Conflict management

What is a dependency conflict?

A conflict occurs when pip cannot find any combination of versions satisfying all constraints.

Conflict example:

# requirements.txt
requests == 2.28.0
demo_pkg    # requiert idna < 2.5

requests 2.28.0 requires idna > 2.5, but demo_pkg requires idna < 2.5. There is no version of idna that satisfies both constraints simultaneously.

pip then displays an explicit error message:

ERROR: Cannot install requests==2.28.0 and demo_pkg because these package versions have conflicting dependencies.

The conflict is caused by:
    requests 2.28.0 depends on idna<4,>=2.5
    demo_pkg depends on idna<2.5

Resolution strategies

pip itself suggests solutions:

1. Relax version constraint: replace requests == 2.28.0 with requests ~= 2.10
2. Remove version constraint: let pip choose
3. Find an alternative to one of the conflicting dependencies
4. Fork the problematic package and modify its dependencies

Resolution by relaxation:

# Avant (conflit)
requests == 2.28.0
demo_pkg

# Après (résolu)
requests ~= 2.10
demo_pkg

pip can now find a version of requests compatible with demo_pkg, for example requests 2.28.2.

4. Beyond pip: pipenv and poetry

4.1 Module Introduction

Why were these tools created?

Although pip and venv are essential standard tools, the Python community has developed alternative tools to fill certain gaps.

Advantages of modern tools (pipenv and poetry):

• A single tool to replace pip + venv
• Better dependency management thanks to dependency locking (deterministic builds)
• More security thanks to hash verification
• Additional features to simplify workflow

The problem of non-determinism

Scenario 1 — Top-level dependencies only:

# requirements.txt (simplifié, sans pin)
flask
sqlalchemy
requests

Problem: Subdependencies are not listed. By installing tomorrow, some subdependencies may have changed version. The list is not a complete specification.

Scenario 2 — All pinner (exit from pip freeze):

# requirements.txt (pip freeze)
flask == 2.3.2
werkzeug == 2.3.6
jinja2 == 3.1.2
click == 8.1.6
itsdangerous == 2.1.2
sqlalchemy == 2.0.20
requests == 2.31.0
certifi == 2023.7.22
charset-normalizer == 3.2.0
idna == 3.4
urllib3 == 2.0.4
...

Problem: Very long list, difficult to maintain. For most subdependencies, you just want the latest version — you don’t want to be responsible for every update.

The ideal solution: two file levels

• A file for top-level dependencies (what we really choose)
• A lock file for all resolved versions (automatically generated)

This is exactly what pipenv and poetry do.

4.2 Introduction to pipenv

Installation

# Méthode recommandée : via pipx
pipx install pipenv

# Alternative
pip install --user pipenv

First project with pipenv

# Dans le répertoire du projet
cd my_project

# Créer un fichier script
# script.py :
import arrow
print(arrow.now())

# Installer une dépendance
pipenv install arrow

pipenv performs several actions simultaneously:

1. Creates the virtual environment in ~/.virtualenvs/ with a unique name related to the project
2. Install dependencies in this environment
3. Creates the Pipfile file
4. Creates the Pipfile.lock file

Note: The virtual environment being created outside the project directory, if you move or rename the project, pipenv will no longer be able to find this environment.

The Pipfile

TOML format, similar to pyproject.toml:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
arrow = "*"      # "*" = toute version

[dev-packages]
black = "~= 22"  # dépendance de développement

[requires]
python_version = "3.11"

The Pipfile.lock file

Self-generated JSON file containing the exact versions of all packages (top-level + subdependencies), with their hashes for integrity checking. Do not edit manually.

Install a development dependency

# Installer comme dev-dependency (avec -d)
pipenv install --dev black "~= 22"
# ou
pipenv install -d black "~= 22"

4.3 pipenv workflow

After a git clone

# Installer exactement les versions du lock file (recommandé)
pipenv sync

# Installer aussi les dev-dependencies
pipenv sync --dev
# ou
pipenv sync -d

The pipenv sync command ensures that everyone has exactly the same packages based on the Pipfile.lock.

View dependency tree

pipenv graph

Shows all dependencies with their version specifiers and installed versions — very useful for understanding relationships between packages.

Execute commands in the environment

Option 1 — pipenv run (for a single command):

# Exécuter un script Python
pipenv run python script.py

# Exécuter Black sur le code
pipenv run black my_package/

# Exécuter les tests
pipenv run pytest

Option 2 — pipenv shell (for multiple commands):

# Ouvrir un shell avec l'environnement actif
pipenv shell

# Dans le shell, on peut exécuter directement
python script.py
pytest

# Pour quitter : NE PAS utiliser "deactivate" !
exit   # ← correct, car pipenv shell ouvre un sous-processus

Important: pipenv shell creates a subprocess shell. To exit, use exit and not deactivate. deactivate would not close the subshell properly.

Uninstall a package

pipenv uninstall arrow

This updates the Pipfile and Pipfile.lock, and uninstalls the package. But — like pip — package dependencies are not automatically uninstalled.

4.4 pipenv: keep packages up to date

Security check

pipenv check

pipenv scans installed packages and reports known security vulnerabilities (via the PyUp Safety database).

Example output:

Checking PEP 508 requirements...
Passed!
Checking installed packages for vulnerabilities...
7 vulnerabilities found:
  flask 0.12.2: CVE-2023-... 
  werkzeug 1.0.0: CVE-2023-...

Update a package

# Mettre à jour un package spécifique (et ses sous-dépendances)
pipenv update flask

# Mettre à jour tous les packages
pipenv update

pipenv update flask:

1. Update Flask in the Pipfile (return * if it was pinner to an old version)
2. Installs the latest version of Flask and its subdependencies
3. Updates the Pipfile.lock

Note: Updating a top-level dependency modifies its version in the Pipfile.

4.5 Revision: pipenv

Summary table of pipenv commands:

Specific features to remember:

• pipenv install arrow uses all pip version specifiers (GitHub URLs, version operators, etc.)
• pipenv uninstall does not remove transitive dependencies
• To synchronize with a lock file, use pipenv sync (not pipenv install)
• To exit pipenv shell, use exit and not deactivate
• pipenv is best suited for applications (not libraries) because it always assumes a committed lock file

4.6 Introduction to poetry

Installation

# Méthode recommandée : via pipx
pipx install poetry

# Alternative
pip install --user poetry

Create a new project

# Crée un dossier avec une structure de projet complète
poetry new poetry_demo

Structure generated:

poetry_demo/
├── README.md
├── poetry_demo/          ← package Python
│   └── __init__.py
├── tests/
│   └── __init__.py
└── pyproject.toml        ← dépendances et configuration

Install dependencies

# poetry add (pas install !)
poetry add arrow requests

poetry performs several actions:

1. Create the virtual environment in ~/.cache/pypoetry/virtualenvs/
2. Resolves and installs all dependencies
3. Creates the poetry.lock file
4. Update pyproject.toml

Structure of pyproject.toml with poetry

[tool.poetry]
name = "poetry-demo"
version = "0.1.0"
description = "Ma description"
authors = ["Prénom Nom <email@example.com>"]
packages = [{ include = "poetry_demo" }]

[tool.poetry.dependencies]
python = "^3.11"
arrow = "^1.2.3"
requests = "^2.31.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

• [tool.poetry]: project metadata
• [tool.poetry.dependencies]: main dependencies
• [build-system]: tells pip to use poetry to build the wheel

When installing this project with pip, pip reads the [build-system] section, calls poetry to build the wheel, which then reads the other sections to build the package.

4.7 Poetry workflow

Project sharing (git)

For an application (that you deploy yourself):

# Committer les deux fichiers
git add pyproject.toml poetry.lock
git commit -m "..."
git push

For a library (which is distributed):

• Do not commit poetry.lock (we want to support a wide range of versions)
• Analogy with pip + pyproject.toml: we don’t pin the versions

Difference with pipenv: pipenv doesn’t really support “library” mode — it always assumes pinned versions. poetry supports both cases.

After a git clone

# Installe tout depuis le lock file
poetry install

Unlike poetry add, poetry install does not accept no package name — it is only to synchronize with the lock file.

View installed packages

# Liste simple
poetry show

# Affichage en arbre
poetry show --tree

# Détails d'un package spécifique
poetry show requests

Install / uninstall

# Désinstaller (supprime aussi les dépendances transitives !)
poetry remove arrow

This is an important difference from pip and pipenv: poetry remove also removes dependencies from the removed package.

Update

# Mettre à jour un package (dans les limites des specifiers de pyproject.toml)
poetry update requests

# Mettre à jour tout
poetry update

Difference with pipenv: poetry respects the version specifiers of pyproject.toml. If we allow ^1.1 (compatible with 1.x), poetry will never upgrade to 2.x. pipenv modifies the Pipfile during an update.

Strict synchronization

# Installer ET désinstaller les packages non présents dans le lock file
poetry install --sync

Without --sync, poetry install installs new packages but does not remove obsolete packages.

4.8 poetry: execute commands

Automatic installation of the project in editable mode

When poetry installs the dependencies, it also automatically installs the current project in editable mode:

Installing the current project: poetry-demo (0.1.0)

This means that the package is still available for import, and changes to the source code are immediately active.

Declare scripts

In pyproject.toml:

[tool.poetry.scripts]
showtime = "poetry_demo.script:main"

This registers a showtime command that calls the main function in poetry_demo/script.py.

# poetry_demo/script.py
import arrow

def main():
    print(f"L'heure actuelle est : {arrow.now()}")

After poetry install, the showtime command is available in the environment.

Execute commands

Option 1 — poetry run:

# Exécuter une commande enregistrée
poetry run showtime

# Exécuter un script Python
poetry run python poetry_demo/script.py

# Exécuter pytest
poetry run pytest

Option 2 — poetry shell:

# Ouvrir un shell dans le venv
poetry shell

# Dans le shell
showtime
pytest

# Pour quitter
exit   # NE PAS utiliser "deactivate" — même principe que pipenv

Note on pip show in poetry

poetry hides some packages (setuptools, wheel, pip itself). To see them all:

pip show setuptools   # visible via pip mais pas via poetry show

4.9 poetry: groups and extras

Dependency groups

poetry allows you to organize dependencies into groups:

# Ajouter dans le groupe "dev"
poetry add --group dev black

# Ajouter dans le groupe "test"
poetry add --group test pytest pytest-cov

In pyproject.toml:

[tool.poetry.dependencies]
python = "^3.11"
arrow = "^1.2.3"

[tool.poetry.group.dev.dependencies]
black = "^23.7.0"

[tool.poetry.group.test.dependencies]
pytest = "^7.4.0"
pytest-cov = "^4.1.0"

Install with/without groups

# Installer tout (incluant dev et test par défaut)
poetry install

# Installer sans le groupe test
poetry install --without test

# Installer UNIQUEMENT les dépendances principales (sans aucun groupe)
poetry install --only main

# Installer avec un groupe optionnel
poetry install --with docs

Optional group (not installed by default):

[tool.poetry.group.docs]
optional = true

[tool.poetry.group.docs.dependencies]
mkdocs = "^1.5.0"

Strict synchronization with groups

# Supprimer les packages des groupes non inclus
poetry install --without test --sync

Without --sync, poetry install --without test does not uninstall pytest if it was already installed. With --sync it uninstalls it.

Extras vs groups

• Groups: for developers who work with source code (via poetry). Invisible to end users.
• Extras: for end users installing via pip. Allows you to offer optional features.

# Déclaration d'extras dans pyproject.toml
[tool.poetry.extras]
redis = ["redis"]
all = ["redis", "celery"]

[tool.poetry.dependencies]
redis = { version = "^4.0", optional = true }
celery = { version = "^5.0", optional = true }

Use by end users:

# Installer avec l'extra "redis"
pip install "my-library[redis]"

# Installer avec tous les extras
pip install "my-library[all]"

4.10 Revision: poetry

Summary table of poetry commands:

Specific features to remember:

• poetry add (not poetry install) to install a new package
• poetry install is only used to synchronize with the lock file
• poetry remove also removes transitive dependencies (unlike pip/pipenv)
• poetry update respects the specifiers of pyproject.toml
• To exit poetry shell, use exit and not deactivate
• The current project is always installed in automatically editable mode

4.11 Final course review

Comparison pip / pipenv / poetry

When to use what?

• pip + venv: always useful to know (standard, universally available)
• pipenv: ideal for applications (projects that you deploy yourself). Simple workflow, integrated security verification.
• poetry: ideal for both applications and libraries. More features, group management, package publishing support.

Additional Resources

• pip documentation: pip.pypa.io
• pipx: pypa.github.io/pipx
• pipenv: pipenv.pypa.io
• poetry: python-poetry.org
• pyenv: management of several versions of Python on the same machine
• PEP 508: peps.python.org/pep-0508
• PEP 440: peps.python.org/pep-0440
• pyproject.toml (getting started guide)

Search Terms

development · environments · package · management · python · foundations · data · analysis · engineering · analytics · dependencies · install · pip · poetry · environment · pipenv · requirements · virtual · dependency · groups · installation · packages · commands · application