PiP-Tool

Mastering pip-compile: A pip-tools Tutorial Managing Python dependencies can quickly turn into a headache. As your project grows, your requirements.txt file becomes a messy list of direct dependencies and their hundreds of sub-dependencies (dependencies of dependencies). If you don’t lock these versions, your project might work today but break tomorrow when a dependency updates.

pip-tools is the solution to this problem, and pip-compile is its flagship command. It allows you to maintain a clean list of top-level dependencies while generating a fully pinned, deterministic requirements.txt file.

This tutorial will show you how to master pip-compile to keep your environments stable and reproducible. What is pip-compile?

pip-compile (part of the pip-tools package) acts as a bridge between your high-level requirements (requirements.in) and the specific, pinned dependencies required for production (requirements.txt). Why not just use pip freeze?

pip freeze dumps everything in your environment, making it hard to know which packages you actually installed and which are dependencies.

pip-compile reads your direct dependencies and resolves all sub-dependencies, creating a file that allows you to easily upgrade packages later without losing track of what’s what. 1. Installation First, install pip-tools in your virtual environment: pip install pip-tools Use code with caution. 2. Getting Started: The .in File

Instead of creating a requirements.txt directly, create a requirements.in file. This file only lists the top-level packages you need for your project. requirements.in example: flask requests pyyaml>=6.0 Use code with caution. 3. Compiling Requirements

Run pip-compile on your .in file to generate the requirements.txt: pip-compile requirements.in Use code with caution. What happens next? pip-compile reads requirements.in. It resolves all sub-dependencies and their versions.

It generates a requirements.txt with all packages pinned (e.g., requests==2.31.0), including comments explaining why each package is there. 4. Installing the Compiled Requirements Now that you have a requirements.txt, install it as usual: pip install -r requirements.txt Use code with caution. 5. Updating Dependencies

This is where pip-tools shines. When you need to update your dependencies, you don’t manually edit the requirements.txt. A. Upgrade a specific package

To upgrade only requests to the latest version while keeping everything else locked: pip-compile –upgrade-package requests requirements.in Use code with caution. B. Upgrade all packages

To unlock all packages and update everything to the latest versions: pip-compile –upgrade requirements.in Use code with caution. 6. Advanced Usage & Best Practices

Generate Multiple Requirements Files: You can maintain dev-requirements.in (for testing) and requirements.in (for production) and compile them into separate .txt files.

Use pip-sync: The sister command pip-sync ensures your virtual environment perfectly matches the requirements.txt, uninstalling any packages that are not listed. pip-sync requirements.txt Use code with caution. Compile pip-compile requirements.in Upgrade All pip-compile –upgrade requirements.in Upgrade One pip-compile –upgrade-package requirements.in Sync Env pip-sync requirements.txt

By mastering pip-compile, you move away from chaotic dependency management and toward predictable, stable Python projects. If you’d like, I can show you how to set up

pip-tools with GitHub Actions for automatic dependency management. pip-tools · PyPI