Managing Requirements with pip-tools¶
The golden rule¶
Only ever edit ``.in`` files. Never edit compiled ``requirements.txt`` files by hand.
The compiled files are regenerated by the workflow (and locally via pip-compile). Any manual changes will be overwritten.
Adding a new direct dependency¶
Find the right .in file for where the dependency belongs:
Where is it used? |
Edit this file |
|---|---|
In |
|
In |
|
In docs build |
|
Dev tooling |
|
Add the package with a minimum version constraint:
...
my-new-package>=2.0
Then recompile starting from the level you edited and downward:
# If you edited src/importer/requirements.in:
pip-compile --generate-hashes --upgrade --strip-extras --output-file src/importer/requirements.txt src/importer/requirements.in
pip-compile --generate-hashes --upgrade --strip-extras --output-file src/requirements.txt src/requirements.in
pip-compile --generate-hashes --upgrade --strip-extras --output-file docs/requirements.txt docs/requirements.in
# If you edited src/requirements.in:
pip-compile --generate-hashes --upgrade --strip-extras --output-file src/requirements.txt src/requirements.in
pip-compile --generate-hashes --upgrade --strip-extras --output-file docs/requirements.txt docs/requirements.in
# If you edited docs/requirements.in:
pip-compile --generate-hashes --upgrade --strip-extras --output-file docs/requirements.txt docs/requirements.in
# If you edited requirements-dev.in:
pip-compile --allow-unsafe --generate-hashes --upgrade --strip-extras --output-file requirements-dev.txt requirements-dev.in
Commit both the .in file and the recompiled .txt file(s).
Removing a dependency¶
Remove it from the relevant .in file, then recompile the same way as adding. pip-compile will drop it (and any indirect deps that are no longer needed) from the compiled output automatically.
Upgrading a specific package¶
# Upgrade only one package, leave everything else pinned
pip-compile --generate-hashes --upgrade-package [package] --output-file src/requirements.txt src/requirements.in
Then recompile downstream files as needed
Upgrading all packages at once¶
This is what the automated workflow does every Monday. To do it locally:
pip-compile --generate-hashes --upgrade --output-file src/importer/requirements.txt src/importer/requirements.in
pip-compile --generate-hashes --upgrade --output-file src/requirements.txt src/requirements.in
pip-compile --generate-hashes --upgrade --output-file docs/requirements.txt docs/requirements.in
pip-compile --generate-hashes --upgrade --output-file requirements-dev.txt requirements-dev.in
Pinning a package to a specific version¶
Add a constraint in the .in file:
# Prevent upgrading past 1.x due to breaking API change
some-package>=1.0,<2.0
pip-compile will respect this and never resolve beyond the constraint.
What the automated workflow does vs what you do manually¶
Task |
Who does it |
|---|---|
Adding a new dependency |
You — edit |
Removing a dependency |
You — edit |
Upgrading all deps weekly |
Workflow — opens a PR automatically |
Upgrading a specific dep now |
You — |
Checking for unused deps |
Workflow — runs deptry, reports in PR |
Checking for conflicts |
Workflow — runs |
Updating hashes after version change |
Automatic — pip-compile always regenerates hashes |
Compile order reference¶
Always compile in this order when changes affect multiple levels:
1. src/importer/requirements.in → src/importer/requirements.txt
2. src/requirements.in → src/requirements.txt
3. docs/requirements.in → docs/requirements.txt
4. requirements-dev.in → requirements-dev.txt (independent)
Steps 3 and 4 are independent of each other and can be run in any order relative to each other, but both require step 1 to have run first. Step 3 requires steps 1 and 2.
deptry false positives¶
If deptry flags a package as unused but you know it’s used (e.g. a CLI tool, a pytest plugin, or a package imported via a string), add it to pyproject.toml:
[tool.deptry.per_rule_ignores]
DEP002 = ["the-package-name"]
Do not add packages here speculatively — only when deptry actually flags them.