Skip to content

2019-07-10 Python

Dependency Hell

Dependency Hell#

  1. Breaking changes in a minor release

If a user uses version pinning, then required dependencies requiring a different version will break.

dependency_a==1.2.1

So we use narrow ranges (downside is package gets outdated):

dependency_a>=1.3.0,<1.4.0

A broad range means too many versions to support:

dependency_a>1.2.0,<1.9.0

Use Semantic Versioning#

Semver.org

1.2.0
Major.Minor.Path
  • Major - Breaking / incompatible API change (reset other digits)
  • Minor - Add functionality in a backwards compatible way. Bump minor version and reset patch.
  • Patch - Security or bug fix that doesn’t break the API

Avoid API Churn#

You don’t want to force user’s of your API to change the way they use it

  • Removing an API
  • Renaming an API
  • Changing the parmameters of an API

Setting your Requirements correctly#

Ensure your install_requires uses a range and does not pin versions.

Eg. six>=1.11 is better than six==1.11.*

If six uses semver, you could have done six>1.11,<2

Test supported versions#

Test your library with every supported python version and with earliest versions of all packages you depend on.

By default pip installs the latest versions of your dependencies

Example#

apache-beam: httplib2>=0.8,<=0.11.3
google-api-python-client: httplib2>=0.9.2, <1

With:

pip install apache-beam google-api-python-client

In this case the installation is successful and version that is installed is the upper bound of the overlapping range. It will install 0.11.3

With:

pip install google-api-python-client apache-beam

Gives an error about incompatibilities

Pip resolves dependencies on a first come first serve basis

The order your dependencies are installed if you use requirements.txt are arbitrary

Summary#

Help your users:

  • Use semantic versioning
  • Avoid API churn
  • Support as large a version range as possible
  • Support the latest version of your dependencies

Source#