Resolving private PyPI configuration issues

The Story

This week I spent a few days wrestling with the serverless framework, growing fonder and fonder of AWS’ CDK at every turn. I appreciate that serverless has been, until recently, very necessary and very useful, but I feel confident in suggesting that when deployment mechanisms become more complex than the code they’re deploying things aren’t moving in the right direction.

That said, this post is about overcoming what for us was the final hurdle: getting Docker able to connect to a private PyPI repository for our Python lambdas and lambda layers.

Python is great, kind of, but is marred by two rather severe complications: the split between python and python3, and pip… so it’s a great language, with an awful tooling experience. Anyway, our Docker container couldn’t communicate with our PyPI repo, it took way too long to figure out why, here’s what we learned:

The Solution

If you want to use a private PyPI repository without typing in your credentials at every turn, there are two options:

1. ~/.pip/pip.conf:

2. ~/.pip/pip.conf and ~/.netrc:

What’s not obvious — or even discussed anywhere — is that special characters are problematic and are handled differently by the two mechanisms.

In pip.conf, the password MUST be URL encoded.
In .netrc, the password MUST NOT be URL encoded.