Packaging the packages

I'll be so bold as to assume that you are using pip to manage your packages. It gives you not only the option to install packages, but merely download them to storage aswell. So let's do that and try to serve the packages.

$ pip download faker
[...]
Successfully downloaded faker python-dateutil six text-unidecode
$ ls
Faker-8.1.0-py3-none-any.whl  python_dateutil-2.8.1-py2.py3-none-any.whl  six-1.15.0-py2.py3-none-any.whl  text_unidecode-1.3-py2.py3-none-any.whl
$ python -m RangeHTTPServer

$ python -m venv .venv
$ . .venv/bin/activate
(.venv) $ pip install --index http://localhost:8000 faker
Looking in indexes: http://localhost:8000
ERROR: Could not find a version that satisfies the requirement faker (from versions: none)
ERROR: No matching distribution found for faker

Dag nabbit. Apparently not that simple. And indeed, if we read up on the spec for PEP 503 spec for Simple Repository API, we learn that we are going to stick those package files under directories named after the packages.

Well, nothing that a bit of bash scripting can't sort out:

#!/usr/bin/env

s=$1
d=$2
if [ -z $d ]; then
        >&2 echo "usage: pep503.sh <source_dir> <dest_dir>"
        exit 1
fi

for df in `find $s -name "*.whl" -type f`; do
        f=`basename $df`
        pd=`echo $f | sed -e "s/^\(.*\)-[[:digit:]]*\.[[:digit:]].*$/\1/g" | tr "[:upper:]" "[:lower:]" | tr "_" "-"`
        mkdir -v $d/$pd
        cp -v $df $d/$pd/
done
for df in `find $s -name "*.tar.gz" -type f`; do
        f=`basename $df`
        pd=`echo $f | sed -e "s/^\(.*\)-[[:digit:]]*\.[[:digit:]].*$/\1/g" | tr "[:upper:]" "[:lower:]" | tr "_" "-"`
        mkdir -v $d/$pd
        cp -v $df $d/$pd/
done
for df in `find $s -name "*.zip" -type f`; do
        f=`basename $df`
        pd=`echo $f | sed -e "s/^\(.*\)-[[:digit:]]*\.[[:digit:]].*$/\1/g" | tr "[:upper:]" "[:lower:]" | tr "_" "-"`
        mkdir -v $d/$pd
        cp -v $df $d/$pd/
done

Armed with this, let's try again:

$ sh /home/lash/bin/shell/pep503.sh . packages
$ mkdir packages
$ bash /home/lash/bin/shell/pep503.sh . packages
mkdir: created directory 'packages/text-unidecode'
'./text_unidecode-1.3-py2.py3-none-any.whl' -> 'packages/text-unidecode/text_unidecode-1.3-py2.py3-none-any.whl'
mkdir: created directory 'packages/six'
'./six-1.15.0-py2.py3-none-any.whl' -> 'packages/six/six-1.15.0-py2.py3-none-any.whl'
mkdir: created directory 'packages/python-dateutil'
'./python_dateutil-2.8.1-py2.py3-none-any.whl' -> 'packages/python-dateutil/python_dateutil-2.8.1-py2.py3-none-any.whl'
mkdir: created directory 'packages/faker'
'./Faker-8.1.0-py3-none-any.whl' -> 'packages/faker/Faker-8.1.0-py3-none-any.whl'
$ find packages/ -type f
packages/faker/Faker-8.1.0-py3-none-any.whl
packages/python-dateutil/python_dateutil-2.8.1-py2.py3-none-any.whl
packages/six/six-1.15.0-py2.py3-none-any.whl
packages/text-unidecode/text_unidecode-1.3-py2.py3-none-any.whl
k $ python -m RangeHTTPServer

$ pip install --index http://localhost:8000/packages faker
Looking in indexes: http://localhost:8000/packages
Collecting faker
[...]
Successfully installed faker-8.1.0 python-dateutil-2.8.1 six-1.15.0 text-unidecode-1.3

Extra prepping

There are some basic packages you will most always need, and which pip often will expect to find in at least one of its available repositories, even regardless of whether its installed or not. If you don't have this on your local offline repository when the internet goes poof, then that will block any build you are trying to do.

Let's make sure we have the packages around all the time:

$ pip download pip setuptools setuptools-markdown wheel
[...]
$ bash /home/lash/bin/shell/pep503.sh . packages
[...]

Choosing a server

As I tend to favor the classics, I still use Apache Web Server to host stuff to my local environment. One practical (if not all too safe) thing about it is that it will automatically bind to all interfaces. So to make the repository available, we simply link or add the packages directory to the virtual root, and restart the server e.g. with

systemctl restart httpd

Of course you can use any HTTP server you like, as long as you know how to bind it to the virtual interface.

To verify that the service is providing what's needed, simply point your web browser to the location, e.g.

lynx http://10.1.2.1/packages/

Serving the packages

Now that we are all prepped, the next step is to do install packages from within a Docker container.

Let's start with an Archlinux base image with basic python provisions.

FROM archlinux:latest

RUN pacman -Sy && \
        pacman -S --noconfirm python python-pip

Build and tag it with pythonbase, and then let's add the Dockerfile to test the repository:

[...]

RUN pip install --index http://10.1.2.1/packages --trusted-host 10.1.2.1 faker

Now, turn internet off (and lights, too, if you'd like some extra suspense), and build the second file.

$ docker build .
Sending build context to Docker daemon  3.072kB
[...]
Successfully installed faker-8.1.0 python-dateutil-2.8.1 six-1.15.0 text-unidecode-1.3
Removing intermediate container 2c10ffcdf3ad
---> 1ba83bb8e111
Successfully built 1ba83bb8e111