Davide Pugliese's Engineering Blog

Machine learning: Containerize Jupyter

Even though there’s pipenv, poetry, pyenv, these solutions are not bullet proof and sometimes it is not possible to install all the packages with those solutions. I wanted something independent of my local environment and stable. If you know Docker it is not a big deal, if you don’t know it, then it can be quite time consuming. In this case, I got you covered, bear with me until the end.

The main steps are 2: create the image, create and start the container.

Create the image

To create the image, we need first to create a Dockerfile file which contains the needed instructions.

Instead of installing a Linux distribution by hand on Virtualbox, Parallels, VM Ware, we can do all of this with just a few statements in our terminal. In the Docker world the Dockerfile file is like the musical score, the Docker image contains read-only blueprints that include-container creation instructions, the Docker container is like the music being played.

  1. Create a folder in your user directory for example:
mkdir jupyter-docker
  1. Create a Dockerfile file inside jupyter-docker
# Use Ubuntu latest as the base image
FROM ubuntu:latest

# Install Python and pip
RUN apt-get update && apt-get install -y python3 python3-pip

# Install Jupyter
RUN pip3 install jupyter
RUN pip3 install tensorflow
RUN pip3 install numpy
RUN pip3 install keras

# Create a new user and add to sudoers
RUN useradd -m jupyteruser && echo "jupyteruser:password" | chpasswd && adduser jupyteruser sudo


# Switch to the new user
USER jupyteruser

# Set the working directory for the new user
WORKDIR /home/jupyteruser

# Expose the port Jupyter will run on
EXPOSE 8888

# Run Jupyter Notebook on container startup
CMD ["jupyter", "notebook", "--ip='*'", "--port=8888", "--no-browser", "--allow-root"]
  1. Create the image
docker build -t jupyter-container .

Create the contaienr

  1. Create a docker-compose file which simplifies your life so that you don’t have to remember the command “docker run” with a bunch of options.
version: '3'
services:
  jupyter:
    build: .
    image: jupyter-container
    ports:
      - "8888:8888"
    volumes:
      - "~/notebooks:/home/jupyteruser"

As you can see, you have to create a notebook folder and include it in volumes. The contents of the folder will be synced automatically with the folder /home/jupyteruser.

  1. Run the container with docker-compose up -d

  2. Find the ID of the container with docker ps

  3. Read the logs to copy the HTTP address along with the login token by using docker logs <container_id> This will look like

http://localhost:8888/tree?token=<the_token>
  1. At this point paste the url in your browser and you will be logged in into Jupyter. You can now create your notebooks using tensorflow, numpy, keras.