MLops for beginners

Day15 — Containerizing our Dev Environment

Building container image for jupyter with all required packages

In most of my previous articles, I was using jupyter notebook for the development. And recently my environment got a bit messy due to multiple versions for the same packages and other stuff.

Steps at a glance:

  • create a base OS
  • Install Python3 on it
  • use pip to install all the requirements
  • create and work directory and change into that directory
  • Launch jupyter on startup
list all docker images and grep out python and centos image
python:3.6-slim —  python imagecentos:latest — without python installed
absl-py==0.9.0
astunparse==1.6.3
attrs==19.3.0
backcall==0.1.0
bleach==3.1.5
cachetools==4.1.0
certifi==2020.4.5.1
chardet==3.0.4
decorator==4.4.2
defusedxml==0.6.0
entrypoints==0.3
gast==0.3.3
google-auth==1.15.0
google-auth-oauthlib==0.4.1
google-pasta==0.2.0
grpcio==1.29.0
h5py==2.10.0
idna==2.9
importlib-metadata==1.6.0
ipykernel==5.3.0
ipython==7.14.0
ipython-genutils==0.2.0
ipywidgets==7.5.1
jedi==0.17.0
Jinja2==2.11.2
jsonschema==3.2.0
jupyter==1.0.0
jupyter-client==6.1.3
jupyter-console==6.1.0
jupyter-core==4.6.3
Keras==2.3.1
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.2
Markdown==3.2.2
MarkupSafe==1.1.1
mistune==0.8.4
nbconvert==5.6.1
nbformat==5.0.6
notebook==6.0.3
numpy==1.18.4
oauthlib==3.1.0
opt-einsum==3.2.1
packaging==20.4
pandocfilters==1.4.2
parso==0.7.0
pexpect==4.8.0
pickleshare==0.7.5
prometheus-client==0.8.0
prompt-toolkit==3.0.5
protobuf==3.12.2
ptyprocess==0.6.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
Pygments==2.6.1
pyparsing==2.4.7
pyrsistent==0.16.0
python-dateutil==2.8.1
PyYAML==5.3.1
pyzmq==19.0.1
qtconsole==4.7.4
QtPy==1.9.0
requests==2.23.0
requests-oauthlib==1.3.0
rsa==4.0
scipy==1.4.1
Send2Trash==1.5.0
six==1.15.0
tensorboard==2.2.1
tensorboard-plugin-wit==1.6.0.post3
tensorflow==2.2.0
tensorflow-estimator==2.2.0
termcolor==1.1.0
terminado==0.8.3
testpath==0.4.4
tornado==6.0.4
traitlets==4.3.3
urllib3==1.25.9
wcwidth==0.1.9
webencodings==0.5.1
Werkzeug==1.0.1
widgetsnbextension==3.5.1
wrapt==1.12.1
zipp==3.1.0
FROM python:3.6-slim# install requirements
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
# workspace
WORKDIR /home/workspace
# Exposing ports
EXPOSE 8888
# Running jupyter notebook
# --NotebookApp.token ='demo' is the password
CMD ["jupyter", "notebook", "--no-browser", "--ip=0.0.0.0", "--allow-root", "--NotebookApp.token='demo'"]
  • COPY requirements.txt requirements.txt → Copy requierments.txt file inside the container.
  • RUN pip install -r requirements.txt→ Install all the requirements from requirements.txt file
  • WORKDIR /home/workspace → This is used to change my working directory to /home/workspace.
  • EXPOSE 8888 → Expose port 8888 so that we can use jupyter via web browser.
  • CMD ["jupyter", "notebook", "--no-browser", "--ip=0.0.0.0", "--allow-root", "--NotebookApp.token='demo'"] → Start jupyter notebook with some additional flags.
docker build -t <dockerid>/<imagename>:<tag> -f “<filename>” .
docker build -t ayedaemon/my_jupyter:v1 -f “dockerfile.jupyter” .
docker container run -d -P -v $PWD/notebooks:/home/workspace --name myjenkins my_jupyter:v1
  • -P → map a random port to the exposed container. Unline -p, which is used to map exposed port to some specific port.
  • -v → to add volume to the container.
  • --name → to give some name to the container.
On Browser
  • Set up you docker hub client on your terminal by docker login command.
  • use docker push <dockerID>/<containername> command to push this container to docker hub.

Connecting the dots and rest is magic. https://ayedaemon.github.io/