MLops for beginners

Day19: Getting started with S2I (Source to Image)

What is S2I? Why you need it? How can you use it to leverage the power of complete devops life-cycle?

In my previous few articles, We have seen multiple tools that helps to complete the devops lifecycle. S2I is yet another tool that helps in achieving the same .

As the name suggests, S2I is a framework that helps us to build docker images directly from the source code!! It means we don’t need to write a Dockerfile everytime we have a new patch or new updates in the code, or a completely new project.

We can just give our project source code to this framework, it’ll do all the heavy work for us and provide us with a good ready to deploy docker image.

Let’s see what S2I maintainers say about it. (And yesss… S2I is opensource!!)

Overview

S2I produces ready-to-run images by injecting source code into a container image and letting the container prepare that source code for execution

Goals

- Reproducibility

- Flexibility

- Speed

- Security

This is just what we have always wanted. For better understanding of this tool you should read the complete README file provided on their github.

Also If you are familiar with the workflow of how a dockerfile is created and managed then the below diagram will give you more insights of how s2i does what it does.

Let’s try out the tool before we get into much details of everything how this works internally.

For the demo, I’ll be using a basic flask application.

ayedaemon/VoiceAIM-website

After looking into many documents of the openshift and s2i I got to know that we need a specific directory structure to make things work properly.

So if my project looks like this…

flask-repo
|-- run.py
|-- sampleapp/
    |-- __init__.py
    |-- views.py

… then I’ll have to make some modifications to make it look like this.

flask-repo
|-- run.py
|-- sampleapp/
    |-- __init__.py
    |-- views.py
|-- .s2i                # new folder
    |-- environment     # new file

Ques. What to write in .s2i/environment file?
Ans. APP_FILE=run.py

Now S2I will know what file is to run as entrypoint. But wait. How it’ll know what extra packages and dependencies to install in order to make my code work??

To solve this issue we need to give it a requirements file. Every programming language has their own flavor of requirements file, like for ruby there are gems file, etc. Since my code is in python so I am using requirements.txt file.

Now everything seems to be at place to make it work. We have our source code, a requirements file (that will install all the requirements for the source code), a .s2i/environment file (That tells me which is the entry point program for the project).

Now the rough idea is to use a base image, install all the required dependencies to it (using requirements.txt file) and then start the program which is mentioned in the .env/environment file). This is easy. But there is a lot more than this what S2I does for you. We’ll see that later.

(I’ll be using RHEL8 machine for the demo.)

Steps

• Install docker.

sudo dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf install docker-ce-3:18.09.1-3.el7 -y
sudo systemctl start docker

• Download and Install S2I binaries.

sudo yum install unzip git wget -y
wget https://github.com/openshift/source-to-image/releases/download/v1.3.0/source-to-image-v1.3.0-eed2850f-linux-amd64.tar.gz
tar xvf source-to-image-v1.3.0-eed2850f-linux-amd64.tar.gz

• Add binaries to bin folder, so it is accessible from any directory.

sudo cp s2i /usr/local/bin/
    or (You can do both if you are not sure)
sudo cp s2i /usr/bin

Now we are ready with the setup. It’s time to do some magic.

# This will download the source code from github and use 
# "centos/python-36-centos7" as the base image and will finally
# give you a "voiceaim-website" docker image.
sudo s2i build https://github.com/ayedaemon/VoiceAIM-website centos/python-36-centos7 voiceaim-website

# Here you run the image to create a new container and check it
sudo docker run -p 80:80 voiceaim-website

Now the fun thing I tried with this is to create a docker container that contains jenkins and S2I using docker in docker(dind) concept.

This is what my Dockerfile, for the same, looks like.

Below code might not work for your case because of the version issues of docker and the docker.repo file. You can look up to my one of the previous articles about containerized jenkins and add some required layers to it for S2I (specifically the section of download and install s2i)

from centos:latest
run yum install wget -y
# Installs jenkins
run wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo
run rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
run yum install java-11-openjdk jenkins -y
## Installs Docker
add docker.repo /etc/yum.repos.d/
run yum install docker-ce containerd.io git -y --nobest
run yum install unzip git wget -y
## download and install s2i
run wget https://github.com/openshift/source-to-image/releases/download/v1.3.0/source-to-image-v1.3.0-eed2850f-linux-amd64.tar.gz
run tar xvf source-to-image-v1.3.0-eed2850f-linux-amd64.tar.gz
run cp s2i /usr/local/bin/
run cp s2i /usr/bin
# starts jenkins
expose 8080
entrypoint ["java"]
cmd ["-jar", "/usr/lib/jenkins/jenkins.war", "--httpPort=8080"]

Or you can pull the pre-built image from my docker hub registry.

docker pull ayedaemon/s2ijenkins:1

After pulling the image you can create a container and run it

sudo docker run -d -p 8080:8080 ayedaemon/s2ijenkins:1

But why would I ever do this?? Why we need S2I with jenkins in a container?

The idea is simple enough. Let me break the idea into smaller steps to make it more understandable.

• Jenkins will download the code from github.
• S2I will build a image from that code.
• And then I’ll push the image to docker hub (Or you can do the same for your private repositories too).

In this way, whenever there is any update in my code, it’ll automatically fetch the code and then create a ready to go image out of it and we have a ready to deploy image. And this whole setup is now portable.

Steps →

• Setup docker in the system.

• Pull the S2I-jenkins container from my dockerhub (or build your own if you need some changes)

• Launch a s2i-Jenkins container

sudo docker run -d -p 8080:8080 -v /var/run/docker.sock:/var/run/docker.sock — name s2ijenk ayedaemon/s2ijenkins:1

• Download the required plugins in jenkins.

• Create a job and set up the project url

• I am setting up the build triggers as Poll SCM

• In Build Phase, we need some changes now…. We’ll use docker build and publish along with execute shell as in earlier examples.

• First build phase is to build the image using s2i jenkins

mkdir -p /dockerfiles
s2i build . centos/python-36-centos7 ayedaemon/voiceaim_website --as-dockerfile /dockerfiles/Dockerfile

• Second build phase is to push the image to docker hub.

add the path to the dockerfile too (check in advance section)

• And once the build is complete, You can see the changes in the docker hub.

Here are some resources which helped me to begin with S2I.

Source-to-Image (S2I) Deep Dive with Ben Parees - OpenShift Commons Briefing #43

Getting started with Source-to-Image for building container images

openshift/source-to-image

How to Create an S2I Builder Image

geordgez/flask-s2i

And done…!!