MLops for beginners
Day19: Getting started with S2I (Source to Image)
What is S2I? Why you need it? How can you use it to leverage the power of complete devops life-cycle?
In my previous few articles, We have seen multiple tools that helps to complete the devops lifecycle. S2I is yet another tool that helps in achieving the same .
As the name suggests, S2I is a framework that helps us to build docker images directly from the source code!! It means we don’t need to write a Dockerfile everytime we have a new patch or new updates in the code, or a completely new project.
We can just give our project source code to this framework, it’ll do all the heavy work for us and provide us with a good ready to deploy docker image.
Let’s see what S2I maintainers say about it. (And yesss… S2I is opensource!!)
S2I produces ready-to-run images by injecting source code into a container image and letting the container prepare that source code for execution
Also If you are familiar with the workflow of how a dockerfile is created and managed then the below diagram will give you more insights of how s2i does what it does.
Let’s try out the tool before we get into much details of everything how this works internally.
For the demo, I’ll be using a basic flask application.
Website for VoiceAIM. Contribute to ayedaemon/VoiceAIM-website development by creating an account on GitHub.
After looking into many documents of the openshift and s2i I got to know that we need a specific directory structure to make things work properly.
So if my project looks like this…
… then I’ll have to make some modifications to make it look like this.
|-- .s2i # new folder
|-- environment # new file
Ques. What to write in
Now S2I will know what file is to run as entrypoint. But wait. How it’ll know what extra packages and dependencies to install in order to make my code work??
To solve this issue we need to give it a
requirements file. Every programming language has their own flavor of requirements file, like for ruby there are gems file, etc. Since my code is in python so I am using
Now everything seems to be at place to make it work. We have our source code, a requirements file (that will install all the requirements for the source code), a .s2i/environment file (That tells me which is the entry point program for the project).
Now the rough idea is to use a base image, install all the required dependencies to it (using
requirements.txt file) and then start the program which is mentioned in the
.env/environment file). This is easy. But there is a lot more than this what S2I does for you. We’ll see that later.
(I’ll be using RHEL8 machine for the demo.)
- Install docker.
sudo dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.reposudo dnf install docker-ce-3:18.09.1-3.el7 -ysudo systemctl start docker
- Download and Install S2I binaries.
sudo yum install unzip git wget -ywget https://github.com/openshift/source-to-image/releases/download/v1.3.0/source-to-image-v1.3.0-eed2850f-linux-amd64.tar.gztar xvf source-to-image-v1.3.0-eed2850f-linux-amd64.tar.gz
- Add binaries to
binfolder, so it is accessible from any directory.
sudo cp s2i /usr/local/bin/ or (You can do both if you are not sure)sudo cp s2i /usr/bin
Now we are ready with the setup. It’s time to do some magic.
# This will download the source code from github and use
# "centos/python-36-centos7" as the base image and will finally
# give you a "voiceaim-website" docker image.sudo s2i build https://github.com/ayedaemon/VoiceAIM-website centos/python-36-centos7 voiceaim-website
# Here you run the image to create a new container and check itsudo docker run -p 80:80 voiceaim-website
Now the fun thing I tried with this is to create a docker container that contains jenkins and S2I using docker in docker(dind) concept.
This is what my Dockerfile, for the same, looks like.
Below code might not work for your case because of the version issues of docker and the docker.repo file. You can look up to my one of the previous articles about containerized jenkins and add some required layers to it for S2I (specifically the section of
download and install s2i)
from centos:latestrun yum install wget -y# Installs jenkinsrun wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo
run rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
run yum install java-11-openjdk jenkins -y## Installs Docker
add docker.repo /etc/yum.repos.d/
run yum install docker-ce containerd.io git -y --nobest
run yum install unzip git wget -y## download and install s2irun wget https://github.com/openshift/source-to-image/releases/download/v1.3.0/source-to-image-v1.3.0-eed2850f-linux-amd64.tar.gzrun tar xvf source-to-image-v1.3.0-eed2850f-linux-amd64.tar.gz
run cp s2i /usr/local/bin/
run cp s2i /usr/bin# starts jenkins
cmd ["-jar", "/usr/lib/jenkins/jenkins.war", "--httpPort=8080"]
Or you can pull the pre-built image from my docker hub registry.
docker pull ayedaemon/s2ijenkins:1
After pulling the image you can create a container and run it
sudo docker run -d -p 8080:8080 ayedaemon/s2ijenkins:1
But why would I ever do this?? Why we need S2I with jenkins in a container?
The idea is simple enough. Let me break the idea into smaller steps to make it more understandable.
- Jenkins will download the code from github.
- S2I will build a image from that code.
- And then I’ll push the image to docker hub (Or you can do the same for your private repositories too).
In this way, whenever there is any update in my code, it’ll automatically fetch the code and then create a ready to go image out of it and we have a ready to deploy image. And this whole setup is now portable.
- Setup docker in the system.
- Pull the S2I-jenkins container from my dockerhub (or build your own if you need some changes)
- Launch a s2i-Jenkins container
sudo docker run -d -p 8080:8080 -v /var/run/docker.sock:/var/run/docker.sock — name s2ijenk ayedaemon/s2ijenkins:1
- Download the required plugins in jenkins.
- Create a job and set up the project url
- I am setting up the build triggers as Poll SCM
- In Build Phase, we need some changes now…. We’ll use docker build and publish along with execute shell as in earlier examples.
- First build phase is to build the image using s2i jenkins
mkdir -p /dockerfiless2i build . centos/python-36-centos7 ayedaemon/voiceaim_website --as-dockerfile /dockerfiles/Dockerfile
- Second build phase is to push the image to docker hub.
- And once the build is complete, You can see the changes in the docker hub.
Here are some resources which helped me to begin with S2I.
Source-to-Image (S2I) Deep Dive with Ben Parees - OpenShift Commons Briefing #43
In this Video In this latest OpenShift Commons Briefing, Ben Parees, lead developer at Red Hat on Source-to-Image (S2I)…
Getting started with Source-to-Image for building container images
Source-to-Image is an excellent tool for building container images for applications in a fast, flexible, and…
Source-to-Image (S2I) is a toolkit and workflow for building reproducible container images from source code. S2I…
How to Create an S2I Builder Image
Source-To-Image (S2I) is a standalone tool which is very useful when creating builder images. It also happens that S2I…
This repo shows how to migrate an existing Flask app to one that can use OpenShift's tool to create a Docker image of…