DevOps:(P1/2)CI-CD Pipeline With Jenkins, Python, Docker, GitHub, DockerHub
DevOps:(P1/2)CI-CD Pipeline With Jenkins, Python, Docker, GitHub, DockerHub
What actually is CI/CD? Continuous integration (CI), continuous delivery (CD) and continuous deployment (CD). In this session lets go thru each of them and see a live working example of CI/CD in actual world. We use Python Flask application, with docker, Github and Jenkins as the main tool which is running in a Linux server.
In this video, we get to know all the components used in the pipeline, please also refer p2 of the video for the Jenkins CI/CD declarative pipeline where we go into each step/stages of the pipeline.
GitHub Repo: https://github.com/kss7/SimpleFlaskUI
Hit the Like button if this was helpful!
Thanks🙏
Part2 CI-CD Pipeline With Jenkins: • DevOps:(P2/2)CI-CD Pipeline With Jenk…
+++++++++++++
About SSH-agent Plugin: • Using SSH Agent Plugin On Jenkins | L…
VirtualBox Installation Windows: • VirtualBox Installation: Install Orac…
+++++++++++++
THANKS A LOT FOR WATCHING!! Please give this video a 👍 THUMBS UP if you found it helpful and 📢 COMMENT if you would like to say something or ask a question. This will motivate me to make more such videos 🙏🙏
BTW, DON’T FORGET TO SUBSCRIBE!!
+++++++++++++
0:00 Intro
00:33 CI/CD Definition
01:03 Continuous Delivery def
02:29 Continuous Deploy def
04:20 CICD Pipeline
06:04 Software Tools
07:48 Python Flask Application
08:18 Running application in Pycharm
09:40 Dockerfile
10:11 Building Docker Image
11:21 run docker Image
12:19 Github repo
12:50 About Jenkins server
13:25 Docker daemon permission issue
14:11 Jenkins plugins
15:25 Jenkins Credentials
16:41 Summary \u0026 Conclude
+++++++++++++
#devopsCICD
#PythonDocker
#JenkinsPipeline
Content
0.16 -> hi welcome i'm kumar thanks for joining this
video session on ci cd with jenkins in this video
6.48 -> we will see about ci cd with jenkins docker Git
docker hub and with python code, most of devops
13.2 -> components most of the components in the devops
environment is covered and what actually we do
17.52 -> in devops we'll see that in this video session
without further ado let's get started on this so
22.8 -> what are the main things in devops and cicd let's
try to understand a bit of ci what we do in there
28.96 -> what we do in cd continuous delivery what we do in
continuous deployment so in continuous integration
34.32 -> we know that we merge all our code into the master
branch or the main project branch so developers
39.04 -> might be working on different branches of their
own for example a feature branch or a hotfix or
44.32 -> some defect fixes branch so they would be merging
all their code in the master branch and then the
48.96 -> code is getting reviewed before getting merged
and then it's getting build compiled build and
54 -> unit tests are done and integration tests are done
so that's what happens in a minimal continuous
58.8 -> integration stage of the ci cd or in the whole
devops environment okay so the continuous delivery
65.28 -> it's the next stage of continuous integration
and it's supposed to be automatically deploying
70.96 -> all the code changes to a testing or production
like environment after the build stage of course
77.36 -> continuous delivery is actually the process
where teams release in an automated way the
82.56 -> applications they develop to the qa environment
so basically you are delivering the application
88.96 -> to a qa environment or a test environment to be
tested by the qa team of course in simple terms
94.96 -> actually this means that there is an automated
release process on top of the automated testing
99.44 -> process and that developers can deploy their
changes at any time by simply clicking a button
105.04 -> after the completion of the continuous integration
so you don't manually make a release by putting
110.08 -> the artifact or build package to a specified
location is getting deployed in an automated way
115.28 -> on a qa server it may also be a tomcat server or
any other web server or application server we are
121.76 -> delivering the build to be consumed by whoever the
case may be in this case this is the qa we will be
127.84 -> using this build for their testing and then in the
continuous delivery also we see that the bills are
133.28 -> delivered for every fixes or whenever there is
a fix whenever there is some changes in the code
138.32 -> and then bills are getting deployed it's kind of
you are delivering the bills every every build or
143.36 -> every fix or every changes whatever the developers
whatever the developer does it goes to the qa
148.32 -> servers or the test servers next is the continuous
deployment so the continuous deployment takes
154.24 -> the process one step ahead of the continuous
delivery so it sounds similar to continuous
159.92 -> delivery cd but in case of continuous deployment
it is the actual deployment that happens in the
165.68 -> production environment in an automated way so
in most cases after delivery of the application
171.68 -> there is a final a manual approval step before
production goes live or production deployment you
177.68 -> can say the continuous delivery stage is complete
once it is deployed and tested successfully in the
182.64 -> keyword labs or your engineering environment
there is well in most cases a manual approval
188.16 -> required from stakeholders or customers before
deployment can be done in the staging production
194.32 -> environments wherein again the automated test
a set of automated tests are run before final
199.36 -> push to production or before it goes live or opens
to all the customers think of it in terms of once
205.04 -> you've ordered a tv and it's delivered at your
doorstep but you need to approve it by reviewing
210 -> and signing some documents so once it's manually
approved by you it's then being used or put
215.52 -> into work now coming back to our cicd process the
project team will review merge code do all types
221.04 -> of testing and then package it into an artifact
or image this package is then moved to production
225.68 -> like environment say staging environment
where it awaits approval to be opened for
230.32 -> deployment in this deployment phase the package
is opened and reviewed with automated checks or
236.64 -> tests if the checks fail the package is rejected
of course and if the check passes the package is
241.68 -> automatically deployed to production or made live
or open to all the customers checks means the set
247.44 -> of tests you can call it uat from customer's point
of view that actually runs on the final build
252.56 -> mostly automated test here so that's about the ci
cd continuous integration and continuous delivery
257.76 -> continuous deployment a brief intro on that so
this is the flow which we are going to see in this
262.96 -> video session most of the things we have already
done so mostly i will be covering the jenkins the
269.52 -> cicd pipeline the declarative pipeline which which
we are going to see in this video session usually
274.8 -> in a cicd flow whenever a developer checks in
the code into one of the repository servers
280.96 -> in this case we are going to see it on github
whenever the developer checks in the code
285.44 -> in the repository server the jenkins bill gets
triggered and most of the tasks are being done
291.6 -> by the jenkins server so a developer checks
in a code into github and then the jenkins
297.04 -> will be listening to that and then jenkins will
clone that code and it will make the docker image
302.16 -> and make the docker build and it will deploy
the image in a qa server so in in this case we
308.56 -> are going to deploy the image in the same
machine where jenkins is running and then
313.04 -> if it is successful then it is going to publish it
to the docker hub and then the next stage jenkins
320.64 -> will actually deploy this image to this
server so i'm trying to mimic the whole
327.6 -> flow of ci cd here starting from checking out
code and then deploying to one of these servers
334.64 -> in a usual ci cd environment basically this is how
the things work so i'm trying to mimic that in our
341.28 -> local uh systems so my server would be running
actually ubuntu 20 server here i will deploy the
348.4 -> docker build and then it will start the image on
this ubuntu server we'll see that by accessing the
355.28 -> console of the of our application for doing this
we are going to need all these software components
360.96 -> and all so let us see that what all the things
what are the things which we are going to need
364.8 -> so the first thing which we will need is python
3x we are using python 3x and of course pycharm
370.8 -> for making the software pycharm ide for writing
the code the next thing which we will need is
376.48 -> the docker which will be running in my windows
both python and docker are running on my will be
381.6 -> running on my windows machine and then of course
you will need a docker hub account to push in your
387.36 -> code into docker hub registry next thing is we
will need a git in your windows in our windows
393.36 -> machine and git bash or github desktop i use
both of them both git bash and github desktop for
400.16 -> pushing our code into the git repository also
we will need a github account to store our code
406.88 -> in the github repository of course it's free so
anybody can create a account go ahead and create
411.76 -> an account if you don't have one next we will need
a jenkins server which will be running on a linux
416.56 -> machine i'm in my case i'm running it in a virtual
box in my same windows and it's running the ip is
423.36 -> accessible through my windows host machine since
i've configured my linux machine as a bridged
430.08 -> network also we'll need a linux server with
docker installed so this linux server serves as
436.64 -> my production server so i will be deploying my
docker image into this server and run it and
443.84 -> we'll see how we can access that application
through our browser well there's too many
448.32 -> of these components but you know it's the same
almost here city is actually all about cooking
452.96 -> or stitching of the whole solution with different
softwares and services so before starting to write
459.36 -> the jenkins pipeline let me show you about the
application which we are going to use and which we
465.2 -> are going to dockerize and run it so let me open
my pycharm okay so this is the application this is
470.72 -> actually a very very simple flask application
and it will just present you two pages in this
477.36 -> application i'll show you that so this application
has got most of the things basically which we need
483.12 -> in an ui application it has got some css elements
and then also we have got some html pages in here
490.08 -> of course the main file is this one so let us try
to run this application and then i'll show you
496.72 -> actually what is this application about so python
my app dot py main file and let me run this and
505.36 -> okay so it got started so this
is the ip which it's showing
509.28 -> ip for my internet connection but can actually
use a localhost 127 12701 so let me 127.0.0.1
520.08 -> and then the port is 5000 and this is the home
page welcome to the flask app so this is one of
526.96 -> the pages and the other page is actually hello
page so this is the other one hello and then we
533.6 -> can provide some some string here so let me just
put and hit enter so you can see here hello cases
539.68 -> and this is the other page and this site has been
visited one time so whenever you refresh this page
545.92 -> this count keeps on increasing and i'm storing
this account in a file i'll show you that so
551.76 -> you see here let me just do a refresh and you see
this count gets increased so if i come to python
558.16 -> and it would be saving it in this file so
count.txt so let me do one thing let me
563.92 -> stop this application and what i can do is i can
also clear this to make it a zero so that whenever
571.2 -> next time i run it i get a one in there otherwise
it will start from the next number okay so now let
578.16 -> us build the docker image for this so for that i
have already written a docker file so this is the
582.88 -> docker file and so in this docker file are using
the python the smallest python 3 python 3.8 and
590 -> then i'm using this slim buster because this is
on the smallest python image and the other things
595.84 -> like add add all the files to this folder and then
building this image and the other component code
602.24 -> about building the image okay yeah and then
finally when command is basically when you
607.28 -> run this image what command it should run it's
all about that in the last line so let us build
611.68 -> this docker image so for building docker image
docker okay i cleared the screen so let me make
617.52 -> this bigger i cleared this window so we need to
do docker build minus t for tag my by app and then
627.52 -> give a tag and then with the docker file let's
see it's in the same folder so let's give a dot
633.36 -> here and hit enter okay so looks like my docker
is not yet started so let me start the docker
641.92 -> desktop app so let's see okay so this is my
docker engine this windows machine and looks
650.88 -> like it's starting now okay so i have my docker
engine started this windows machine to minimize
656.64 -> this now i will run this command and now it's
building this okay so it looks like it completed
663.92 -> uh building my docker image so i cannot see it
from here but i can go to the docker desktop
669.52 -> and go to the images you see here my pi app
it shows up starts to show in here and 0.1
676.56 -> is the version and this was built a minute back so
and the size is 144 mb let me try to run it from
683.04 -> console by jump console so so this is the
command which will use docker run minus d
688.48 -> then minus port 5000 was the port and
we'll again use the 5000 port externally
693.44 -> and this is my app so let me start this okay
so now let's go to the docker engine docker
700.48 -> desktop and let's see that here see now we see one
container is running on port 5000 now let us go to
710.32 -> our browser again and let us go back and let us
refresh this page and let us do a hello and let's
720.16 -> put some other name jenkins and now it goes to
visit at one time so let us do it okay it keeps
727.6 -> on increasing so this is fine now so let's come
back here and let's do one thing let's stop this
733.12 -> one and it got exited i can probably undelete this
like to keep my system clean okay the next thing
740.88 -> is the github code base get repository actually
so i have already posted my code into github
747.04 -> and this is the github repo simple flask ui i'll
put this link in the description under this video
754.88 -> and i have used the github desktop to push in this
code i can do that from gitbash also but um i use
762.4 -> github for most of the pushing and pulling seems
to be more easier in windows machine working with
769.2 -> github repository so okay so the next one is the
jenkins server so my jenkins server is running on
776.24 -> this linux vm in this virtual box so this is my
virtualbox and i'm running this centos machine
783.6 -> centos 7 actually and this is my jenkins server
system ctl status jenkins and my jenkins server
790.8 -> is running in this machine with 192 168 1.15 ip
address and once i log in here okay and login
799.36 -> into this uh jenkins server and let me also show
you the jenkins configuration okay so before that
807.76 -> let me tell you one thing in the jenkins in
the linux machine we need to install git and
813.84 -> we need to install docker after installing
both of them you need to run the user mode
821.12 -> command user mode minus or dash a g docker jenkins
and then you need to restart your jenkins server
831.04 -> so otherwise you will get a permission denied
issue permission denied error while you try to
838.48 -> connect to the docker demon so we need to run
this command which you see on the screen basically
844.8 -> after running this command you are allowing
jenkins user to access the docker socket next
852 -> we'll see the plugins in the jenkins so for that
we have to go to jenkins manage jenkins and we can
860.64 -> go to manage plugins i have already installed
it here by the way so can cloud so you need to
869.36 -> install this cloud bs docker build and publish
plugin this is one of them and the other one
877.12 -> is you need to install ssh agent plugin so sss
asset plugin is actually used for pushing my
886.72 -> docker image to the remote server to the remote
server which in my case will be this ubuntu 20
894.72 -> machine so if you remember in this diagram this
is server this in my case is the production server
901.6 -> so this is a ubuntu 220 machine i would be pushing
my docker image to this server and my sss agent
908.56 -> plugin will be helping me to connect to that
remote server and running the docker image in that
915.52 -> server of course after pulling it down from docker
hub so my jenkins pipeline will actually do all
920.88 -> those those code so going back to jenkins let's go
back again to manage jenkins so we need to add the
927.6 -> credentials for connecting to the remote docker
hub docker hub and the other ubuntu server so in
935.6 -> this case i've set an idea of docker test but let
me show you so once you come here and let me show
942.24 -> you this one so this configuration we need to do
in the in the jenkins we need to select global
947.92 -> jenkins and put an id description and username
in here and we need to put the ssh keys which
955.12 -> you need to generate i already have a video for
this how to generate ssh keys and how to use it
960.8 -> in the ssh agent so you can go through that video
i'll put it in the description under this video so
966.48 -> we need to add these credentials in jenkins and
then the other credential which we need to put
972.88 -> is for the docker hub login so if i come here
and let me show you this one as well so for my
979.2 -> docker hub login i need to put in my username
and password and then the docker have login id
985.76 -> and the description okay so this is just the login
information but the most of the other things will
991.2 -> go into that pipeline definition coming back again
this is my whole jenkins pipeline which i would be
997.28 -> going through each of these steps while letting
it down in the jenkins pipeline so that is about
1002.08 -> the whole setup the all the necessary things which
i would be using and all the configurations which
1007.28 -> is required for creating this whole pipeline so
that's it in this video i wanted to do the whole
1012.8 -> session in one video but as this is already over
16 minutes now i decided to make it in two parts
1018.08 -> you know it's easier to learn or grasp in smaller
modules so see you in the next video where i will
1023.6 -> go through the pipeline steps in detail it's going
to be super informative, thank you, do hit the
1028.16 -> like button it surely motivates me to be committed
and please do subscribe for more such videos
Source: https://www.youtube.com/watch?v=jHkbtzemsGs