How to Automate service deployment to Docker Swarm using Jenkins

Introduction

Jenkins is a wonderful tool for continuous integration and continuous deployment. The plethora of plugins available makes it really powerful. In this tutorial, I will show you how to use Jenkins to automate swarm deployment.

How to do it

To do a Docker Swarm deployment all you need is a docker-compose file which will contain the references to docker images along with the configuration settings like port, network name, labels, constraints etc and to run this file you need to execute a command called “docker stack deployment”

So all we need to do is to send this docker-compose.yml file over SSH to Manager node and execute the command 

docker stack deploy 

Jenkins plugin – Publish over SSH

We need to install a Jenkins plugins “Publish over SSH”, this plugin will allows us to

• Sends files over SSH(SFTP)
• Execute commands on a remote server

To add this plugin you need to go to Jenkins -> Manage Jenkins -> Manage Plugins -> Available and search for “Publish
Over SSH”. Install this plugin and restart Jenkins.

Adding Remote Hosts

Navigate to Jenkins -> Manage Jenkins -> Configure System and scroll down until you find Publish over SSH.

Since we need to execute the docker stack deploy command on any manager node, we need to connect to a manager node in Docker Swarm. This plugin offers various ways to connect to remote hosts, I prefer SSH public/private key value pair. Keys can be generated with ssh_keygen. The private key must be kept on Jenkins server and the public key must be stored on the manager node.

Click on Test configuration and see if the connection is successful.
Have a look at the screenshot below ( please note that Remote Directory contains a swarm directory if it doesn’t exist either leave Remote Directory as blank or create a directory on manager node, in the next step we will use this directory to publish docker-compose file)

You can add multiple manager nodes if you want.

Configuring the Jenkins Job

In this step, we need to tell Jenkins from where to get our docker-compose file and how to transfer using SSH to remote server and execute subsequently.

• • Under Source Code Management and add the repository(GIT/SVN) where you have checked-in or stored the docker-compose file.
  • Under Build section Select “Add Build Step” -> “Send files or execute commands over SSH”

Now Under SSH server Select the manager node where you want to publish/send the docker-compose file. In the Transfers Set block,

• In the Source, files enter the path for the docker-compose file. This folder is relative to your Jenkins Workspace so if say docker compose file exists in directory structure as swarm/dev/docker-compose.yml it can be written as swarm/dev/**/*
• In the Remove Prefix, enter the path that should not be created on the remote server.
• Now In exec Command, enter the command as shown below ( Please note that cd /swarm is only needed in if remote the configuration you have added swarm as a remote directory)

  cd /swarm docker stack deploy -c docker-compose.yml

This is how my configuration looks like for reference.

Run the Jenkins Job

Now run this Jenkins Job using Build now and then check in Console Output to see the output of remote server. Hopefully, it will run fine !!

The post How to Automate Docker Swarm Service deployment using Jenkins appeared first on Little Big Extra.

Convert JSON schema to Java classes using maven plugin

Introduction

We often need to convert JSON schema’s to Java classes, there is a jsonschema2pojo-maven-plugin which helps in generating POJO(plain old java objects) from JSON or JSON schema.

To use it all we need is add this plugin, update dependencies and run mvn generate

Add jsonschema2pojo-maven-plugin in POM.XML

Add this plugin to your POM.XML under plugin section

<plugin>
				<groupId>org.jsonschema2pojo</groupId>
				<artifactId>jsonschema2pojo-maven-plugin</artifactId>
				<version>0.4.34</version>
				<configuration>
					<sourceDirectory>${basedir}/src/main/resources/schema</sourceDirectory>
					<targetPackage>com.test.gen</targetPackage>
					<useCommonsLang3>true</useCommonsLang3>
				</configuration>
				<executions>
					<execution>
						<goals>
							<goal>generate</goal>
						</goals>
					</execution>
				</executions>
			</plugin>

Add Maven dependencies

The generated types depend on Commons Lang library for equals, hashCode and toString. Also, you need to add Jackson-databind which contains the data-binding functionality and tree-model for Jackson Data Processor, this jar has a transitive dependency on jackson-core and jackson-annotations so they will be downloaded automatically.

<dependency>
			<groupId>org.apache.commons</groupId>
			<artifactId>commons-lang3</artifactId>
			<version>3.0</version>
		</dependency>
		
		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-databind</artifactId>
		</dependency>

Run the Maven Goal

You can run

mvn generate-sources

${basedir}/src/main/gen

Make sure that generated folder is added as a source folder in your IDE (Properties ->Java build Path -> Add source folder in eclipse).

Generating classes from JSON and not from schema

If you want to generate Java classes from JSON rather than JSON schema add

<sourceType>json</sourceType>

inside the configuration tag as default source type is JSON Schema.

The post How to convert JSON schema to Java classes using maven plugin appeared first on Little Big Extra.

How to use Jenkins to Build and Deploy docker images without Jenkins Docker plugin

Introduction

Jenkins is one of the most popular continuous integration tools and is widely used. It can be used to create pipelines, run jobs and do automated deployments.
There are many plugins available which can be added to Jenkins and makes it really powerful.

Recently I wanted to do an automated deployment of Docker Images to docker server and tried using docker-plugin but after spending some time it looked to me that if it asking for too much of information and for each operation you need to provide arguments, I would prefer a solution which is more dynamic and picks things automatically with user providing the bare essentials parameters arguments.
This is how it looks like

 

How to do it

• Well, the first step is is to enable Jenkins to auto build with BitBucket server or other code repository server. If you want your build process to start as soon as the code is committed to GIT repository.
  Here is an example of how to enable with BitBucket Server, for other GIT repositories servers like GITHUB Jenkins does have different plugins.In case you want to build process to start manually or periodically you can skip this step.
  Step-1: Enable Jenkins to auto build with BitBucket server
• Once the build is automated.The second step is to enable Remote API on Docker Host.Docker provides remote REST API which is beneficial if you want to connect to a remote docker host, this API allows you do to set of operations from creating and managing containers to start to stop containers.This step is pivotal before we move on to next step.
  Step-2: Enable Remote API on Docker Host
• The third step is to modify your pom to build and deploy Docker Image, well this is the key step as in this step you will be literally doing all the stuff from creating the image to build the image and start the container.
  Step-3: Modify POM to Build and Deploy Docker Image

  Once all 3 steps have been tested and completed all is left to be done in Jenkins is to Invoke the clean install goal as shown below.

If you want to push the docker image to some repository after building, testing and deploying.Please follow this link

Push a Docker image to DockerHub using docker maven plugin

Share this article on

The post Using Jenkins to Build and Deploy Docker images appeared first on Little Big Extra.

]]> http://littlebigextra.com/using-jenkins-to-build-and-deploy-docker-images/feed/ 0 http://littlebigextra.com/push-docker-images-docker-hub-using-maven/ http://littlebigextra.com/push-docker-images-docker-hub-using-maven/#comments Sat, 25 Mar 2023 12:15:07 +0000 http://littlebigextra.com/?p=501 Push a Docker image to DockerHub using docker maven plugin fabric8.io   Introduction If you want to push a docker image to Docker Hub repository, it can be achieved using docker maven plugin from fabric8.io. This plugin lets you build images, start and stop containers and push it to Docker repositories In case you are wondering […]

The post How To Push Docker Images To Docker Hub Repository Using Docker Maven plugin appeared first on Little Big Extra.

]]>

Share this article on

Push a Docker image to DockerHub using docker maven plugin fabric8.io

Introduction

If you want to push a docker image to Docker Hub repository, it can be achieved using docker maven plugin from fabric8.io. This plugin lets you build images, start and stop containers and push it to Docker repositories

&nsbp;

1. For the below example, I have used Spring Boot Rest service project which is connected to MongoDB. See below link for reference.
2. Using io.fabric8 plugin, under the plugin section add the io.fabric8 plugin
   

   <plugin>
   	<groupId>io.fabric8</groupId>
   	<artifactId>docker-maven-plugin</artifactId>
   	<version>0.20.0</version>....

   
   
3. We need to first build the image from Dockerfile, this plugin supports all the command which are in Dockerfile and you can directly create a docker image by giving arguments in pom.xml but In my view making docker images from Docker file is much easier and simpler. So we will ask maven plugin in below step to look for Dockerfile in root( project base directory)
   

   <name>springboot-mongo-dockerimage:${project.version}</name>
       <alias>springboot-mongo-dockerimage</alias>
   <build>
       <dockerFileDir>${project.basedir}</dockerFileDir>
   </build>

4. Next we specify the docker registry address
   

   <registry>registry.hub.docker.com/YOUR_DOCKER_HUB_USERNAME</registry>

5. Now we need to add authentication credentials in <authconfig> tag
   

   <authConfig>
   	<username>ENTER YOUR DOCKER HUB USERNAME LIKE abhishek</username>
   	<password>ENTER YOUR DOCKER HUB PASSWORD</password>
   </authConfig>

6. We will add execution phases, where we will push the image
   

   <execution>
   	<id>push</id>
   	<phase>post-integration-test</phase>
   	<goals>
   		<goal>push</goal>
   	</goals>
   </execution>

7. Now run the pom.xml using mvn clean install and check your docker hub repo after some time to see if image has been pushed. If you are pushing the image for first time its gonna take some time  depending on the size of image.

Here is the complete build section from the plugin, hope this help.

<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
			<plugin>
				<groupId>io.fabric8</groupId>
				<artifactId>docker-maven-plugin</artifactId>
				<version>0.20.0</version>
				<configuration>
					<registry>registry.hub.docker.com/YOUR_DOCKER_HUB_USERNAME</registry>

					<images>
						<image>
							<name>springboot-mongo-dockerhub:${project.version}</name>
							<alias>springboot-mongo-dockerhub</alias>
							<build>
								<dockerFileDir>${project.basedir}</dockerFileDir>

							</build>
							<run>
								<namingStrategy>alias</namingStrategy>
								<dependsOn>
									<container>mongo</container>
								</dependsOn>

								<links>
									<link>mongo</link>
								</links>
								<ports>
									<port>9876:8080</port>
								</ports>
								<log>
									<prefix>TC</prefix>
									<date>default</date>
									<color>cyan</color>
								</log>
							</run>
						</image>
					</images>
					<authConfig>
						<username>ENTER YOUR DOCKER HUB USERNAME LIKE abhishek</username>
						<password>ENTER YOUR DOCKER HUB PASSWORD</password>
					</authConfig>
				</configuration>

				<executions>
					<execution>
						<id>start</id>
						<phase>pre-integration-test</phase>
						<goals>
							<goal>stop</goal>
							<goal>build</goal>
							<goal>start</goal>
						</goals>
					</execution>
					<execution>
						<id>push</id>
						<phase>post-integration-test</phase>
						<goals>
							<goal>stop</goal>
							<goal>push</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>

The post How To Push Docker Images To Docker Hub Repository Using Docker Maven plugin appeared first on Little Big Extra.

Build a Docker Image and Run a Docker Container with docker maven plugin fabric8.io

Introduction

 The maven plugin for docker fabric8io/docker-maven-plugin helps us in building docker images and running containers. In case you are thinking about the difference between an image and container then please note that a running image is called container.

1. For below steps, I have used Spring Boot Rest service project which is connected to MongoDB. See below link for reference.

   Read Here: How to Connect to Spring Boot Rest Service Image to MongoDB

2. Using io.fabric8 plugin, under the plugin section add the io.fabric8 plugin
   

   <plugin>
   	<groupId>io.fabric8</groupId>
   	<artifactId>docker-maven-plugin</artifactId>
   	<version>0.20.0</version>....

3. We need to first build the image from Dockerfile, this plugin supports all the command which are in Dockerfile but I think making docker image from Docker file is much easier and simpler. So I telling the maven plugin in below step to look for Dockerfile in root( project base directory)
   

   <name>springboot-mongo-dockerimage:${project.version}</name>
       <alias>springboot-mongo-dockerimage</alias>
   <build>
       <dockerFileDir>${project.basedir}</dockerFileDir>
   </build>

4. Next step is to specify what docker container needs to do when it runs.
   1. Since we want to make sure that mongo DB container should be running before this image starts running. We add a tag called <dependsOn>.
   2. If the MongoDB is started we would like to link this container to Mongo so both can talk to each other, we specify that by adding a tag called <link>
      

      <dependsOn>
      	<container>mongo</container>
      </dependsOn>
      <links>
      	<link>mongo</link>
      </links>

   3. Next, we will specify on which port we would like to run this container and which port it should map to localhost machine
      

      <ports>
      <port>9876:8080</port>
      </ports>

   4. We enable all the logging using <log> tag, so we could see any errors or debug if some problem arises.
5. Next, we will add execution phases, we will cover these steps
   1. Remove an existing image, if it exists
   2. Build the Image
   3. Deploy the image
      

      <executions>
      	<execution>
      	        <id>start</id>
      		<phase>pre-integration-test</phase>
      		<goals>
                          <goal>stop</goal>
      		    <goal>build</goal>
      		    <goal>start</goal>
      							
      	</goals>
      </execution>

6. Now run this pom.xml using mvn clean install and check your docker containers so see if it running.

Here is the complete build section from the plugin, hope this help.

<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
			<plugin>
				<groupId>io.fabric8</groupId>
				<artifactId>docker-maven-plugin</artifactId>
				<version>0.20.0</version>
				<configuration>
					<!--<dockerHost>tcp://REMOTE_IP:2375</dockerHost>-->
					<images>
						<image>
							<name>springboot-mongo-dockerimage:${project.version}</name>
							<alias>springboot-mongo-dockerimage</alias>
							<build>
								<dockerFileDir>${project.basedir}</dockerFileDir>
 
							</build>
							<run>
								<namingStrategy>alias</namingStrategy>
								<dependsOn>
									<container>mongo</container>
								</dependsOn>
 
								<links>
									<link>mongo</link>
								</links>
								<ports>
									<port>9876:8080</port>
								</ports>
								<log>
									<prefix>TC</prefix>
									<date>default</date>
									<color>cyan</color>
								</log>
							</run>
 
						</image>
					</images>
				</configuration>
 
				<executions>
					<execution>
						<id>start</id>
						<phase>pre-integration-test</phase>
						<goals>
							<goal>stop</goal>
							<goal>build</goal>
							<goal>start</goal>
						</goals>
					</execution>
                               <!-- Uncommment this execution phase if you wan to remove container after testing ->
					<!--<execution>
						<id>clean image</id>
						<phase>post-integration-test</phase>
						<goals>
							<goal>stop</goal>
						</goals>
					</execution> -->
				</executions>
			</plugin>
		</plugins>
	</build>

The post Build and deploy Docker Image with Docker Maven plugin appeared first on Little Big Extra.

 Integrate BitBucket webhook with Jenkins to auto build on push of code to a bitbucket repository

One of the goals of Continuous Integration andContinuous deployment is that once the code is pushed to the repository it should be tested, built and then packaged for deployment. BitBucket webhook makes is possible to trigger real-time notifications to Jenkins server so the auto build starts as soon as the code is pushed.

Add Jenkins add on to BitBucket server

Once you have installed BitBucket server on your machine, make sure that it is working. The default port is 7990, so you should be able to access it by using localhost: 7990.

• Click on Settings and on the left side under ADD-ONS, click on Find new add-ons.
• Search Jenkins and add the plugin

Install BitBucket Plugin in Jenkins

• Make sure Jenkins is up and Running, in my case, it is running on localhost: 8080
• In Jenkins go to Jenkins->Plugin Manager and install BitBucket Plugin and then restart Jenkins.

Configure Jenkins

• Now go the Job, which you have created and under Source Code Management Plugin, give the GIT Clone URL as shown below in the screenshot.
  

•  Under Build, Triggers make sure two boxes are checked
  a. Build when a change is published to BitBucket
  b. Poll SCM

Configure BitBucket

• Now go to BitBucket -> Repositories and select the repo for which are Jenkins has been configured

• Click on the setting (icon on the left-hand side) and then hooks. Edit the Bitbucket Server Webhook to Jenkins
  a. Enter your Jenkins URL
  b. Repo Clone URL: Select the URL and the same protocol which BitBucket is providing you to clone. In case username is missing from the URL, we need to add it manually.I had to use Chrome Developer tools and remove the disabled flag from the input box and then add username manually.
  Ignore above step if the username is in URL
  c. Once it is enabled edit the URL with http://UserName@Server/gitrepo

• Click on Trigger Jenkins if it says successful everything is fine.
  

•  Commit some code and Jenkins should automatically start the build. Refresh the Jenkins page is auto refresh is disabled

The post How to enable jenkins to auto build with BitBucket server appeared first on Little Big Extra.

Installing Docker on Jenkins

Jenkins is a great tool to build and package artifacts, it can be either deployed independently as a service in docker, lets go through the steps required to install Jenkins on docker as a container.

Get official Jenkins Image

• Go to your terminal/command prompt and get the Jenkins latest’s image from docker hub
  

  docker pull Jenkins

Run Docker Image

Map localhost ports

• Then we will try to run the image, we will map Jenkins on localhost port 8080 and port 50000

Map Volumes to persist data

• If it is important to persist the Jenkins data you can choose between the 2 options.
  1. Save data within the docker container, the docker volume persists upon restarts and even after if the container is deleted.
  2. Save data on your host/disk/laptop/machine. Weigh in your requirements and see what fits in with your requirements.
• Option-1> To persist the data within docker container
  

  docker volume create --name jenkins_data
  docker run --name jenkins -d -v jenkins_data:/var/jenkins_home -p 8080:8080 -p 50000:50000 jenkins:latest

OR

• Option-2>To persist the data within host( host machine/laptop) container(option-2)
  

  docker run --name jenkins -d -v /Users/jenkins:/var/jenkins_home -p 8080:8080 -p 50000:50000 jenkins:latest

  Don’t forget to change the path to your local drive  after -v tag /Users/…. in case of Mac and C:/Users…. in case of Windows)

Check Jenkins is running

• Check if docker container Jenkins is running
  

  docker ps -a

  
  If all is fine, it should show output like below
  

  CONTAINER    ID    IMAGE    COMMAND    CREATED   STATUS   PORTS    NAMES
  041465f0254e jenkins:latest "/bin/tini -- /usr..." 43 hours ago Up 58 seconds 0.0.0.0:50000>50000/tcp, 0.0.0.0:8082->8080/tcp jenkins

Configure Jenkins

• Go to Jenkins -> Globals tools configurations and add Java/GIT/Maven installations; download from the internet is an option which you can choose if there are no available installations on your disk as shown below.

The post How to Install Jenkins on Docker appeared first on Little Big Extra.

Installing Artifactory on Docker

JFrog Artifactory is a repository manager used for versioning deployable like a jar, war, docker image etc. Artifactory Pro is a licensed/paid version which will let you create repositories for docker images and much more.Artifactory OSS is open source and a restrective version where you can create maven repositories and few others.

Get official Artifactory Image

Pull the Artifactory OSS Docker Image, it’s an open source image. For this tutorial, we will go ahead with Artifactory OSS

docker pull docker.bintray.io/jfrog/artifactory-oss:latest

Run Docker Image

Map localhost ports

• Then we will try to run the image, we will map Docker on localhost port 8081

docker run --name artifactory -d -p 8081:8081 docker.bintray.io/jfrog/artifactory-pro:latest

Map Volumes to persist data

It is important to persist the artifacts which will be fed to Artifactory, so we have two options here

• If it is important to persist the Jenkins data you can choose between the 2 options.
  1. Save data within the docker container, the docker volume persists upon restarts and even after if the container is deleted.
  2. Save data on your host/disk/laptop/machine. Weigh in your requirements and see what fits in with your requirements.

docker stop artifactory
docker rm artifactory

• Option-1> To persist the data within docker container
  

  docker volume create --name artifactory5_data
  
  docker run --name artifactory-5.0.0 -d -v artifactory5_data:/var/opt/jfrog/artifactory -p 8081:8081 docker.bintray.io/jfrog/artifactory-pro:latest

  OR

• Option-2> To persist the data within host( host machine/laptop) container
  

  docker run --name artifactory-5.0.0 -d -v  /Users/abhishek/jfrog/artifactory:/var/opt/jfrog/artifactory -p 8081:8081 docker.bintray.io/jfrog/artifactory-oss:latest

Verify Installation

• Since Artifactory was mapped to localhost port 8081, open your browser and use localhost:8081 if accessing from the same machine.

http://SERVER_DOMAIN:8081/artifactory

The post How to Install Artifactory on Docker appeared first on Little Big Extra.

Add Maven Remote catalogue in eclipse

Maven remote catalogues are very useful in starting a project quickly and neatly. The project structure including src, test directory are created along with pom dependencies.

Follow below steps to add remote archetype catalogue

1. Go to Windows -> Preferences -> Maven ->Archetype
2. Click on Add Remote Catalog
   
3. Add catalogue file as
   • http://repo1.maven.org/maven2/archetype-catalog.xml
4. Description as
   • Remote Archetype catalogue
5. Click on Verify, you should see a message like Found XXX archetype’s

If you are behind a proxy you might need to go to Windows -> Preference-> Network and add the proxy details

Why not use this remote archetype to create your first Spring Boot Project

Follow this Video for more details

The post How to add remote archetype catalog in Eclipse appeared first on Little Big Extra.