DevOps tutorials
Contact us

DevOps for small / medium web apps - Part 2 - Continuous Integration

Summary

  1. Introduction
  2. Simple application
  3. GitLab project creation
  4. Run the application locally
  5. Commit and first CI pipeline

Introduction

This tutorial part introduces a simple Continuous Integration pipeline based on GitLab CI/CD. Although we keep it simple now, this pipeline will be extended in the next parts.

Simple application

This tutorial is based on a simple web application written on top of Spring Boot (for the backend) and React (for the frontend).

The application consists in a todo list where a user can add or remove items. The goal is to have a simple 3-tier architecture with enough features that allow us to explore important concepts:

GitLab project creation

Let’s start by creating a project on GitLab:

We now have a project but we cannot download it on our computer yet; for that we need to generate and register a SSH key:

You can now configure git and clone the project on your computer. Enter the following commands in your terminal:

# Set your real name
git config --global user.name "John Doe"

# Set the same email address as the one you set in your GitLab profile
git config --global user.email "john.doe@your-company.com"

# Create a directory for your projects
mkdir ~/projects
cd ~/projects

# Clone the empty project on your computer (set your GitLab domain name and username)
git clone git@gitlab.my-sample-domain.xyz:johndoe/todolist.git

# Change directory and check the ".git" folder is present
cd todolist
ls -la

Copy all the files from the folder “sample-app/version1/*” of this tutorial into “~/projects/todolist”. You should have a directory with the following top files:

The “src” folder is organized like this:

Run the application locally

Install the JDK 8 and Maven on your computer, and build your application with the following command:

mvn clean package

This command should end with a “BUILD SUCCESS” message: it compiles, runs the tests and packages the application.

Notes:

The next step is to setup a database locally:

Now that we have a database up and running, we need to configure the application. Have a look at the backend configuration file “src/main/resources/application.properties” and check that the DB configuration corresponds to your installation:

spring.datasource.url=jdbc:mysql://localhost:3306/todolist?useSSL=false
spring.datasource.username=todolist
spring.datasource.password=P@ssw0rd

Note 0: The spring.datasource.url property is in the format “jdbc:mysql://HOSTNAME:PORT/DATABASE_NAME?useSSL=false”.

Note 1: If you modified this file you need to re-run mvn clean package.

You can now launch the application locally with the following command:

mvn spring-boot:run

If everything went well, the application should print several lines of logs in the console. Look at the two last lines:

2018-11-02 13:56:18.139  INFO 87329 --- [main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2018-11-02 13:56:18.145  INFO 87329 --- [main] com.alibaba.intl.todolist.Application    : Started Application in 5.305 seconds (JVM running for 17.412)

Open a new tab in your web browser and open the url “http://localhost:8080”. You should normally get something like this:

Sample application version 1

Note: you can add new tasks by filling a description and by clicking on the “Add” button.

Congratulation if you managed to get the application up and running! The source code has been written with the IntelliJ IDEA IDE (the ultimate edition is necessary for frontend development, you can evaluate it for free for 30 days).

Before we move on and create our first CI pipeline, there is still an important point to talk about: we didn’t create any table in the database, so how does the application work? Let’s have a look at our database with a terminal:

# Connect to the database (use your new root password)
mysql -u root -p

The command above opens a prompt; please enter the following instructions:

-- Use our database
USE todolist;

-- Display the tables
SHOW TABLES;

The last command should display something like this:

+-----------------------+
| Tables_in_todolist    |
+-----------------------+
| flyway_schema_history |
| task                  |
+-----------------------+
2 rows in set (0.00 sec)

Now we can understand why the application works: because the database schema has been created. The “task” table corresponds to the Java class “src/main/java/com/alibaba/intl/todolist/model/Task.java”. Let’s study “flyway_schema_history”:

-- Look at the content of the flyway_schema_history table
SELECT * FROM flyway_schema_history;

The result should look like this:

+----------------+---------+-------------------+------+-----------------------------+------------+--------------+---------------------+----------------+---------+
| installed_rank | version | description       | type | script                      | checksum   | installed_by | installed_on        | execution_time | success |
+----------------+---------+-------------------+------+-----------------------------+------------+--------------+---------------------+----------------+---------+
|              1 | 001     | Create task table | SQL  | V001__Create_task_table.sql | -947603613 | todolist     | 2018-10-31 17:57:51 |             24 |       1 |
+----------------+---------+-------------------+------+-----------------------------+------------+--------------+---------------------+----------------+---------+
1 row in set (0.00 sec)

The “flyway_schema_history” table has been created by Flyway, a tool that allows us to create and update our database schema. As you can see, the table contains the names of the scripts from “src/main/resources/db/migration” that have been successfully executed.

Working with Flyway requires us to follow this procedure:

Flyway is automatically started when the applications starts, if you check the application logs, you can see that Spring calls Flyway during its initialization. For more information about this integration, please read the official documentation.

Commit and first CI pipeline

It is now time to save the project in the git repository. Please enter the following command in your terminal:

# Go to the project folder
cd ~/projects/todolist

# Check files to commit
git status

The last command should print something like this:

On branch master

No commits yet

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	.gitignore
	.gitlab-ci.yml
	package.json
	pom.xml
	src/
	webpack.config.js

Add all these files and commit them:

# Add the files
git add .gitignore .gitlab-ci.yml package.json pom.xml src/ webpack.config.js

# Commit the files and write a comment
git commit -m "Initial commit."

# Push the commit to the GitLab server
git push origin master

Pushing your code to GitLab triggers something interesting:

You should see something like this:

First pipeline

Clicking on the “Artifacts” button on the left allows you to download the generated “.jar” file containing your ready-for-production application.

Clicking on the icon in the “Stages” column and then selecting “build” allows you to see the commands and logs used to compile and package the application.

This pipeline is triggered when somebody pushes code to the server. It is configured by the “.gitlab-ci.yml” file:

image: maven:3.6.0-jdk-8

variables:
  MAVEN_OPTS: "-Dmaven.repo.local=./.m2/repository"

cache:
  paths:
    - ./.m2/repository

stages:
  - build

build:
  stage: build
  script: "mvn package"
  artifacts:
    paths:
      - target/*.jar

The first line “image: maven:3.6.0-jdk-8” defines the Docker image used to execute the build command (as you can see, using Docker relieves us to setup the JDK 8 and Maven on the GitLab runner manually).

The “MAVEN_OPTS” variable and the “cache” block are an optimization: because Maven takes a lot of time to download dependencies, these definitions allow us to re-use these dependencies among pipelines.

The “stages” block defines only one stage “build”, we will add new ones later in this tutorial.

The “build” block is the most important one: it instructs the GitLab runner to execute “mvn package” in order to compile, run the tests and package the application. The “artifacts” block instructs GitLab to save the generated “.jar” file.

Note: even if this pipeline is simple, it is already quite useful for a team since it can immediately inform the team that somebody committed something bad (for example he missed a file, or some test fail unexpectedly). GitLab automatically sends an email to the person who made the mistake: this rapid feedback can save us a lot of time because the error cause has a great chance to be located in the code that we just modified.