DevOps tutorials
Contact us

DevOps for small / medium web apps - Part 1 - GitLab installation and configuration

Summary

  1. Introduction
  2. Cloud resources creation
  3. GitLab installation
  4. HTTPS configuration
  5. Mail server configuration
  6. Automatic backup configuration
  7. GitLab runner installation and configuration
  8. User management
  9. Maintenance
  10. Upgrade

Introduction

GitLab CE edition is a free open-source tool that will help us to host Git repositories and run our CI/CD pipeline.

In order to keep it simple, we will install GitLab on an ECS instance with a direct access to internet. Although the servers will be protected via encryption and restrictive security group rules, you might also want to isolate your virtual machines from internet by using a VPN Gateway.

The following diagram illustrates the architecture we will put in place for GitLab:

GitLab architecture

Cloud resources creation

The first step is to buy a domain name. This is necessary if you want to enable security on your servers:

Important: due to a limitation in Direct Mail, please choose a domain name with less than 28 characters.

The second step is to create ECS instances and related resources:

The ECS instance is ready for GitLab, let’s register a sub-domain for this machine:

GitLab installation

We can now finally install GitLab! Open a terminal on your computer and type (note: if you use MAC OSX, you first need to disable the setting “Set locale environment variables on startup” in “Preferences > Profiles > Advanced”):

# Connect to the ECS instance
ssh root@gitlab.my-sample-domain.xyz # Use the password you set when you have created the ECS instance

# Update the machine
apt-get update
apt-get upgrade

# Add the GitLab repository for apt-get
cd /tmp
curl -LO https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh
bash /tmp/script.deb.sh

# Install GitLab
apt-get install gitlab-ce

# Open GitLab configuration
nano /etc/gitlab/gitlab.rb

In the GitLab configuration file, replace the value of external_url by “http://gitlab.my-sample-domain.xyz” (the domain you have just purchased and configured), then save and quit by pressing CTRL+X.

Now let’s start GitLab and try it. In your terminal, run the following command:

gitlab-ctl reconfigure

Open your web browser on “http://gitlab.my-sample-domain.xyz”. You should have a screen like this: First GitLab screen

Congratulation if you get a similar screen! In case it doesn’t work, please first make sure you didn’t miss a step, then raise an issue if the problem persists.

Note: if you get automatically redirected to HTTPS, please continue to the next section first.

Do not enter your new password yet because you are using an unencrypted connection. Let’s fix this problem now.

HTTPS configuration

Open your terminal and enter the following commands:

# Connect to the ECS instance
ssh root@gitlab.my-sample-domain.xyz # Use the password you set when you have created the ECS instance

# Install dependencies
apt-get install ca-certificates openssh-server
apt-get install postfix # During the installation, select "Internet Site" and set your domain (e.g. gitlab.my-sample-domain.xyz)

# Open GitLab configuration
nano /etc/gitlab/gitlab.rb

The last command allows you to edit GitLab configuration:

You can now enter your new password and sign in with the username “root” and your new password. You should be able to access to the GitLab dashboard.

Before going further we still need to configure two things:

Mail server configuration

Note: Direct Mail is not available in all regions, but you can configure it in a different one from where you have created your ECS. At the time of writing, Direct Mail is available in China (Hangzhou), Singapore and Australia (Sydney). Please contact us if you need it in another region.

Go back to the Alibaba Cloud web console and execute the following instructions:

You should probably have a domain configuration that looks like that:

Domain configuration

Continue the email server configuration:

Now that the email server is ready, let’s configure GitLab to use it. Open a terminal on your computer and enter the following commands:

# Connect to the ECS instance
ssh root@gitlab.my-sample-domain.xyz # Use the password you set when you have created the ECS instance

# Open GitLab configuration
nano /etc/gitlab/gitlab.rb

Scroll down to “### Email Settings” and insert the following lines:

gitlab_rails['gitlab_email_enabled'] = true
gitlab_rails['gitlab_email_from'] = 'gitlab@mail.my-sample-domain.xyz' # The sender address you have just created
gitlab_rails['gitlab_email_display_name'] = 'GitLab'
gitlab_rails['gitlab_email_reply_to'] = 'gitlab@mail.my-sample-domain.xyz'

Scroll down to “### GitLab email server settings” and insert the following lines:

gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtpdm-ap-southeast-1.aliyun.com"   # SMTP address written in the Direct Mail console
gitlab_rails['smtp_port'] = 465                                     # SMTP port written in the Direct Mail console
gitlab_rails['smtp_user_name'] = "gitlab@mail.my-sample-domain.xyz" # Sender address
gitlab_rails['smtp_password'] = "HangzhouMail2018"                  # SMTP password for the sender address
gitlab_rails['smtp_domain'] = "mail.my-sample-domain.xyz"           # Your email domain
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = false
gitlab_rails['smtp_tls'] = true

Apply the configuration change and restart GitLab:

gitlab-ctl reconfigure

You can test the configuration like this:

Automatic backup configuration

Backups are important because they prevent data loss in case of accident and allow you to migrate to another ECS instance if you need.

In order to run backups automatically, please open a terminal and run the following commands:

# Make sure rsync is installed
apt-get install rsync

# Install the latest version of tar
cd /tmp
wget http://ftp.gnu.org/gnu/tar/tar-latest.tar.gz
tar -xvzf tar-latest.tar.gz
cd tar-*/
FORCE_UNSAFE_CONFIGURE=1 ./configure
make
make install

# Restart the machine to make sure all users use the same version of tar
reboot

# Re-connect after few seconds to the ECS instance
ssh root@gitlab.my-sample-domain.xyz # Use the password you set when you have created the ECS instance

# Check the tar version is >= 1.30
tar --version

# Install OSSFS, a tool that allow us to mount an OSS bucket as a local folder: https://github.com/aliyun/ossfs
apt-get install gdebi-core
cd /tmp
wget https://github.com/aliyun/ossfs/releases/download/v1.80.5/ossfs_1.80.5_ubuntu16.04_amd64.deb
gdebi ossfs_1.80.5_ubuntu16.04_amd64.deb

Note: the GitLab documentation requires tar version to be equals to or greater than 1.30.

Let’s now create an OSS bucket where we will store our backups:

You will also need an access key id and secret:

In your terminal, mount your OSS bucket as a folder:

# Save your bucket name, access key id and access key secret in the file /etc/passwd-ossfs
# The format is my-bucket:my-access-key-id:my-access-key-secret
echo gitlab-my-sample-domain-xyz:LTAIsP66uJ8zujwZ:rc15yggaCX08AiYKe2BGnX49wNUGpk > /etc/passwd-ossfs
chmod 640 /etc/passwd-ossfs

# Create a folder where we will mount the OSS bucket
mkdir /mnt/gitlab-bucket

# Mount the OSS bucket
# The -ourl come from the last "Endpoint" for VPC Network Access
ossfs gitlab-my-sample-domain-xyz /mnt/gitlab-bucket -ourl=http://oss-ap-southeast-1-internal.aliyuncs.com

# Check it works
echo "It works" > /mnt/gitlab-bucket/test.txt

# Unmount the OSS bucket
umount /mnt/gitlab-bucket

Check that the test file is present in your bucket:

Configure the OSS bucket so that it is automatically mounted when the ECS machine starts. Create the following file:

nano /etc/systemd/user/ossfs.service

Adapt and copy the following content:

[Unit]
Description=OSSFS mounts
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/bin/ossfs gitlab-my-sample-domain-xyz /mnt/gitlab-bucket -o allow_other -ourl=http://oss-ap-southeast-1-internal.aliyuncs.com
ExecStop=/bin/umount /mnt/gitlab-bucket

[Install]
WantedBy=default.target

Make sure you set the right bucket name and endpoint. Quit and save by pressing CTRL+X. Configure Systemd to run this script at startup:

systemctl --user enable ossfs.service

# Restart the machine to test
reboot

# Re-connect after few seconds to the ECS instance
ssh root@gitlab.my-sample-domain.xyz # Use the password you set when you have created the ECS instance

# Create a sample file
echo "Second test" > /mnt/gitlab-bucket/test2.txt

Go to the OSS console, check that the test2.txt file is present in your bucket and delete it.

Let’s now configure GitLab to put its backup files in the mounted folder. Open the terminal and run:

# Open GitLab configuration
nano /etc/gitlab/gitlab.rb

Scroll to “### Backup Settings” and insert the following line:

gitlab_rails['backup_path'] = "/mnt/gitlab-bucket/backup/"

Quit and save by pressing CTRL+X, then check if it works:

# Apply GitLab configuration
gitlab-ctl reconfigure

# Manually launch a first backup
gitlab-rake gitlab:backup:create

The last command should have created a backup. Go to the OSS console and check you have a file with a path like “backup/1540288854_2018_10_23_11.3.6_gitlab_backup.tar”.

Let’s now configure automatic backup to be executed automatically every night. For that we will create two types of cron jobs: one to execute the backup command above, one to save the GitLab configuration files.

Open your terminal and execute:

# Edit the CRON configuration file. Select nano as the editor.
crontab -e

Enter the following lines into this file:

0 2 * * * /opt/gitlab/bin/gitlab-rake gitlab:backup:create CRON=1
0 2 * * * /bin/cp /etc/gitlab/gitlab.rb "/mnt/gitlab-bucket/backup/$(/bin/date '+\%s_\%Y_\%m_\%d')_gitlab.rb"
0 2 * * * /bin/cp /etc/gitlab/gitlab-secrets.json "/mnt/gitlab-bucket/backup/$(/bin/date '+\%s_\%Y_\%m_\%d')_gitlab-secrets.json"

Save and quit by pressing CTRL+X.

You now have configured automatic backup every night at 2AM. If you want to test this configuration you can replace “0 2 * * *” by the current time + 2 min; for example if the current time is 14:24, then set “26 14 * * *”; after that you need to wait about 2 min and check whether new files have been created in your OSS bucket.

The restoration process is well described in the official documentation (section “Restore for Omnibus installations”). Note that it is considered as a good practice to test your backups from time to time.

GitLab runner installation and configuration

It is a good practice to run CI/CD jobs (code compilation, unit tests execution, application packing, …) on a different machine from the one that run GitLab.

Thus, we need to setup one runner on a new ECS instance. Please execute the following instructions:

Execute the following commands in this “web-terminal”:

# Update the machine
apt-get update
apt-get upgrade

# Add a new repository for apt-get for GitLab Runner
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash

# Add a new repository for apt-get for Docker
apt-get install software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

# Update the machine
apt-get update

# Install GitLab runner
apt-get install gitlab-runner

# Install dependencies for Docker
apt-get install apt-transport-https ca-certificates curl software-properties-common

# Install Docker
apt-get install docker-ce

As you can see we setup two applications: GitLab Runner and Docker. We will keep things very simple with Docker: it is a very powerful tool, but for the moment we will just use it as a “super installer”, for example we will not setup any tool, compiler or SDK on this machine; instead we will be lazy and let Docker to download the right images for us. Things will become more clear later in this tutorial when we will configure our CI/CD pipeline.

Now we need to connect the runner with GitLab:

The bottom of the page contains an URL and a token:

URL and token for the runner

Go back to the “web-terminal” connected to the runner machine, and type:

gitlab-runner register

This tool needs several information in order to register the runner. Enter the following responses:

After the tool gives you back the hand, you should be able to see this runner in the GitLab web browser tab. Refresh the page and check at the bottom, you should see something like this: Registered runner

Our GitLab is now ready to be used! But there are few more points to consider before creating our first project:

User management

As administrator, there are few steps you need to follow in order to improve your GitLab account:

You may also want to control who can register on your GitLab server (the default configuration allows anyone on internet to register):

Now only administrators can create new users. This can be done by navigating to the “Overview > Users” menu in the “Admin area”.

Maintenance

Linux servers need to be upgraded from time to time: security patches must be installed as soon as possible and applications should be updated to their latest versions.

On Ubuntu instances, the following commands allow you to safely update your server:

apt-get update
apt-get upgrade

Other commands such as apt-get dist-upgrade or do-release-upgrade are less safe, especially the last one since it can update Ubuntu to a newer LTS version that is not yet supported by Alibaba Cloud.

For more complex upgrade it may be more practical to replace the ECS instance:

Security updates can be automatically installed thanks to unattended-upgrades. For each ECS instance (GitLab and its runner), open a terminal (via SSH or via the web-terminal console) and enter the following commands:

# Install unattended-upgrades
apt-get install unattended-upgrades

# Check the default configuration is fine for you. Press CTRL+X to quit.
nano /etc/apt/apt.conf.d/50unattended-upgrades

# Enable automatic upgrades
dpkg-reconfigure --priority=low unattended-upgrades

# Edit the related configuration
nano /etc/apt/apt.conf.d/20auto-upgrades

The last configuration file can be modified in order to look like this:

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::AutocleanInterval "7";

Save and quit by pressing CTRL+X. You can launch unattended-upgrades manually for testing:

unattended-upgrade -d

The logs of unattended-upgrades are printed in /var/log/unattended-upgrades.

More information about automatic update can be found here.

Upgrade

The described architecture for GitLab is fine as long as the number of users is not too large. However there are several solutions when things start to get slow:

When a single GitLab instance become unacceptable, maybe because of performance issues or because high-availability is required, the architecture can evolve into a distributed system involving the following cloud resources:

As you can see the complexity can quickly increase. Tools such as Packer (virtual machine image builder), Terraform (infrastructure as code software) or Chef / Puppet / Ansible / SaltStack (configuration management) can greatly help managing it: they require an initial investment but allow organizations to better manage their systems.

Another solution is to let other companies to manage this complexity for you. There are many SaaS vendors such as GitLab.com or GitHub. Alibaba Cloud offers Codepipeline, but it is currently only available in Chinese.