0 – Principle
The main idea is to interconnect the GITLAB (SCM) and the CI (continous integration) services, in order to:
- automatically run jobs on CI, then some events occur on the GITLAB side, such as: new push to branches, merge request on feature branches…
- automatically publish the CI build status on the GITLAB server.
To achieve these features, we need to create shared authentication tokens on each side, and configure each service to use these tokens.
Remarks:
- the "merge request" configuration assumes that we use the gitflow paradigm (https://jeffkreeftmeijer.com/git-flow/)
- the name of the CI platform is ci.inria.fr, you may replace it by your own CI URL
- the name of the GITLAB platform is gitlab.inria.fr, you may…
- in the rest of the document PROJECT is the name of your software project.
1 – Prerequisite on gitlab.inria.fr
Your gitlab project needs to be enabled for CI :
- log on gitlab.inria.fr
- select Settings/General (left menu)
- select Visibility, project features, permissions
- check CI/CD and then Save changes
2 – Prerequisite on ci.inria.fr
Firstly, we need to install the gitlab plugin on ci.inria.fr:
- log on ci.inria.fr (https://ci.inria.fr/dashboard)
- scoll to your project section (PROJECT )
- select Jenkins (green button)
- you must authenticate yourself on this new page, use the same credentials as for ci.inria.fr
- select Manage Jenkins (left menu)
- select Manage Plugins (right side of the screen)
- select the Available tab
- search and install the gitlab plugin
- follow the instructions (install the dependencies, restart jenkins…)
In some cases, you may need to upgrade the jenkins version.
- log on ci.inria.fr (https://ci.inria.fr/dashboard)
- scroll to your project section (PROJECT )
- select Manage project (orange button)
- you may need to authenticate on the new page https://ci.inria.fr/project/PROJET/show
- select Manage Jenkins (blue button)
- select a new jenkins version on your qualification session (we advise you use the last LTS version)
- push the qualification session to the production session
- restart the production jenkins service.
3 – Configure the link from ci to gitlab
3.1 – on the gitlab.inria.fr side
On the gitlab side, we must create an access token and register it on ci.inria.fr.
You have two options to create this token here:
- use your own account
- create a specific account on gitlab.inria.fr, and use it to create the token.
Depending on the project status, we advise:
- in case of a "personal" project, use your own account,
- in case of a "shared" project, create a dedicated account.
3.1.1 – create a dedicated account on gitlab (for a shared project)
(this section must be skipped if you use your personal account to create the token)
- create a maillist on sympa.inria.fr with the name PROJECT-gitlab@inria.fr (or whatever name suits you)
- add your own login in the list of this maillist subscribers
-
WARNING: this maillist should be public to receive emails from the gitlab server
- connect to https://gitlab.inria.fr (do not log on with your account)
- create a new user with the email PROJECT-gitlab@inria.fr
- choose a password (you may share this password with the rest of the team)
- the mailist PROJECT-gitlab@inria.fr will receive a confirmation email (as a subscriber, you will also receive it)
- click on the link provided by this email
-
the new account is now fully declared
- log off gitlab.inria.fr
- and log back on with an administrator account (aka your own account)
- add the user PROJECT-gitlab as member of the project
- give the developer privileges to this account (if a HTTP 403 error occurs when running a CI build give maintainer privileges; this case may happen eg when using protected branches)
3.1.2 – create an access token
- log on https://gitlab.inria.fr with the proper identity (your account or the PROJECT-gitlab account)
- select Profile Settings : in the Profile menu activated by clicking on your gravatar in the top right corner
- select Access Token (5th item on the left menu)
-
create a new token with the following attributes:
- name: PROJET-token
- Expires at: leave empty (no end date)
- Scopes: api
- Click on Create personal access token
- WARNING: you must immediately save the token value (eg: 9kUp5GGq7chCK8DmessB) because this token cannot be retrieved later.
3.2 – on the ci.inria.fr side
We will now declare the above token on ci.inria.fr:
- log on https://ci.inria.fr (using your personal account)
- go to the dashboard (https://ci.inria.fr/dashboard)
- scroll to the PROJECT
- select Jenkins (green button)
- you must land to https://ci.inria.fr/PROJET/
- authenticate if necessary (same email/password that on ci.inria.fr)
-
to add the token, you need to:
- select Manage Jenkins (left menu)
- select Manage Credentials (on the main panel), which will expand this menu
- select Credentials and then System (breadcrumb on the top of the panel)
- select Global credentials (on the main panel)
- select Add credentials (left menu)
- fill the form with:
- Type: Gitlab API Token
- Scope: Global (Jenkins, nodes,…) (default value)
- API token: copy here the previous authentication token (eg: 9kUp5GGq7chCK8DmessB)
- ID: Gitlab API token (or anything you like, this field is mandatory)
- Description: any description you like
- click on OK
- if "Gitlab API Token" is not available for Type, it must be because you did not install the Gitlab plugin in Jenkins (see above).
We will now configure Jenkins to use this token.
-
get back to https://ci.inria.fr/PROJET/
- select Manage Jenkins (left menu)
- select Configure System (1st item of the right part of the page)
- scroll to Gitlab
-
configure:
- Connection name: gitlab-inria
- Gitlab Host URL: https://gitlab.inria.fr/
-
Credential: Gitlab API token (the previous ID)
- select Save
- select Test Connection (a success message should appear).
3.3 – Share the deploy key
This key is needed for Jenkins to be authorized for git clone.
In CI / slaves https://ci.inria.fr/project/PROJECT/slaves, click public SSH key Download (orange button), save or open this public key.
Then in Gitlab / Project Settings (last item of the left column, you need to be maintainer to have this button): click Repository, then next to Deploy Keys, click Expand, paste the public key, and do not check Write access allowed.
4 – Slaves configuration
In the CI dashboard https://ci.inria.fr/dashboard, click Manage project (orange button), then Slaves (red button), and Add Slave (green button). Choose a template (successfully tested on January 7th 2020: Featured/fedora-29-amd64, Community/Mac OS 10 13). The Mac OS 10 13 image needs more disk space, for example 10 GB in the Additional Disk menu.
Test the slave: in the slave dashboard page https://ci.inria.fr/project/PROJECT/slaves : make sure it is running, and try to connect with the blue Connect button.
For MacOS slaves : use this machine, not a normal slave from a featured or community template :
Go to Jenkins dashboard : https://ci.inria.fr/PROJECT/
Click Build Executor Status (it opens https://ci.inria.fr/PROJECT/computer/)
Click New Node
Name it inextremis-zie
Give these properties :
Remote root directory = /users/ci
Labels = macos
Launch method / host = inextremis-zie.inria.fr
Launch method / credentials = ci (Inria CI generic authentification to slaves)
(If this value is not available, you probably missed step 3.3 Share the deploy key).
Launch method / host key verification strategy = Non verifying Verification Strategy
Node properties : check Environment variables, add PATH with value "/Users/ci/miniconda3/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin"
To log into this machine : username is ‘ci’, password must be asked to a SED team member. Then you can add your CI public SSH key to the authorized_keys file.
5 – Jobs configuration
Remark: the next two chapters should be repeated for all jobs that you want to automatize.
5.1 – Jobs configuration on ci.inria.fr
This step will configure the ci.inria.fr jobs, and allow them to receive signals from gitlab.inria.fr
In order to achieve this, you should:
- log on ci.inria.fr (using your personal account)
- scroll to your projet, and select Jenkins
- if you want to create a new job, select New Item (left menu) and Freestyle project (main panel), choose a name for your job, click Ok and then Save
- from the dashboard, now select the job you want to trigger with gitlab
- select Configure (left menu)
- Tab: General (in the tab list)
- Item Gitlab Connection: select gitlab-inria in the menu (in fact, the name is the "Connection Name" above)
- Tab: Source Code Management
- Select: git
- Item Branch Specifier (blank for ‘any’): origin/${gitlabSourceBranch}
- Tab: Build Triggers
- select Build when a change is pushed to GitLab
- WARNING: you must memorize the GitLab CI Service URL of the job (ex: https://ci.inria.fr/windpos/project/JOB ) which will be reported on the next section
- select Advanced
- go to the end of this section, until the Secret token item
- select Generate
- this will generate the token associated to the GitLab CI Service URL (eg: 3d28ce2e9df42fb44d994752ce142b57)
- item Filter branches by name: Include develop,master
- Tab: Post-build actions
- add a new action by clicking Add post-build action
- action: Publish build status to GitLab
5.2 – webhook configuration on gitlab.inria.fr
The last action is to configure gitlab.inria.fr to send signals to ci.inria.fr for each job it want to schedule. This is done with webhooks configured to use the GitLab CI Service URL and token.
- log on to https://gitlab.inria.fr (use an administrator account, yours for example)
- select Settings/Webhooks (left menu)
- copy the GitLab CI Service URL (ex: https://ci.inria.fr/PROJET/project/JOB)
- copy the token (ex: 3d28ce2e9df42fb44d994752ce142b57)
-
select the following triggers:
- Push events
- Merge Request events
-
Comments
- select Add webhook
- select Test (Push events)
- this should display a pale blue banner with the Hook executed successfully HTTP 200 message
- if you are quick enough, you can go back to ci.inria.fr and verify that a build process is in action
That’s all folks
