Documentation

Configure Operator Service for Jenkins® on Azure

This tutorial outlines a step by step guide on how to configure Jenkins instance after you successfully deployed it from the Azure Marketplace.

How Jenkins Configuration Works

The current configuration mechanism is based on Kubernetes Custom Resource which is automatically created during the installation phase and then used as a configuration file by operator.

Every time you want to configure Jenkins, it needs to be done in code by modifying existing Custom Resource file. Any manual changes from the web interface will be overridden by automation or after the Jenkins restart.

In the future, we are planning to introduce web interface and infrastructure as code SDK to make these changes programmatically without Kubernetes-specific knowledge required.

Getting Jenkins Custom Resource

In order to get existing Jenkins configuration, run the following command:

1
kubectl -n jenkins get jenkins -o yaml > jenkins_cr.yaml

This command will save current Jenkins Custom Resource into jenkins_cr.yaml file. Every time you want to change the configuration, you have to get the current version first and update it.

1
kubectl -n jenkins apply -f jenkins_cr.yaml


Sections below will show you how to configure Jenkins using Groovy Scripts and Configuration as Code (CasC).

Customization via Groovy Scripts and yamls

Jenkins instance can be customized using Groovy scripts or Configuration as Code (thanks to pre-installed configuration as code plugin).

CasC scripts are more readable and simplier to write, this should be your default choice.

However, when something is not supported by the CasC plugin or for more complex and low-level configuration Groovy scripts are better. They allow you to use Jenkins internal API.

Let’s take a look at some examples below.

Customization of Jenkins with Groovy Scripts

The overall process of configuration can be divided into:

  1. Creating a Kubernetes ConfigMap, which contains Groovy script you want to execute.
  2. Optionally creating a Kubernetes Secret if you need to store secrets like password or certificates.
  3. Referencing a Kubernetes ConfigMap and corresponding Secret in Jenkins Custom Resource.

1. Creating ConfigMap

We need to create a ConfigMap config file containing the configuration we want to apply. In the data section we can use Groovy scripts to write configuration code. Since the secret is already present in the Cluster before ConfigMap we can safely reference its value.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
apiVersion: v1
kind
: ConfigMap
metadata
:
  name
: jenkins-operator-user-configuration
data
:
  1-system-message.groovy
: |
   import jenkins.*
    import jenkins.model.*
    import hudson.*
    import hudson.model.*
    Jenkins jenkins = Jenkins.getInstance()

    jenkins.setSystemMessage(secrets["SYSTEM_MESSAGE"])
    jenkins.save()

Now we can safely create this ConfigMap as a Kubernetes Resource.

2. Creating Secret

In case we want to use confidential data, we have to start by creating a Secret resource to wrap it in. Since it’s a secret, it would be a good idea to encode it. Let’s write into console:

1
echo-n"Hello World"| base64

The output is

1
SGVsbG8gd29ybGQ=

Now we can place it in a Secret config file as the value of SYSTEM_MESSAGE key.

1
2
3
4
5
6
7
8
apiVersion: v1
kind
: Secret
type
: Opaque
metadata
:
  name
: jenkins-conf-secrets
  namespace
: jenkins
data
:
  SYSTEM_MESSAGE
: SGVsbG8gd29ybGQ=

We have to create the Secret resource in Kubernetes using this config file, saved somewhere in the catalogue, for example in deploy folder.

1
kubectl -n jenkins apply -f deploy/secret.yaml

3. Referencing ConfigMap and Secret in Jenkins Custom Resource

Finally in the Jenkins Custom Resource, under spec we have to reference Secrets and ConfigMaps which hold configuration code we want to apply to our instance. Under groovyScripts we’ll place name of the ConfigMap and name of the corresponding secret.

1
2
3
4
5
6
7
8
9
10
apiVersion: jenkins.io/v1alpha2
kind
: Jenkins
metadata
:
  name
: jenkins
spec
:
  groovyScripts
:
    configurations
:
    - name
: jenkins-operator-user-configuration
    secret
:
      name
: jenkins-conf-secrets

Our Jenkins instance will see our previously created resources thanks to references we created. This event will trigger Jenkins restart and our configuration will be applied automatically. You will see a tiny “Hello World” on the main page.

If you store secrets, it’s even simplier, because you only have to create and reference ConfigMaps.

Customization of Jenkins with Configuration as Code (CasC)

The overall process of customization can be divided into:

  1. Creating a Kuberentes ConfigMap, which contains Configuration as Code yaml.
  2. Optionally creating a Kubernetes Secret if you need to store secrets like passoword or certificates.
  3. Referencing a Kubernetes ConfigMap and corresponding Secret in Jenkins Custom Resource.

1.Creating Secret config file and creating it in Kubernetes as a resource

In case we want to use confidential data, we have to start by creating a Secret resource to wrap it in. Since it’s a secret, it would be a good idea to encode it. Let’s write into console:

1
echo-n"Hello World"| base64

The output is

1
SGVsbG8gd29ybGQ=

Now we can place it in a Secret config file as the value of SYSTEM_MESSAGE key.

1
2
3
4
5
6
7
8
apiVersion: v1
kind
: Secret
type
: Opaque
metadata
:
  name
: jenkins-conf-secrets
  namespace
: jenkins
data
:
  SYSTEM_MESSAGE
: SGVsbG8gd29ybGQ=

We have to create the Secret resource in Kubernetes using this config file, saved somewhere in the catalogue, for example in deploy folder.

1
kubectl -n jenkins apply -f deploy/secret.yaml

2. Creating ConfigMap config file and creating it in Kubernetes as a resource

We need to create a ConfigMap config file containing the configuration we want to apply. In the data field value we can use yaml syntax to add fields with configuration code. Since the secret is already present in the Cluster before ConfigMap we can safely reference its value.

1
2
3
4
5
6
7
8
apiVersion: v1
kind
: ConfigMap
metadata
:
  name
: jenkins-operator-user-configuration
data
:
  1-system-message.yaml
: |
    jenkins
:
      systemMessage
: ${SYSTEM_MESSAGE}

Now we can safely create this ConfigMap.

1
kubectl -n jenkins apply -f configmap.yaml

3. Referencing ConfigMap and corresponding Secret in Jenkins Custom Resource

Finally in the Jenkins Custom Resource, under spec we have to reference Secrets and ConfigMaps which hold configuration code we want to apply to our instance. Under configurationAsCode we’ll place name of the ConfigMap and name of the corresponding secret.

1
2
3
4
5
6
7
8
9
10
apiVersion: jenkins.io/v1alpha2
kind
: Jenkins
metadata
:
  name
: jenkins
spec
:
  configurationAsCode
:
    configurations
:
    - name
: jenkins-operator-user-configuration
    secret
:
      name
: jenkins-conf-secrets

When previous steps are ready, we can safely create the ConfigMap as a Kubernetes Resource. Our Jenkins instance will see it and can bind it to previously created Secret, thanks to references we created. This event will trigger pod restart and our configuration will be applied. You will see a tiny “Hello World” on the main page.

If you don’t store secrets, it’s even simplier, because you only have to create and reference ConfigMaps.

Installing Plugins

The following plugins are pre-installed and required to provide fully operational Jenkins instance:

  • configuration-as-code
  • git
  • job-dsl
  • kubernetes-credentials-provider
  • kubernetes
  • workflow-aggregator
  • workflow-job

If you would like to install an additional plugin from the official repository, you only need to update custom resource file.

1
2
3
4
5
6
7
8
9
apiVersion: jenkins.io/v1alpha2
kind
: Jenkins
metadata
:
  name
: jenkins
spec
:
master
:
  plugins
:
  - name
: jacoco
    version
: "3.0.6"

The Jenkins Operator Service will then automatically install plugins.

Troubleshooting

At startup Jenkins Operator Service needs time to configure itself and download all necessary dependencies. Log saying the “User configuration phase is complete” indicates instance is ready to use.

1
2
3
4
5
6
7
8
9
10
2020-11-20T12:59:41.982Z    INFO    controller-jenkins  jenkins/jenkins_controller.go:264   Base configuration phase is complete, took 1m31s    {"cr": "jenkins"}
2020-11-20T12:59:43.126Z    INFO    controller-jenkins  seedjobs/seedjobs.go:248    Waiting for Seed Job Agent `seed-job-agent`...  {"cr": "jenkins"}
2020-11-20T12:59:43.467Z    INFO    controller-jenkins  seedjobs/seedjobs.go:248    Waiting for Seed Job Agent `seed-job-agent`...  {"cr": "jenkins"}
2020-11-20T12:59:43.825Z    INFO    controller-jenkins  seedjobs/seedjobs.go:248    Waiting for Seed Job Agent `seed-job-agent`...  {"cr": "jenkins"}
2020-11-20T12:59:44.117Z    INFO    controller-jenkins  seedjobs/seedjobs.go:248    Waiting for Seed Job Agent `seed-job-agent`...  {"cr": "jenkins"}
2020-11-20T12:59:44.406Z    INFO    controller-jenkins  seedjobs/seedjobs.go:248    Waiting for Seed Job Agent `seed-job-agent`...  {"cr": "jenkins"}
2020-11-20T12:59:44.878Z    INFO    controller-jenkins  seedjobs/seedjobs.go:248    Waiting for Seed Job Agent `seed-job-agent`...  {"cr": "jenkins"}
2020-11-20T12:59:45.182Z    INFO    controller-jenkins  seedjobs/seedjobs.go:248    Waiting for Seed Job Agent `seed-job-agent`...  {"cr": "jenkins"}
2020-11-20T12:59:45.476Z    INFO    controller-jenkins  seedjobs/seedjobs.go:248    Waiting for Seed Job Agent `seed-job-agent`...  {"cr": "jenkins"}
2020-11-20T13:00:10.024Z    INFO    controller-jenkins  jenkins/jenkins_controller.go:324   User configuration phase is complete, took 2m0s {"cr": "jenkins"}

In case of an error you can just delete the jenkins pod and it will come back healthy, without losing any configurations.

1
kubectl -n jenkins delete pod jenkins-jenkins

Operator Service for Jenkins is full of helpful messages which can, more often than not, directly state the reason for an error.
To check the logs of Operator Service for Jenkins you need to run:

1
kubectl -n jenkins logs deployment/jenkins-operator-<hash>

And to check the logs of the Jenkins instance you need to run:

1
kubectl -n jenkins logs jenkins-jenkins jenkins-master