Cloud Manager » CI/CD Builds

Tekton is a reference extension to Kubernetes to support Continuous Integration/Delivery (CICD) pipelines. CloudManager provides UI integration with the Tekton run-time, providing a highly simplified One-Click experience to CICD and image building. Managed Deployment can be configured with a source code GIT repository and a target image, and have new versions of an application be build and redeployed on user request or automatically.

About CICD and Tekton

Continuous Integration/Delivery (CICD) is the practice of setting automated build-deploy pipelines for a software project, that includes all the steps involved in making software available. This includes, compilation, testing, building binary images, and deploying for end-users or manual QA testers. This practice is considered a gold standard in the industry as it allows updates to software to quickly available, including bugs and/or security fixes and new features.

Tekton is a reference extension to Kubernetes to support Continuous Integration/Delivery (CICD) pipelines. Tekton runtime adds several new Custom Resources (CRD) to Kubernetes, used to model CICD pipelines. The general approach used by Tekton is to model a CICD pipeline as a set of one or more Tasks, that produce some output artifacts from input resources. The abstractions introduced by Tekon are:

• Pipeline Resources — A resource used in defining a pipeline (e.g. a GIT source repository from a VCS, or a target Docker image to push to some Registry)
• Tasks/ClusterTasks — A procedure in a pipeline with some input(s) and some output(s), defined as a sequence of one or more steps (actions) (e.g. a task to build a Docker image from a source code repository with the help of a build/project-management tool).
• Task Runs — An actual configuration, instantiation, and execution of a Task (e.g. a Docker image built from a Java/Go/C#/etc. source code)
• Pipeline — A more complex pipeline chaining multiple Tasks
• Pipeline Runs — And actual execution of a Pipeline

Tasks and ClusterTasks are semantically equivalent and installed from similar YAML files, but Tasks are installed at the Space/Namespace level, while ClusterTasks are installed at the *Cluster level.

While the low-level definition of Tekton tasks and pipelines can be a bit unyieldy and complex, high-level tools such as CloudManager can make the use Tekton very straightforward and simple in practice. Additionally, a set of commonly used Tasks/Pipelines definitions are provided as part of a Catalog. Thus, in many/most use-cases, tools and devs need to configure only the execution of these Tasks/Pipelines (e.g. source and target resource locations, and required credentials). Furthermore, many useful build pipeline can be modeled and perform with a single task definition, rather than requiring a more complex multi-task definition.

Installing Tekton and Task/Pipeline Catalog

Tekton runtime needs to be configured in a cluster before it can be used by CloudManager. For shared clusters pre-available in EInnovator Cloud cloud.einnovator.org this is already done. For on-premises clusters where CloudManager is being used, Tekton needs to be installed before the the CICD support in CloudManager can be used. This can be done very easily by applying a provide YAML (operator) manifest file with kubectl tool, as show in the command below:

kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.6.0/release.yaml

Alternatively, the YAML file can be download and installed from the local disk:

wget https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.6.0/release.yaml
kubectl apply release.yml

CloudManager latest version is tested for Tekton version 0.6. Is part of the road-map of CloudManager to support newer versions of Tekton. Status updates will be published in this documentation as they unfold.

In addition to the Tekton runtime, Catalog Task/Pipeline definition need to be available as well as well. For shared clusters pre-available in EInnovator Cloud cloud.einnovator.org several ClusterTasks are pre-installed and ready for us. For on-premises clusters where CloudManager is being used, Tekton needs to be installed before the the CICD support in CloudManager can be used. This can be done very easily by applying a provide YAML (operator) manifest file with kubectl tool, as show in the command below:

To install a Task/Pipeline the provide YAML file should be used. To install a Task definition as a ClusterTask simply edit edit manifest file an replace line kind: Task by kind: ClusterTask. Below, we show some examples how to install Task/ClusterTask/Pipeline.

kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/master/jib-maven/jib-maven.yaml

kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/master/jib-gradle/jib-gradle.yaml

Table below show-cases some some of the commonly used Tasks/ClusterTasks available in Tekton Catalog and supported by CloudManagger.

All manifest URL above are prefixed by https://raw.githubusercontent.com/tektoncd/catalog/master/.

The complete list of Tasks/Pipelines available in the Catalog can be found in the Tekton Catalog (GIT Hub Repo).

Configuring a Source GIT Repositoryy

The setup and management of CI/CD pipeline for managed Deployments is available in tab CI/CD > Options. The first step is to configure a a source GIT repository by clicking button Add Repository and open the repository modal dialog. For public GIT repositories, the generic GIT VCS can be selected. The URL of the repository should be specified, and optionally a repository branch. For private GIT repositories, a pre-configured VCS should be selected and a repository picked up from the ones found or the URL entered manually. Selecting a VCS is important because Tekton TaskRuns/PipelineRuns need to be setup to use the configured VCS credentials otherwise they git pull will fail due to authorization error.

After the GIT repository is specified, it can later be edit/modified if needed. The repository branch can also be edit/modified to reflect the branching evolution of the project (if any).

Images below shows an example of a Repository being specified for a manage Deployment from pulic GitHub repository:.

Configuring Target Images

The second configuration step in setting up a CI/CD pipeline for managed Deployments is to specify the target image qualified [domain/]name[:tag] and selecting a pre-configured Registry. Selecting a Registry is necessary because pushing a new image to a repository always requires credentials. This is done also in the CI/CD > Options tab. The Save button should be clicked whenever the CICD options are updated.

Images below show an example of the CI/CD options after the target image is selected (to be pushed to GitHub):

Selecting Build Methodd

The last configuration step is selecting the build method. This is a Task/ClusterTask/Pipeline from the ones available in the space/cluster. If a stack was specified in the Deploument options tab, a default build method is pre-selected/suggested. Notice that the CloudManager does not pull the GIT repository to infer the appropriate build method, so developers should make sure that the selected build method is appropriate. For example, if jib-maven or other Maven based method is selected, then a pom.xml should be available in the root directory of the project.

Builds as TaskRuns/PipelineRuns

Once the configuration is done, a new build can be triggered by clicking on the toolbar button Build. This is create the necessary PipelineResources, and create and submit a new TaskRun/PipelineRun for Tekton runtime, and show the build progress in a list of all recent TaskRuns/PipelineRuns. Status updates can be tracked by clicking on Refresh button in the TaskRuns/PipelineRuns tab.

Images below show a list of recent TaskRuns for a Deployment (left-hand side), and the underlying PipelineResources created (right-hand side).

The execution of the build is done by a Pod created on-the-fly. This pod will pull all the necessary resources (e.g. GIT repo pull), and run the Task(s) steps as per the specification (e.g. compile project source code, build Docker image, and push it to the configured Registyr). The status and logs this Pod can be inspected by clicking in the Log. The Pod and its log are kept by Tekton runtime even after the TaskRuns/PipelineRuns completed execution — successful or not. This allow for easier trouble-shooting of failed builds. On the other hand, its is recommend to delete old and completed TaskRuns/PipelineRuns to release unused resources.

Images below show an example of a Pod details dashboard and log of a TaskRun running a jib-maven task, in this case that completed with success.

Using Built Images

Once a Docker image is build and pushed to the configured Registry is can be used are any other image. Three cases can be considered:

• If the built image name:tag is the same as is currently deployed:
  • If the Deployment is running, pressing the Restart button (in top-right toolbar) is sufficient to have Kubernetes rollout a new ReplicaSet/Pods that use the new image.
  • If the Deployment is stopped, pressing the Start button will create a new Deployment/ReplicaSet/Pods that use the new image.
  • The above described behavior assumes that the PullAlways image pulling policy is configured for the Deployment, which is the default.
• If the name:tag of the built image is different from the one set in the Deployment settings panel, it should be updated there, followed by a Save and a Start/Restart.

The pushed image will be available in selected Registry for future and other uses as well. So it can also be used in other deployments, in same or different space, in same or different cluster. For public image repositories, its coordinates can also be shared with the community.

Learning Morere

• Tekton Website
• Tekton CD Project Pipeline (GitHub)
• Tekton CD Project Collection (GitHub)
• Catalog of Tekton Tasks/Pipelines
• Tekton Docs
• Tekton Tutorial