Gluu CI CD pipeline blueprint

Problem statement

Gluu as any other top company has the customers satisfaction is primary goal. Given the number of Gluu products, it is difficult to meet that goal without a complete, clear and comprehensive workflow.

Below is the list of problems we have to fix:

1. How to provide patch as fast as possible
2. How to catch bugs as soon as possible before it reach production
3. How to react quickly to production outage
4. How to monitor the production server
5. Etc

Solution

DevOps is a software development approach which involves continuous development, continuous testing, continuous integration, continuous deployment, and continuous monitoring of the software throughout its development lifecycle.

Gluu is a 100% software company and we have to take advantage of philosophy like DevOps, tools and technologies like Jenkins, GitLab, Jira, SCM, etc.

The solution for all problems listed above is what is called a CI/CD pipeline. CI stands for Continuous Integration and CD stands for Continuous Delivery/Continuous Deployment. You can think of it as a process similar to a software development lifecycle.

CI/CD pipeline

Version control

Developers team use version control to track changes on files, the code is store in GitHub. When a developer have done with a feature or a fix, the code is pushed to GitHub.

Build

When code is push to GitHub the Build phase can start. Most of our products is Java based and use Maven* as build tool. This phase most of time check if there is compilation error.

Unit testing

When the previous phase is okay, the unit test phase run all unit tests defined in the application. Unit test is intended to validate domain object class. JUnit or TestNG can be use to write such test.

Deploy to Dev

When the Unit test is okay, the artifact generated by the build phase can be use to deploy the application in development environment.

. There are two possible scenario:

1. The dev servers are up and ready: In this case a script can be use to move artifact to dev servers.
2. The dev servers aren't ready: In this case we may have to use configuration management tool like Chef, Ansible to provision the servers and then move move

QA

This is the last verification phase, Qa test cases are done during this phase. Cucumber is the de facto tool to write and automated Qa validation.

Deploy to Prod

When the QA validation is okay, the final product can safely be deploy to production server for clients/users. At this stage we are confident about the product quality because it has pass all verification?

Monitoring/Validation(performance monitoring for example)

This phase can be ignore depending on the project, for transactions applications it is very important to automated the monitoring of the application state/metrics.

Tools

Our task is to automate the entire process, from the time a developer commits a changes to the time we get it into production. We will automate the pipeline in order to make the entire software development lifecycle in agile, maintainable and spotless . For this, we will need automation tools.

Jenkins:

The leading open source automation server, Jenkins provides hundreds of plugins to support building, deploying and automating any project

Docker:

Building and deploying new applications is faster with containers. Docker containers wrap up software and its dependencies into a standardized unit for software development that includes everything it needs to run: code, runtime, system tools and libraries. This guarantees that your application will always run the same and makes collaboration as simple as sharing a container image.

Maven:

Apache Maven is a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project's build, reporting and documentation from a central piece of information.

Junit:

JUnit 5 is the next generation of JUnit. The goal is to create an up-to-date foundation for developer-side testing on the JVM. This includes focusing on Java 8 and above, as well as enabling many different styles of testing.

Selenium:

Selenium automates browsers. That's it! What you do with that power is entirely up to you. Primarily, it is for automating web applications for testing purposes, but is certainly not limited to just that. Boring web-based administration tasks can (and should!) be automated as well.

Cucumber:

Cucumber is a tool that supports Behaviour-Driven Development(BDD). Cucumber reads executable specifications written in plain text and validates that the software does what those specifications say. The specifications consists of multiple examples, or scenarios. For example:

Git:

Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency.

Provisioning tool

SaltStack:

Use event-driven automation to transform complex systems into an organized and intelligent digital infrastructure controlled by a central nervous system that senses, learns, and reacts to important IT events across your environment.

Ansible:

Ansible is a universal language, unraveling the mystery of how work gets done. Turn tough tasks into repeatable playbooks. Roll out enterprise-wide protocols with the push of a button. Give your team the tools to automate, solve, and share.

Teraform:

HashiCorp Terraform enables you to safely and predictably create, change, and improve infrastructure. It is an open source tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.

Questions

1. What is the best provisioning tool to use?
2. How can we modified the setup.py command to run automatically without user intervention?