Microservice Continuous Integration
Microservice Continuous Integration is not the same as our old CI ‘check-in and build’ process. Not even close. We started down the Continuous Integration path to solve the “really big build” problem. Builds that ran for hours required a new approach, and so CI was born. Well, with microservices we don’t have builds that run for hours, in fact there may not be a compile/link step at all.
With microservices you must think ‘functions.’ Microservices are functions that are written in a few hundred lines of code, not thousands, using languages like Python that is interpreted (no compile needed). Even if a language like GoLang is used, and a compile is required, it is minuscule compared to the days of C++ or C#. In other words, a microservice continuous integration most likely will have NO compile step. In addition, microservices are loosely coupled, so NO link step is required.
In our monolithic CI process, source code is checked out intermittently. The build executes in a ‘build directory’ and linking is performed. Linking pulls in external libraries that are needed by references defined in the source code or by the run-time environment. In really smart shops a software bill of material (SBOM) is created to validate the contents of the build. All that goes away with microservices. In microservice continuous integration the complete business of running a monolithic build is disrupted to a point of non-existence.
The Microservice Continuous Integration Shift
With microservices, we shift the important step of the software configuration management process (creating the monolithic binary) from the development side of the house to the operations side of the house. While I would like to say that there is no such thing as a CI process with microservices, I know that statement will cause some techies to just flat out disagree – yes, we do ‘build’ a container image. But that is different. The important understanding with microservices is that we are no longer managing a binary with all of the linking sorted out. Decoupling is the core of a service based architecture. This fact changes the purpose of a traditional CI process.
Re-thinking Continuous Integration
In your microservice approach, you need to rethink what microservice continuous integration means. If we stick with the CI basics and pinpoint a feature of traditional CI that continues to be important with microservices, you will find ‘software configuration management.’ The challenge is to re-think where and how this step is done. We still need to understand the configuration of our ‘logical application’ and we require the tracking of application to service dependencies. Yes, these basic features of CI are still important in this new modern architecture. In fact, in many ways it is even more important. Because our microservices are immutable, meaning unchanging, you could have multiple versions of a microservice running and consumed in production at the same time, by different sets of users. So managing the configuration of microservice versions to application versions becomes the challenge. This is exactly the same as our traditional build process. In the old build, we defined what versions of libraries would be linked into our binary. Some build experts would refer to this as ‘packaging,’ and it is still important.
DeployHub and Microservice Configuration Management
So with microservice continuous integration we seem to be throwing out the bath water but we want to keep the baby. DeployHub provides a method to perform an initial ‘packaging’ of your serviced based applications for the purpose of managing configurations. DeployHub does this through blueprinting. The blueprinting process involves defining the ‘base’ application with particular versions of microservices. If you integrate DeployHub to your CD pipeline process, DeployHub will automatically update your Application version each time a microservice which it consumes has a new version available. So each time a new container build is performed for the microservice, DeployHub tracks the new version of both the microservice and the consuming applications. In essence, DeployHub creates a SBOM that tracks all changes and relationships overtime. DeployHub provides visual maps of the relationships between deployed application versions to deployed microservice versions as well as dependency reports.
While we are starting to move away from executing long builds as part of our continuous integration, their is still a process for microservice continuous integration. At the core is configuration management. DeployHub supports your migration to microservices by providing a central hub of deployed microservice information that tracks the configuration of application versions. While this information is lost when we stop doing traditional builds, DeployHub finds and keeps the configuration data and supports it through your CI/CD process.
- A Microservice Service Catalog for Incident Response
- Domain Driven Design Catalog to Tame Microservice Sprawl
- Microservice Architecture and Your Logical Application
- Microservice Configuration Management
- Continuous Deployment and DevOps at Scale
- Register for DeployHub Team
- Reverse Proxy Setup for SaaS Deployments
- DeployHub Team User Guide
- The Hipster Store Tutorial
Ortelius Open Source Project
All Continuous Delivery and DevOps Blogs
- Continuous Deployment Blogs
- Understanding a Microservice Pipeline
- Microservice Continuous Integration – Where’s the Build?
- Working with Helm for your Microservices Releases
- Kubernetes Pipeline Challenges
- Managing Embedded Configurations
- Questions and Answers on managing a K8s Pipeline
- Running Safe Blue / Green Deployments
- The DeployHub Jenkins Plug-in
- Setting Up the Jenkins Plug-in
- The DeployHub CircleCI Orb
- Creating a Continuous Feedback Loop
- Release Agents – the Enemy of Continuous Deployments
- DeployHubs Release Architecture
- Why we need Application Packages for CD
- Agentless Deployments with DeployHub’s Engine
- Version Jumps and Tracking
- Microservice Configuration Management
- Database Updates in Your Continuous Delivery Process
- Database DevOps and DeployHub