Software Supply Chain Configuration the Newest Trend
Software Supply Chain configuration is the newest shiny object in both the DevOps and the DevSecOps community discussions. But what does it mean in relation to software development?
Historically, Supply Chain has been defined as a ‘commerce’ term that refers to tracking the logistics of goods and services moving between producer and consumer. This would include the storage and flow of raw materials with a focus on the channels that support the process.
When we look at this concept from the software development perspective, we will soon see that it is a new form of Software Configuration Management. Yes, the new SCM is SCM. The core of Supply Chain Management is the process of controlling changes in software. The new SCM expands our old practices to consider all ‘raw materials’ that we manage as part of our software build and package step. Companies purchase and download open-source libraries used in their in-house software.
Software Supply Chain Management includes interrogating these ‘raw materials’ and all of the internal code and libraries. We will soon be hearing more about Software Bill of Material reports and Difference Reports, as they are critical for understanding the objects that flow through the software we create.
Historically, not much has been discussed around the software supply chain or software configuration management practice. This area of expertise has never been ‘sexy.’ That is until the SolarWinds ‘supply chain’ hack. The Solar Winds hack was a hard lesson to learn, but it taught us to take a closer look at how we consume our ‘raw materials’ and to be more serious about how we build and package our software.
According to Fireeye, the firm who discovered the ‘backdoor’ Supply Chain hack:
“SolarWinds.Orion.Core.BusinessLayer.dll is a SolarWinds digitally-signed component of the Orion software framework that contains a backdoor that communicates via HTTP to third-party servers. We are tracking the trojanized version of this SolarWinds Orion plug-in as SUNBURST.”
It was obvious something went very wrong with the creation of that .dll. I don’t pretend to understand fully how this breach occurred. However I understand the build process and have been discussing the potential of these types of breaches for over 25 years.
Software Supply Chain and the Build
There have always been big security gaps and vulnerabilities in our most basic software development practice – the compile/link process. Most build processes are imperatively defined using an Ant, Make, or Maven script. Each time the build runs, it rebuilds all the binaries, even if the corresponding source code has not changed.
This is a problem. Instead, we need a build that is incremental. Incremental builds are difficult to script because they need more advanced logic. Most developers are not given time to create a better build process, one that only re-builds the changes. This means that it is next to impossible to audit why a binary was re-compiled because the answer is they are always recompiled. If we only recompiled the binaries that had corresponding code updates, we could more carefully audit the code and make sure that only approved coding updates were included.
This may not have caused the SolarWinds attack, but it could have. Once the hackers have gotten past the firewall, they can then find the build directory, read through the build scripts, and identify the perfect piece of code to update with their ‘bad function.’ The next time the build runs, a ‘clean all’ recompiles everything, and like magic, a SUNBURST is created.
In our not too distant past the developer community had tools that controlled software builds beyond what an imperative script could do. OpenMake Software provided a build automation solution called Meister, ensuring that incremental builds were easy. As part of the build, it generated Software Bill of Material reports and Differences reports allowing teams to easily audit and compare the actual source changes to confirm that only the approved code updates were included. Rational Software’s ClearCase used a program called ClearMake that used a process called ‘winkin’ to reuse ‘derived’ objects that had not changed. This process created an incremental build that could be easily audited.
I realize that I am simplifying the overall problem in this case. But it is a good example of the basic Software Supply Chain practice and how potentially vulnerable our processes are.
It is time to carefully think about managing our ‘raw materials’ and overall supply chain. I’m guessing that most monolithic practices will not see much change. It is too costly and impactful to re-write hundreds of CI pipelines. But that does not mean we can’t solve the supply chain puzzle as we shift into cloud-native development in a decoupled architecture.
In a decoupled application, we need to begin thinking about the supply chain. How do we track ‘raw materials’ that are now in the form of hundreds of small ‘bounded context’ functions? The good news is that an emerging software supply chain management catalog market can be part of the supply chain solution. A software supply chain management catalog, like DeployHub Pro, can track versions of services to versions of their consuming applications. Your new Software Bill of Material report will have two levels. First, an SBOM for the microservice. The second level is aggregating this data at an application level. Remember, in this new architecture a full build is not done.
It is important to understand that decoupled components are naturally ‘incremental’ and are independently built, packaged, and deployed. Managing these small services opens the door to more closely auditing their source code before they are released. As we shift from monolithic development to decoupled architectures, we have an opportunity to incorporate supply chain and configuration management best practices. This includes being able to audit the code carefully before a release is done. And remember, we must do this as fast as possible. It sounds like a big task, but well worth the effort. Collectively we will find the answers as we explore new ways to manage our DevOps pipeline and incorporate configuration management and DevSecOps principles.
About the Author:
Tracy Ragan is CEO and Co-Founder of DeployHub. She is an expert in software configuration management and pipeline practices with a hyper-focus on software supply chain security. She currently serves as a board member of the Continuous Delivery Foundation (CDF) and is the Executive Director of the Ortelius Open Source Project incubating at the CDF. Tracy often speaks at industry conferences such as DevOps World and CDCon. She was recognized by TechBeacon as one of the top 100 DevOps visionaries in 2020.
Further Reading on Software Supply Chain Management:
- Software Supply Chain Management Catalogs Explored Whitepaper
- Aggregated SBOM Reports
- Open-Source Inventory and Risk Management
- Supply Chain Versioning with Historical Trends
- Component Impact and Blast Radius
- Logical Application Views in a Decoupled Architecture
- SBOMs and Cybersecurity
- Federated Software Composition Analysis Data