Submitted
By Mike Hansen
Software developers can learn a lot from the example of car manufacturing. Both stand to benefit from reducing the complexity in their supply chains and gaining more control over the parts they use.
The software supply chain is growing increasingly complicated, and with that complexity comes challenges. As complexity continues to grow, software supply chain automation will usher in a new era for application development efficiency that drives increases in innovation, productivity, cost savings, and control over risk.
Over the last 15 years, software development has benefited greatly from practices and tooling created as part of the agile, lean product development and DevOps movements. These movements have led to marked increases in development efficiency. However, most of the optimization we’ve seen centers around the production of software, without real consideration for the supplier dimension. During this same period, there has been little focus on the suppliers that now feed software development—primarily from the open source software ecosystem.
A Software Supplier Sea Change
15 years ago, the vast majority of applications were written by the organization producing them. Today, with the huge and rapidly expanding body of freely available open source software, the opposite is true. Modern applications involve a significant portion of open source elements, often upwards of 80 to 90 percent. This shift in balance happened somewhat gradually. Organizations continued to witness incremental gains in productivity by leveraging more of the diverse specialization that the open source ecosystem offers.
This unchecked diversity of supply, which enables development teams to choose whatever technology is appropriate, as well as whatever version might be in vogue at the time of selection, causes unnecessary complexity. For a real-world example of this, we need look no further than the car manufacturing industry.
To produce the Prius, Toyota leverages 125 total suppliers. To produce the Chevy Volt, General Motors (GM) uses over 800 suppliers. 73 percent of the Prius is sourced from Toyota’s suppliers, whereas only 46 percent of the Volt is outsourced. This extraordinary difference in complexity for GM causes it to have significantly higher overhead costs, more variability in the flow of supplies, and greater challenges in quality management, among other disadvantages. It is no wonder the Prius outsells the Volt by 10 to one.
The software development practices of the majority of organizations today tend to have more in common with GM than Toyota. This leads to a dark matter version of technical debt, which creates a drag on overall development throughput.
If Developers Built Cars
As a thought experiment, let’s transpose modern software development practices into the production of an automobile. In this universe, Big Auto teams independently choose whatever supplier they want for a car’s transmission and whatever revision that might be available at the time. They base their decisions on what they hear and read from peers, and no one is complaining much, so the quality of the transmission is presumed to be good.
This transmission has numerous other parts inside it that were chosen from even more suppliers, which supposedly have good reputations and produce good parts. There are ways to see what parts are being used, but no one really bothers with that. Each team just wants a transmission that will work and trust that someone else tested it. Worrying about the internals is not part of their job description. If something is wrong that the supplier won’t fix, the teams think they will be able to fix it themselves.
Each team is given complete autonomy to use any supplier for any part and to change suppliers whenever they want, without any system for tracking this activity. Big Auto sensed this might not be optimal and tried to institute a tight set of controls on their supply chain several times. But without understanding the benefits of high autonomy and unobstructed flow, these attempts ended up crushing productivity and innovation and undermining its competitive position. Forced to contend with this ad hoc approach to sourcing, Big Auto was left unable to perform any kind of orderly recall. Since there weren’t too many catastrophes, Big Auto considered this approach to be “good enough.”
Ripe for Optimization
This is essentially how most software is written today. While empowering development teams to choose their own suppliers—i.e. software components—has enabled speed, increased throughput, and unlocked innovation, it also involves a significant amount of complexity, and thus inefficiency and risk.
Complexity leads to burdensome technical debt, quality issues, and challenges in keeping a system secure. The magnitude of unnecessary complexity in modern software supply chains is significant. In a large portfolio it is often enormous.
We worked with one organization that used 81 out of the 85 available versions of a popular Web framework, spanning years of release history. While that is an extreme case, the pattern of greater-than-necessary version dispersion is common. After all, the average open source project releases a new version four times each year and it is common practice to only update in response to a known issue that impacts a particular application. Given that the average application also involves hundreds of open source components, any given day can see multiple new versions across the inventory of components in use. Furthermore, issues that may be known to a particular open source project are often unknown to its consumers—i.e. the “no recall” problem mentioned earlier.
In addition to version dispersion, there is also technology dispersion, where different components that perform the same basic functions are used indiscriminately—e.g. logging frameworks and Web frameworks. Viewed in the context of a single application, such dispersion does not usually present a serious issue, but impact quickly becomes apparent when you add more.
More recently, open source vulnerabilities have become exploit targets because commonly used components with known vulnerabilities are often the path of least resistance for attackers. These vulnerabilities are now getting catchy monikers like HeartBleed, ShellShock, POODLE, and GHOST. Unfortunately, a superfluous set of suppliers with no real visibility not only leads to greater risk potential, but to incremental challenges and greater costs in securing them.
Despite the problems inherent in drawing from a large pool of suppliers, attempts to centralize control and decision making—often instituted in reaction to the manifestation of some previously unidentified risk—have failed to produce positive results and often make situations worse.
Answer: Software Supply Chain Automation
As software development evolves, we continue to create situations where complexity blocks further increases in productivity per unit cost. For instance, the introduction of high-level languages, object oriented programming, agile, lean, DevOps, and dependency management are all examples of how we have responded upon reaching the limits of scale. Unnecessary complexity causes problematic symptoms that inspire important software development innovations.
Today, there are significant opportunities to make performance gains by optimizing supply chains. By allowing for appropriate constraints or rules to be established up front, automating conformance and tooling, and verifying the information needed to remediate issues or context, organizations can markedly improve efficiency and control risk, thereby unleashing their capacity for innovation.
These gains can be achieved without limiting the current speed of production and can actually allow it to proceed faster. The use of automation has already provided significant benefits in areas of testing, build, and deployment. It is time for further automation throughout the software supply chain. Organizations can achieve this by introducing non-intrusive guardrails that factor in the necessity for autonomy and the imperative for speed. Solutions that facilitate comprehensive software supply chain automation are poised to usher in the next wave of development productivity, on par or greater with the gains possible with agile, lean, and DevOps. SW
Mike Hansen is the SVP of products for Sonotype. He offers nearly 20 years of experience building and leading global product development organizations ranging from start-ups to the Fortune 100 to this position. Most recently Hansen served as VP of development for Hanley Wood, where he led development across five product lines, including the creation of advanced business analytics platforms for the residential construction industry.
