Submitted
By Bill Dickenson
As software development becomes more value based, the need for a fluid promotion to production process has emerged as DevOps. As the number of release cycles rise, so does the risk of disruptive code entering the system and eroding the very value DevOps created.
As described in The Failure of Risk Management, by Douglas W. Hubbard, risk assessment for release processes are rooted in traditional risk management techniques—e.g. high, medium, low—which reduce clarity and create a false sense of security.
Assessing risk requires an adjustment to the release packaging process. Risk is not a simple go or no-go decision. It is an input to an informed decision. The need for more extensive risk management is compelling.
Risk Management for DevOps
In a recent CAST software blog, Improving Code Quality in DevOps, the need to benchmark the whole environment to establish baseline metrics was discussed. The benchmark produces a set of values, which are used to guide the process. One goal of DevOps is to continue to improve the code base and not allow defective code to creep in and destroy the value of more frequent releases.
Risk is assessed in five major categories, quality, defect density, complexity, performance, and security. These metrics are discussed below.
Quality
Quality is a good indicator of risk. As code quality decreases, it opens the code up to security and performance issues as well as undue complexity. Simplicity takes time, yet time spent in development tends to pay off quickly, usually before users test.
Individual metrics should include changeability, robustness, and transferability.
Badly written code tends to manifest itself as being unclear or “spaghetti” code. Assessing the changeability of the code provides insight into the risk of deployment.
Over time, code designed for one purpose may be used in another. As a result, the robustness, and the ability of the code to stay self-contained, handle errors, and new data becomes essential to quality.
Recent data suggests that 50 percent of the time spent in maintenance is reading the existing code and making sure the fix does not do more harm than good. The faster the code can be read, the lower support costs will be.
Defect Density
Defect density is a measure of how many critical violations are present in the code versus the technical size. In any language or environment, there are programming practices deemed too poor or dangerous. Constructs with these practices are known as critical violations and represent the single largest area of risk, not only for operations but also for support.
New code should have a defect density no larger than .2 percent (2 CV per 1,000 function points) depending on tolerance. In new function points, there is no reason for a larger defect density.
For existing code, as a rule, the defect density cannot grow; it should only be managed down. Base rule of thumb is that any work packet should eliminate .2 percent of the pre-existing defect density. This amounts to fixing 2 CV per 1,000 function points.
A critical violation may open the door to hacks, serious failures, even outages. It is imperative to eliminate these. To score, divide the number of CV by 1,000 function points.
Complexity
Complexity assesses the propagation of risk across the application web. Complexity makes change more costly and lengthens change cycle. For this process, we use multiple methods of risk management.
Any given program can take a limited number of linear paths. Cyclomatic complexity—an Integration complexity for subroutines—measures the number of these paths and assesses the complexity based on those branches.
SQL complexity looks at the SQL language constructs and evaluates them for ease of understanding, number of tables invoked, and the granularity.
A poorly designed module used once may be an issue. When it’s called multiple times throughout the application, it can seriously weaken the application’s effectiveness. This is assessed through the propagated risk and transaction risk index.
Structural quality looks at whether the source code is consistent with best practices in software engineering.
Performance
Performance is an area that is underexplored. As machine environments becomes more cloud based, performance tuning is enjoying a renaissance. Every aspect of performance is under scrutiny, including code base. Performance is specific to language and there are structures that can be avoided.
Suggested methods include the Green Index, efficiency, and security.
Developed to help data centers embrace a lower carbon footprint, the Green Index looks at high physical or electrical consumption structures.
The efficiency metric examines the code for poorly structured loops, condition traces, and other structures that either reduce speed or cause unnecessary delays.
Security
The importance of the security index cannot be overstated. The Security index examines code for vulnerabilities and weaknesses related to poor or incorrect coding practices based on the CWx set of security briefs. Security holes of any type, like critical violations, are unacceptable.
Also like defect density, there are two sets of thresholds, one for new work and one for maintenance on old work.
To score, divide the number of security issues by 1,000 function points. For existing code, reductions in security issues are good.
Closing the Loop – The Decision Process
It is not advisable to combine the five areas and the related metrics into a single release metric. The packet may have one metric, which is risky, while other metrics may be within tolerance. However, the risk profile is the highest, not the average. A major packet risk should be mitigated either through reductions in the issues, or careful management of the effects.
Similarly, the whole release—multiple packets—takes on the rating of the highest risk packet, not the average. Risk is reduced by either removing or re-mediating the packet and in classic gated funnel logic, the reduction will be to the next highest packet. Security and critical violations can easily outweigh the balance of the metrics.
The packet is then released as is if the risk values are low enough and the severity and the probability are within tolerance.
Alternatively, it is possible that the code is better off being re-mediated before promotion to production, so it is deferred to the next release.
Complexity is particularly difficult to assess manually and having the information will, and should, change the behavior. Recent studies indicate that once in production, remediation is significantly more expensive than remediation in development.
CAST is an Essential Component of DevOps
Engineering for DevOps success requires balancing flexibility with speed. More importantly, it requires data. Using CAST as the basis for risk assessment has many advantages over alternative approaches. Based on a proven set of standards, CAST provides a deterministic assessment of quality that balances out the severity/probability equation. The metrics embedded within CAST support a range of tools, from packaging and code management to release and risk assessment. SW
Bill Dickenson is an independent consultant with Strategy On The Web, and former VP of application management services for IBM. He brings decades of experience in application development, maintenance and integrated operations. Dickenson offers professional experience with multiple industry verticals at IBM, PwC, CSC, and DuPont in operations, new business development, and executive leadership.
Sources http://www.castsoftware.com/products/code-analysis-tool http://blog.castsoftware.com/improving-code-quality-in-devops/ http://www.castsoftware.com/research-labs/code-quality-application-quality http://www.strategyontheweb.com
