INTRODUCTION:
Inspections including code reviews have excellent return on investment. For example, one hour spent in inspection saves up to 33 hours in product maintenance. In addition, defect removal rate – the percentage of defects found early — with inspections can reach 95%. Projects with little use of inspections have a considerably lower defect removal rate. In our previous post, we discussed how to enhance code reviews with static analysis, this post looks at the financial impact of improving your defect removal rate.
Related:
- Enhancing Code Reviews with Static Analysis
- Measuring Defect Potentials and Defect Removal Efficiency
- Software Defect Origin and Removal Methods
A typical project following a relatively light process such as CMMI (Software Engineering Institute’s Capability Maturity Model) level 1, may have a defect removal rate of 78%. A project that makes use of modern processes that include inspections and automated tools such as advanced static analysis, can achieve defect removal rates of 95% and above. Although the difference between 95% and 78% may seem small, it makes a big difference when defects cost five times more to fix in product maintenance than they do in development – a relatively conservative figure when outlier defects can be much more expensive.
The main discussion here is around software defects. However, we need to keep in mind that often software defects are the root cause of security vulnerabilities. A security vulnerability is typically built up around one of more software defects that allow an attacker to gain foothold into a software system. The cost of a exploited security vulnerability is often in the millions of dollars, which further builds a case to drive towards higher defect removal rates.
The cost benefit for removing defects early in the development lifecycle is well known. Let’s look at how this relates to the defect removal rate. The graph in Figure 1 is based on the following assumptions:
- The engineering hourly loaded labor rate is $75 per hour.
- Defects typically take four hours to fix in development.
- Defects caught after a product has been released are five times more expensive to fix, in other words, a single defect is $300 to fix in development and $1500 in product maintenance.
- Approximately 1000 defects are introduced into the project during requirements, analysis and development phases. This is also a conservative estimate for a typical project with up to a 1000 unique function points.
Figure 1: The relative cost savings per defect per defect removal rate versus the baseline of 0.75 or 75%.
Clearly, improving defect removal rate results in cost savings. As discussed in the previous section, the role of advanced static analysis in the code review process is to augment and improve the defect removal rate. Figure 2 translates the numbers from Figure 1 into dollars assuming that a project has 1000 defects introduced in the requirements and development phases.
For example, a software team already using code reviews can gain a reasonable increase of 10% in defect removal, say moving from 85% to 95% by using advanced static analysis tools, results in double the cost savings ($240,000 versus $120,000.) Note that the biggest influencer in both graphs is the cost multiplier of fixing bugs and security defects after a product is released (conservatively set at 5:1 here.)
Figure 2: The relative cost savings in dollars versus the baseline defect removal rate of 75% (0.75) for every 1000 defects introduced during development.
CONCLUSION:
Unsurprisingly, finding and removing bugs and security vulnerabilities early in development saves money. Software teams making use of advanced static analysis tools can easily improve their defect removal rates and see a significant impact on the amount of savings.
Interested in reading more on this topic?
Download the full white paper ‘Enhancing Code Reviews with Static Analysis’ here.
{{cta(‘c2c45658-dc38-4210-a8dd-c8033ad6e3e1’)}}
