INTRODUCTION:
Safety-critical software has hit the “unaffordable” wall due to increasing complexity and growing reliance on software to perform mission-critical functions (Redman et. al. 2010). Software developer productivity on safety-critical systems hasn’t really changed at 5 lines of code (LOC) a day and roughly 1000 LOC per year (O. Benediktsson 2000). However, the reliance on software and the code size for safety-critical software has skyrocketed. Static analysis tools are recommended by various standards and experts in the safety-critical software field and remains an essential tool for tackling the software affordability problem.
Related:
- Detecting Bugs in Safety-Critical Code
- Static versus Dynamic Detection of Bugs in Safety Critical Code
- Finding defects using Holzmann’s “Power of 10” rules for writing safety critical code
- Four Pillars for Improving the Quality of Safety-Critical Software-Reliant Systems
The Safety-Critical Software Affordability Wall
Software has become the leading cost of safety-critical systems. One third of new airplane costs are in software and software development. In automobiles, 25% of the capital costs of a new vehicle are now electronics (and software). Software has afforded amazing new capabilities, but its exponential growth and associated costs have made it effectively unaffordable:
Source: SEI, “Virtual Integration for Improved System Design“, Redman et. al, 2010
Clearly the in-step increase in development costs for larger systems has to change and various processes and techniques have been proposed. Amongst those is the use of tools and specifically static analysis to improve the test coverage and to detect defects that traditional testing cannot. In fact, both SEI and NASA recommend static analysis as an indispensable tool in safety critical software development.
The Role of Static Analysis in Safety Critical Software Development
- Code coverage isn’t everything: Many safety standards require high levels of code coverage (proof that tests executed most, if not all, statements and conditions). Although this is very exhaustive, it’s very expensive to do and must be repeated in each major phase of development (unit, integration and system testing). The criticality of the software dictates the level of coverage with some less critical software requiring no formal test coverage (e.g. aircraft on-board entertainment). Testing code coverage is one metric to evaluate software quality by, but there are cases where it doesn’t catch everything.
- Bugs that coverage-based testing miss: Testing software based on coverage metrics is inherently unit-based (although coverage is evaluated at a system level as well). Concurrency errors and security vulnerabilities are two key instances of defects that can be missed even during rigorous testing. Concurrency is often tough to program correctly and can yield errors (e.g. race conditions) that are undetected until some unforeseen condition during operation. Security vulnerabilities do manifest as bugs in the code — the conditions creating the error are often due to types of input not considered during testing. Static analysis can discover these errors early and prevent them from being show-stoppers late in the development cycle.
- Detect defects early: Rigorous testing can discover most defects in software, but it’s expensive and extremely time-consuming. Discovering and fixing these bugs when writing the code is also considerably cheaper than later in the development cycle (defect discovery is exponentially more expensive over time). Static analysis can detect bugs in the code as it is written — as part of a developer’s development environment — greatly reducing the downstream cost of defects.
- Analyzing SOUP: Use of third party code such as commercial off-the-shelf software (COTS) and open source software is a fact of life in embedded software development. Some safety standards consider any software that isn’t developed to the specific standard as software of unknown pedigree (SOUP) — software that needs to be looked at carefully for inclusion in the system. Static analysis tools can analyze third party source and binaries to discover defects and security vulnerabilities in software that could be impossible to test otherwise (without including it and running it, an expensive option).
CONCLUSION:
Safety-critical software is becoming excessively expensive to develop and manufacturers are looking for solutions that increase developer productivity. Static analysis tools are indispensable for safety-critical software development, so much so that experts in the field make them pillars of their software development processes. By increasing development and testing productivity and finding bugs that are missed by regular testing, static analysis plays a key part in breaking through the safety-critical software affordability crisis.
