This week we added a new dynamic testing tool to our product line.
Many of our customers, most especially those who are working in embedded safety-critical development, are deeply committed to finding and eliminating software defects in their products. They often have to meet very stringent testing requirements in order to get certification, and tool support for achieving those requirements is essential. We have often been asked for guidance in selecting a dynamic testing tool that complements our static analysis offering.
Cantata
Rather than build testing technology ourselves, we chose to partner with European company QA Systems, the makers of the highly-respected testing tool named Cantata. Cantata was originally developed in the UK by a company named IPL. That division was then purchased by QA Systems, so the engineering team remained intact. They are all top-notch technically and are great to work with.
Cantata has been a successful product for years, so it is already highly mature and capable. It helps you create and manage test suites, and measures code coverage for those test cases so you can understand what parts of the code need more testing. Perhaps most importantly, Cantata produces the reports that are necessary to achieve certification for various safety-critical development standards.
Coding Standard Certification
There are now several such standards for safety-critical software development, detailed on our site here. The use of these standards is increasing worldwide, not just because they are a good idea, but because more regulatory authorities are requiring them. Cantata is out-of-the-box ready to make it straightforward to satisfy the testing requirements and has itself been certified to be capable of supporting these standards.
We are finding that our static analysis customers wish to use CodeSonar results to support their compliance, so we have started the effort to get that certification as well, and are working with TÜV-SGS in Germany to do so. When complete, we will be able to supply our customers with a compliance “kit” tailored for each of the standards we support. CodeSonar will be able to automatically generate the reports that will form part of the submission to the regulatory authority.
MC/DC coverage
It is worth pointing out one of the ways that static and dynamic analysis can complement each other very well. Some coding standards require that code be tested to MC/DC coverage. This can be difficult to achieve because the test engineer must craft exactly the right set of inputs to make boolean expressions evaluate to both true and false. This can involve a painstaking effort to reason about how data flows through the code, and how the values affect control flow.
It is fundamentally impossible to achieve MC/DC if the code contains conditions that are either always true or always false. If the test engineers don’t know that, then they can waste precious hours attempting to achieve the impossible. This is exactly the kind of property that static analysis tools are good at uncovering automatically. CodeSonar’s Redundant Condition check is just the ticket. Other checks such as Unreachable Code or Useless Assignment (where a variable is assigned a value it is guaranteed to already have) are useful because they tell the engineer that the code is not ready to undergo stringent compliance testing, thereby saving lots of time.
We’re excited to offer Cantata, to provide our customers with a well-rounded solution for achieving the highest quality code.
