GrammaTech Wins IEEE SCAM 2019 Distinguished Paper Award for Bug-Injector Research

January 2020

ITHACA, NY. & BETHESDA, MD

During the International Working Conference on Source Code Analysis & Manipulation (SCAM), a GrammaTech research publication was awarded the Institute of Electrical and Electronics Engineers (IEEE) Computer Society TCSE (Technical Council on Software Engineering) Distinguished Paper Award.

The publication was chosen through further rounds of review and voting among program committee members. The paper, available here for viewing, details Bug-Injector, a system that automatically creates benchmarks for customized evaluation of static analysis tools.

Bug-Injector works by inserting bugs based on bug templates into real-world host programs. It runs tests on the host program to collect dynamic traces, searches the traces for a point where the state satisfies the preconditions for some bug template, then modifies the host program to “inject” a bug based on that template. Injected bugs were used as test cases to build a static analysis tool evaluation benchmark. Bug-Injector pairs every injected bug with the program input that exercises that bug. The team identified a broad range of requirements and desiderata for bug benchmarks; their approach generated on-demand test benchmarks to meet these requirements. It also allowed them to create customized benchmarks suitable for evaluating tools for a specific use case (e.g., a given codebase and class of bug). Their experimental evaluation demonstrates the suitability of the generated benchmark for evaluating static bug-detection tools and for comparing the performance of different tools.

According to their website, “[t]he aim of the International Working Conference on Source Code Analysis & Manipulation (SCAM) is to bring together researchers and practitioners working on theory, techniques and applications which concern analysis and/or manipulation of the source code of computer systems. While much attention in the wider software engineering community is properly directed towards other aspects of systems development and evolution, such as specification, design and requirements engineering, it is the source code that contains the only precise description of the behavior of the system. The analysis and manipulation of source code thus remains a pressing concern.”

For more information, check out our blog post detailing Bug-Injector.

Bug-Injector research was sponsored by the Defense Advanced Research Projects Agency (DARPA) under Contract No. D17PC00096 and the Department of Homeland Security (DHS) Science and Technology Directorate, Cyber Security Division (DHS S&T/CSD) via interagency agreements HSHQDC-16-X-00076 and 70RSAT18KPM000161 with the Department of Health and Human Services (HHS) resulting in contract No. HHSP233201600062C. The views, opinions, findings, and conclusions or recommendations contained herein are those of the authors and should not be interpreted as necessarily representing the official views, policies, or endorsements, either expressed or implied, of DARPA or DHS.

About GrammaTech:

GrammaTech’s advanced static analysis tools are used by software developers worldwide, spanning a myriad of embedded software industries including avionics, government, medical, military, industrial control, automotive and other applications where reliability and security are paramount. Originally developed within Cornell University, GrammaTech is now a leading research center for software security and a commercial vendor of software-assurance tools and advanced cyber-security solutions. With both static and dynamic analysis tools that analyze source code as well as binary executables, GrammaTech continues to advance the science of superior software analysis, providing technology for developers to produce safer software. For more information, visit www.grammatech.com or follow GrammaTech on LinkedIn.