Bayesian Networks for software testing

Abstract

A software design methodology is proposed that involves development of approximate models based on Bayesian Networks capturing probabilistic representations of expected behavior, which are further used in developing and running tests that can dynamically diagnose bugs and attacks during production. While automation of Software design is still a very remote goal, it can already benefit from AI tools and ideas. One of the main challenges with automating software design methods, for any product with modest complexity, is the mere intractability of enumerating all scenarios of the product usage when also taking into account user intentions.

This leads to an intractability of generating exact specifications and exhaustive tests.

We show how approximate models of the design can exploit AI techniques to represent the system and to derive meaningful tests, warning when the environment is not behaving as designed, detecting bugs and attacks. The representation can use Bayesian Networks that are rather simple, enabling usage by novice practitioners. We validate the methodologies with on two different applications: a device driver for Wi-Fi Direct, and a website, MindBlog.com.
In the Wi-Fi Direct use case, we successfully built a test ensuring the connection is fair and contrasted it experimentally to earlier work where we created a robust Bayes network based on expert knowledge.
In the MindBlog.com use case, we show that the procedure is flexible and can detect when the developers found a bug and were attempting to debug their application yielding anomalous behavior.