Metamorphic Testing for Sensor Based Applications (Thesis)

Thesis (Computer Science)

Abstract

The field of Software Engineering has always strived to enable the creation of more reliable and accurate software by implementing a range of software testing techniques to ensure source code executes as intended. Traditional software testing is done by evaluating results against a known oracle, consisting of a set of acceptable outputs for each test case. A test case is another program created to emulate real world inputs and scenarios a particular software might encounter. This is an effective method of testing and is summarily an industry standard of today; but as we all know, no program is without its bugs and glitches. Detecting theses errors more effectively has become one of the most pressing objectives for many computer science industries. Perhaps the chief error detection obstacle software engineers face today is known as the oracle problem. The oracle problem arises from one of two situations. The first is when the answer to the problem the software under test is solving is difficult to constrain. This issue occurs most often in machine learning software, where a machine must perform a task without being explicitly programmed, such as the self-driving car. In this case a source code must learn how to complete a task from the input of the world around it. The second situation is when it is either impossible or too expensive to create a test for all reasonable inputs a software might encounter. Both situations leave the software developer without a means to test their software effectively. In the case of sensor data calculations, it is very difficult to calculate accurate results when given wide range of possible sensor inputs. The goal of this thesis is to evaluate a testing technique known as Metamorphic Testing within sensor based Android applications. Metamorphic Testing is a software testing technique that takes already existing test cases for a particular software and builds new test cases. This method essentially reuses test cases to apply different mathematical properties until an error is found. Using this technique we were able to detect 100% of errors introduced into a particular source code using a set of nine particular metamorphic transforms. We also performed a study on open source android application to determine how many developers are currently using testing metrics. We concluded that only 10% of the applications we evaluated contained in software tests.