College of Engineering Unit:
The Problem
Regression testing is part of any good software development cycle. It is critical to make sure that new additions do not break existing features, and equally important to let developers know as quickly as possible. What do you do when the regression testing suite becomes too large to run quickly for your developers, and your project too important to have errors? This is exactly the situation Tektronix found themselves in with a regression testing suite that takes 8+ hours to run overnight, and developers losing work to unknown mistakes and bugs.
Our Solution: Smart Test Selection
Our solution to this particular problem is to leverage the power of machine learning to smartly select which of the tests are most likely to fail with a given code change. Our model is built on the historical overnight test results as well as meta data from each code change. The model in the end produces a list of all of the tests, ranked by their likelihood of failure based on the incoming meta data of a code change. This allows the developers to run a small subset of the most risky tests, and be fairly confident that tests that are going to fail will be among them. The model is also designed to retrain itself automatically with each new nightly regression test, improving the overall accuracy and accounting for new tests and code changes.
Industry Sponsor(s):