A Comparative Study on Testing Optimization Techniques with Combinatorial Interaction Testing for Optimizing Software Product Line Testing

Abstract

A software product line (SPL) is a combination of software products that have similarities in features and functions. These combinations usually result in many feature combinations that challenge the testing process. The explosion of the combination of features can lead to exhaustive testing. This exhaustive testing will affect the time and cost for the product to be delivered to the market. This paper aims to identify the best algorithm and interaction strength to avoid exhausting testing and reduce the time and cost of the testing process. An experiment has been conducted on the most commonly used optimization algorithms in previous studies. The optimization algorithms we explored are the Genetic Algorithm, Cuckoo Search algorithm, Ant Colony algorithm, and Particle Swarm Optimization algorithm. Each algorithm has been tested with different combinatorial interaction strengths from two to six. This paper aims to get the best meta-heuristic algorithm and the optimum number of interaction strengths for optimizing the number of configurations in the SPL testing. Results show the best optimization algorithm is the Genetic Algorithm and the optimum interaction strength is t=5. This interaction strength achieves the optimum number of features combination that is sufficient for the testing process and thus can avoid the exhaustive testing in SPL testing. By using the best optimization algorithm with the optimum number of interaction strengths, the complexity of the SPL testing process could be reduced without prejudicing the quality of the software system itself.