RT Journal Article
JF IEEE Transactions on Software Engineering
YR 2013
VO 39
IS 6
SP 757
TI A large-scale empirical study of just-in-time quality assurance
A1 Y. Kamei,
A1 E. Shihab,
A1 B. Adams,
A1 A. E. Hassan,
A1 A. Mockus,
A1 A. Sinha,
A1 N. Ubayashi,
K1 Measurement
K1 Quality assurance
K1 Predictive models
K1 Software
K1 Entropy
K1 Object oriented modeling
K1 Accuracy
K1 just-in-time prediction
K1 Maintenance
K1 software metrics
K1 mining software repositories
K1 defect prediction
AB Defect prediction models are a well-known technique for identifying defect-prone files or packages such that practitioners can allocate their quality assurance efforts (e.g., testing and code reviews). However, once the critical files or packages have been identified, developers still need to spend considerable time drilling down to the functions or even code snippets that should be reviewed or tested. This makes the approach too time consuming and impractical for large software systems. Instead, we consider defect prediction models that focus on identifying defect-prone (“risky”) software changes instead of files or packages. We refer to this type of quality assurance activity as “Just-In-Time Quality Assurance,” because developers can review and test these risky changes while they are still fresh in their minds (i.e., at check-in time). To build a change risk model, we use a wide range of factors based on the characteristics of a software change, such as the number of added lines, and developer experience. A large-scale study of six open source and five commercial projects from multiple domains shows that our models can predict whether or not a change will lead to a defect with an average accuracy of 68 percent and an average recall of 64 percent. Furthermore, when considering the effort needed to review changes, we find that using only 20 percent of the effort it would take to inspect all changes, we can identify 35 percent of all defect-inducing changes. Our findings indicate that “Just-In-Time Quality Assurance” may provide an effort-reducing way to focus on the most risky changes and thus reduce the costs of developing high-quality software.
PB IEEE Computer Society, [URL:http://www.computer.org]
SN 0098-5589
LA English
DO 10.1109/TSE.2012.70
LK http://doi.ieeecomputersociety.org/10.1109/TSE.2012.70