Improved Software Defect Prediction with Gated Tab Transformer

Saranya Manikandan, Duksan Ryu

http://doi.org/10.5626/JOK.2025.52.3.196

Software Defect Prediction (SDP) plays a crucial role in ensuring software quality and reliability. Although, traditional machine learning and deep learning models are widely used for SDP, recent advancements in the field of natural language processing have paved the way for applying transformer-based models in software engineering tasks. This paper investigated transformer-based model as a potential approach to improve SDP model quality, ultimately aiming to enhance software quality and optimize testing resource allocation. Inspired by the Gated Tab Transformer’s (GTT) ability to effectively model relationship within features, we evaluated its effectiveness in SDP. We conducted experiments using 15 software defect datasets and compared results with other state-of-the-art machine learning and deep learning models. Our experiments showed that GTT outperformed state-of-the-art machine learning models in terms of recall, balance, and AUC (increase by 42.1%, 10.93%, and 7.1%, respectively). Cohen's d confirmed this advantage with large and medium effect sizes for GTT on these metrics. Additionally, an ablation study assessed the impact of hyperparameter variations on performance. Thus, GTT's effectiveness address the challenges of SDP, potentially leading to more effective testing resource allocation and improved software quality.