Machine Learning Technique Boosts Requirements Classification Accuracy by 27%

A recent academic study demonstrates how data preprocessing can dramatically improve the accuracy of automated software requirements classification. By addressing a fundamental imbalance in training datasets, researchers achieved a 16-point jump in predictive performance compared to baseline approaches.

The Class Imbalance Problem#

In software engineering, requirements are typically sorted into two broad categories: functional (what the system must do) and non-functional (performance, security, and other quality attributes). However, many real-world datasets contain far more examples of one class than the other, making it difficult for machine learning models to learn patterns in the underrepresented group. This class imbalance causes models to perform poorly at recognizing minority-class requirements, even when they appear accurate overall.

Solution: SMOTE-Tomek Preprocessing#

The research applies SMOTE-Tomek, a hybrid technique that combines two data-balancing strategies. SMOTE (Synthetic Minority Over-sampling Technique) generates synthetic examples of the minority class, while Tomek links remove redundant or noisy samples near decision boundaries. When paired with stratified K-fold cross-validation—a method that preserves class proportions across training folds—this approach maintains data integrity while improving model robustness.

Results and Impact#

The team tested their method on the PROMISE dataset, containing 969 manually categorized requirements. A logistic regression classifier trained with SMOTE-Tomek preprocessing achieved 76.16% accuracy, up from a baseline of 59.85%. This performance gain demonstrates that preprocessing choices can matter as much as algorithm selection in building scalable, interpretable solutions for real-world software engineering challenges.