AI
New Framework Bridges Categorical and Dimensional Emotion Labels in Music Recognition
Researchers develop a unified multitask learning approach that combines different emotion labeling systems to train music emotion recognition across multiple datasets.
A team of researchers has tackled a persistent problem in music emotion recognition—the inconsistency in how emotions are labeled across different datasets. Their solution: a unified framework that simultaneously handles both categorical emotion labels (like "happy" or "sad") and dimensional representations (such as valence-arousal scores).
The Core Challenge#
Music emotion recognition systems have long struggled with fragmented datasets that use incompatible labeling schemes. Some datasets classify emotions into discrete categories, while others use continuous dimensions. This mismatch prevents researchers from effectively combining multiple datasets in a single training pipeline, ultimately limiting model generalization and performance.
The Unified Framework#
The proposed multitask learning system enables training on heterogeneous datasets by treating categorical and dimensional emotion labels as complementary learning objectives. The architecture leverages both musical structural features—such as key and chord progressions—alongside MERT embeddings, a sophisticated representation that captures timbral and acoustic properties.
To enhance learning across datasets, the team incorporated knowledge distillation, a technique where teacher models trained on individual datasets transfer their learned insights to a single student model. This approach helps the model generalize more effectively across different datasets and emotion representation styles.
Experimental Results#
The framework was validated on several major music emotion datasets, including MTG-Jamendo, DEAM, PMEmo, and EmoMusic. The results demonstrate measurable improvements from each component:
- Musical features (key and chords) provided meaningful structural context
- Multitask learning improved performance across all datasets
- Knowledge distillation enhanced generalization capabilities
Most notably, the final model outperformed the winning submission from MediaEval 2021 on the MTG-Jamendo benchmark, suggesting it represents a meaningful step forward for the field.
