P1-10: How Music features and Musical Data Representations Affect Objective Evaluation of Music Composition: A Review of CSMT Data Challenge 2020
Li, Yuqiang*, Li, Shengchen, Fazekas, George
Subjects (starting with primary): Evaluation, datasets, and reproducibility -> evaluation metrics ; Musical features and properties -> melody and motives ; Musical features and properties -> representations of music ; Musical features and properties -> rhythm, beat, tempo ; Evaluation, datasets, and reproducibility -> evaluation methodology
Presented Virtually: 4-minute short-format presentation
Tools and methodologies for distinguishing computer-generated melodies from human-composed melodies have a broad range of applications from detecting copyright infringement through the evaluation of generative music systems to facilitating transparent and explainable AI. This paper reviews a data challenge on distinguishing computer-generated melodies from human-composed melodies held in association with the Conference on Sound and Music Technology (CSMT) in 2020. An investigation of the submitted systems and the results are presented first. Besides the structure of the proposed models, the paper investigates two important factors that were identified as contributors to good model performance: the specific music features and the music representation used. Through an analysis of the submissions, important melody-related music features have been identified. Encoding or representation of the music in the context of neural network modes has also been found to significantly impact system performance through an experiment where the top-ranked system was re-implemented with different input representations for comparison purposes. Besides demonstrating the feasibility of developing an objective music composition evaluation system, the investigation presented in this paper also reveals some important limitations of current music composition systems opening opportunities for future work in the community.