M6: Fantastic AI Sinawi

Danbinaerin Han Danbinaerin Han graduated from Gugak National Middle School, Gugak National High School, and Seoul National University with a degree in Korean traditional music(Gugak) and is currently active as a Haegeum performer. She performed at Chamber Hall in Sejong Art Center and National Gugak Center and winning prizes in numerous music competitions (Dong-A Korean Traditional Music Competition, Incheon Korean Traditional Music Competition, 21st Korean Music Project, etc.). Under the direction of Professor Dasaem Jeong, she is presently conducting deep learning research on Korean traditional music at the Department of Art & Technology of Sogang University. She is eager to identify the similarities and differences in Eastern music as well as the distinctive melodic characteristics of Korean music. Through analysis and automatic generating of Korean music using cutting-edge technology, she looks for new directions for the genre's evolution.

Hannah Park Hannah Park is majoring in Computer Science & Engineering and double majoring in Art & Technology. She is interested in the various traces in the world. She expressed our lives using a variety of tools and languages, not limited to just one way. , an interactive media art that changes graphics and sound through audience participation, was presented in HCI Korea 2022 Interactive Art Gallary. She hopes her activities can bring happiness to many people, just like the moonlight that lights up the darkness as happiness.

Chaeryeong Oh Chaeryeong Oh is a graduate candidate with a Bachelor of Art and Science in Art & Technology and a Bachelor of Science in Computer Science and Engineering. They have focused on human interactions with and various expressions of music, such as using music as a medium to explain diverse topics, a straightforward method of shaping up ideas, a communication system, or exploring fun ways to teach the concept of music theory, under their basis of "Music for All" - easy and amusing access to music for everyone. They are currently interested in creating a generalized music framework to aid music creativity and draw entertaining musical interactions. Their work , a VR experience in which users can visually draw the melody, was awarded second place at the KT Super VR game challenge in 2020 and released as game content in KT Super VR. , the experience of the possibilities of harmonic grammar, and , the experience of using music as a communication tool, are presented in HCI Korea in 2019 and 2022. They also had an internship for the media artist group Seoul Open Media, whose leader is media artist Byungjun Kwon and enhanced their view of the fusion of Korean traditional culture and high technology. During the apprenticeship, they participated in , held by Gyeonggi Sinawi Orchestra, a Korean traditional music group of Gyeonggi-do, Korea, by using deep learning to generate new sound with Korean folk music. In Contents Impact: Music Meets AI at Korea Creative Content Agency, they also had an experience using deep learning to generate neo-Korean traditional music and held a concert with the results.

Daseam Jeong Dasaem Jeong graduated Ph.D. in Culture Technology from KAIST, researching expressive performance modeling with deep learning under the guidance of Prof. Juhan Nam. He is currently an assistant professor at the Dept. of Art & Technology, Sogang University.

“Fantastic AI Sinawi” is a live performance of a haegeum (해금) player with visuals and accompaniment. The music was composed using melodies generated by a GRU-based language model. With the form of “sinawi(시나위),” one of the most frequently played genres among Korean traditional ensemble music, a human haegeum player performs improvisation along with the accompaniment part based on AI-generated melodies. Through this project, we try to discover the future and means to transform Korean folk music with modern technology.

Sinawi has two essential features: the musical scale and improvisation. For the scale, sinawi uses “Yukjabaegi Tori(육자배기 토리).” It is characterized by deep, slow vibrato and the use of specific notes. For improvisation, each instrument player freely plays by imitating or contrasting the melodies.

We collected a sinawi melody dataset for this project, which consists of traditional sinawi and other monophonic folk melodies using the same scale. We also employed a larger monophonic melody dataset to train a base model. By fine-tuning this model with the sinawi melody dataset, we generated melodies in the style of sinawi. The generated melodies are then polished and composed into the form of one sinawi song. The accompaniment part consists of four instruments, gayageum (가야금), daegeum (대금), janggu (장구), and jing (징), pre-rendered by VSTi with rule-based MIDI CC modification.

“Fantastic AI Sinawi” consists of two jangdans (장단), a musical concept of patterned tempo and rhythm in traditional Korean music. The former part of the song is slow going, while the latter part is fast and lively. The visualization focuses on conveying different sensations between two different jangdans, goodgeori (굿거리) and jajinmori (자진모리), in the style of Korean folk painting interacting with a live performance.