Melody Saber

• Documentation

• Github Repo

• Python (pretty_midi), C#, Unity3D, Oculus, Jupyter Notebook

• Chenhe Gu, Keyin Wu, Ziying Wang

• Dec, 2021

This is an immersive music game which, unlike traditional ones, allows the user to experience alternative renditions of pop songs and perceive real-time modulations to the music they hear as a result of their actions. Specifically, notes are mapped from Deep Learning-generated melody sequences to blocks which the player hits along with the music. The perceived music will also be altered if the hit direction is incorrect.

Melody Saber is an immersive music game inspired by the popular rhythm game Beat Saber. Different from Beat Saber which plays out the entire music piece regardless of the player’s performance, our Melody Saber allows the user to generate a customized game map of pop songs and play different melody clips depending on how they hit the game block. The user can generate accompaniments based on deep learning models Chorderator and Accomontage, choose a track to chop into blocks, and generate the music game map in a Virtual Reality (VR) environment through Unity. By enabling players to generate their unique melody track, Melody Saber creates an innovative music game experience combined with music game map generator.

Pipeline Design, Melody Processing, Data Transformation & Mapping, Unity Debugging, Visual Illustration