Search and Filter

Musicair: A Multimodal Ai Music Generation Framework Powered By An Algorithm-Driven Core

Date

AI Overview

Why it matters

Deep learning models for music generation rely heavily on large datasets, raising risks of copyright infringement and high computing costs while often diverging from music theory conventions.

Study at a glance
  • Question: Can an algorithm-driven system generate musically coherent songs from lyrics and images without using training data?
  • Setting and data: 24 songs from children's piano books and 517 music samples generated by anonymous users in the GenAIM database.
  • Method: Prototype system development and music-theory-based metric evaluation (using music21).
  • AI tools: MusicAIR, GenAIM
Key findings
  • AI-generated songs achieved an average key confidence of 85%, outperforming human composers who averaged 79%.
  • Generated songs demonstrated high rhythmic alignment with original songs, achieving a 73.6% average rhythm matching accuracy overall.
  • Across 517 user-generated pieces, the system maintained consistent melodic smoothness, showing an average interval of 2.56 and key confidence of 0.828.

Summary drafted with AI from the full paper and may contain errors. See the full paper for details.

Recent advances in generative AI have made music generation a prominent research focus. However, many neural-based models rely on large datasets, raising concerns about copyright infringement and high-performance costs. In contrast, we propose MusicAIR, an innovative multimodal AI music generation framework powered by a novel algorithm-driven symbolic music core, effectively mitigating copyright infringement risks. The music core algorithms connect critical lyrical and rhythmic information to automatically derive musical features, creating a complete, coherent melodic score solely from the lyrics. The MusicAIR framework facilitates music generation from lyrics, text, and images. The generated score adheres to established principles of music theory, lyrical structure, and rhythmic conventions. We developed Generate AI Music (GenAIM), a web tool using MusicAIR for lyric-to-song, text-to-music, and image-to-music generation. In our experiments, we evaluated AI-generated music scores produced by the system using both standard music metrics and innovative analysis that compares these compositions with original works. The system achieves an average key confidence of 85%, outperforming human composers at 79%, and aligns closely with established music theory standards, demonstrating its ability to generate diverse, human-like compositions. As a co-pilot tool, GenAIM can serve as a reliable music composition assistant and a possible educational composition tutor while simultaneously lowering the entry barrier for all aspiring musicians, which is innovative and significantly contributes to AI for music generation.