YuE: Open-Source AI Generates Full Songs from Lyrics

Ruibin Yuan has introduced YuE, an open-source AI model designed to generate full-length songs from lyrics, a capability that has so far been dominated by closed commercial platforms like Suno and Udio.

YuE, pronounced “yeah”, can create up to five-minute-long compositions with both vocals and instrumental accompaniment. The system supports multiple languages, including English, Mandarin, Cantonese, Japanese, and Korean, and can even switch between languages mid-song.

How YuE Works

YuE consists of two specialized models—one focused on vocals and music generation, and another for production elements. The system includes:

• Lyrics-to-Song AI – Generates music that follows lyrical content throughout the entire song, maintaining a coherent structure, melody, and accompaniment.

• Advanced Vocal Capabilities – Supports complex techniques such as scatting, mixed-voice performances, a cappella, riffs, and runs, mimicking skills that take years to master.

• Genre & Mood Control – Users can customize instrument choices, vocal style, mood, and musical structure to fine-tune their creations.

• Long-Context Modeling – Unlike earlier music AIs that struggled with short clips, YuE can generate full-length tracks up to five minutes long.

Breaking New Ground in AI Music

The biggest challenge in AI-generated music has been handling long-form compositions with vocals. Previous AI models excelled at generating short instrumental clips, but producing entire structured songs with lyrics, melody, and accompaniment has remained a difficult technical hurdle.

To overcome this, YuE introduces several key innovations:

• Semantically Enhanced Audio Tokenizer – Reduces training costs and speeds up learning. Dual-Token Technique – Synchronizes vocals and instrumentals without modifying the underlying AI architecture.

• Lyrics Chain-of-Thoughts – Allows the model to progressively generate a full song within a single context, ensuring that lyrics and melody stay connected.

• Three-Stage Training – Improves scalability, musicality, and lyrics adherence, making the AI more responsive to lyrical content.

While YuE showcases impressive multilingual and vocal capabilities, the specific training data used to develop the model has not been disclosed. Additionally, there is no available information on how YuE handles data privacy or storage, raising questions about where user data is processed and whether it adheres to global data protection standards

You can see the research here and the GitHub page here.

YuE is available to try for yourself here.

What This Means

The introduction of open-source AI music generation like YuE could disrupt the AI music industry, which is currently dominated by closed platforms like Suno and Udio. While these commercial systems have achieved impressive results, they are facing growing legal challenges from record labels over copyright concerns.

An open-source alternative like YuE democratizes access to AI-generated music, allowing musicians, producers, and independent creators to experiment without relying on proprietary systems. Whether from YuE or another open-source competitor, AI-powered music composition is evolving rapidly—and the music industry is already grappling with the impact.

Editor’s Note: This article was created by Alicia Shapiro, CMO of AiNews.com, with writing, image, and idea-generation support from ChatGPT, an AI assistant. However, the final perspective and editorial choices are solely Alicia Shapiro’s. Special thanks to ChatGPT for assistance with research and editorial support in crafting this article.