Get Started

Music Generation, MusicGen, Stable Audio, Suno, and the Licensing Earthquake

2026 music generation: Suno v5 and Udio v4 dominate commercial; MusicGen, Stable Audio Open, and ACE-Step lead open-source. The technical problem is mostly solved. The legal problem (Warner Music $500M settlement, UMG settlement) reshaped the field in 2025-2026.

75 minutesBuildPythonFree lesson

The Problem

Text → a 30-second to 4-minute music clip, with lyrics, vocals, and structure. Three sub-problems:

  1. Instrumental generation. Text like "lo-fi hip-hop drums with warm keys" → audio. MusicGen, Stable Audio, AudioLDM.
  2. Song generation (with vocals + lyrics). "Country song about rainy Texas nights" → full song. Suno, Udio, YuE, ACE-Step.
  3. Conditional / controllable. Extend an existing clip, regenerate a bridge, swap genre, stem-separate, or inpaint. Udio's inpainting + stem separation is the 2026 feature to match.

The Concept

Music generation: token-LM vs diffusion, the 2026 model map

Token LM over neural-codec tokens

Meta's MusicGen (2023, MIT) and many derivatives: condition on text/melody embeddings, autoregressively predict EnCodec tokens (32 kHz, 4 codebooks), decode with EnCodec. 300M - 3.3B params. Strong baseline; struggles past 30 seconds.

ACE-Step (open-source, 4B XL released April 2026) extends this for full-song lyric-conditioned generation. The open community's closest thing to Suno.

Diffusion over mels or latents

Stable Audio (2023) and Stable Audio Open (2024): latent diffusion on compressed audio. Excels at loops, sound design, ambient textures. Not great at structured full songs.

AudioLDM / AudioLDM2: text-to-audio via T2I-style latent diffusion, generalized to music, sound effects, speech.

Hybrid (production), Suno, Udio, Lyria

Closed weights. Likely AR codec LM + diffusion-based vocoder with specialized voice / drum / melody heads. Suno v5 (2026) is the ELO 1293 quality leader. Udio v4 adds inpainting + stem separation (bass, drums, vocals separate downloads).

Evaluation

  • FAD (Fréchet Audio Distance). Embedding-level distance between generated vs real audio distribution using VGGish or PANNs features. Lower is better. MusicGen small: 4.5 FAD on MusicCaps; SOTA ~3.0.
  • Musicality (subjective). Human preference. Suno v5 ELO 1293 leads.
  • Text-audio alignment. CLAP score between prompt and output.
  • Musicality artifacts. Off-beat transitions, vocal-phrase drift, loss of structure past 30 s.

2026 model map

ModelParamsLengthVocalsLicense
MusicGen-large3.3B30 snoMIT
Stable Audio Open1.2B47 snoStability non-commercial
ACE-Step XL (Apr 2026)4B> 2 minyesApache-2.0
YuE7B> 2 minyes, multilingualApache-2.0
Suno v5 (closed)?4 minyes, ELO 1293commercial
Udio v4 (closed)?4 minyes + stemscommercial
Google Lyria 3 (closed)?real-timeyescommercial
MiniMax Music 2.5?4 minyescommercial API
  • Warner Music vs Suno settlement. $500M. WMG now has oversight of AI-likeness, music rights, and user-generated tracks on Suno. Similar UMG settlement on Udio.
  • EU AI Act + California SB 942: AI-generated music must be disclosed.
  • Riffusion / MusicGen under MIT have no compliance baggage but also no commercial vocals.

Safe-to-ship patterns:

  1. Generate instrumental only (MusicGen, Stable Audio Open, MIT/CC0 outputs).
  2. Use commercial APIs (Suno, Udio, ElevenLabs Music) with per-generation license.
  3. Train on owned or licensed catalog (most enterprises end up here).
  4. Tag generations with watermarks + metadata.

Build It

Step 1: generate with MusicGen

Python
from audiocraft.models import MusicGen
import torchaudio

model = MusicGen.get_pretrained("facebook/musicgen-small")
model.set_generation_params(duration=10)
wav = model.generate(["upbeat synthwave with driving drums, 128 BPM"])
torchaudio.save("out.wav", wav[0].cpu(), 32000)

Three sizes: small (300M, fast), medium (1.5B), large (3.3B). Small is enough for "does the idea land."

Step 2: melody conditioning

Python
melody, sr = torchaudio.load("humming.wav")
wav = model.generate_with_chroma(
    ["jazz piano cover"],
    melody.squeeze(),
    sr,
)

MusicGen-melody takes a chromagram and preserves the tune while swapping timbre. Useful for "give me this melody as a string quartet."

Step 3: FAD evaluation

Python
from frechet_audio_distance import FrechetAudioDistance
fad = FrechetAudioDistance()

fad.get_fad_score("generated_folder/", "reference_folder/")

Computes VGGish-embedding distance. Useful for genre-level regression tests; not a substitute for human listeners.

Step 4: adding to the LLM-music workflow

Combine with the ideas from Lessons 7-8:

Python
prompt = "Write a 30-second jazz loop. Describe the drums, bass, and piano voicing."
description = llm.complete(prompt)
music = musicgen.generate([description], duration=30)

Use It

GoalStack
Instrumental sound designStable Audio Open
Game / adaptive musicGoogle Lyria RealTime (closed)
Full songs with vocals (commercial)Suno v5 or Udio v4 with explicit license
Full songs with vocals (open)ACE-Step XL or YuE
Short ad jingleMusicGen melody-conditioned on a hummed reference
Music-video backgroundMusicGen + Stable Video Diffusion

Pitfalls that still ship in 2026

  • Copyright-laundering prompts. "Song in the style of Taylor Swift", commercial Suno/Udio filter these now, open models do not. Add your own filter list.
  • Repetition / drift past 30 s. AR models loop. Crossfade multiple generations, or use ACE-Step for structural coherence.
  • Tempo drift. Models wander off the BPM. Use BPM tags in the prompt and post-filter with librosa's beat_track.
  • Vocal intelligibility. Suno is excellent; open models are often mushy on words. If lyrics matter, use a commercial API or fine-tune.
  • Mono output. Open models generate mono or fake-stereo. Upgrade with a proper stereo reconstruction (ezst, Cartesia's stereo diffusion).

Ship It

Save as outputs/skill-music-designer.md. Pick model, license strategy, length / structure plan, and disclosure metadata for a music-gen deployment.

Exercises

  1. Easy. Run code/main.py. It produces a "generative" chord progression + drum pattern as ASCII symbols, a music-gen cartoon. Play it back via any MIDI renderer if you want.
  2. Medium. Install audiocraftgenerate 10-second clips across 4 genre prompts with MusicGen-small, measure FAD against a reference genre set.
  3. Hard. Using ACE-Step (or MusicGen-melody), generate three variations of the same tune with different timbre prompts. Compute CLAP similarity to the prompt to verify alignment.

Key Terms

TermWhat people sayWhat it actually means
FADAudio FIDFréchet distance between embedding distributions of real vs generated.
ChromagramMelody as pitches12-dim per-frame vector; input to melody conditioning.
StemsInstrument tracksSeparated bass / drums / vocals / melody as WAV.
InpaintingRegen a sectionMask a time window; model regenerates just that.
CLAPText-audio CLIPContrastive audio-text embedding; eval text-audio alignment.
EnCodecMusic codecMeta's neural codec used by MusicGen; 32 kHz, 4 codebooks.

Further Reading

The full code

Every file for this lesson. Copy it, run it, then change it.

main.py
"""Music-generation cartoon: symbolic chord/drum generation from a prompt.

This is a pedagogical stand-in. Real music-gen uses neural codec LM
(MusicGen / ACE-Step) or latent diffusion (Stable Audio). Here we walk
through the "tokens over time" idea at a symbolic level so the shape is
visible.

Stdlib only. Run: python3 code/main.py
"""

import random


MAJOR_KEYS = {
    "C": ["C", "Dm", "Em", "F", "G", "Am", "Bdim"],
    "G": ["G", "Am", "Bm", "C", "D", "Em", "F#dim"],
    "D": ["D", "Em", "F#m", "G", "A", "Bm", "C#dim"],
    "A": ["A", "Bm", "C#m", "D", "E", "F#m", "G#dim"],
}

COMMON_PROGRESSIONS = {
    "pop":     [1, 5, 6, 4],
    "ballad":  [1, 6, 4, 5],
    "jazz":    [2, 5, 1, 6],
    "rock":    [1, 4, 5, 1],
    "lofi":    [6, 4, 1, 5],
}

DRUM_PATTERNS = {
    "pop":    "X.o.X.o.X.o.X.o.",
    "rock":   "X..oX..oX..oX..o",
    "lofi":   "X...o...X...o.o.",
    "jazz":   "X.oox.oxX.oox.ox",
    "trap":   "Xooox.oxXooox.ox",
}


def chord_progression(key, genre, bars=8):
    scale = MAJOR_KEYS[key]
    pat = COMMON_PROGRESSIONS.get(genre, COMMON_PROGRESSIONS["pop"])
    repeats = bars // len(pat) + 1
    seq = (pat * repeats)[:bars]
    return [scale[i - 1] for i in seq]


def drum_pattern(genre, bars=8):
    base = DRUM_PATTERNS.get(genre, DRUM_PATTERNS["pop"])
    return (base * bars)[: bars * 16]


def fake_generate(prompt, rng=None):
    rng = rng or random.Random(0)
    prompt_lower = prompt.lower()
    key = "C"
    for k in MAJOR_KEYS:
        if f" {k.lower()}" in " " + prompt_lower:
            key = k
            break
    genre = "pop"
    for g in COMMON_PROGRESSIONS:
        if g in prompt_lower:
            genre = g
            break
    bars = 8
    bpm = 120
    for token in prompt_lower.split():
        if token.endswith("bpm"):
            try:
                bpm = int(token[:-3])
            except ValueError:
                pass
    return {
        "key": key,
        "genre": genre,
        "bpm": bpm,
        "bars": bars,
        "chords": chord_progression(key, genre, bars),
        "drums": drum_pattern(genre, bars),
    }


def visualize(piece):
    print(f"  key: {piece['key']}  genre: {piece['genre']}  tempo: {piece['bpm']} bpm  bars: {piece['bars']}")
    print(f"  chords: {' | '.join(piece['chords'])}")
    drum = piece["drums"]
    print(f"  drums (kick=X snare=o): {drum}")


def main():
    prompts = [
        "upbeat pop in G major at 128 bpm",
        "slow lofi groove in C",
        "rock anthem in D at 140 bpm",
        "jazz swing in A",
    ]

    print("=== Step 1: prompt → symbolic music piece (toy) ===")
    for p in prompts:
        print(f"prompt: {p!r}")
        piece = fake_generate(p)
        visualize(piece)
        print()

    print("=== Step 2: 2026 music-gen model cheatsheet ===")
    models = [
        ("MusicGen-large",     3300, "30 s",  "no",  "MIT"),
        ("Stable Audio Open",  1200, "47 s",  "no",  "non-commercial"),
        ("ACE-Step XL (Apr 26)", 4000, "2 min+", "yes", "Apache-2.0"),
        ("YuE",                7000, "2 min+", "yes", "Apache-2.0"),
        ("Suno v5 (closed)",      0, "4 min",  "yes", "commercial"),
        ("Udio v4 (closed)",      0, "4 min",  "yes + stems", "commercial"),
    ]
    print("  | model               | params (M) | length | vocals | license        |")
    for name, p, length, v, lic in models:
        print(f"  | {name:<20} | {p:>10} | {length:>6} | {v:<12} | {lic:<14} |")

    print()
    print("takeaways:")
    print("  - open models: MusicGen (instrumental), ACE-Step / YuE (full song)")
    print("  - commercial: Suno v5 = quality leader; Udio v4 = producer tools (stems + inpaint)")
    print("  - legal: Warner + UMG settlements (2025-2026) define safe zones")
    print("  - always tag AI-generated music with watermark + metadata disclosure")


if __name__ == "__main__":
    main()
You've finished the free half of Speech & Audio

Most people stop here. Don't be most people.

The next 8 lessons are where Phase 06 turns from understanding into ability: the harder builds, the production patterns and the quizzes that prove you've actually got it. Unlock the full AI Expert Masterclass: all 542 expert lessons, every line of code, every phase.

  1. 10Audio-Language Models, Qwen2.5-Omni, Audio Flamingo, GPT-4o Audio
  2. 11Real-Time Audio Processing
  3. 12Build a Voice Assistant Pipeline, The Phase 6 Capstone
  4. 13Neural Audio Codecs, EnCodec, SNAC, Mimi, DAC and the Semantic-Acoustic Split
  5. + 4 more in this phase, and 542 across the whole masterclass