add stable audio tools as a library + code snippets.

[Vaibhavs10]

Adds inference snippets, download count support for stable audio tools.

[Vaibhavs10]

The failing test is unrelated to this PR.

[Wauplin]

Metadata in model-libraries.ts looks good to me. Added a comment about the snippet. Better to wait review from others :)

[Wauplin]

Code snippets are usually much smaller than this (basically from stable_audio_tools import get_pretrained_model + model, model_config = get_pretrained_model("${model.id}")).

Are we sure the whole snippet is valid for all models tagged as stable-audio-tools? (or is there only one model?). I'm not against a more complete snippet if you think it makes sense though. Happy to get opinion from others as well

[Vaibhavs10]

This is only one model - but they support other models (although there are no pre-trained models supported)

Let me see if I can reduce the snippet size.

[Vaibhavs10]

Here's the shortened snippet.

import torch
import torchaudio
from einops import rearrange
from stable_audio_tools import get_pretrained_model
from stable_audio_tools.inference.generation import generate_diffusion_cond

device = "cuda" if torch.cuda.is_available() else "cpu"

# Download model
model, model_config = get_pretrained_model("stabilityai/stable-audio-open-1.0")
sample_rate = model_config["sample_rate"]
sample_size = model_config["sample_size"]

model = model.to(device)

# Set up text and timing conditioning
conditioning = [{
    "prompt": "128 BPM tech house drum loop",
}]

# Generate stereo audio
output = generate_diffusion_cond(
    model,
    conditioning=conditioning,
    sample_size=sample_size,
    device=device
)

# Rearrange audio batch to a single sequence
output = rearrange(output, "b d n -> d (b n)")

# Peak normalize, clip, convert to int16, and save to file
output = output.to(torch.float32).div(torch.max(torch.abs(output))).clamp(-1, 1).mul(32767).to(torch.int16).cpu()
torchaudio.save("output.wav", output, sample_rate)

[osanseviero]

This is too complex imo. Usually the snippets don't have things such as loading into GPU for example. We usually also don't show the actual inference (and just do the loading), but it might be fine for this case.

Although it's just one model, the library is fine-tuning, so we need to make sure our snippet would work for those.

[Vaibhavs10]

I updated the code snippet to the minified code snippet as mentioned above!

[pcuenca]

That's a good point, happy to hear other opinions :)

[julien-c]

no preference on my side!

[Wauplin]

No strong opinion either especially since it can be reassessed later on if we see it brakes on some finetunes

[Vaibhavs10]

Cool! Should we merge with the current snippet for now? I'm setting a reminder to revisit this in 20 or so days.

[Wauplin]

Fine for me!

[Wauplin]

Fine for me!

[Vaibhavs10]

Given there is more or less agreement here, I will merge this!

[NielsRogge]

@Vaibhavs10 does the model card also require "library_name: "stable_audio" to be added? https://huggingface.co/stabilityai/stable-audio-open-1.0

[Wauplin]

@NielsRogge Yes it does! Actually, it requires library_name: stable-audio-tools in the model card metadata.

[NielsRogge]

Ok, opened a PR here: https://huggingface.co/stabilityai/stable-audio-open-1.0/discussions/25

[Vaibhavs10]

Thanks for the ping, @NielsRogge, and to take of this - I made a note in my internal doc not to forget about this!