Skip to main content
Riffusion featured image

Riffusion

Free

Riffusion generates music from text prompts. Try your favorite styles, instruments like saxophone or violin, modifiers like Arabic or Jamaican, genres like jazz or gospel, sounds like church bells or rain, or any combination

Riffusion featured image

About

Riffusion is a neural network designed to generate music from text prompts using images of sound rather than audio. It was created by Seth Forsgren and Hayk Martiros as a fine-tuning of Stable Diffusion, an existing open-source model for generating images from text prompts, on spectrograms.

The Riffusion model uses text prompts to generate image files, which can then be put through an inverse Fourier transform to convert them into audio files. The resulting music is described as “de otro mundo,” or “out of this world.” Although unlikely to replace man-made music, Riffusion can generate unique music that can be customized with various parameters such as style, instruments, modifiers, genres, and sounds.

The Riffusion model is also capable of interpolating between outputs to generate longer music pieces by using the latent space between outputs. This functionality is accomplished using the Stable Diffusion model’s img2img feature.

Riffusion is classified as a subset of AI text-to-music generators, along with other models such as Mubert and Google’s MusicLM. The Riffusion model was made available on December 15, 2022, with the code freely available on GitHub.

Read more
Release Date: 28 March 2023
Category:

Reviews
0.0
0.0 out of 5 stars (based on 0 reviews)
Excellent0%
Very good0%
Average0%
Poor0%
Terrible0%

Please LoggIn To Write A Review

There are no reviews yet. Be the first one to write one.

View all reviews

Frequently Asked Questions

How does Riffusion work?

Riffusion uses text prompts to generate image files, which can be put through an inverse Fourier transform and converted into audio files. Users can customize various parameters such as style, instruments, modifiers, genres, and sounds to generate music that fits their preferences.

Can Riffusion generate music in any style or genre?

Riffusion can generate music in a variety of styles and genres, but the quality and accuracy of the generated music may vary depending on the complexity of the requested style or genre.

Can Riffusion replace human composers?

Riffusion is not designed to replace human composers or musicians but rather to provide a new and innovative tool for music creation and exploration.

Is Riffusion free to use?

The Riffusion code is freely available on GitHub, but access to the platform itself may require payment or subscription, depending on the specific version or use case.

How long can the generated music pieces be?

The generated music pieces are typically only several seconds long, but the Riffusion model is capable of interpolating between outputs to generate longer pieces.

What is the output format of the generated music?

The generated music is output in the form of an audio file that can be played back on any standard media player or software.