Dreaming Computers

Deforum Stable Diffusion

What is Deforum Stable Diffusion?

Deforum Stable Diffusion (Design Forum) builds upon Stability AI’s Stable Diffusion Model and add’s a lot of additional functionality not seen in the default notebook by Stability.  Since Stability AI (blog post) has released this model for free and commercial usages a lot of amazing new notebooks have come out that push this technology further.

Deforum Stable Diffusion (DSD) as of this writing has additional features such as animation in the form of 2D and 3D, Video Init, and a few other masking options.

Release Notes 0.4:

  • New diffusion_cadence parameter (under Animation Coherence section) to run diffusion every 3,4, whatever steps instead of every frame and interpolate the in between frames for improved coherence and animation render speed. –
  • New save_depth_maps checkbox under Animation 3D Depth Warping section to write out the depths for later use in post processing
  • Support from Doggettx for running at higher resolutions without running out of VRAM
  • Bug fixes (to prompt sanitization, video init paths, 3D animation aspect ratio)

What Is Stable Diffusion?

Stable Diffusion is a latent text-to-image diffusion model. Thanks to a generous compute donation from Stability AI and support from LAION, we were able to train a Latent Diffusion Model on 512×512 images from a subset of the LAION-5B database. Similar to Google’s Imagen, this model uses a frozen CLIP ViT-L/14 text encoder to condition the model on text prompts. With its 860M UNet and 123M text encoder, the model is relatively lightweight and runs on a GPU with at least 10GB VRAM. See this section below and the model card.
-source Stable Diffusion Github

I love how fast the Stable Diffusion (SD) model is.  In the past I would have to wait a few minutes to render on my Titan RTX, but with this new model it takes seconds to generate amazing results.  One of the most particular things I love is the fact its great at creating more human type of characters with way better facial features than other models.

I started creating pieces of art with style transfer about 2 1/2 years ago and have watched this space quickly grow and am super excited about what we have now and what’s coming. Also a super big shoutout to everyone involved in creating, updating and maintaining this stuff.

Deforum Stable Diffusion

As noted on their official GitHub page and working Google Colab Notebook

Please also help grow their Discord by joining their sever here.

You can check out the rest of the contributors here.

Most of you who have stumbled onto this page have been using Deforum Stable Diffusion and looking for hints and tips or just getting started and you want a jumping off point to get you rolling!

List of Links I recommend to get you going or advance your understanding of DD

Here are a few of my DSD Image generated So far.