Starstruk Logo MarkStarstruk

v1.5.0

Henry-23/VideoChatv1.5.0Mar 25, 2025by lawrence-cj

AI Summary

A major update merging SANA-1.5 and SANA-Sprint, featuring significant performance improvements, support for 4K images, and new training capabilities.

Key Highlights

  • Merged SANA-1.5 and SANA-Sprint releases
  • Added support for 4K images with VAE-tiling to prevent Out Of Memory errors
  • Integrated `diffusers` support for Sana models
  • Added ControlNet and Sana-LoRA training capabilities
  • Implemented FSDP training code and lower VRAM inference requirements

New Features

  • Lower VRAM requirements for inference
  • Fix config loading for Jupyter environments
  • Fixed Dockerfile
  • Fixed relative imports and triton dependencies
  • Added ComfyUI related files and T2I2V video link
  • Added diffusers support
  • Added support for various image extensions in SanaImgDataset
  • Added Sana-LoRA training and guidance
  • Added config files for 2K models
  • Updated code for better quality
  • Fixed sample num bug in inference
  • Fixed RNG state resume bug with different GPU numbers
  • Added ImgDataset2WebDatasetMS tool
  • Fixed 4K OOM with VAE-tiling
  • Updated README
  • Fixed CSS properties and documentation
  • Fixed AutoencoderDC encode training bug
  • Added 8-bit CAME optimizer code
  • Added ControlNet support
  • Added FSDP training code
  • Added inference scaling update
  • Added guidance exploration
  • Added diffusers conversion script

Full Release Notes

## What's Changed
* Inference with lower VRAM requirements by @frutiemax92 in https://github.com/NVlabs/Sana/pull/18
* Fix config loading for jupyter environments by @odusseys in https://github.com/NVlabs/Sana/pull/30
* Fix Dockerfile by @JimothyJohn in https://github.com/NVlabs/Sana/pull/34
* Relative import xformer triton depend fix by @lawrence-cj in https://github.com/NVlabs/Sana/pull/37
* remove triton dependency of DC-AE and fix bugs; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/38
* Refine gradio by @yujincheng08 in https://github.com/NVlabs/Sana/pull/39
* fix the bug of stop training around 4 hours. by @lawrence-cj in https://github.com/NVlabs/Sana/pull/66
* add comfyui related files by @lawrence-cj in https://github.com/NVlabs/Sana/pull/77
* add a T2I2V video link into comfyui readme by @lawrence-cj in https://github.com/NVlabs/Sana/pull/78
* 1. `diffusers` support Sana; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/88
* receive any kinds of image extension in `SanaImgDataset`; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/94
* add Sana-LoRA training and guidance by @lawrence-cj in https://github.com/NVlabs/Sana/pull/98
* 1. update for 2Kpx models release; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/103
* add config file for 2K models; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/104
* update code for better quality by @lawrence-cj in https://github.com/NVlabs/Sana/pull/105
* fix the sample num bug in inference. by @lawrence-cj in https://github.com/NVlabs/Sana/pull/108
* Nv labs GitHub repo/nv labs GitHub repo main by @lawrence-cj in https://github.com/NVlabs/Sana/pull/124
* fix the bug when resume rng_state with diffuerent gpu numbers. by @lawrence-cj in https://github.com/NVlabs/Sana/pull/131
* support 4K images by @lawrence-cj in https://github.com/NVlabs/Sana/pull/135
* [Tool]ImgDataset2WebDatasetMS by @Pevernow in https://github.com/NVlabs/Sana/pull/130
* fix 4K OOM with VAE-tiling by @lawrence-cj in https://github.com/NVlabs/Sana/pull/144
* docs: update README.md by @eltociear in https://github.com/NVlabs/Sana/pull/151
* fix: correct CSS property & missing articles by @CharlesCNorton in https://github.com/NVlabs/Sana/pull/152
* fix the bug for `AutoencoderDC` encode training; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/158
* Nv labs GitHub repo/nv labs GitHub repo main by @lawrence-cj in https://github.com/NVlabs/Sana/pull/163
* update CI script; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/165
* Sana1.5 8 bit CAME optimizer code release by @lawrence-cj in https://github.com/NVlabs/Sana/pull/167
* Nv labs GitHub repo/nv labs GitHub repo main adding controlnet (#23) by @lawrence-cj in https://github.com/NVlabs/Sana/pull/177
* update README by @lawrence-cj in https://github.com/NVlabs/Sana/pull/195
* SANA-1.5 && FSDP training code by @lawrence-cj in https://github.com/NVlabs/Sana/pull/196
* inference scaling update by @hills-code in https://github.com/NVlabs/Sana/pull/208
* Merging SANA-Sprint release code by @lawrence-cj in https://github.com/NVlabs/Sana/pull/198
* add a guidance to explore; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/212
* update diffusers conversion script; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/213

## New Contributors
* @frutiemax92 made their first contribution in https://github.com/NVlabs/Sana/pull/18
* @odusseys made their first contribution in https://github.com/NVlabs/Sana/pull/30
* @JimothyJohn made their first contribution in https://github.com/NVlabs/Sana/pull/34
* @lawrence-cj made their first contribution in https://github.com/NVlabs/Sana/pull/37
* @yujincheng08 made their first contribution in https://github.com/NVlabs/Sana/pull/39
* @Pevernow made their first contribution in https://github.com/NVlabs/Sana/pull/130
* @eltociear made their first contribution in https://github.com/NVlabs/Sana/pull/151
* @CharlesCNorton made their first contribution in https://github.com/NVlabs/Sana/pull/152
* @hills-code made their first contribution in https://github.com/NVlabs/Sana/pull/208

**Full Changelog**: https://github.com/NVlabs/Sana/commits/v1.5.0

## What's Changed
* Inference with lower VRAM requirements by @frutiemax92 in https://github.com/NVlabs/Sana/pull/18
* Fix config loading for jupyter environments by @odusseys in https://github.com/NVlabs/Sana/pull/30
* Fix Dockerfile by @JimothyJohn in https://github.com/NVlabs/Sana/pull/34
* Relative import xformer triton depend fix by @lawrence-cj in https://github.com/NVlabs/Sana/pull/37
* remove triton dependency of DC-AE and fix bugs; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/38
* Refine gradio by @yujincheng08 in https://github.com/NVlabs/Sana/pull/39
* fix the bug of stop training around 4 hours. by @lawrence-cj in https://github.com/NVlabs/Sana/pull/66
* add comfyui related files by @lawrence-cj in https://github.com/NVlabs/Sana/pull/77
* add a T2I2V video link into comfyui readme by @lawrence-cj in https://github.com/NVlabs/Sana/pull/78
* 1. `diffusers` support Sana; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/88
* receive any kinds of image extension in `SanaImgDataset`; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/94
* add Sana-LoRA training and guidance by @lawrence-cj in https://github.com/NVlabs/Sana/pull/98
* 1. update for 2Kpx models release; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/103
* add config file for 2K models; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/104
* update code for better quality by @lawrence-cj in https://github.com/NVlabs/Sana/pull/105
* fix the sample num bug in inference. by @lawrence-cj in https://github.com/NVlabs/Sana/pull/108
* Nv labs GitHub repo/nv labs GitHub repo main by @lawrence-cj in https://github.com/NVlabs/Sana/pull/124
* fix the bug when resume rng_state with diffuerent gpu numbers. by @lawrence-cj in https://github.com/NVlabs/Sana/pull/131
* support 4K images by @lawrence-cj in https://github.com/NVlabs/Sana/pull/135
* [Tool]ImgDataset2WebDatasetMS by @Pevernow in https://github.com/NVlabs/Sana/pull/130
* fix 4K OOM with VAE-tiling by @lawrence-cj in https://github.com/NVlabs/Sana/pull/144
* docs: update README.md by @eltociear in https://github.com/NVlabs/Sana/pull/151
* fix: correct CSS property & missing articles by @CharlesCNorton in https://github.com/NVlabs/Sana/pull/152
* fix the bug for `AutoencoderDC` encode training; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/158
* Nv labs GitHub repo/nv labs GitHub repo main by @lawrence-cj in https://github.com/NVlabs/Sana/pull/163
* update CI script; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/165
* Sana1.5 8 bit CAME optimizer code release by @lawrence-cj in https://github.com/NVlabs/Sana/pull/167
* Nv labs GitHub repo/nv labs GitHub repo main adding controlnet (#23) by @lawrence-cj in https://github.com/NVlabs/Sana/pull/177
* update README by @lawrence-cj in https://github.com/NVlabs/Sana/pull/195

## New Contributors
* @frutiemax92 made their first contribution in https://github.com/NVlabs/Sana/pull/18
* @odusseys made their first contribution in https://github.com/NVlabs/Sana/pull/30
* @JimothyJohn made their first contribution in https://github.com/NVlabs/Sana/pull/34
* @yujincheng08 made their first contribution in https://github.com/NVlabs/Sana/pull/39
* @Pevernow made their first contribution in https://github.com/NVlabs/Sana/pull/130
* @eltociear made their first contribution in https://github.com/NVlabs/Sana/pull/151
* @CharlesCNorton made their first contribution in https://github.com/NVlabs/Sana/pull/152

**Full Changelog**: https://github.com/NVlabs/Sana/commits/v1.0.0

## What's Changed
* SANA-1.5 && FSDP training code by @lawrence-cj in https://github.com/NVlabs/Sana/pull/196
* inference scaling update by @hills-code in https://github.com/NVlabs/Sana/pull/208
* Merging SANA-Sprint release code by @lawrence-cj in https://github.com/NVlabs/Sana/pull/198
* add a guidance to explore; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/212
* update diffusers conversion script; by @lawrence-cj in https://github.com/NVlabs/Sana/pull/213

## New Contributors
* @hills-code made their first contribution in https://github.com/NVlabs/Sana/pull/208

**Full Changelog**: https://github.com/NVlabs/Sana/compare/v1.0.0...v1.5.0