Starstruk Logo MarkStarstruk

v0.1.0

microsoft/playwrightv0.1.0Mar 21, 2025by a-r-r-o-w

AI Summary

The initial stable release of FineTrainers introduces support for new diffusion models like Wan 2.1 and CogView4, while restoring Accelerate as a parallelization backend and optimizing dataset loading.

Key Highlights

  • Support for new diffusion models: Wan 2.1 and CogView4.
  • Fully restored Accelerate as a parallelization backend.
  • Opt-in precomputation to reduce memory usage on smaller datasets.
  • Better remote dataset loading support.
  • PyPI package availability.

New Features

  • Wan 2.1 T2V model support.
  • CogView4 model support.
  • Accelerate backend restoration.
  • Opt-in precomputation feature.
  • Remote dataset loading improvements.

Full Release Notes

# Finetrainers v0.1.0 🧪

https://github.com/user-attachments/assets/37f27188-727a-4e9b-a1b9-baa621ddbb88

### New models supported
- CogView4
- Wan 2.1

### Checkpoints released
- https://huggingface.co/finetrainers/Wan2.1-T2V-1.3B-crush-smol-v0
- https://huggingface.co/finetrainers/Wan2.1-T2V-1.3B-3dgs-v0
- https://huggingface.co/finetrainers/CogView4-6B-rider-waite-tarot-v0
- https://huggingface.co/finetrainers/CogView4-6B-rider-waite-tarot-v0-shifted-sigmas

### Other major changes
- Fully support for Accelerate as a parallelization backend again
- Opt-in precomputation. Helpful for smaller datasets
- Better remote dataset loading support
- Support for `datasets>=3.4.0`
- Bug fix: Layerwise Casting now works with Wan
- Bug fix: LTX Video training now works with batch_size > 1

## What's Changed
* calling save_model_card correctly by @vivkul in https://github.com/a-r-r-o-w/finetrainers/pull/275
* 3D Parallel + Model Spec API by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/245
* CogVideoX ModelSpec by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/280
* Remove unused CogVideoX code by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/282
* Wan T2V ModelSpec by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/281
* update docs by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/284
* Improve local dataset loading by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/289
* Add 3DGS dataset example for Wan by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/290
* HunyuanVideo ModelSpec by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/287
* Fix full rank config by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/292
* update docs by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/294
* CogView4 ModelSpec by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/297
* PyPI package by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/298
* Minor cleanup and doc updates by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/304
* Remove forceful precomputation behaviour; Improvements to data loading by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/303
* Webdataset improvements; CogView4 example with The Simpsons webdataset by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/305
* Fix for #306 by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/307
* Fix Wan scaling due to upstream changes by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/308
* Fix CogVideoX config error for invert_scale_latents by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/309
* add valid names to dataset docs by @neph1 in https://github.com/a-r-r-o-w/finetrainers/pull/318
* Fix Layerwise Casting  by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/316
* Fix enable_model_cpu_offload problems by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/320
* Patch WanTimeTextImageEmbedding forward only with fp8 by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/327
* Lower memory requirements on single GPU by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/321
* Respect report_to if set to none by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/331
* Cleanup model load methods by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/333
* add __init__.py to sft_trainer/ to fix #335 by @jbilcke-hf in https://github.com/a-r-r-o-w/finetrainers/pull/336
* Add version guard for datasets 3.4.0; Fix argument documentation by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/332
* Update dataset.py by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/337
* Remove hard-coded batch size configuration in LTXModel by @Linyou in https://github.com/a-r-r-o-w/finetrainers/pull/340
* Fix missing comma in loading hunyuan video tokenizer by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/341
* Add back accelerate compatibility by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/339
* Relative -> Absolute imports by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/343
* Prepare for v0.1.0 release by @a-r-r-o-w in https://github.com/a-r-r-o-w/finetrainers/pull/322

## New Contributors
* @vivkul made their first contribution in https://github.com/a-r-r-o-w/finetrainers/pull/275
* @neph1 made their first contribution in https://github.com/a-r-r-o-w/finetrainers/pull/318
* @jbilcke-hf made their first contribution in https://github.com/a-r-r-o-w/finetrainers/pull/336
* @Linyou made their first contribution in https://github.com/a-r-r-o-w/finetrainers/pull/340

**Full Changelog**: https://github.com/a-r-r-o-w/finetrainers/compare/v0.0.1...v0.1.0