Starstruk Logo MarkStarstruk

August-2024

unslothai/unslothAugust-2024Aug 21, 2024by danielhanchen

AI Summary

Adds Phi 3.5 support and various bug fixes.

Key Highlights

  • Phi 3.5 support
  • Llama 3.1 support
  • Gemma support
  • Torch 2.4 and Xformers > 0.0.27 support

New Features

  • Phi 3.5 support
  • Llama 3.1 support
  • Gemma support
  • Python 3.12 support

Full Release Notes

## Phi 3.5 is here!
Try it out here: https://colab.research.google.com/drive/1lN6hPQveB_mHSnTOYifygFcrO8C1bxq4?usp=sharing

## What's Changed
* Llama 3.1 by @danielhanchen in https://github.com/unslothai/unsloth/pull/797
* Better debugging by @danielhanchen in https://github.com/unslothai/unsloth/pull/826
* fix UnboundLocalError by @xyangk in https://github.com/unslothai/unsloth/pull/834
* Gemma by @danielhanchen in https://github.com/unslothai/unsloth/pull/843
* Fix ROPE extension issue and device mismatch by @xyangk in https://github.com/unslothai/unsloth/pull/840
* Fix RoPE extension by @danielhanchen in https://github.com/unslothai/unsloth/pull/846
* fix: fix config.torch_dtype bug by @relic-yuexi in https://github.com/unslothai/unsloth/pull/874
* pascal support by @emuchogu in https://github.com/unslothai/unsloth/pull/870
* Fix tokenizers by @danielhanchen in https://github.com/unslothai/unsloth/pull/887
* Torch 2.4, Xformers>0.0.27, TRL>0.9, Python 3.12 + bug fixes by @danielhanchen in https://github.com/unslothai/unsloth/pull/902
* Fix DPO stats by @danielhanchen in https://github.com/unslothai/unsloth/pull/906
* Fix Chat Templates by @danielhanchen in https://github.com/unslothai/unsloth/pull/916
* Fix chat templates by @danielhanchen in https://github.com/unslothai/unsloth/pull/917
* Bug Fixes by @danielhanchen in https://github.com/unslothai/unsloth/pull/920
* Fix mapping by @danielhanchen in https://github.com/unslothai/unsloth/pull/921
* untrained tokens llama 3.1 base by @danielhanchen in https://github.com/unslothai/unsloth/pull/929
* Bug #930 by @danielhanchen in https://github.com/unslothai/unsloth/pull/931
* Fix NEFTune by @danielhanchen in https://github.com/unslothai/unsloth/pull/937
* Update README.md by @danielhanchen in https://github.com/unslothai/unsloth/pull/938

## New Contributors
* @relic-yuexi made their first contribution in https://github.com/unslothai/unsloth/pull/874
* @emuchogu made their first contribution in https://github.com/unslothai/unsloth/pull/870

**Full Changelog**: https://github.com/unslothai/unsloth/compare/July-Mistral-2024...August-2024