2025-01

* Please update Unsloth if you're seeing significant or unusual loss results—the latest update fixes an issue caused by the new transformers version. See our new updating instructions [here](https://docs.unsloth.ai/get-started/installing-+-updating/updating)

Phi-4 is now supported! We also fixed a few bugs in Phi-4 including:

1. EOS token should not be `<|endoftext|>` but rather `<|im_end|>` 
2. Chat template did not have a `add_generation_prompt`
3. Pad token should not be `<|endoftext|>` but rather `<|dummy_87|>`

Try Phi-4 finetuning out in our [Colab Notebook](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Phi_4-Conversational.ipynb) for conversational workloads.

We also **llama-fied** Phi-4 (converted to Llama arch), and uploaded all fixed versions and GGUFs to https://huggingface.co/unsloth

Using Hugging Face's Open LLM Leaderboard, we can see our fixes and changes do work, and sometimes increase accuracy vs Microsoft's original Phi-4 model!
![Phi](https://unsloth.ai/cgi/image/unsloth-phi4-mmlu_uq1ktQuBxxVDM20qUJJO-.png?width=3840&quality=80&format=auto)

Other community members show our fixes do in fact work:
<img src="https://unsloth.ai/cgi/image/unsloth_gguf_vs_old_phi4_acqt4upP8f1ARPET-Vd56.png?width=1920&quality=80&format=auto" width="300" height="200" />
More details on bug fixes here: https://unsloth.ai/blog/phi4

We also uploaded dynamic 4bit quants as well, with error plots here:
![Error](https://unsloth.ai/cgi/image/phi-4_Quantization_Errors_EDrGvwloSg67tLJKWnNYO.webp?width=2048&quality=80&format=auto)

## Bug Fixes
* Gradient accumulation bug fix was not applied on Llama 3.2 Vision
* Fix attention backwards performance regression for T4s, V100s for Xformers < 0.0.29
* Enable gradients without `make_inputs_require_grad` - ie not just on `model.get_input_embeddings()` - this meant only language models had correctly applied `requires_grad_(True)`, and all vision models had their PEFT modules on the vision part essentially be untrained.
* Fix `train_on_responses_only` for Phi-4

## New features
* Auto enable gradient accumulation fixes for all models
* Windows support via Triton Windows and Xformers for Windows
* Xformers==0.0.29 support makes H100 training 10% faster
* Updated all notebooks to use Github links

## Update instructions
Please update Unsloth with no dependency updates via
```bash
pip install --upgrade --force-reinstall --no-deps --no-cache-dir unsloth unsloth_zoo
```

To get the latest Unsloth version, go to https://pypi.org/project/unsloth/ or do:
```bash
pip install unsloth
```

## What's Changed
* Feat/kto by @Erland366 in https://github.com/unslothai/unsloth/pull/1316
* Fix orpo/dpo trainer  by @dame-cell in https://github.com/unslothai/unsloth/pull/1286
* Update README.md by @shimmyshimmer in https://github.com/unslothai/unsloth/pull/1383
* Fix vision model tokenizer padding side. by @ZewenShen in https://github.com/unslothai/unsloth/pull/1384
* Add citation section to README.md by @Erland366 in https://github.com/unslothai/unsloth/pull/1377
* Granite support by @Datta0 in https://github.com/unslothai/unsloth/pull/1218
* Llama 3.3 by @danielhanchen in https://github.com/unslothai/unsloth/pull/1393
* Update README.md by @shimmyshimmer in https://github.com/unslothai/unsloth/pull/1401
* Update README.md by @shimmyshimmer in https://github.com/unslothai/unsloth/pull/1411
* Update README.md by @shimmyshimmer in https://github.com/unslothai/unsloth/pull/1412
* Fix loader.py to work on Windows by @fundthmcalculus in https://github.com/unslothai/unsloth/pull/1453
* Update save.py warning message by @qingy1337 in https://github.com/unslothai/unsloth/pull/1425
* Change _fix_chat_template in case a template has both endif and endfor by @giuliabaldini in https://github.com/unslothai/unsloth/pull/1388
* Pass position embeddings explicitly from decoder layer. by @Datta0 in https://github.com/unslothai/unsloth/pull/1442
* Bug fixes by @danielhanchen in https://github.com/unslothai/unsloth/pull/1458
* Name Error Bug Fix - import from packaging.version import Version by @developer0hye in https://github.com/unslothai/unsloth/pull/1468
* Bug Fixes by @danielhanchen in https://github.com/unslothai/unsloth/pull/1470
* Bug fixes by @danielhanchen in https://github.com/unslothai/unsloth/pull/1473
* Bug fixes by @danielhanchen in https://github.com/unslothai/unsloth/pull/1484
* Create CONTRIBUTING.md by @shimmyshimmer in https://github.com/unslothai/unsloth/pull/1472
* Update CONTRIBUTING.md by @NinoRisteski in https://github.com/unslothai/unsloth/pull/1507
* Bug fixes by @danielhanchen in https://github.com/unslothai/unsloth/pull/1516
* Update __init__.py by @sebaxakerhtc in https://github.com/unslothai/unsloth/pull/1520
* Phi-4 by @danielhanchen in https://github.com/unslothai/unsloth/pull/1523
* Update README.md for Notebooks by @shimmyshimmer in https://github.com/unslothai/unsloth/pull/1515

## New Contributors
* @ZewenShen made their first contribution in https://github.com/unslothai/unsloth/pull/1384
* @fundthmcalculus made their first contribution in https://github.com/unslothai/unsloth/pull/1453
* @qingy1337 made their first contribution in https://github.com/unslothai/unsloth/pull/1425
* @developer0hye made their first contribution in https://github.com/unslothai/unsloth/pull/1468
* @NinoRisteski made their first contribution in https://github.com/unslothai/unsloth/pull/1507
* @sebaxakerhtc made their first contribution in https://github.com/unslothai/unsloth/pull/1520