July-2024

## Gemma 2 support
We now support Gemma 2! It's 2x faster and uses 63% less VRAM than HF+FA2!
![image](https://github.com/unslothai/unsloth/assets/23090290/8274d9e1-c923-434f-9a61-28ca412c8fc3)
We have a Gemma 2 9b notebook here: https://colab.research.google.com/drive/1vIrqH5uYDQwsJ4-OO3DErvuv4pBgVwk4?usp=sharing

To use Gemma 2, please update Unsloth:
```bash
pip uninstall unsloth -y
pip install --upgrade --force-reinstall --no-cache-dir git+https://github.com/unslothai/unsloth.git
```

Head over to our blog post: https://unsloth.ai/blog/gemma2 for more details.

We uploaded 4bit quants for 4x faster downloading to:

https://huggingface.co/unsloth/gemma-2-9b-bnb-4bit

https://huggingface.co/unsloth/gemma-2-27b-bnb-4bit

https://huggingface.co/unsloth/gemma-2-9b-it-bnb-4bit

https://huggingface.co/unsloth/gemma-2-27b-it-bnb-4bit


## Continued pretraining
You can now do continued pretraining with Unsloth. See https://unsloth.ai/blog/contpretraining for more details!

Continued pretraining is 2x faster and uses 50% less VRAM than HF + FA2 QLoRA. We offload `embed_tokens` and `lm_head` to disk to save VRAM!

You can now simply use both in the target modules like below:
```python
model = FastLanguageModel.get_peft_model(
    model,
    r = 128, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
                      "gate_proj", "up_proj", "down_proj",
                      "embed_tokens", "lm_head",], # Add for continual pretraining
    lora_alpha = 32,
    lora_dropout = 0, # Supports any, but = 0 is optimized
    bias = "none",    # Supports any, but = "none" is optimized
    # [NEW] "unsloth" uses 30% less VRAM, fits 2x larger batch sizes!
    use_gradient_checkpointing = "unsloth", # True or "unsloth" for very long context
    random_state = 3407,
    use_rslora = True,  # We support rank stabilized LoRA
    loftq_config = None, # And LoftQ
)
```

We also allow 2 learning rates - one for the embedding matrices and another for the LoRA adapters:
```python
from unsloth import is_bfloat16_supported
from unsloth import UnslothTrainer, UnslothTrainingArguments

trainer = UnslothTrainer(
    args = UnslothTrainingArguments(
        ....
        learning_rate = 5e-5,
        embedding_learning_rate = 5e-6,
    ),
)
```

We also share a free Colab to finetune Mistral v3 to learn Korean (you can select any language you like) using Wikipedia and the Aya Dataset: https://colab.research.google.com/drive/1tEd1FrOXWMnCU9UIvdYhs61tkxdMuKZu?usp=sharing

And we're sharing our free Colab notebook for continued pretraining for text completion: https://colab.research.google.com/drive/1ef-tab5bhkvWmBOObepl1WgJvfvSzn5Q?usp=sharing

## What's Changed

* Ollama Chat Templates by @danielhanchen in https://github.com/unslothai/unsloth/pull/582
* Fix case where GGUF saving fails when model_dtype is torch.float16 ("f16") by @chrehall68 in https://github.com/unslothai/unsloth/pull/630
* Support revision parameter in FastLanguageModel.from_pretrained by @chrehall68 in https://github.com/unslothai/unsloth/pull/629
* clears any selected_adapters before calling internal_model.save_pretr… by @neph1 in https://github.com/unslothai/unsloth/pull/609
* Check for incompatible modules before importing unsloth by @xyangk in https://github.com/unslothai/unsloth/pull/602
* Fix #603 handling of formatting_func in tokenizer_utils for assitant/chat/completion training  by @Oseltamivir in https://github.com/unslothai/unsloth/pull/604
* Add GGML saving option to Unsloth for easier Ollama model creation and testing. by @mahiatlinux in https://github.com/unslothai/unsloth/pull/345
* Add Documentation for LoraConfig Parameters by @sebdg in https://github.com/unslothai/unsloth/pull/619
* llama.cpp failing by @bet0x in https://github.com/unslothai/unsloth/pull/371
* fix libcuda_dirs import for triton 3.0 by @t-vi in https://github.com/unslothai/unsloth/pull/227
* Nightly by @danielhanchen in https://github.com/unslothai/unsloth/pull/632
* README: Fix minor typo. by @shaper in https://github.com/unslothai/unsloth/pull/559
* Qwen bug fixes by @danielhanchen in https://github.com/unslothai/unsloth/pull/639
* Fix segfaults by @danielhanchen in https://github.com/unslothai/unsloth/pull/641
* Nightly by @danielhanchen in https://github.com/unslothai/unsloth/pull/646
* Nightly by @danielhanchen in https://github.com/unslothai/unsloth/pull/648
* Nightly by @danielhanchen in https://github.com/unslothai/unsloth/pull/649
* Fix breaking bug in save.py with interpreting quantization_method as a string when saving to gguf by @ArcadaLabs-Jason in https://github.com/unslothai/unsloth/pull/651
* Revert "Fix breaking bug in save.py with interpreting quantization_method as a string when saving to gguf" by @danielhanchen in https://github.com/unslothai/unsloth/pull/652
* Revert "Revert "Fix breaking bug in save.py with interpreting quantization_method as a string when saving to gguf"" by @danielhanchen in https://github.com/unslothai/unsloth/pull/653
* Fix GGUF by @danielhanchen in https://github.com/unslothai/unsloth/pull/654
* Fix continuing LoRA finetuning by @danielhanchen in https://github.com/unslothai/unsloth/pull/656

## New Contributors
* @chrehall68 made their first contribution in https://github.com/unslothai/unsloth/pull/630
* @neph1 made their first contribution in https://github.com/unslothai/unsloth/pull/609
* @xyangk made their first contribution in https://github.com/unslothai/unsloth/pull/602
* @Oseltamivir made their first contribution in https://github.com/unslothai/unsloth/pull/604
* @mahiatlinux made their first contribution in https://github.com/unslothai/unsloth/pull/345
* @sebdg made their first contribution in https://github.com/unslothai/unsloth/pull/619
* @bet0x made their first contribution in https://github.com/unslothai/unsloth/pull/371
* @t-vi made their first contribution in https://github.com/unslothai/unsloth/pull/227
* @shaper made their first contribution in https://github.com/unslothai/unsloth/pull/559
* @ArcadaLabs-Jason made their first contribution in https://github.com/unslothai/unsloth/pull/651

**Full Changelog**: https://github.com/unslothai/unsloth/commits/June-2024