July-Llama-2024

unslothai/unslothJuly-Llama-2024Jul 23, 2024by danielhanchen

AI Summary

Adds support for Llama 3.1, claiming 2.1x faster training and 60% less VRAM usage.

Key Highlights

Llama 3.1 support
2x faster training and inference
New chat UI on Colab
4bit quants uploaded to HuggingFace

New Features

Llama 3.1 support
faster training and inference
Colab chat UI
4bit quants

Full Release Notes

# Llama 3.1 Support
Excited to announce Unsloth makes finetuning Llama 3.1 2.1x faster and use 60% less VRAM! Read up on our release here: https://unsloth.ai/blog/llama3-1
![image](https://github.com/user-attachments/assets/5c0b861c-d9f4-4533-9f15-f10f7431057b)

We uploaded a Google Colab notebook to finetune Llama 3.1 (8B) on a free Tesla T4: [Llama 3.1 (8B) Notebook](https://colab.research.google.com/drive/1Ys44kVvmeZtnICzWz0xgpRnrIOjZAuxp?usp=sharing). We also have a [new UI on Google Colab](https://colab.research.google.com/drive/1T-YBVfnphoVc8E2E854qF3jdia2Ll2W2?usp=sharing) for chatting with your Llama 3.1 Instruct models which uses our own 2x faster inference engine.

# Run UI Preview
![unsloth_chat_ui_cHN5s0tryafdUM6nzXjgf](https://github.com/user-attachments/assets/8c503357-0f97-40b4-8a71-051b5b4ed14d)
We created a [new chat UI](https://colab.research.google.com/drive/1T-YBVfnphoVc8E2E854qF3jdia2Ll2W2?usp=sharing) using Gradio where users can upload and chat with their Llama 3.1 Instruct models online for free on Google Colab.

We uploaded 4bit bitsandbytes quants here: https://huggingface.co/unsloth
To finetune Llama 3.1, please update Unsloth:
```bash
pip uninstall unsloth -y
pip install --upgrade --force-reinstall --no-cache-dir git+https://github.com/unslothai/unsloth.git
```