v0.7.0

hiyouga/LlamaFactoryv0.7.0Apr 27, 2024by hiyouga

AI Summary

Adds LLaVA Multimodal LLM support, 2x faster generation via UnslothAI optimization, and native Transformers and vLLM inference support for LLaVA models.

Key Highlights

LLaVA-1.5 SFT/PPO/DPO/ORPO support
2x faster QLoRA generation via UnslothAI
Support for LLaVA with native Transformers and vLLM
vLLM+LoRA inference for partial models

New Features

LLaVA-1.5 support
UnslothAI long-context QLoRA optimization
New special tokens support
Device selection for LoRA merge in LlamaBoard
SDPA attention and fast tokenizer

Full Release Notes

### Congratulations on 20k stars 🎉 We are the 1st of the *GitHub Trending* at Apr. 23rd 🔥 Follow us at *[X](https://twitter.com/llamafactory_ai)* 

### New features

- Support SFT/PPO/DPO/ORPO for the **LLaVA-1.5** model by @BUAADreamer in #3450 
- Support inferring the LLaVA-1.5 model with both native Transformers and vLLM by @hiyouga in #3454 
- Support **vLLM+LoRA** inference for partial models (see [support list](https://docs.vllm.ai/en/latest/models/supported_models.html))
- Support 2x faster generation of the QLoRA model based on [UnslothAI](https://github.com/unslothai/unsloth)'s optimization
- Support adding new special tokens to the tokenizer via the `new_special_tokens` argument
- Support choosing the device to merge LoRA in LlamaBoard via the `export_device` argument
- Add a Colab notebook for getting into fine-tuning the Llama-3 model on a free T4 GPU
- Automatically enable SDPA attention and fast tokenizer for higher performance

### New models

- Base models
  - OLMo-1.7-7B
  - Jamba-v0.1-51B
  - Qwen1.5-110B
  - DBRX-132B-Base
- Instruct/Chat models
  - Phi-3-mini-3.8B-instruct (4k/128k)
  - LLaVA-1.5-7B
  - LLaVA-1.5-13B
  - Qwen1.5-110B-Chat
  - DBRX-132B-Instruct

### New datasets

- Supervised fine-tuning datasets
  - LLaVA mixed (en&zh) by @BUAADreamer in #3471 
- Preference datasets
  - DPO mixed (en&zh) by @hiyouga 

### Bug fix

- Fix #2093 #3333 #3347 #3374 #3387