v0.1.8

hiyouga/LlamaFactoryv0.1.8Sep 11, 2023by hiyouga

AI Summary

Focuses on performance optimization with FlashAttention-2 and adds Baichuan2 training capabilities with automatic LoRA module selection.

Key Highlights

FlashAttention-2 support for LLaMA models
Baichuan2 model training support
Automatic LoRA target module selection

New Features

FlashAttention-2 support
Baichuan2 training
Right-padding fix for fp16 training
Reward score alignment with DeepSpeed-Chat
Automatic LoRA target module selection via --lora_target all

Full Release Notes

### New features

- Support [**FlashAttention-2**](https://github.com/Dao-AILab/flash-attention) for LLaMA models. (RTX4090, A100, A800 or H100 GPU is required)
- Support training the Baichuan2 models
- Use right-padding to avoid overflow in fp16 training (also mentioned [here](https://github.com/huggingface/trl/blob/22a90198e5e2652161dbf955e196a97033decf40/examples/research_projects/stack_llama_2/scripts/sft_llama2.py#L168))
- Align the computation method of the reward score with DeepSpeed-Chat (better generation)
- Support `--lora_target all` argument which automatically finds the applicable modules for LoRA training

### Bug fix

- Use efficient EOS tokens to align with the Baichuan training ( https://github.com/baichuan-inc/Baichuan2/issues/23 )
- Remove PeftTrainer to save model checkpoints in DeepSpeed training
- Fix bugs in web UI by @beat4ocean in #596 by @codemayq in #644 #651 #678 #741 by @kinghuin in #786 
- Add dataset explanation by @panpan0000 in #629 
- Fix a bug in the DPO data collator
- Fix a bug of the ChatGLM2 tokenizer in right-padding
- #608 #617 #649 #757 #761 #763 #809 #818