v0.8.2

hiyouga/LlamaFactoryv0.8.2Jun 19, 2024by hiyouga

AI Summary

Adds support for GLM-4 tools and parallel function calling, introduces PiSSA fine-tuning, and supports DeepSeek-Coder-V2 models. Includes new datasets for supervised fine-tuning.

Key Highlights

Support for GLM-4 tools and parallel function calling
Support for PiSSA fine-tuning
Support for DeepSeek-Coder-V2 models

New Features

GLM-4 tools support
PiSSA fine-tuning support
DeepSeek-Coder-V2 (Base and Instruct) support
Neo-sft dataset
Magpie-Pro-300K-Filtered dataset
WebInstruct dataset

Full Release Notes

### New features

- Support GLM-4 tools and parallel function calling by @mMrBun in #4173 
- Support **PiSSA** fine-tuning by @hiyouga in #4307 

### New models

- Base models
  - DeepSeek-Coder-V2 (16B MoE/236B MoE) 📄
- Instruct/Chat models
  - MiniCPM-2B 📄🤖
  - DeepSeek-Coder-V2-Instruct (16B MoE/236B MoE)  📄🤖

### New datasets

- Supervised fine-tuning datasets
  - Neo-sft (zh)
  - Magpie-Pro-300K-Filtered (en) by @EliMCosta in #4309 
  - WebInstruct (en) by @EliMCosta in #4309 

### Bug fix

- Fix DPO+ZeRO3 problem by @hiyouga 
- Add MANIFEST.in by @iamthebot in #4191 
- Fix eos_token in llama3 pretrain by @dignfei in #4204 
- Fix vllm version by @kimdwkimdw and @hzhaoy in #4234 and #4246 
- Fix Dockerfile by @EliMCosta in #4314 
- Fix pandas version by @zzxzz12345 in #4334 
- Fix #3162 #3196 #3778 #4198 #4209 #4221 #4227 #4238 #4242 #4271 #4292 #4295 #4326 #4346 #4357 #4362