Starstruk Logo MarkStarstruk

v0.8.2

MinishLab/model2vecv0.8.2May 29, 2026by stephantul

AI Summary

This release focuses on fixing multiword token handling, refactoring the training pipeline, and upgrading the underlying skeletoken library. It also introduces new options for training configuration and trusted publishers.

Key Highlights

  • Refactor training pipeline
  • Fix multiword and tokenizer issues
  • Upgrade skeletoken library
  • Add frozen-weights option to training
  • Add trusted publisher support

New Features

  • Add frozen-weights option to training
  • Add trusted publisher
  • Convert output to float in distill

Full Release Notes

## What's Changed
* fix: multiword broken by @stephantul in https://github.com/MinishLab/model2vec/pull/317
* feat: refactor training by @stephantul in https://github.com/MinishLab/model2vec/pull/316
* fix: upgrade skeletoken by @stephantul in https://github.com/MinishLab/model2vec/pull/318
* fix: Incorrect added token can cause issues when adding token as multiword token by @stephantul in https://github.com/MinishLab/model2vec/pull/319
* fix: don't set weights for non-quantized models by @stephantul in https://github.com/MinishLab/model2vec/pull/321
* fix: correctly set weights when w is initialized by @stephantul in https://github.com/MinishLab/model2vec/pull/322
* feat: convert output to float in distill by @stephantul in https://github.com/MinishLab/model2vec/pull/320
* clean up post-processing code by @stephantul in https://github.com/MinishLab/model2vec/pull/323
* chore: fix pre-commit by @stephantul in https://github.com/MinishLab/model2vec/pull/326
* chore: add exclude-newer by @stephantul in https://github.com/MinishLab/model2vec/pull/328
* feat: add frozen-weights option to training by @stephantul in https://github.com/MinishLab/model2vec/pull/327
* fix: small typing issue by @stephantul in https://github.com/MinishLab/model2vec/pull/329
* chore: Update email by @Pringled in https://github.com/MinishLab/model2vec/pull/331
* feat: add trusted publisher by @stephantul in https://github.com/MinishLab/model2vec/pull/332


**Full Changelog**: https://github.com/MinishLab/model2vec/compare/v0.8.1...v0.8.2