Starstruk Logo MarkStarstruk

v0.3.8

MinishLab/model2vecv0.3.8Jan 27, 2025by stephantul

AI Summary

Focuses on improving distillation for modernBERT, fixing tokenizer issues, and adding optional embedding normalization to StaticModel loading.

Key Highlights

  • Fix modernBERT tokenizer
  • Improve distill for modernBERT
  • Add optional embedding normalization to StaticModel loading

New Features

  • Float PCA dims
  • Add optional embedding normalization to StaticModel loading

Full Release Notes

## What's Changed
* docs: fix docstrings in distill by @stephantul in https://github.com/MinishLab/model2vec/pull/157
* remove unnecessary import by @stephantul in https://github.com/MinishLab/model2vec/pull/161
* remove deduplication tutorial by @stephantul in https://github.com/MinishLab/model2vec/pull/159
* fix: issue with modernbert tokenizer, add token pattern to _distill by @stephantul in https://github.com/MinishLab/model2vec/pull/158
* fix: fix typing issue by @stephantul in https://github.com/MinishLab/model2vec/pull/162
* feat: float pca dims by @stephantul in https://github.com/MinishLab/model2vec/pull/163
* feat: Add optional embedding normalization to StaticModel loading by @davidberenstein1957 in https://github.com/MinishLab/model2vec/pull/164
* feat: Improve distill for modernBERT by @stephantul in https://github.com/MinishLab/model2vec/pull/165
* increase version by @stephantul in https://github.com/MinishLab/model2vec/pull/166

## New Contributors
* @davidberenstein1957 made their first contribution in https://github.com/MinishLab/model2vec/pull/164

**Full Changelog**: https://github.com/MinishLab/model2vec/compare/v0.3.7...v0.3.8