v0.2.0

MinishLab/model2vecv0.2.0Sep 28, 2024by stephantul

AI Summary

A feature-rich release introducing safetensors support, hybrid tokenizers, and model cards.

Key Highlights

  • Added support for safetensors
  • Implemented hybrid tokenizers
  • Added model cards
  • Removed sentence transformers dependency

New Features

  • Safetensors support
  • Hybrid tokenizers
  • Model cards
  • Retrieval tutorial

Full Release Notes

## What's Changed
* remove broken dim property by @stephantul in https://github.com/MinishLab/model2vec/pull/26
* Fix issue with hasattr vs getattr by @stephantul in https://github.com/MinishLab/model2vec/pull/27
* Add retrieval tutorial by @Pringled in https://github.com/MinishLab/model2vec/pull/28
* docs: Add readme headline by @Pringled in https://github.com/MinishLab/model2vec/pull/29
* [`fix`] Various fixes for non-Posix machines by @tomaarsen in https://github.com/MinishLab/model2vec/pull/30
* Add model cards by @Pringled in https://github.com/MinishLab/model2vec/pull/31
* Hybrid tokenizers by @stephantul in https://github.com/MinishLab/model2vec/pull/25
* docs: Added new image by @Pringled in https://github.com/MinishLab/model2vec/pull/32
* [`enh`] Rely on Jinja for model card, use model id/path in snippet by @tomaarsen in https://github.com/MinishLab/model2vec/pull/37
* remove sentence transformers dependency by @stephantul in https://github.com/MinishLab/model2vec/pull/35
* Fix bug where we accidentally pass a pretrainedtokenizer by @stephantul in https://github.com/MinishLab/model2vec/pull/40
* Add support for safetensors by @stephantul in https://github.com/MinishLab/model2vec/pull/36
* Make token optional and private an argument, add template by @stephantul in https://github.com/MinishLab/model2vec/pull/39
* tests: Add distill tests and CI by @Pringled in https://github.com/MinishLab/model2vec/pull/42
* docs: Added codecov badge by @Pringled in https://github.com/MinishLab/model2vec/pull/43
* Logos by @stephantul in https://github.com/MinishLab/model2vec/pull/44
* Add model card loading by @stephantul in https://github.com/MinishLab/model2vec/pull/45

## New Contributors
* @tomaarsen made their first contribution in https://github.com/MinishLab/model2vec/pull/30

**Full Changelog**: https://github.com/MinishLab/model2vec/compare/v0.1.2...v0.2.0