Starstruk Logo MarkStarstruk

v0.3.0

MinishLab/model2vecv0.3.0Oct 21, 2024by stephantul

AI Summary

A major feature release introducing numpy inference, local model loading, and faster tokenization capabilities.

Key Highlights

  • Added Numpy inference support
  • Enabled local model loading
  • Implemented faster tokenization
  • Added Python 3.9 support

New Features

  • Numpy inference
  • Local loading
  • Faster tokenization
  • Dynamic versioning
  • Explained variance messages

Full Release Notes

## What's Changed
* fix: Fix token type ids not supported by @Pringled in https://github.com/MinishLab/model2vec/pull/77
* docs: Add deduplication tutorial by @Pringled in https://github.com/MinishLab/model2vec/pull/72
* Fix distill model bos and eos token by @zechengz in https://github.com/MinishLab/model2vec/pull/78
* docs: Added Sentence Transformers example code by @Pringled in https://github.com/MinishLab/model2vec/pull/80
* docs: Update readme by @Pringled in https://github.com/MinishLab/model2vec/pull/81
* docs: Move results and add blogpost by @Pringled in https://github.com/MinishLab/model2vec/pull/82
* docs: Fixed broken link by @Pringled in https://github.com/MinishLab/model2vec/pull/84
* fix: move tensor to cpu by @stephantul in https://github.com/MinishLab/model2vec/pull/86
* feat: Numpy inference by @stephantul in https://github.com/MinishLab/model2vec/pull/87
* feat: local loading by @stephantul in https://github.com/MinishLab/model2vec/pull/88
* feat: faster tokenization by @stephantul in https://github.com/MinishLab/model2vec/pull/89
* enhancement: Add dynamic version by @stephantul in https://github.com/MinishLab/model2vec/pull/91
* enhancement: Add explained variance messages by @stephantul in https://github.com/MinishLab/model2vec/pull/92
* docs: Updated slogan by @Pringled in https://github.com/MinishLab/model2vec/pull/94
* feat: Add python3.9 support by @Pringled in https://github.com/MinishLab/model2vec/pull/96
* enh: remove CLI command by @stephantul in https://github.com/MinishLab/model2vec/pull/98
* fix: rename show progress bar argument by @stephantul in https://github.com/MinishLab/model2vec/pull/99
* fix: Reverted eos bos change by @Pringled in https://github.com/MinishLab/model2vec/pull/101
* docs: Added results link by @Pringled in https://github.com/MinishLab/model2vec/pull/102
* docs: Fix broken link by @Pringled in https://github.com/MinishLab/model2vec/pull/103
* increment version by @stephantul in https://github.com/MinishLab/model2vec/pull/104

## New Contributors
* @zechengz made their first contribution in https://github.com/MinishLab/model2vec/pull/78

**Full Changelog**: https://github.com/MinishLab/model2vec/compare/v0.2.4...v0.3.0