v1.5.4

feyninc/chonkiev1.5.4Jan 28, 2026by chonk-lain

AI Summary

This release introduces the Chonkie CLI and completes the migration from Cython to Rust for improved performance and maintainability.

Key Highlights

  • Official migration from Cython to Rust (chonkie-core)
  • New Chonkie CLI for command-line operations
  • Added GroqGenie and CerebrasGenie for fast inference
  • Fixed FastChunker UTF-8 byte offset mismatch

New Features

  • Chonkie CLI
  • Rust core implementation
  • GroqGenie and CerebrasGenie

Full Release Notes

## Chonkie v1.5.4 🎉

### Chonkie CLI 💻
checkout the [docs](https://docs.chonkie.ai/oss/experimental/chonkie-cli) for more details about CLI commands and usage


<img width="1467" height="353" alt="image" src="https://github.com/user-attachments/assets/a585f7ee-e30c-4c22-8868-9a356bec7172" />

### Migrate from Cython to rust âš¡
we have officially migrated from Cython towards rust 🦀

---

## What's Changed
* feat: add chonkie CLI by @chonk-lain in https://github.com/chonkie-inc/chonkie/pull/401
* Register Milvus handshake by @akx in https://github.com/chonkie-inc/chonkie/pull/436
* Use component registry in CLI by @akx in https://github.com/chonkie-inc/chonkie/pull/435
* CI: speed up & simplify by @akx in https://github.com/chonkie-inc/chonkie/pull/420
* perf: defer Hubbie recipe schema download and parsing by @akx in https://github.com/chonkie-inc/chonkie/pull/428
* Fix FastChunker UTF-8 byte offset mismatch by @chonknick in https://github.com/chonkie-inc/chonkie/pull/440
* feat: Add text extraction mode to SlumberChunker by @chonknick in https://github.com/chonkie-inc/chonkie/pull/443
* fix: Use group_end_index as fallback when extraction fails by @chonknick in https://github.com/chonkie-inc/chonkie/pull/444
* fix: enable Ruff B904; use `raise ... from ...` when reraising by @akx in https://github.com/chonkie-inc/chonkie/pull/424
* chore(deps): bump aiohttp from 3.13.0 to 3.13.3 by @dependabot[bot] in https://github.com/chonkie-inc/chonkie/pull/448
* chore(deps): bump pyasn1 from 0.6.1 to 0.6.2 by @dependabot[bot] in https://github.com/chonkie-inc/chonkie/pull/449
* chore(deps): bump langchain-core from 0.3.78 to 0.3.81 by @dependabot[bot] in https://github.com/chonkie-inc/chonkie/pull/450
* chore(deps): bump urllib3 from 2.5.0 to 2.6.3 by @dependabot[bot] in https://github.com/chonkie-inc/chonkie/pull/446
* chore(deps): bump authlib from 1.6.5 to 1.6.6 by @dependabot[bot] in https://github.com/chonkie-inc/chonkie/pull/447
* Bump azure-core from 1.35.1 to 1.38.0 by @dependabot[bot] in https://github.com/chonkie-inc/chonkie/pull/442
* Bump filelock from 3.19.1 to 3.20.3 by @dependabot[bot] in https://github.com/chonkie-inc/chonkie/pull/441
* feat: Add GroqGenie and CerebrasGenie for fast inference by @chonknick in https://github.com/chonkie-inc/chonkie/pull/453
* docs: Add GroqGenie and CerebrasGenie documentation by @chonknick in https://github.com/chonkie-inc/chonkie/pull/454
* refactor: replace Cython split/merge with chonkie-core (Rust) by @chonknick in https://github.com/chonkie-inc/chonkie/pull/456
* refactor: replace memchunk with chonkie-core in FastChunker by @chonknick in https://github.com/chonkie-inc/chonkie/pull/457
* feat: Register official SentenceTransformer models to resolve TODO by @chimchim89 in https://github.com/chonkie-inc/chonkie/pull/445
* Format chunker modules with ruff by @chimchim89 in https://github.com/chonkie-inc/chonkie/pull/467
* add lain to assignees list by @chonk-lain in https://github.com/chonkie-inc/chonkie/pull/468
* update cli by @chonk-lain in https://github.com/chonkie-inc/chonkie/pull/439
* fix: add exception chaining to CLI error handling by @chonknick in https://github.com/chonkie-inc/chonkie/pull/470
* Add tenacity retry logic to OpenAI API calls by @emerose in https://github.com/chonkie-inc/chonkie/pull/198
* feat: update publish workflow by @chonk-lain in https://github.com/chonkie-inc/chonkie/pull/474
* feat: remove api docs by @chonk-lain in https://github.com/chonkie-inc/chonkie/pull/475
* bump version by @chonk-lain in https://github.com/chonkie-inc/chonkie/pull/477

## New Contributors
* @chimchim89 made their first contribution in https://github.com/chonkie-inc/chonkie/pull/445
* @emerose made their first contribution in https://github.com/chonkie-inc/chonkie/pull/198

**Full Changelog**: https://github.com/chonkie-inc/chonkie/compare/v1.5.2...v1.5.4