Starstruk Logo MarkStarstruk

v0.2.11

datalab-to/markerv0.2.11May 28, 2024by VikParuchuri

AI Summary

Ensures Python 3.12 compatibility by removing the Ray dependency, which results in reduced VRAM usage and increased throughput.

Key Highlights

  • Removed Ray to enable Python 3.12 compatibility
  • Reduced average VRAM usage to 3GB per process
  • Increased throughput compared to previous versions

Full Release Notes

- Remove ray to enable python 3.12 compatibility
- Removing ray frees a lot of VRAM (since we can use torch shared tensors), so on average with `convert.py` each process takes 3GB VRAM.  This enables much higher throughput (was between 4.5GB and 5GB before).