v0.1.72
allenai/olmocrv0.1.72Jun 17, 2025by github-actions[bot]
AI Summary
A major infrastructure update migrating the inference backend from SGLang to vLLM and updating the Docker environment.
Key Highlights
- Migrated from SGLang to vLLM backend for better performance
- Updated to Ubuntu 22.04
- Added FP8 compression support
- Enabled passing AWS credentials for benchmarking custom models in S3
Breaking Changes
- Switched inference backend from SGLang to vLLM
New Features
- vLLM backend integration
- FP8 compression
- AWS credentials support
- Ubuntu 22.04 updates
Full Release Notes
## What's new ## Commits 5e5c31b Bump version to v0.1.72 for release 715b841 0.1.72 b03feb3 Fixed b588ae2 Remvoing sglang tests, switch to vllm 6e3fba3 Lints e489b28 Lints 6fcd26d Updating readme 8c62072 Merge remote-tracking branch 'origin/main' into jakep/vllm_perf 3eda2c0 updated vllm to 0.9.1 a83a0da Cleanup of vllm perf branch with @amanr 316d0af added dtype functionality c8a5361 fixing packages of 22.04 c5d075c fixed apt_pkg module 08fd82f made changes wrt ubuntu 22.04 6507a65 updated ubuntu to 22.04 for glbc 2.32 25dfe0b Weird glibc error 9539eab AWs creds fix e0fda1a Passing aws creds to benchmark so we can run custom models stored in s3 ecf0d48 Dont allow uncomitted changes 134bba9 Run benchmark adjustments 7009a7a Trying out FP8 compression aad8428 Reverting custom pipeline image 5c52e01 Include cuda 12.8 5c524b5 Cleaning up stats reportng 916f0cb Trying with flash infer installed 2ccef7d Ugh, this code is bad 2f1957b Performance fixes with vllm backend d717033 Fixing parse for waiting d1baa51 Python alternatives 581915f Fixes for docker image 153f1e5 Final uv fixes 97da87a Hopefully a much better dockerfile 04dd71c Trying to get onto vllm latest 106070d Moving pipeline to vllm 2235b82 Beaker tests 967c83d Better way to setup beaker