v0.54.0
roboflow/inferencev0.54.0Aug 29, 2025by PawelPeczek-Roboflow
AI Summary
Architectural redesign of the inference server to eliminate event-loop blocking and introduction of a new dashboard for monitoring.
Key Highlights
- Redesigned request handling to be non-blocking
- New Inference Dashboard for server logs and metrics
- Higher throughput and comparable latency
New Features
- Non-blocking handlers
- Inference Dashboard
Full Release Notes
# 💪 Added ## `inference` server with non-blocking handlers We redesigned request handling to eliminate event-loop blocking in FastAPI. The server now processes requests non-blocking, enabling higher concurrency and improved throughput while keeping latency about the same. * 🛠️ **Removed event-loop bottlenecks** in the request pipeline * 📈 **Higher throughput** under load * ⏱️ **Comparable** latency ℹ️ No action required. ## 💻 Discover server logs with ease A fresh, at-a-glance dashboard for your Roboflow Inference server—status, metrics, and live logs in one place. All thanks to @digaobarbosa * 🟢 **Health & Status:** Server health badge, version, server ID, uptime, and memory usage. * 📦 **Loaded Models:** Clear empty state when no models are active. * 📈 **API Requests:** Totals and success rate to monitor reliability. * 🪵 **Live Logs:** Real-time tailing with level filter, text search, auto-scroll, and “latest” timestamp. * 🔄 **Manual Refresh:** Update metrics on demand (Refresh Data). **Where to find it:** open the new Inference Dashboard from `inference` landing page > [!TIP] > To get access to dashboard, start your `inference` server with the following environmental variables `ENABLE_IN_MEMORY_LOGS=true` and `ENABLE_DASHBOARD=True` <p align="center"> <img width="60%" alt="image" src="https://github.com/user-attachments/assets/b9c200ba-e809-4fe9-836f-f3abb8b3e0be" /> </p> # 🔧 Fixed * Fix copy paste in local builder by @yeldarby in https://github.com/roboflow/inference/pull/1513 * Add missing page size parameter to list-ingest-details CLI command by @PawelPeczek-Roboflow in https://github.com/roboflow/inference/pull/1516 **Full Changelog**: https://github.com/roboflow/inference/compare/v0.53.0...v0.54.0