v1.5.1

# 🚀 Added

## 🪣 Optional S3-compatible shared cache for model files

Multi-node deployments no longer have to download the same model weights from origin once per node. An opt-in cache layer sits between the local filesystem cache and the original download source: on a local miss, inference checks a shared S3-compatible object store first, so an artifact downloaded by one node is reused by every other node — cutting cold-start latency and repeated origin downloads (@ecarrara, https://github.com/roboflow/inference/pull/2769).

**How to use it.** Set `INFERENCE_MODELS_MODEL_BLOB_CACHE_ENABLED=True` (default is off) and point `INFERENCE_MODELS_MODEL_BLOB_CACHE_BUCKET` at your bucket; endpoint URL, region, prefix, credentials, timeouts and circuit-breaker thresholds are all configurable through the `INFERENCE_MODELS_MODEL_BLOB_CACHE_*` family. The layer is designed to never take a deployment down: cache misses, timeouts, corrupted objects and service failures all fall back to the original source, and misconfiguration fails open.

## 🔢 Token-usage outputs on remote VLM Workflow blocks

Remote VLM blocks now tell you what they cost: new `input_tokens` and `output_tokens` outputs report billed token counts, with both Roboflow-key pass-through and user-supplied API keys (@SkalskiP, https://github.com/roboflow/inference/pull/2858, fast-tracked into the v1.5.0 post-releases). Shipped as new block versions — `openrouter@v2`, `google_gemma@v3`, `meta_vlm@v2`, `qwen_vlm@v3`, `anthropic_claude@v4`, `google_gemini@v5`, `open_ai@v6`, `spacexai@v2` — so existing workflows keep their behavior byte-for-byte. The OpenRouter v2 and Gemma v3 blocks additionally gain a `reasoning_effort` parameter (`none`/`low`/`medium`/`high`/`xhigh`), OpenRouter v2 exposes the reasoning trace as a `thinking` output, and both raise the default `max_tokens` from 500 to 2048 so reasoning models don't burn the whole budget on internal thinking.

## 🧪 Experimental CUDA 13 server build

A new x86 GPU server image, `roboflow/roboflow-inference-server-gpu-cu13`, built on a CUDA 13.2.1 / Ubuntu 22.04 base with CUDA-13 builds of FFmpeg, GStreamer and OpenCV (@PawelPeczek-Roboflow, https://github.com/roboflow/inference/pull/2884 and https://github.com/roboflow/inference/pull/2885). Verified end-to-end on an RTX 6000; published with the same versioned tagging as the other server images. Experimental for now — one known limitation is that YOLACT ONNX models show systemic errors on this build, to be resolved separately.

## 📦 Custom file names for the Write Vision Event Bundle sink

The Write Vision Event Bundle sink gains an optional `file_name` field — a literal or a selector — so exported bundles can carry meaningful, downstream-friendly names instead of generated ones (@rvirani1, https://github.com/roboflow/inference/pull/2887). Names are strictly validated (safe character set, no path separators, length-capped) at both the manifest and runtime layers, and the sink refuses to overwrite an existing bundle of the same name — the collision check is atomic, so concurrent writers cannot clobber each other even on removable media.

## 📹 Grab-cadence telemetry for the GStreamer CUDA video producer

The GStreamer CUDA producer now reports per-producer frame-gap statistics (count/mean/max plus under/over-period buckets) and bounded source-stream metadata through `tensor_bridge_stats`, and a new `ROBOFLOW_GSTREAMER_CUDA_APPSINK_SYNC` env var can opt the appsink into clock-synced delivery for controlled comparisons — the low-latency default is unchanged, and the frame path itself is untouched (@hansent, https://github.com/roboflow/inference/pull/2868).

# 🔧 Fixed

- **Batch- and list-shaped data now cross the Modal remote-execution boundary intact** — Custom Python blocks executed remotely on Modal received broken data whenever a batch shape crossed the wire: a `Batch[...]` input (for example `Batch[WorkflowImageData]` from a crop step) arrived in the sandbox as a stringified placeholder, and a list-shaped `BlockResult` returned over the HTTP transport failed to deserialize on the way back. Both directions are fixed, on both the HTTP and msgpack transports, with batch indices preserved exactly so downstream index-based filtering keeps working (@jeku46, https://github.com/roboflow/inference/pull/2870). The fix spans the inference client and the hosted execution sandbox; the hosted side is rolled out alongside this release.
- **Workflow step workers now run in the request's full context** — steps executed in parallel (including nested block worker pools) run inside a per-task snapshot of the submitting thread's `contextvars` context, so block code and instrumentation observe the request's values for every `ContextVar`, and context set inside a step no longer leaks into later requests that reuse the same pool thread (@SolomonLake, https://github.com/roboflow/inference/pull/2843).
- **MQTT Writer sink lifecycle rebuilt** — the enterprise MQTT sink's client management is rewritten, closing 13 confirmed defects: a failed first connect permanently poisoning the client, rejected CONNACKs logged as "Connected", reconnect races, silently publishing to the wrong broker after host or credential changes, skipped username-only auth, and hangs on invalid timeout or port values. Failures now log and return `error_status=True` without stopping the workflow, and a new opt-in `fail_fast` field raises instead (@grzegorz-roboflow, https://github.com/roboflow/inference/pull/2876).
- **TRT CUDA-graph capture no longer trips over concurrent pipelines** — graph capture ran in process-wide mode, so any other thread touching the default CUDA stream during a capture failed with CUDA error 906. Capture is now thread-local and serialized under a lock; same graph, same replay path, no change to predictions (@shntu, https://github.com/roboflow/inference/pull/2866).
- **`MemoryCache` per-key locks validate their generation** — a waiter could acquire a lock whose cache entry had already expired and been replaced, letting two callers into a critical section meant for one. Stale-generation locks are now released and retried within the original timeout budget, at under 0.25 µs of overhead per operation (@voropaevv, https://github.com/roboflow/inference/pull/2804).
- **Cache Set / Cache Get blocks clean up every namespace they touch** — an instance serving multiple videos only released the last video's namespace, and cleanup from one instance could wipe keys still in use by another instance on the same video. Namespaces are now tracked per instance and reference-counted, making cross-instance sharing safe (@davidnichols-ops, https://github.com/roboflow/inference/pull/2834).
- **`from inference import Model` works again** — `Model` was advertised in `__all__` but missing from the lazy loader's registry, so the import passed type checking and failed at runtime (@davidnichols-ops, https://github.com/roboflow/inference/pull/2844).
- **`google_gemini` blocks forward `temperature` to thinking-level models** — previously an explicitly set temperature was silently dropped for Gemini 3.x models, so structured-extraction workflows ran at Gemini's default temperature 1.0 with no way to opt out (@gavwin, https://github.com/roboflow/inference/pull/2813).
- **Jetson runtime env compatibility** — JetPack 6+ images set `RUNNING_ON_JETSON`, but the legacy `VideoSource` path only checked `RUNS_ON_JETSON`; the latter now falls back to the former, restoring correct RTSP/GStreamer producer selection on those images (@hfsc2004, https://github.com/roboflow/inference/pull/2806).
- **Secure-gateway gate applies to per-run `step_execution_mode`** — with `SECURE_GATEWAY` set, a caller passing `step_execution_mode=remote` per run could bypass the gate that blocks remote step execution against the hosted Roboflow API. The restriction is now enforced where the parameter is consumed, and such runs raise a clear error; self-hosted remote targets behind the gateway remain allowed (@adhavan18, https://github.com/roboflow/inference/pull/2805).
- **Block-documentation links repaired** — old `/workflows/blocks/<slug>` gallery URLs on inference.roboflow.com forward to the right docs.roboflow.com page instead of dropping the slug (@Erol444, https://github.com/roboflow/inference/pull/2839), and generated block-page slugs no longer break markdown cross-references for block names containing brackets like `PTZ Tracking (ONVIF)` (@adhavan18, https://github.com/roboflow/inference/pull/2830).

# ⚙️ Execution Engine `v1.15.1`

The Workflows Execution Engine version moves from `v1.15.0` to `v1.15.1`, claiming the Modal boundary serialization fix (#2870) and the step-worker context propagation change (#2843) described above. The full version-by-version record lives in the [Execution Engine changelog](https://docs.roboflow.com/workflows/developer-guide/developer-guide/execution-engine-changelog).

# 🚧 Maintenance

- **Quieter serverless logs** — the duplicated per-request "Request received" line and health/probe access-log lines are demoted to DEBUG; access logs for real requests stay at INFO, since the dedicated-deployment auto-pause daemon uses them as its activity signal (@bigbitbus, https://github.com/roboflow/inference/pull/2877).
- **Webexec Modal deployment CI** — deployment workflow and environment wiring for the hosted Custom Python execution app (@grzegorz-roboflow, https://github.com/roboflow/inference/pull/2848, https://github.com/roboflow/inference/pull/2849).
- **Post-v1.5.0 fast-track packaging** (@PawelPeczek-Roboflow, https://github.com/roboflow/inference/pull/2863, https://github.com/roboflow/inference/pull/2874, https://github.com/roboflow/inference/pull/2881), linter cleanup (https://github.com/roboflow/inference/pull/2867), a CI fix (https://github.com/roboflow/inference/pull/2890), and the PR review agent moves to Claude Opus 4.8 (@grzegorz-roboflow, https://github.com/roboflow/inference/pull/2888).
- **Usage telemetry:** A four-PR overhaul of usage records by @SolomonLake. (https://github.com/roboflow/inference/pull/2851, https://github.com/roboflow/inference/pull/2855, https://github.com/roboflow/inference/pull/2852, https://github.com/roboflow/inference/pull/2843)

## 🏅 New Contributors

* @hfsc2004 made their first contribution in https://github.com/roboflow/inference/pull/2806
* @davidnichols-ops made their first contribution in https://github.com/roboflow/inference/pull/2844
* @gavwin made their first contribution in https://github.com/roboflow/inference/pull/2813

**Full Changelog**: https://github.com/roboflow/inference/compare/v1.5.0...v1.5.1