v1.3.7
roboflow/inferencev1.3.7Jul 27, 2026by PawelPeczek-Roboflow
AI Summary
This release introduces air-gapped deployment support, adds the YOLO26 depth estimation model, and introduces new workflow blocks like Detections Nearest Neighbor alongside performance optimizations for memory usage and TensorRT engine builds.
Key Highlights
- Air-gapped deployment support via OFFLINE_MODE
- YOLO26 depth estimation model with ONNX, TorchScript, and TensorRT backends
- New Detections Nearest Neighbor workflow block
- Per-run workflow sink disabling
- Significant memory reduction for dense-mask post-processing
New Features
- OFFLINE_MODE for air-gapped deployments
- YOLO26 depth estimation models
- Detections Nearest Neighbor workflow block
- Grid Visualization multiple image inputs
- Per-run workflow sink disabling
- CUDA allocator memory breakdown
- Billable and errored usage aggregation
Full Release Notes
# 🚀 Added
## ✈️ `OFFLINE_MODE` — air-gapped deployments
Inference servers can now run fully air-gapped (@alexnorell, https://github.com/roboflow/inference/pull/2263). The supported flow is deliberately simple:
1. **Warm** a mounted cache while network access and a Roboflow API key are available.
2. **Restart** the same deployment with the same cache, `OFFLINE_MODE=True`, and no API key.
3. Cached model metadata, weights, and Workflow specifications load with no built-in Roboflow API calls, no retries, and no cache-expiration failures.
## 📏 YOLO26 depth estimation
Public-pretrained **YOLO26 depth estimation** lands end-to-end as a drop-in alternative to Depth Anything (@leeclemnet,
https://github.com/roboflow/inference/pull/2691): `inference_models` backends for **ONNX, TorchScript, and TensorRT** (Ultralytics `-depth` pretrains, 768×768, metric log-depth head), public aliases `yolo26{n,s,m,l,x}-depth-768`, and the same five variants selectable in the `roboflow_core/depth_estimation@v1` Workflow block. Outputs are normalized to Depth Anything's disparity-style convention (larger = closer), so downstream tooling is interchangeable; the block's default model is unchanged.
Alongside it, **depth maps got dramatically cheaper on the wire** (@leeclemnet, https://github.com/roboflow/inference/pull/2693): the SDK now requests the normalized depth map as a **base64 PNG16** (uint16 quantization of the normalized map) instead of a JSON list of floats — roughly **17 MB → ~1 MB** for a typical single-image response, with matching decode-time wins.
Raw HTTP callers are unaffected (the server-side default response format is still `json`), and the SDK decoder transparently accepts responses from older servers.
## 🔥 New workflow blocks
- **Detections Nearest Neighbor** (`roboflow_core/detections_nearest_neighbor@v1`) — a nearest-neighbor spatial join between two detection sets: for each query detection it finds the closest target detection(s) by 2D pixel distance between configurable anchor points (bbox corners/edges/center, or a named keypoint), with tie-awareness, self-match exclusion, and an optional `max_distance` cutoff (@bczifra, https://github.com/roboflow/inference/pull/2698). Emits enriched query predictions (`nearest_target_distance`) plus index-aligned matched-query/matched-target sets ready for standard downstream blocks.
- **Grid Visualization** now accepts **multiple image inputs** directly (@leeclemnet, https://github.com/roboflow/inference/pull/2694).
## 🎛️ Execution & operations
- **Per-run workflow sink disabling** — a `disable_sinks` API parameter, delivered to sink blocks through the existing dependency-injection system, turns off built-in sink side effects for a single run while the rest of the workflow executes normally; all 22 built-in side-effect sinks honor it with no manifest or spec changes (@joaomarcoscrs, https://github.com/roboflow/inference/pull/2697).
- **TensorRT engine builds announce themselves** — with the TRT execution provider and a cold engine cache, the first inference silently compiles an engine (measured ~9.5 minutes for a single COCO object-detection model on an Orin 16 GB) while requests time out. The server now logs an explicit warning at session-configuration time, pointing at the mitigation: persist `MODEL_CACHE_DIR` (@sberan, https://github.com/roboflow/inference/pull/2703).
- **Billable and errored usage are aggregated separately** — usage records now partition by billability, request outcome, structured `error_type`, and bounded HTTP `error_status_code` (400–599), so a failed request can no longer blend into a billable aggregate; billing *intent* is preserved and policy stays with the Roboflow backend. Payload schema is additive-only (@hansent, https://github.com/roboflow/inference/pull/2692).
- **Roboflow API calls now default to a 120-second timeout** (`ROBOFLOW_API_REQUEST_TIMEOUT`) instead of waiting indefinitely (@sberan, https://github.com/roboflow/inference/pull/2702).
# 🔧 Fixed
- **Instance-segmentation dense-mask post-processing memory bounded** — mask upscaling now runs in fixed-size chunks and RF-DETR segmentation applies the standard `max_detections` cap (default 300, overridable via `INFERENCE_MODELS_RFDETR_DEFAULT_MAX_DETECTIONS`) *before* the expensive full-resolution work. Peak CUDA memory on a dense-instance workload dropped **17.1 GiB → 4.2 GiB** with no wall-time regression (@bigbitbus, https://github.com/roboflow/inference/pull/2682).
- **Model-access failures keep their real HTTP statuses** — Execution Engine v1.12.1 propagates upstream model-access errors (401/402/403/404…) through workflow runs instead of collapsing them into generic 500s (@hansent, https://github.com/roboflow/inference/pull/2690; @dkosowski87, https://github.com/roboflow/inference/pull/2709).
- **PP-OCR** — authorization fixed on Serverless (@Erol444, https://github.com/roboflow/inference/pull/2646), and the route now returns a clean 404 when the `inference_models` stack is disabled instead of an internal error (@dkosowski87, https://github.com/roboflow/inference/pull/2707).
- **Phantom keypoints eliminated** — padded keypoint slots are no longer emitted as real keypoints in predictions (@kounelisagis,
https://github.com/roboflow/inference/pull/2677).
- **`detections_overlaps` kind round-trips** — serializer and deserializer registered, so the kind survives workflow JSON output/input (@kounelisagis, https://github.com/roboflow/inference/pull/2638).
- **Grounding DINO accepts both canonical and legacy BERT cache layouts**, so existing warmed caches keep working after the cache-layout changes (@dkosowski87, https://github.com/roboflow/inference/pull/2716).
- **SAM3 package load no longer requires `sam_configuration.json`** (@grzegorz-roboflow, https://github.com/roboflow/inference/pull/2678).
- **Local package imports no longer write `.pyc` bytecode during module execution**, keeping mounted/read-only caches byte-stable (@dkosowski87, https://github.com/roboflow/inference/pull/2713).
- **Pre-release security patches** — landing-page npm dependency chain moved to patched versions and Python dependency bumps rolled up ahead of the release (@PawelPeczek-Roboflow, https://github.com/roboflow/inference/pull/2715).
# 🚧 Maintenance
- **`inference-models` 0.33.0 / 0.34.1 releases and pins** (@PawelPeczek-Roboflow, https://github.com/roboflow/inference/pull/2706; @dkosowski87, https://github.com/roboflow/inference/pull/2711).
- **Windows build fixes** (@PawelPeczek-Roboflow, https://github.com/roboflow/inference/pull/2689).
- **CI** — PP-OCR T4 regression tests skipped when `USE_INFERENCE_MODELS=False` (@grzegorz-roboflow, https://github.com/roboflow/inference/pull/2712) and the workflows integration-test job timeout raised to 25 minutes (@grzegorz-roboflow, https://github.com/roboflow/inference/pull/2719).
- **README contributing-guide link fixed** (@bczifra, https://github.com/roboflow/inference/pull/2704).
---
### 📦 Side note: dual GPU build for Cosmos 3
There is still no released `transformers` version that ships the NVIDIA Cosmos 3 model code, so this release again publishes **two GPU server builds**:
- `roboflow/roboflow-inference-server-gpu:1.3.7` — the standard build, with the regular dependency stack (no Cosmos 3).
- `roboflow/roboflow-inference-server-gpu:1.3.7-cosmos3` — identical server, but with the custom `transformers` dependency set required by **NVIDIA Cosmos 3 Edge**:
```bash
docker pull roboflow/roboflow-inference-server-gpu:1.3.7-cosmos3
```
Use the `-cosmos3` tag only if you need the Cosmos 3 Edge preview self-hosted; all other images are single-build.
---
**Full changelog:** [`v1.3.6...v1.3.7`](https://github.com/roboflow/inference/compare/v1.3.6...v1.3.7)