v1.6.1

# 🚀 Added

## ✂️ Instance segmentation from the OpenAI block

<img alt="astra-segmentation" src="https://github.com/user-attachments/assets/06bec079-8337-40b9-b41c-7896f1a605e6" />


`roboflow_core/open_ai@v7` gains a new task type, `instance-segmentation`: name the classes, and the model returns one outline polygon per instance, which the block turns into real segmentation predictions that feed mask, polygon and halo visualizations, crops and dataset upload. It is the first VLM block that produces masks. `classes` is required for the task, and when you do not set `model_version` the block picks `gpt-6-astra` for it; an explicit value or a selector is always honoured, and the block-wide default stays `gpt-5.1` for every other task. The image is uploaded at its original resolution (not the 2048 px downscale used for object detection), and the answer is enforced through OpenAI structured outputs as absolute-pixel vertex lists. All thanks to @Erol444 

```json
{
  "type": "roboflow_core/open_ai@v7",
  "name": "segmentation",
  "images": "$inputs.image",
  "task_type": "instance-segmentation",
  "classes": ["chip", "connector"],
  "api_key": "$inputs.openai_api_key"
}
```

## 🪪 Responses say which model actually served them

HTTP inference responses gain an optional top-level `resolved_model` object, so a client can see what ran instead of assuming it was the id it sent: `model_id` (the canonical id of the loaded model), `model_package_id` (the package that loaded successfully and produced the result, including after a loading fallback), `backend` (`onnx`, `trt`, `torch`, ...) and `quantization` (`fp32`, `fp16` or `unknown`; this describes the package, not the input tensor dtype or the precision of every operator). The field sits on the base `InferenceResponse`, so object detection, classification, instance and semantic segmentation, keypoints, LMM, OCR, depth estimation, action recognition, CLIP, Perception Encoder, Grounding DINO, YOLO-World and the SAM, SAM2 and SAM3 responses all carry it (@leeclemnet, https://github.com/roboflow/inference/pull/2999).

```json
{
  "inference_id": "...",
  "time": 0.087,
  "resolved_model": {"model_id": "my-workspace/my-project-131", "model_package_id": "c70f32369a54d61e06ef4e6b56c82524", "backend": "trt", "quantization": "fp16"},
  "image": {"width": 2048, "height": 1371},
  "predictions": []
}
```

## 🧬 Anomaly detection with PatchCore and FoundAD

Anomaly-detection models trained on Roboflow can now be served by inference. Two architectures are supported, PatchCore and FoundAD, both fitted on images of your normal product only, and both addressed like any other custom model (`my-project/2`). They are registered as `classification` models with the two fixed classes `normal` and `anomalous`, so they work on `POST /infer/classification`, on the legacy `/{dataset_id}/{version_id}` route and through `inference_models.AutoModel.from_pretrained(...)`. Next to the usual classification fields the response carries `anomaly_score` (the raw score), `anomaly_threshold` (the threshold stored with the model at training time), `is_anomalous` (the decision) and, on request, `anomaly_map`, a per-pixel heatmap resized to the input image (@probicheaux, https://github.com/roboflow/inference/pull/3007).

**How to use it.** Send `include_anomaly_map: true` in the classification request (or `?include_anomaly_map=true` on the legacy route, or `InferenceConfiguration(include_anomaly_map=True)` in `inference_sdk`) only when you are going to render the heatmap: it is a full-resolution float array and it is large. Both classes are always returned and confidence filtering does not apply; `confidence` is a monotonic transform of the score margin where the threshold maps to exactly 0.5, not a probability.

```json
{
  "predictions": [{"class": "anomalous", "class_id": 1, "confidence": 0.73}, {"class": "normal", "class_id": 0, "confidence": 0.27}],
  "top": "anomalous",
  "confidence": 0.73,
  "anomaly_score": 1.42,
  "anomaly_threshold": 1.10,
  "is_anomalous": true,
  "anomaly_map": null
}
```


## 📨 Kafka Producer and Kafka Consumer blocks

Two new enterprise blocks connect Workflows to Apache Kafka - producer and consumer. Make your workflow connected today 🛜 (@tewanchia, https://github.com/roboflow/inference/pull/3008).

**At this moment, Kafka blocks are not available on the serverless platform**

# 🔒 Security hardening

## 🪝 Webhook sink validates where it connects

`roboflow_core/webhook_sink@v1` used to hand the workflow-supplied URL straight to `requests`, redirects included. It now sends through a guarded transport: only `http` and `https` URLs with a real host are accepted, a backslash in the URL authority is refused, the hostname is resolved once and the connection is pinned to the validated address (with the original name kept for TLS SNI, certificate checks and the `Host` header, which closes DNS rebinding), and **redirects are never followed**: any `3xx` answer is reported as a failed notification with `Webhook redirects are not allowed`. A new switch, `ALLOW_WEBHOOK_WORKFLOWS_SINK_TO_NON_GLOBAL_ADDRESSES`, controls whether the sink may reach non-global destinations: loopback, RFC 1918 private ranges, link-local including the cloud metadata address `169.254.169.254`, CGNAT, IPv6 ULA, reserved, unspecified and multicast addresses, including IPv4-mapped IPv6 forms. When it is `False`, every A/AAAA record of the target must be global, so a mixed public/private DNS answer is rejected outright (@grzegorz-roboflow, https://github.com/roboflow/inference/pull/2995).

**What administrators should do.**

- **The switch ships as `True` (permissive)**, so webhooks pointing at LAN and private addresses keep working after the upgrade. Servers that execute workflows written by people you do not fully trust should set `ALLOW_WEBHOOK_WORKFLOWS_SINK_TO_NON_GLOBAL_ADDRESSES=False`.
- **Webhook endpoints that answer with a redirect now fail.** Point the block at the final URL.

# 🔧 Fixed

- **Two video sources opening at once no longer hang the first one** — the helper that captures OpenCV/FFmpeg error text redirected the process-wide stderr around each `cv2.VideoCapture` open with no synchronisation. When two opens overlapped (two pipelines starting together is enough), one thread saved the other's pipe as "the real stderr": the first source sat in `INITIALISING` at 0 fps with no error, forever, the process lost its real stderr, and a restart re-ran the race. A backend writing more than 64 KiB during one open could block the same way. Capture is now a single reference-counted, continuously drained redirect shared by all overlapping opens, with an at-fork reset and a graceful no-capture fallback when stderr is closed. `DISABLE_NATIVE_STDERR_CAPTURE=True` (new, default `False`) leaves stderr untouched, at the cost of less detailed `SourceConnectionError` messages. Unchanged and worth knowing: while a source is opening, anything the process writes to stderr is captured rather than logged (@alexnorell, https://github.com/roboflow/inference/pull/3014).
- **YOLO-NAS on TensorRT no longer returns phantom detections under GPU load** — `YOLONasForObjectDetectionTRT` concatenated the engine outputs on the default CUDA stream while NMS ran on the post-processing stream, with nothing ordering the two, so NMS could read a half-written buffer and emit extra boxes non-deterministically when several workers shared a GPU. The concatenation now runs on the inference stream and is synchronised before post-processing. The ONNX backend was never affected. Shipped in `inference-models` `0.37.2` (@grzegorz-roboflow, https://github.com/roboflow/inference/pull/3009).
- **Usage is recorded for requests authenticated only by the `Authorization` header** — on routes that resolve the key inside the handler, such as the legacy `/{dataset_id}/{version_id}` route, a request carrying its key only as `Authorization: Bearer <api_key>` passed authentication but its usage row was silently dropped, because the collector looked for the key in bound call arguments only. It now falls back to the request-scoped header key. Expect reported usage of header-authenticated clients to rise to its true level after the upgrade (@grzegorz-roboflow, https://github.com/roboflow/inference/pull/3020).
- **Workspaces whose id starts with `-` or `_` can use inference again** — the workspace id pattern required an alphanumeric first character, so a real workspace such as `-li3oe` failed validation: serverless authorization answered 500 ("workspace lookup returned an invalid identity"), routes resolving the workspace answered 502, and the `X-Roboflow-Workspace-Id` header was dropped. The pattern is now `[A-Za-z0-9_-]+`; everything else is still rejected (@mkaic, https://github.com/roboflow/inference/pull/3027).
- **Polygon, halo and label visualizations draw RLE masks** — RLE-first segmentation predictions (the new `open_ai@v7` task above, and the `rle_instance_segmentation_prediction` kind in general) carry their masks in `data["rle_mask"]` with `mask=None`, and only `mask_visualization@v1` decoded them. `label_visualization@v1`/`@v2` and `rich_label_visualization@v1` with `text_position: CENTER_OF_MASS` failed with "Cannot use Position.CENTER_OF_MASS without a detection mask", and `polygon_visualization@v1`/`@v2` and `halo_visualization@v1`/`@v2` silently drew the bounding box. A shared `ensure_dense_masks` helper now decodes the masks on a shallow copy in all of them. Note the visible change: polygon and halo now draw the real contour where they used to draw a box (@SkalskiP, https://github.com/roboflow/inference/pull/3013).
- **Empty detections keep their image dimensions in two more places** — v1.6.0 taught seven VLM parsers and the serialiser to carry `image.width` / `image.height` on a zero-detection result. Two gaps remained. The Florence-2 object-detection parser in `roboflow_core/vlm_as_detector@v1` and `@v2` has its own code path and still produced `{"width": null, "height": null}`; it now stores the dimensions in object-level `metadata` like the others (@davidnichols-ops, https://github.com/roboflow/inference/pull/2983). And two consumers dropped what the producers had stored: the deserialiser returned early for zero rows, so an empty result lost its dimensions on a serialise/deserialise round trip (custom Python blocks, Modal and remote execution), and the numpy-to-tensor converter rebuilt image metadata from per-row columns only. Both now read the object-level `metadata` (@L4XB, https://github.com/roboflow/inference/pull/2982).
- **`describe_interface` accepts outputs that pass an input through** — a workflow with an output such as `{"name": "img", "selector": "$inputs.image"}` executed correctly, but `POST /workflows/describe_interface` and `POST /{workspace_name}/workflows/{workflow_id}/describe_interface` answered 400 with "Could not find step referred in outputs (`inputs`)". Output discovery now resolves input selectors and reports the input's kinds; a selector that names a missing input still returns a clear 400 (@atirna, https://github.com/roboflow/inference/pull/2936).
- **Detection filters accept `class_name` stored as a NumPy object array** — a custom Python block that filled `sv.Detections.data["class_name"]` with a `dtype=object` array made every downstream `class_name` expression (detections filter, property extraction, conditions) crash with `AttributeError: 'str' object has no attribute 'item'`. Plain Python strings are now accepted next to `numpy.str_` (@hansent, https://github.com/roboflow/inference/pull/3022).
- **Track Class Lock bounds its state per track** — the caps added in v1.6.0 limit the number of tracks, but a single tracker id that carried a new class name on every detection grew its vote tables without bound and re-sorted them on every detection, which is quadratic. `roboflow_core/track_class_lock@v1` now admits at most 256 distinct class names per track (classes already tallied always count, new ones beyond the cap are ignored), sweeps unreachable class ids at the cap, and ranks with a top-2 selection instead of a full sort. Real objects flicker between a handful of classes, so legitimate use is unaffected (@alexeialexandrovich, https://github.com/roboflow/inference/pull/2981).
- **The landing page hides the builder link when the builder is off** — a self-hosted server always showed "Start building locally", which led to a 404 unless `ENABLE_BUILDER=True`. The page now probes `GET /build` once and shows the link only when the builder is mounted. Defaults are unchanged: the builder is still enabled with `ENABLE_BUILDER=True` or `inference server start --dev`, and the `--dev` help text now says that it sets `ENABLE_BUILDER` and `NOTEBOOK_ENABLED` (@mstevens843, https://github.com/roboflow/inference/pull/3001).
- **Model blocks say that the recommended confidence thresholds are per class** — the `Best (Recommended)` option of `confidence_mode` now reads "Use F1-optimal per-class thresholds from model evaluation" in `roboflow_object_detection_model@v3`, `roboflow_instance_segmentation_model@v3`/`@v4`, `roboflow_multi_label_classification_model@v3` and `roboflow_semantic_segmentation_model@v2`. Wording only (@leeclemnet, https://github.com/roboflow/inference/pull/2998).

# 🚧 Maintenance

- **Security dependency refresh (2026-09-18)** — `fastapi` moves from `>=0.100,<0.129` to `>=0.134,<0.142`, and the Workflows lockfile resolves Starlette `1.6.0`, in response to Dependabot alerts on Starlette (@grzegorz-roboflow, https://github.com/roboflow/inference/pull/3024). `inference-models` moves to `0.37.4`, which adds an `anyio>=4.14.2,<5.0.0` floor for security reasons; every requirements file pins `inference-models~=0.37.4` (@PawelPeczek-Roboflow, https://github.com/roboflow/inference/pull/3029).
- **Packaging after the Workflows split** — `requirements/requirements.workflows.txt` is new and pins `roboflow-workflows[enterprise]==0.1.1`. The industrial and database client libraries (`asyncua`, `paho-mqtt`, `pylogix`, `pymodbus`, `psycopg[binary]`) left the shared requirements and now arrive through that `enterprise` extra, so the installed set is the same. `inference-sdk` is no longer bundled inside the `inference` wheels; they depend on `inference-sdk==1.6.1` instead, which removes overlapping files between the two distributions. Every Docker image builds the Workflows wheel from source, and the macOS and Windows app bundles collect `roboflow_workflows` (https://github.com/roboflow/inference/pull/3004).
- **Compatibility repairs made along the way** — two regressions introduced by the refactor were fixed before any release carried them. Segmentation converters briefly returned dataclasses, breaking `model_dump()` for callers; `InferenceResponseImage`, `InferenceResponse`, `CvInferenceResponse`, `WithVisualizationResponse` and `InstanceSegmentationInferenceResponse` are Pydantic models again, defined in `roboflow-workflows` and re-exported from `inference.core.entities.responses.inference` (https://github.com/roboflow/inference/pull/2991). And direct Python users building `WorkflowImageData` from a file path or URL outside an engine lost image loading; `inference` now registers its guarded image codec lazily at import (https://github.com/roboflow/inference/pull/2996).
- **JetPack 5.1.1 image builds again** — the Workflows install step re-resolved `transformers` above the `5.7.0` that the JetPack 5 PyTorch can run; the downgrade now happens after that step. Only `main` builds between 2026-09-16 and 2026-09-18 were affected (@grzegorz-roboflow, https://github.com/roboflow/inference/pull/3030).
- **CI for the new package** — `roboflow-workflows` is published through PyPI trusted publishing on release, with a build-only manual dispatch (https://github.com/roboflow/inference/pull/3003); the `inference-models` integration tests install the Workflows wheel (https://github.com/roboflow/inference/pull/3019); a test import that still used the legacy path inside the standalone package was corrected (https://github.com/roboflow/inference/pull/3023); the TensorRT CUDA-graph cache-reuse test builds its batch through the real pre-processing path (https://github.com/roboflow/inference/pull/3002); linters after https://github.com/roboflow/inference/pull/2936 (https://github.com/roboflow/inference/pull/3021); a direct changelog commit was reverted and its text re-landed under `inference-models` `0.37.2` (https://github.com/roboflow/inference/pull/3011).
- **Version** — `inference` `1.6.1`, `inference-models` `0.37.4`, `roboflow-workflows` `0.1.1` (@PawelPeczek-Roboflow, https://github.com/roboflow/inference/pull/3026).

# 📓  Side-notes

## 📦 Workflows ships as a standalone `roboflow-workflows` package

Marching towards new server we needed to make  Workflows detached from the old `inference`. It's still fully compatible and imports are re-mapped within `inference` package. The change is **purely tactical for now**, the new package is **not intended to be installed stand-alone for now.**

## 🏅 New Contributors

* @mstevens843 made their first contribution in https://github.com/roboflow/inference/pull/3001
* @tewanchia made their first contribution in https://github.com/roboflow/inference/pull/3008
* @L4XB made their first contribution in https://github.com/roboflow/inference/pull/2982
* @atirna made their first contribution in https://github.com/roboflow/inference/pull/2936

**Full Changelog**: https://github.com/roboflow/inference/compare/v1.6.0...v1.6.1