v0.13.0

# SkyPilot v0.13.0: Sky Batch, Hugging Face Storage, Lifecycle Hooks, Cost Caps, Blackwell GPUs, and More

SkyPilot v0.13.0 introduces **Sky Batch** for large-scale batch inference and data processing across worker pools, a native **Hugging Face storage backend** (`hf://`), a generalized **lifecycle-hooks framework** for running custom logic around cluster events, and the `sky debug-dump` command for one-shot troubleshooting. This release also adds **per-instance cost caps**, **NVIDIA Blackwell / CUDA 13** images, and **GKE Autopilot** support.

Get it now with:

```bash
uv pip install "skypilot[all]==0.13.0"
```

Or, upgrade your team SkyPilot API server:

```bash
NAMESPACE=skypilot
RELEASE_NAME=skypilot
VERSION=0.13.0

helm repo update skypilot
helm upgrade -n $NAMESPACE $RELEASE_NAME skypilot/skypilot \
  --set apiService.image=berkeleyskypilot/skypilot:$VERSION \
  --version $VERSION --devel --reuse-values
```

**Deprecations & Breaking Changes**:

- `api_access` renamed to `api_server_access`, now defaulting to `True` ([#9169](https://github.com/skypilot-org/skypilot/pull/9169)). Tasks now automatically receive API server credentials. The legacy `api_access` key is still accepted for backward compatibility.
- `sky.jobs.queue(version=1)` remains deprecated and continues to emit a warning; use `sky.jobs.queue(version=2)`, which returns richer job metadata as dictionaries ([#9118](https://github.com/skypilot-org/skypilot/pull/9118)).
- **Removed `all_includes_in_cluster`** from the SkyPilot config schema ([#9671](https://github.com/skypilot-org/skypilot/pull/9671)).

## Highlights

### [New] [Sky Batch](https://docs.skypilot.co/en/latest/examples/batch/index.html)

`**sky batch**` brings large-scale batch inference and data processing to SkyPilot with a simple Python API ([#8850](https://github.com/skypilot-org/skypilot/pull/8850), [#9438](https://github.com/skypilot-org/skypilot/pull/9438), [#9456](https://github.com/skypilot-org/skypilot/pull/9456)). Given a dataset and a processing function, Sky Batch splits the data into batches, distributes them across a pool of GPU/CPU workers, and collects the results, with no manual sharding, provisioning, or failure handling. Workers are reused across jobs, so expensive setup (installing packages, downloading weights, loading models onto GPUs) happens only once.

```python
import sky

# Point at your input data in cloud storage
ds = sky.batch.Dataset(source='s3://my-bucket/prompts/')

# Map a processing function over the dataset across a worker pool
ds.map(my_inference_fn, pool='my-gpu-pool', output='s3://my-bucket/results/')
```

Inputs and outputs can be read from and written to Amazon S3 (`s3://`) or Google Cloud Storage (`gs://`).

### [New] [Hugging Face Storage Backend](https://docs.skypilot.co/en/latest/reference/storage.html)

SkyPilot now supports **Hugging Face** as a first-class storage backend via the `hf://` scheme ([#9698](https://github.com/skypilot-org/skypilot/pull/9698), [#9763](https://github.com/skypilot-org/skypilot/pull/9763)). It wraps read-write **Hugging Face Buckets** (Xet-backed, S3-like object storage) and, through the same handle, read-only mounting of model, dataset, and space repos. Bucket lifecycle operations go through `huggingface_hub`, and `MOUNT`/`MOUNT_CACHED` modes use the FUSE-based `hf-mount`, consistent with SkyPilot's existing mount infrastructure.

```yaml
file_mounts:
  # Read-write Hugging Face Bucket
  /data:
    source: hf://buckets/my-org/my-bucket
  # Read-only model / dataset / space repo
  /model:
    source: hf://meta-llama/Llama-3.1-8B
    mode: MOUNT
```

URL formats: `hf://buckets/<ns>/<bucket>` (bucket, read-write), `hf://<ns>/<model>[@<rev>]`, `hf://datasets/<ns>/<ds>`, and `hf://spaces/<ns>/<sp>` (repos, read-only). Authentication reads `HF_TOKEN` from the environment or the Hugging Face CLI token file.

### [New] [Lifecycle Hooks Framework](https://docs.skypilot.co/en/latest/reference/yaml-spec.html)

The autostop hook introduced in v0.12 has grown into a **generalized lifecycle-hooks framework** via `resources.hooks` ([#9064](https://github.com/skypilot-org/skypilot/pull/9064)). Each hook declares a script and the events it fires on: `autostop`, `preemption`, and `down`. Use it to checkpoint state, sync experiment trackers, or send notifications around cluster lifecycle transitions.

```yaml
resources:
  hooks:
    - run: |
        wandb sync
        curl -X POST $SLACK_WEBHOOK -d '{"text": "Cluster shutting down"}'
      events: [autostop, down]
      timeout: 300
```

The legacy `autostop.hook` YAML continues to work with a deprecation warning.

### [New] `sky debug-dump`: One-Shot Troubleshooting

The new `**sky debug-dump**` command collects diagnostic information (logs, state, and configuration) for clusters, requests, and managed jobs into a single zip for troubleshooting and bug reports ([#8868](https://github.com/skypilot-org/skypilot/pull/8868), [#9752](https://github.com/skypilot-org/skypilot/pull/9752), [#9753](https://github.com/skypilot-org/skypilot/pull/9753)). Dumps are stored under `~/.sky/debug_dumps/` and automatically cross-link related resources (e.g. a cluster and its associated requests) so you capture complete context in one command.

```bash
sky debug-dump --cluster my-cluster
```

### Cost Caps: Max Hourly Cost

You can now set a **per-instance hourly budget cap** with `resources.max_hourly_cost` ([#9132](https://github.com/skypilot-org/skypilot/pull/9132), [#9170](https://github.com/skypilot-org/skypilot/pull/9170)). Only instances at or below the limit are considered during optimization. When `use_spot` is set, the cap applies against spot prices; otherwise against on-demand prices.

```yaml
resources:
  accelerators: A100
  max_hourly_cost: 10.0
```

### NVIDIA Blackwell & CUDA 13 Images

SkyPilot's default GPU image now ships **NVIDIA driver 580 (open) + CUDA 13** for Blackwell support ([#9639](https://github.com/skypilot-org/skypilot/pull/9639)), with a legacy CUDA 12 image retained for pre-Turing GPUs and an arm64 build fix ([#9789](https://github.com/skypilot-org/skypilot/pull/9789)). RunPod adds RTX PRO Blackwell GPUs to its catalog ([#9187](https://github.com/skypilot-org/skypilot/pull/9187)).

### GKE Autopilot Support

SkyPilot now **detects GKE Autopilot** clusters and trusts Node Auto-Provisioning (NAP) instead of walking existing node pools, so GPU requests provision correctly on Autopilot instead of being incorrectly rejected ([#9565](https://github.com/skypilot-org/skypilot/pull/9565)).

---

## What's new: compared to v0.12.0

### API Server

- **RBAC viewer role** for read-only dashboard/API access ([#9681](https://github.com/skypilot-org/skypilot/pull/9681)).
- **Reconcile internal daemon rows** on API server boot ([#9668](https://github.com/skypilot-org/skypilot/pull/9668)) and skip request env overlay for internal daemon requests ([#9663](https://github.com/skypilot-org/skypilot/pull/9663)).
- **Bundle SSH credentials with the launch response** ([#9586](https://github.com/skypilot-org/skypilot/pull/9586)).
- Refinements for graceful upgrade ([#9317](https://github.com/skypilot-org/skypilot/pull/9317)) and `MIN_COMPATIBLE_API_VERSION` bump for 0.12 clients ([#9121](https://github.com/skypilot-org/skypilot/pull/9121)).
- Reap multiprocess Prometheus files from dead/exited workers ([#9689](https://github.com/skypilot-org/skypilot/pull/9689), [#9690](https://github.com/skypilot-org/skypilot/pull/9690)) and periodic scrape-timeout tuning ([#9480](https://github.com/skypilot-org/skypilot/pull/9480), [#9434](https://github.com/skypilot-org/skypilot/pull/9434)).
- Return 404 for missing dashboard `_next/` assets instead of index.html ([#9722](https://github.com/skypilot-org/skypilot/pull/9722)); don't cache error responses on `/dashboard/_next` paths ([#9498](https://github.com/skypilot-org/skypilot/pull/9498)).
- Fixed incorrect headers passed by oauth2-proxy ([#9468](https://github.com/skypilot-org/skypilot/pull/9468)) and oauth pod scheduling ([#9288](https://github.com/skypilot-org/skypilot/pull/9288)).
- Async engine improvements: hand asyncpg the libpq DSN via `async_creator` ([#9483](https://github.com/skypilot-org/skypilot/pull/9483)), `put_async` for the queue backend ([#9346](https://github.com/skypilot-org/skypilot/pull/9346)).
- Workspace controls: name-length limits ([#9162](https://github.com/skypilot-org/skypilot/pull/9162)), workspace permission checks for `enabled_clouds` and `sky check` ([#9284](https://github.com/skypilot-org/skypilot/pull/9284)), cached permission checks ([#9125](https://github.com/skypilot-org/skypilot/pull/9125)).
- Helm chart: `devMode` flag for in-cluster iteration ([#9609](https://github.com/skypilot-org/skypilot/pull/9609)), persist the whole `sky_logs` dir ([#9454](https://github.com/skypilot-org/skypilot/pull/9454)), optional server log mirror ([#9754](https://github.com/skypilot-org/skypilot/pull/9754)), configurable probes ([#9477](https://github.com/skypilot-org/skypilot/pull/9477)), disable session affinity on ingress ([#9347](https://github.com/skypilot-org/skypilot/pull/9347)), option to disable serving the server log ([#9140](https://github.com/skypilot-org/skypilot/pull/9140)).

### Kubernetes

- **GKE Autopilot support** via `autopilot.enabled` detection and NAP trust ([#9565](https://github.com/skypilot-org/skypilot/pull/9565)).
- **Per-workspace namespaces** via `context_configs` ([#9640](https://github.com/skypilot-org/skypilot/pull/9640)).
- **Container operations for clusters on Kubernetes** ([#9066](https://github.com/skypilot-org/skypilot/pull/9066)).
- `**ephemeral-storage` requests/limits** on pods ([#9053](https://github.com/skypilot-org/skypilot/pull/9053)), `allowed_nodes` config for node-level filtering ([#9151](https://github.com/skypilot-org/skypilot/pull/9151)), pod spread constraints ([#9296](https://github.com/skypilot-org/skypilot/pull/9296)).
- **Configurable APT mirrors** ([#9521](https://github.com/skypilot-org/skypilot/pull/9521), [#9553](https://github.com/skypilot-org/skypilot/pull/9553)) and timeouts on APT commands ([#9375](https://github.com/skypilot-org/skypilot/pull/9375)).
- Surface pod OOM and eviction reasons for jobs and clusters ([#9758](https://github.com/skypilot-org/skypilot/pull/9758)); better provisioning-failure reasons ([#9389](https://github.com/skypilot-org/skypilot/pull/9389)); fix misleading "Pulling image" status during init-container execution ([#9702](https://github.com/skypilot-org/skypilot/pull/9702)).
- Treat pods as scheduled on binding rather than container status ([#9766](https://github.com/skypilot-org/skypilot/pull/9766)); auto-increase provision timeout when an autoscaler is configured ([#9322](https://github.com/skypilot-org/skypilot/pull/9322)).
- Respect configured tolerations in node-health display ([#9750](https://github.com/skypilot-org/skypilot/pull/9750)).
- Allow `pod_config` to override `runtimeClassName` for GPU pods ([#9241](https://github.com/skypilot-org/skypilot/pull/9241)); probe Ray ports when `hostNetwork: true` ([#9644](https://github.com/skypilot-org/skypilot/pull/9644)).
- Fail fast when a cluster has no default StorageClass ([#9713](https://github.com/skypilot-org/skypilot/pull/9713)); auto-clamp CPU/memory for capacity ([#9179](https://github.com/skypilot-org/skypilot/pull/9179)).
- `network_tier: best` for OCI OKE RoCE ([#9622](https://github.com/skypilot-org/skypilot/pull/9622)); allow excluding the in-cluster context from `allowed_contexts: all` ([#9637](https://github.com/skypilot-org/skypilot/pull/9637)).
- Fixed GPU labeller crash preserving method metadata ([#9152](https://github.com/skypilot-org/skypilot/pull/9152)), `volumeMounts` patch-merge key from `name` to `mountPath` ([#9358](https://github.com/skypilot-org/skypilot/pull/9358)), write permissions for all volume mounts at startup ([#9221](https://github.com/skypilot-org/skypilot/pull/9221)), owner-identity check for scoped kubeconfig names ([#9725](https://github.com/skypilot-org/skypilot/pull/9725), [#9190](https://github.com/skypilot-org/skypilot/pull/9190)). Updated default/base k8s images ([#9618](https://github.com/skypilot-org/skypilot/pull/9618), [#9590](https://github.com/skypilot-org/skypilot/pull/9590)).

### Volumes

- **Volume auto-mounts on Kubernetes** ([#9205](https://github.com/skypilot-org/skypilot/pull/9205), [#9353](https://github.com/skypilot-org/skypilot/pull/9353)).
- **New `k8s-hostpath` volume type** for node-local cache ([#9165](https://github.com/skypilot-org/skypilot/pull/9165)).
- **Volume detail page** with saved/shown volume YAML ([#9101](https://github.com/skypilot-org/skypilot/pull/9101)) and refreshed volume page styling ([#9122](https://github.com/skypilot-org/skypilot/pull/9122)).
- Infer `access_mode` from an existing PVC for `use_existing` volumes ([#9359](https://github.com/skypilot-org/skypilot/pull/9359)); check the existing backend when creating volumes ([#9104](https://github.com/skypilot-org/skypilot/pull/9104)); add volume mount check ([#9343](https://github.com/skypilot-org/skypilot/pull/9343)).
- Show `Ti` units for volume sizes ≥ 1024Gi ([#9369](https://github.com/skypilot-org/skypilot/pull/9369)); don't rewrite region to `in-cluster` when in-cluster auth is unavailable ([#9743](https://github.com/skypilot-org/skypilot/pull/9743)).

### Storage

- **Hugging Face storage backend** (`hf://`) for buckets and repos ([#9698](https://github.com/skypilot-org/skypilot/pull/9698), [#9763](https://github.com/skypilot-org/skypilot/pull/9763)).
- **Read-only support for `MOUNT`** ([#9225](https://github.com/skypilot-org/skypilot/pull/9225)).
- Upload v2 improvements: narrowed locking ([#9460](https://github.com/skypilot-org/skypilot/pull/9460)), skip-assemble support ([#9458](https://github.com/skypilot-org/skypilot/pull/9458)), delayed extract ([#9376](https://github.com/skypilot-org/skypilot/pull/9376)), and upload chunk size capped to fit Cloudflare's 100MB limit ([#9360](https://github.com/skypilot-org/skypilot/pull/9360)).
- Upgraded goofys to 0.24.0 ([#9222](https://github.com/skypilot-org/skypilot/pull/9222)); mount-command cleanup for `S3CompatibleStore` ([#9080](https://github.com/skypilot-org/skypilot/pull/9080)); fixed git "dubious ownership" during workdir sync ([#9352](https://github.com/skypilot-org/skypilot/pull/9352)).

### Core & Backend

- **Support specifying both a cloud VM image and a Docker image** on the same resources ([#9759](https://github.com/skypilot-org/skypilot/pull/9759)).
- **Deep-merge workspace config with global cloud config** ([#9788](https://github.com/skypilot-org/skypilot/pull/9788)).
- **Speed up log download and tailing for large jobs** ([#9464](https://github.com/skypilot-org/skypilot/pull/9464)).
- **Turn silent launch misconfigurations into explicit errors** ([#9423](https://github.com/skypilot-org/skypilot/pull/9423)) with improved user-facing messages ([#9403](https://github.com/skypilot-org/skypilot/pull/9403)); detect file paths mistakenly used as cluster names ([#9233](https://github.com/skypilot-org/skypilot/pull/9233)).
- **Fall back to SSH when the skylet gRPC endpoint is unreachable** ([#9287](https://github.com/skypilot-org/skypilot/pull/9287)); cancel in-flight skylet gRPC calls on context cancel ([#9620](https://github.com/skypilot-org/skypilot/pull/9620)); bind gRPC to all interfaces ([#9282](https://github.com/skypilot-org/skypilot/pull/9282)).
- Bumped default cluster event retention to 30 days ([#9462](https://github.com/skypilot-org/skypilot/pull/9462)).
- Deterministic config hash ([#9415](https://github.com/skypilot-org/skypilot/pull/9415)), reproducible wheel builds ([#9368](https://github.com/skypilot-org/skypilot/pull/9368)), and excluding `__pycache__` from the built wheel ([#9413](https://github.com/skypilot-org/skypilot/pull/9413)).
- Allow relative `file_mounts` destinations again ([#9537](https://github.com/skypilot-org/skypilot/pull/9537)); fixed cluster re-provisioning and atomic SSH-key writes ([#9476](https://github.com/skypilot-org/skypilot/pull/9476)); prioritize user SSH keys over cluster keys in `Host *` config ([#9099](https://github.com/skypilot-org/skypilot/pull/9099)).
- Fixed autostop ignoring active SSH after stop/start ([#9539](https://github.com/skypilot-org/skypilot/pull/9539)); close `/dev/fuse` fd after passing it to the client ([#9463](https://github.com/skypilot-org/skypilot/pull/9463)); fixed a flock leak in the `uv_spawn` child process ([#9431](https://github.com/skypilot-org/skypilot/pull/9431)); fixed browser opening on WSL for `sky api login` ([#9178](https://github.com/skypilot-org/skypilot/pull/9178)).
- Moved the catalog dir to `SKY_RUNTIME_DIR` and removed import-time filesystem side effects ([#9605](https://github.com/skypilot-org/skypilot/pull/9605)); let packages expose client SDKs under the `sky` namespace ([#9784](https://github.com/skypilot-org/skypilot/pull/9784)).
- Expanded plugin extension points and config sections for third-party extensibility ([#9494](https://github.com/skypilot-org/skypilot/pull/9494), [#9510](https://github.com/skypilot-org/skypilot/pull/9510), [#9457](https://github.com/skypilot-org/skypilot/pull/9457), [#9616](https://github.com/skypilot-org/skypilot/pull/9616)).

### Managed Jobs & Serve

- `**--tail` for `sky jobs logs`** ([#9449](https://github.com/skypilot-org/skypilot/pull/9449)), `preload_content` for `sky.jobs.tail_logs` ([#9662](https://github.com/skypilot-org/skypilot/pull/9662)), and a `wait` helper in the programmatic SDK ([#9138](https://github.com/skypilot-org/skypilot/pull/9138)).
- **Provisioning spinner and dashboard links** during `sky jobs launch` ([#9761](https://github.com/skypilot-org/skypilot/pull/9761), [#9135](https://github.com/skypilot-org/skypilot/pull/9135), [#9191](https://github.com/skypilot-org/skypilot/pull/9191)); optionally merge cluster launch-progress events into job events ([#9804](https://github.com/skypilot-org/skypilot/pull/9804)).
- **Custom job recovery strategy configuration** ([#9154](https://github.com/skypilot-org/skypilot/pull/9154)); retry transient DB errors in the controller monitor loop ([#9748](https://github.com/skypilot-org/skypilot/pull/9748)); daemon to sweep expired managed-job API access tokens ([#9588](https://github.com/skypilot-org/skypilot/pull/9588)).
- Fixed git commit metadata not captured in managed jobs ([#9184](https://github.com/skypilot-org/skypilot/pull/9184)) and git workdir env vars not set for JobGroup / multi-task DAGs ([#9251](https://github.com/skypilot-org/skypilot/pull/9251)); fixed user-hash races in concurrent job-group launches ([#9248](https://github.com/skypilot-org/skypilot/pull/9248)).
- **Serve**: `ServiceStatusRunner` extension point ([#9801](https://github.com/skypilot-org/skypilot/pull/9801)), new user-configurable options ([#9116](https://github.com/skypilot-org/skypilot/pull/9116)), and improved max-services error messages ([#8897](https://github.com/skypilot-org/skypilot/pull/8897)).
- Consolidation mode: use short workers ([#9442](https://github.com/skypilot-org/skypilot/pull/9442)) and skip the two-hop path ([#9444](https://github.com/skypilot-org/skypilot/pull/9444)).

### Pools & Batch

- **Sky Batch** for distributed batch inference over worker pools ([#8850](https://github.com/skypilot-org/skypilot/pull/8850), [#9438](https://github.com/skypilot-org/skypilot/pull/9438), [#9456](https://github.com/skypilot-org/skypilot/pull/9456)).
- Prioritize idle workers for scale-down ([#9098](https://github.com/skypilot-org/skypilot/pull/9098)); speed up the pool dashboard with batched queries and indexes ([#9636](https://github.com/skypilot-org/skypilot/pull/9636)); fixed pool consolidation under deploy-mode auto-enable ([#9342](https://github.com/skypilot-org/skypilot/pull/9342)).

### Infrastructure & Cloud Integrations

**AWS**

- **Support for `aws login`** (2026 AWS CLI command) ([#9526](https://github.com/skypilot-org/skypilot/pull/9526)) and subnet selection ([#9127](https://github.com/skypilot-org/skypilot/pull/9127)).
- Add tags to security groups for scoped IAM policies ([#8981](https://github.com/skypilot-org/skypilot/pull/8981)); fixed false MISMATCH error when `subnet_names` span multiple AZs ([#9791](https://github.com/skypilot-org/skypilot/pull/9791)); added AZ-fetch timeouts to prevent `sky check` hangs ([#9244](https://github.com/skypilot-org/skypilot/pull/9244), [#9240](https://github.com/skypilot-org/skypilot/pull/9240)).

**GCP**

- **Select specific VPC subnets** ([#9334](https://github.com/skypilot-org/skypilot/pull/9334)); set `provisioningModel=RESERVATION_BOUND` for DENSE reservations ([#9779](https://github.com/skypilot-org/skypilot/pull/9779)); added missing required instance types to the catalog ([#9472](https://github.com/skypilot-org/skypilot/pull/9472)); pinned/bumped the gcloud SDK ([#9587](https://github.com/skypilot-org/skypilot/pull/9587), [#9535](https://github.com/skypilot-org/skypilot/pull/9535)); sped up `sky check` ([#9095](https://github.com/skypilot-org/skypilot/pull/9095)).

**Nebius**

- **Native security groups with `open_ports`** replacing UFW ([#9563](https://github.com/skypilot-org/skypilot/pull/9563)); use managed disks to create instances ([#9432](https://github.com/skypilot-org/skypilot/pull/9432)); parallelize multi-node VM creation ([#9570](https://github.com/skypilot-org/skypilot/pull/9570)); custom `image_id` ([#9133](https://github.com/skypilot-org/skypilot/pull/9133)); fixed autodown ([#9573](https://github.com/skypilot-org/skypilot/pull/9573)).

**Slurm**

- `cpu_partition` config for CPU-only tasks ([#9281](https://github.com/skypilot-org/skypilot/pull/9281)) and `gpu_partition_map` for GRES without a GPU type ([#9237](https://github.com/skypilot-org/skypilot/pull/9237)).
- Let users override `--time` and fall back to `DefaultTime` when `MaxTime` is UNLIMITED ([#9633](https://github.com/skypilot-org/skypilot/pull/9633)); preserve `SLURM_CONF*` env vars when launching `srun` ([#9425](https://github.com/skypilot-org/skypilot/pull/9425)).
- Fixed a proctrack barrier hang on multi-node container jobs ([#9181](https://github.com/skypilot-org/skypilot/pull/9181)), fractional memory truncation in `sbatch` ([#9174](https://github.com/skypilot-org/skypilot/pull/9174)), numeric partition names treated as integers ([#9168](https://github.com/skypilot-org/skypilot/pull/9168)), and `--mem` when memory is not consumable ([#9171](https://github.com/skypilot-org/skypilot/pull/9171)).

**RunPod / Vast / PrimeIntellect**

- RunPod: RTX PRO Blackwell GPUs in the GPU name map ([#9187](https://github.com/skypilot-org/skypilot/pull/9187)); pass formatted image name to `create_template` ([#9550](https://github.com/skypilot-org/skypilot/pull/9550)).
- Vast: fixed API compatibility with `vastai-sdk` v6+ ([#9314](https://github.com/skypilot-org/skypilot/pull/9314)); made account-level SSH key registration best-effort ([#9631](https://github.com/skypilot-org/skypilot/pull/9631)).
- PrimeIntellect: fixed `query_instances` TypeError on retry ([#9259](https://github.com/skypilot-org/skypilot/pull/9259)) and instance update after `_get_instance_info` ([#9327](https://github.com/skypilot-org/skypilot/pull/9327)).

### SSH Node Pools

- **Install Prometheus + node-exporter in `sky ssh up`** ([#9402](https://github.com/skypilot-org/skypilot/pull/9402)).
- **Auto-enable HA K3s for pools of 3+ nodes** ([#9351](https://github.com/skypilot-org/skypilot/pull/9351)).
- Wait for async Ray setup on SSH node pools ([#9299](https://github.com/skypilot-org/skypilot/pull/9299)); support redirect in the WebSocket SSH proxy ([#9167](https://github.com/skypilot-org/skypilot/pull/9167)).

### User Experience

**Dashboard**

- **Batch-edit user roles and workspace `allowed_users`** ([#9696](https://github.com/skypilot-org/skypilot/pull/9696)); show counts above the users and service-account tables ([#9655](https://github.com/skypilot-org/skypilot/pull/9655)).
- **Admin-configurable external links** ([#9603](https://github.com/skypilot-org/skypilot/pull/9603)) and detection of W&B Dedicated Cloud run URLs ([#9383](https://github.com/skypilot-org/skypilot/pull/9383)).
- **Provision log in the cluster detail page** ([#9201](https://github.com/skypilot-org/skypilot/pull/9201), [#9344](https://github.com/skypilot-org/skypilot/pull/9344)).
- Managed Jobs filter polish: More dropdown, Active/All toggle, default to "Mine" ([#9707](https://github.com/skypilot-org/skypilot/pull/9707)), default sort by ID descending ([#9522](https://github.com/skypilot-org/skypilot/pull/9522)), and don't block the table on `/status` when the controller is up ([#9645](https://github.com/skypilot-org/skypilot/pull/9645)).
- Unified filter-bar sizing and styling across list pages ([#9407](https://github.com/skypilot-org/skypilot/pull/9407)); improved performance for cluster detail and job pages ([#9195](https://github.com/skypilot-org/skypilot/pull/9195)); quieter upgrade banner ([#9654](https://github.com/skypilot-org/skypilot/pull/9654)).
- Fixed infinite-spinner states on root and detail pages ([#9139](https://github.com/skypilot-org/skypilot/pull/9139), [#9147](https://github.com/skypilot-org/skypilot/pull/9147)), routing for catch-all pages opened in new tabs ([#9137](https://github.com/skypilot-org/skypilot/pull/9137)), and layout flashes on slow plugin loads ([#9208](https://github.com/skypilot-org/skypilot/pull/9208)).

**CLI**

- `**sky api login --no-browser`** for headless environments, continuing to poll when a browser can't be opened ([#9677](https://github.com/skypilot-org/skypilot/pull/9677), [#9676](https://github.com/skypilot-org/skypilot/pull/9676)).
- `**sky debug-dump**` troubleshooting command ([#8868](https://github.com/skypilot-org/skypilot/pull/8868)).
- Display autostop in hours when minutes are divisible by 60 ([#9453](https://github.com/skypilot-org/skypilot/pull/9453)); JSON output for `cost-report` ([#9504](https://github.com/skypilot-org/skypilot/pull/9504)).

### Dependencies

- Bumped protobuf to 5.29.6 and black to 24.10.0 to close CVE alerts ([#9541](https://github.com/skypilot-org/skypilot/pull/9541)).
- Pinned `kubernetes < 36` due to an RBAC regression, while accepting `dict[K, V]` openapi_types for `kubernetes >= 36` ([#9685](https://github.com/skypilot-org/skypilot/pull/9685), [#9682](https://github.com/skypilot-org/skypilot/pull/9682)).
- Capped `azure-cli < 2.87.0` (azure-mgmt-storage 25.0.0 breaks Azure storage) ([#9774](https://github.com/skypilot-org/skypilot/pull/9774)).
- Dropped the `pyopenssl` upper limit ([#8070](https://github.com/skypilot-org/skypilot/pull/8070)); pinned `click < 8.3.0` in relevant runtime installs ([#9459](https://github.com/skypilot-org/skypilot/pull/9459), [#9763](https://github.com/skypilot-org/skypilot/pull/9763)).

### Documentation & Examples

- **Sky Batch docs** including custom formats ([#9438](https://github.com/skypilot-org/skypilot/pull/9438)); `MOUNT_CACHED` config documentation ([#9130](https://github.com/skypilot-org/skypilot/pull/9130)); `api_server_access` documentation ([#9115](https://github.com/skypilot-org/skypilot/pull/9115)); refreshed managed jobs, quickstart, and agent-skills docs ([#9564](https://github.com/skypilot-org/skypilot/pull/9564), [#9250](https://github.com/skypilot-org/skypilot/pull/9250)).
- **New examples**: autonomous code optimization ([#9182](https://github.com/skypilot-org/skypilot/pull/9182)), parallel autoresearch ([#9092](https://github.com/skypilot-org/skypilot/pull/9092), [#9131](https://github.com/skypilot-org/skypilot/pull/9131)), RL post-training with policy pushback ([#9625](https://github.com/skypilot-org/skypilot/pull/9625)), Cosmos3 fine-tuning and nano ([#9809](https://github.com/skypilot-org/skypilot/pull/9809), [#9790](https://github.com/skypilot-org/skypilot/pull/9790)).
- **"Copy page as Markdown" button** for LLM-friendly content sharing ([#9093](https://github.com/skypilot-org/skypilot/pull/9093)); faster incremental docs builds ([#9396](https://github.com/skypilot-org/skypilot/pull/9396)); added Slurm to the CoreWeave installation guide ([#9674](https://github.com/skypilot-org/skypilot/pull/9674)); refreshed cloud logos ([#9642](https://github.com/skypilot-org/skypilot/pull/9642), [#9602](https://github.com/skypilot-org/skypilot/pull/9602)).

### Testing & CI/CD

- Nightly workflow: run once daily at 00:00 UTC and drop the preflight schedule ([#9739](https://github.com/skypilot-org/skypilot/pull/9739)); trigger a downstream workflow after PyPI publish ([#9738](https://github.com/skypilot-org/skypilot/pull/9738)).
- Fixed a race condition in the release workflow that caused untagged releases ([#9227](https://github.com/skypilot-org/skypilot/pull/9227)).
- Dropped Black in favor of yapf+isort for the IBM provider ([#9557](https://github.com/skypilot-org/skypilot/pull/9557)); numerous test reliability and flakiness fixes.

## Contributors

Thank you to all contributors who made this release possible!

@aylei, @Michaelvll, @DanielZhangQD, @zpoint, @kevinmingtarja, @lloyd-brown, @concretevitamin, @cg505, @SeungjinYang, @rohansonecha, @kevinzwang, @kyuds, @romilbhardwaj, @alex000kim, @andylizf, @cblmemo, @hentt30, @williamsnell, @Jobarion, and many others.

> *Note: this is a draft. Use GitHub's "Generate release notes" on the tagged release to finalize the exact contributor @handles and the "New Contributors" section.*

## Full Changelog

For a complete list of changes, see the [commit history](https://github.com/skypilot-org/skypilot/compare/v0.12.0...v0.13.0rc1).