v0.13.2rc1
skypilot-org/skypilotv0.13.2rc1Sep 18, 2026by Michaelvll
Upgrading?
AI Summary
This release candidate introduces fixes for Kubernetes pod resource request display and job recovery logic, alongside significant dashboard restyling and improvements to cloud provider integrations like RunPod and Nebius.
Key Highlights
- Kubernetes: Improved pod resource request tracking and handling of persistent volume mount failures.
- Dashboard: Complete restyling of the infra page, YAML editor search panel, and updated branding assets.
- Jobs: Enhanced recovery logic and terminal failure handling when pools no longer exist.
- API: Support for running multiple isolated local API servers on a single machine.
- Cloud Providers: Added federated credentials support for GCS COPY and updated Nebius SDK handling.
New Features
- Support running multiple isolated local API servers on one machine
- Support federated credentials for GCS COPY
- Make the settings config editor card plugin-replaceable
- Compute Kubernetes accelerator node affinity in Python
Full Release Notes
## SkyPilot v0.13.2rc1 Get it now with: ```bash uv pip install "skypilot>=0.13.2rc1" ``` --- ## What's Changed * [Kubernetes] Display actual pod resource requests when they diverge from the resource spec by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10214 * [Jobs] Reset transient job-status-check window after recovery by @spiffaz in https://github.com/skypilot-org/skypilot/pull/10156 * [Dashboard] Restyle the YAML editor's search panel by @stephenkevn in https://github.com/skypilot-org/skypilot/pull/10208 * [Dashboard] Fix top-bar logo vertical alignment by @stephenkevn in https://github.com/skypilot-org/skypilot/pull/10207 * [Jobs] Fail jobs terminally when their pool no longer exists by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10203 * [Kubernetes] Fail provisioning on persistent volume mount failures instead of hanging by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10218 * [Test] okta smoke: don't gate login success on the in-app dashboard redirect by @zpoint in https://github.com/skypilot-org/skypilot/pull/10188 * [Dashboard] Make the settings config editor card plugin-replaceable by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10158 * [Dashboard] Use dark circular favicon for better tab visibility by @hopechong in https://github.com/skypilot-org/skypilot/pull/10215 * [Dashboard] Rework infra page: unified table, per-GPU-type summary, responsive by @stephenkevn in https://github.com/skypilot-org/skypilot/pull/10067 * [Test] Cover more documented env vars in managed jobs env smoke test by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10224 * [Kubernetes] Don't install conda by default and remove it from K8s images by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10132 * [Core] Handle missing Git during commit metadata collection by @edmsimplicio in https://github.com/skypilot-org/skypilot/pull/10230 * [RunPod] Label account SSH keys with the SkyPilot user by @ciprianf in https://github.com/skypilot-org/skypilot/pull/9975 * [Core] debug-dump: bound controller SSH work + add best-effort overall deadline by @claude[bot] in https://github.com/skypilot-org/skypilot/pull/10161 * [Nebius] Keep personal pricing fetch failures out of CLI output by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10222 * [Test] Add golden snapshot tests for kubernetes-ray.yml.j2 rendering by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10226 * [Test] Poll for STOPPED after sky stop in test_resize by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10217 * [Test] Pin zone in skyserve auto_restart.yaml so replica termination finds the instance by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10216 * [Kubernetes] Surface which node a terminated cluster lost by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10233 * [Docs] Trim redundant optimization section from Overview; link SkyPilot Platform by @concretevitamin in https://github.com/skypilot-org/skypilot/pull/10235 * [Docs] Update logo to official brand assets by @stephenkevn in https://github.com/skypilot-org/skypilot/pull/10238 * [Test] Add smoke test for k8s accelerator node-affinity scheduling constraint by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10225 * [Test] Add managed jobs smoke test for pod_config targeting the ray-node container by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10223 * [Dashboard] Use new brand favicon by @stephenkevn in https://github.com/skypilot-org/skypilot/pull/10237 * [Metrics] Stop cluster-label stamping from stalling the metrics server by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10240 * [Test] Fix apiserver-start-stop smoke test setup on bare cloud VMs by @zpoint in https://github.com/skypilot-org/skypilot/pull/10242 * [Test] Drop conda-version guard from test_custom_default_conda_env by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10244 * [Core] Guard source ~/.bashrc against a missing ~/.bashrc by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10245 * [Jobs] Retry transient cluster-status errors in controller monitor loop by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10040 * [Core] Restore the config override in `replace_skypilot_config` when the body raises by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10236 * [LLM] slime example: pin image to known-good nightly tag by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10249 * [Test] Pin failed-mount escalation smoke test to a single Kubernetes context by @MadhavJivrajani in https://github.com/skypilot-org/skypilot/pull/10248 * [Core] PostgresLock: run advisory-lock statements in autocommit mode by @MadhavJivrajani in https://github.com/skypilot-org/skypilot/pull/10251 * [Server] Park launch as WAITING on cluster lock contention instead of blocking a worker by @aylei in https://github.com/skypilot-org/skypilot/pull/10210 * [Core] Surface resolved active_workspace to admin policies by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10241 * [API] Support running multiple isolated local API servers on one machine by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10213 * [Core] Compute Kubernetes accelerator node affinity in Python by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10232 * [Nebius] Wait for instance termination before deleting the cluster security group by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10220 * [Jobs] Collapse repeated job-status polls in the controller log by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10255 * [k8s] Do not count queue-gated (Kueue admission) time against provision_timeout by @MadhavJivrajani in https://github.com/skypilot-org/skypilot/pull/10252 * [Server] Fail /api/get definitively for requests interrupted by a server restart by @aylei in https://github.com/skypilot-org/skypilot/pull/10257 * [Launcher] Fix multi-node rank crash on multi-NIC / hostNetwork K8s n… by @yash170106065 in https://github.com/skypilot-org/skypilot/pull/9924 * Update Together AI references in installation guide by @pavneeta in https://github.com/skypilot-org/skypilot/pull/10260 * [Jobs] Include exit code and log pointer in RECOVERING event for user-job failure restarts by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10256 * [Dashboard] Infra table: unify data cell color across columns by @stephenkevn in https://github.com/skypilot-org/skypilot/pull/10267 * Update docstr for `resolved_config` by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10270 * [GCP] Support federated credentials for GCS COPY by @samuelv8 in https://github.com/skypilot-org/skypilot/pull/10212 * [Nebius] Pin nebius SDK below 0.4 by @zpoint in https://github.com/skypilot-org/skypilot/pull/10280 * [Server] Validate and parameterize request-status queries by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10273 * [Kubernetes] Bound pod wait-loop polls with a request timeout by @aylei in https://github.com/skypilot-org/skypilot/pull/10272 * [LLM] slime example: offload optimizer state to host RAM by default by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10269 * [Slurm] Fix node info aggregation for multi-cluster setups by @liuxu623 in https://github.com/skypilot-org/skypilot/pull/10284 * [Core] Clamp open-files (nofile) limit raise to the attainable hard limit by @claude[bot] in https://github.com/skypilot-org/skypilot/pull/10263 * [Server] Park jobs-controller launches on cluster lock contention too by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10291 * [API] Export metrics for the on-demand request thread executor by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10287 * [Slurm] Enrich node info with CPU/memory allocation and sampled usage by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10293 * [Jobs] Release log-tail threads when the client disconnects by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10290 * [Dashboard] Persist rows-per-page for the cluster jobs and pools tables by @zpoint in https://github.com/skypilot-org/skypilot/pull/10279 * [Test] Fix test_minimal job numbering on Slurm by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10298 * [Server] Return the executor slot when the monitor thread fails to start by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10300 * [Server] Make debug dump creation admin-only by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10277 * [Test] Pin stale-pod-cleanup kubectl to the cluster's landed context by @zpoint in https://github.com/skypilot-org/skypilot/pull/10302 * [k8s] Keep the spinner updating while pods await queue admission by @zpoint in https://github.com/skypilot-org/skypilot/pull/10305 * [Analytics] Add analytics to gain insights to usage by @hopechong in https://github.com/skypilot-org/skypilot/pull/10266 * [Slurm] Fix leaked allocation when terminating a cluster with a running task by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10296 * [Slurm] Unset step-scoped SLURM_* env before running user scripts by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10309 * [Slurm] Submit jobs as authenticated users by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10294 * [Slurm] Add SlurmClient.get_all_jobs_info() for per-node job attribution by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10306 * [Dashboard] Break Slurm clusters down by partition on the Infra page by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10317 * [Dashboard] Show CPU-only Slurm clusters on the Infra page by @aylei in https://github.com/skypilot-org/skypilot/pull/10318 * [Slurm] Use node AllocTRES for GPU availability accounting by @liuxu623 in https://github.com/skypilot-org/skypilot/pull/10285 * [API] Give auth its own thread executor so streams cannot starve it by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10313 * [Nebius] Unpin nebius SDK now that 0.4.3 is loop-aware by @zpoint in https://github.com/skypilot-org/skypilot/pull/10320 * [Slurm] Stop the multi-node run-done barrier from failing completed jobs by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10321 * [Test] Stop the auth-pool test from spawning 128 threads by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10322 * [Core] Optionally route the state DB through a transaction pooler by @aylei in https://github.com/skypilot-org/skypilot/pull/10301 * [Dashboard] Add filtering to the volumes page by @stephenkevn in https://github.com/skypilot-org/skypilot/pull/10316 * [Slurm] Make multi-cluster node info aggregation actually best-effort by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10289 * [Slurm] Keep unreachable Slurm clusters listed on the Infra page by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10329 * [Jobs] Fail deterministic task-prep errors fast instead of retrying them by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10326 * [Server] Scope request-listing body to the request owner by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10278 * Fix/k8s bootstrap package manager agnostic by @trevanth24 in https://github.com/skypilot-org/skypilot/pull/10077 * [Client] Fix `sky api logs` traceback on 4xx from /api/stream by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10341 * [Jobs] Fix managed job crash on empty imagePullSecrets in task config by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10325 * [Core] Read-only visibility of private workspaces for non-members by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10228 * [Test] Classify /slurm_cluster_names as read in workspace-access guard by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10343 * [Core] Add plugin hook for task-overrideable config keys by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10339 * [Test] Stop smoke-test teardowns from skipping cleanup on failure by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10342 * [Jobs] Add inter_connection knob for JobGroups: require or skip in-group networking by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10297 * [Jobs] Review fixes for inter_connection: networking hardening, terminal errors, admin-policy header preservation by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10330 * [Server] Scope request-tracking endpoints to the calling user by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10274 * [API] Measure /api/get latency broken out by request name by @aylei in https://github.com/skypilot-org/skypilot/pull/10349 * [Nebius] Add SkyPilot user-agent prefix by @complynx in https://github.com/skypilot-org/skypilot/pull/10345 * [Core] Add lease-based leader election alternative to advisory locks by @aylei in https://github.com/skypilot-org/skypilot/pull/10347 * [Test] Remove the shared GKE API server nightly lane by @zpoint in https://github.com/skypilot-org/skypilot/pull/10358 * [Examples] Serve CIFAR-10 from a SkyPilot-maintained GCS mirror by @cg505 in https://github.com/skypilot-org/skypilot/pull/10261 * [Test] Pin only object stores whose cloud is enabled on the API server by @zpoint in https://github.com/skypilot-org/skypilot/pull/10361 * [Jobs] Don't re-create user buckets in post-job storage cleanup by @aylei in https://github.com/skypilot-org/skypilot/pull/10353 * [Test] Pin enable_docker off in the non-Debian image smoke test by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10366 * [Helm] History dashboard: bound GPU utilization to 0-100 (fix >100%) by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10368 * fix(schemas): accept fractional GPU counts in ManagedJobRecord by @Anai-Guo in https://github.com/skypilot-org/skypilot/pull/10356 * [Core] Lease leader election: heartbeat on a dedicated unpooled connection by @aylei in https://github.com/skypilot-org/skypilot/pull/10372 * [API] Bound auth middleware DB lookups with a deadline by @aylei in https://github.com/skypilot-org/skypilot/pull/10373 * [Test] Make sky status -o json output tests hermetic by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10106 * [Core] Add equality for the Cloud class; fix JobGroup common-infra selection by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10381 * [CI] Don't auto-retry exclusive smoke steps by @aylei in https://github.com/skypilot-org/skypilot/pull/10385 * [Server] Keep /metrics responsive when a collector's data source hangs by @aylei in https://github.com/skypilot-org/skypilot/pull/10377 * [Jobs] Add a managed-job addressed external log read-back hook by @aylei in https://github.com/skypilot-org/skypilot/pull/10376 * [Test] Pin provision_timeout in test_launching_with_pending_pods by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10389 * [k8s] Gracefully skip Docker CLI install on non-Debian images when enable_docker: ALL by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10382 * [API] Throttle per-request service-account last_used_at writes by @MadhavJivrajani in https://github.com/skypilot-org/skypilot/pull/10391 * [k8s] Report node GPU total from capacity, with allocatable alongside by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10395 * [Test] Resolve managed jobs by job id in smoke tests, not by reused names by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10396 * [API] Redact basic-auth password when printing the API server URL by @Khas-Erdene-Tsogtsaikhan in https://github.com/skypilot-org/skypilot/pull/10286 * [Test] Fix cloud-cmd exclusion in test_services_on_kubernetes by @zpoint in https://github.com/skypilot-org/skypilot/pull/10401 * [Core] Remove queue name keys from config instead of setting them to None by @aylei in https://github.com/skypilot-org/skypilot/pull/10134 * [Jobs] Add index on job_info.schedule_state for scheduler queries by @aylei in https://github.com/skypilot-org/skypilot/pull/10259 * [Jobs] Pass task_name to read_managed_job_logs and document id scope by @aylei in https://github.com/skypilot-org/skypilot/pull/10388 * [Kubernetes] Add scope (personal/workspace/global) to auto_mounts by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10411 * [Server] Export SQLite database on-disk size metrics by @aylei in https://github.com/skypilot-org/skypilot/pull/10363 * [Kubernetes] Size the inline-command limit against the exec request URL by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10403 * [Kubernetes] Fail early when an auto-mounted volume is not ready by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10402 * [Test] Fix container-logs smoke tests: bare kubectl misses non-default namespace by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10420 * [Slurm] Batch inventory queries over one SSH connection by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10425 * [Slurm] Fail fast when a task exits nonzero by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10428 * [Docs] Update README news section by @hopechong in https://github.com/skypilot-org/skypilot/pull/10429 * [Test] container-logs: don't require cluster-scoped pod list by @zpoint in https://github.com/skypilot-org/skypilot/pull/10433 * [Kubernetes] Let a new volume settle before recording it as not ready by @zpoint in https://github.com/skypilot-org/skypilot/pull/10443 * fix(slurm): surface invalid GPU partition mappings by @ryxli in https://github.com/skypilot-org/skypilot/pull/10436 * [Jobs] Fix partial state writes on Job Group Phase-2 sync failure; parallelize group teardown by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10415 * [Jobs] Emergency recovery: skip per-task surgery for Job Groups by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10416 * [Test] Give optimizer dryrun launches unique cluster names by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10448 * Revert "[Kubernetes] Let a new volume settle before recording it as not ready" by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10451 * [Logs][Test] Bound apt stalls in the fluent-bit agent install, and raise log-store smoke-test timeouts by @zpoint in https://github.com/skypilot-org/skypilot/pull/10362 * [Dashboard] Drop unknown filter properties from the URL instead of crashing by @zpoint in https://github.com/skypilot-org/skypilot/pull/10441 * [Core] Let volume_refresh refresh a named subset of volumes by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10452 * feat(slurm): support host path volume mounts by @ryxli in https://github.com/skypilot-org/skypilot/pull/10438 * fix(slurm): respect custom task CPU demands by @ryxli in https://github.com/skypilot-org/skypilot/pull/10447 * [Jobs] Keep the local log copy when external delivery is unconfirmed by @aylei in https://github.com/skypilot-org/skypilot/pull/10410 * [Docs] Revamp Sky Batch page visuals by @Michaelvll in https://github.com/skypilot-org/skypilot/pull/10453 * [Test] Make volume smoke tests behave against a shared remote API server by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10461 * [Core] Force sslmode=disable in the DB pooler hostport rewrite by @cg505 in https://github.com/skypilot-org/skypilot/pull/10463 * [API] Process client task YAMLs in memory by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10435 * [Test] Wait for a created volume to be mountable before launching against it by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10467 * [Test] Make cancel-pytorch smoke test work on H100 by @aylei in https://github.com/skypilot-org/skypilot/pull/10418 * [API] Sweep the clients dir off the event loop, and GC legacy task YAMLs by @aylei in https://github.com/skypilot-org/skypilot/pull/10473 * [Kubernetes] Bound the wait for pods to run, and say why a pod went missing by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10460 * fix(k8s): ignore stale Kueue ErrWorkloadCompose in the pending-reason scan (#10379) by @Anai-Guo in https://github.com/skypilot-org/skypilot/pull/10380 * [Slurm] Impersonate submit users via sudo -u instead of sudo su by @claude[bot] in https://github.com/skypilot-org/skypilot/pull/10477 * [Dashboard] Paginate the node table on the infra context detail page by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10480 * users: block non-admin from granting admin role via service account u… by @intagliated in https://github.com/skypilot-org/skypilot/pull/9932 * [API] Validate the role a service account may be granted by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10482 * [Kubernetes] Detect a volume that will not bind while waiting for pods by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10440 * [Jobs] Bound the job-status retry window by retries as well as elapsed time by @aylei in https://github.com/skypilot-org/skypilot/pull/10499 * [Kubernetes] Let the k8s images be rebuilt on a different base by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10500 * [Test] Pin the auto-mount launches to the context their volumes live on by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10470 * [Dashboard] Show why a volume is not ready by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10486 * [Dashboard] Name the clusters filter params in the URL by @zpoint in https://github.com/skypilot-org/skypilot/pull/10455 * [Server] Serve /metrics on its own event loop by @aylei in https://github.com/skypilot-org/skypilot/pull/10503 * [API] Reload the policy under the lock before mutating it by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10488 * [Dashboard] Name the managed-jobs filter params in the URL by @zpoint in https://github.com/skypilot-org/skypilot/pull/10459 * [Dashboard] Name the filter params on users, volumes, recipes and infra by @zpoint in https://github.com/skypilot-org/skypilot/pull/10505 * [Dashboard] Run the dashboard tests in CI by @zpoint in https://github.com/skypilot-org/skypilot/pull/10510 * [Logs] Fall back to the canonical archive when the distro mirror is down by @zpoint in https://github.com/skypilot-org/skypilot/pull/10512 * [CLI] Allow `sky api login` when SKYPILOT_API_SERVER_ENDPOINT is set by @zpoint in https://github.com/skypilot-org/skypilot/pull/10404 * [LLM] slime example: pin swebench/swesmith deps; declare inter_connection in Job Group YAMLs by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10498 * [Core] Validate the volume where it is created, not only in /volumes/validate by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10513 * [API] Deny a principal whose role the policy system does not recognize by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10501 * [CI] Notify the downstream nightly on no-op cycles too by @zpoint in https://github.com/skypilot-org/skypilot/pull/10529 * [Server] Attribute event loop stalls to the code that caused them by @aylei in https://github.com/skypilot-org/skypilot/pull/10523 * [API] Tell the user why a parked request is still waiting by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10534 * [Test] Fix the inline-env queue assertion matching a log line instead of the queue row by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10539 * fix(slurm): run skylet under an allocation-owned keeper step by @ryxli in https://github.com/skypilot-org/skypilot/pull/10516 * [Volumes] Filter the volume listing to the caller's accessible workspaces by @trevanth24 in https://github.com/skypilot-org/skypilot/pull/10531 * [Core] Redact credentials from SkyPilot config debug logs by @zpoint in https://github.com/skypilot-org/skypilot/pull/10511 * [Metrics] Client-side database instrumentation for the API server by @aylei in https://github.com/skypilot-org/skypilot/pull/10525 * [Kubernetes] Let pod_config opt out of OCI RoCE host networking by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10546 * [Volume] Refresh a volume's size from the cloud, not only at creation by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10549 * [Server] Account server and job controller process memory when sizing the executor pools by @aylei in https://github.com/skypilot-org/skypilot/pull/10550 * [Test] Give the Kubernetes volume smoke tests run-unique resource names by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10565 * [Server] Never regenerate the JWT signing secret on a DB read failure by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10566 * [Docs] Document `--num-jobs` without a pool by @Michaelvll in https://github.com/skypilot-org/skypilot/pull/10552 * [Server] Release the GIL during the hourly sky_logs prune (use os.stat over DirEntry.stat) by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10568 * [Metrics][Dashboard] Slurm: federate GPU metrics via the login node, and up-node/power-saved infra handling by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10572 * [Volume] Show a resize that has not landed yet by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10553 * [API server] Run the requests GC daemon once an hour by @aylei in https://github.com/skypilot-org/skypilot/pull/10555 * [Volumes] Make a volume's readiness cheap to ask about by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10564 * [Server] Cache the multiprocess metrics merge across scrapes by @aylei in https://github.com/skypilot-org/skypilot/pull/10573 * [Server] Give the retention daemons a fixed cadence and a startup jitter by @zpoint in https://github.com/skypilot-org/skypilot/pull/10538 * [Volumes] Read only the names when scoping the listing refresh by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10578 * [Dashboard] Let a jobs-table row link to its own detail page by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10580 * [k8s] Recommend set_pod_resource_limits when an OOM'd pod had no memory limit by @zpoint in https://github.com/skypilot-org/skypilot/pull/10551 * [API] Stop rewriting the whole policy table on every casbin write by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10577 * [Docs] Document -b/-s on the k8s image build script by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10583 * [Server] Move the chunked-upload rename and part scan off the event loop by @aylei in https://github.com/skypilot-org/skypilot/pull/10556 * Switch Nebius catalog fetcher to billing v1 by @klenovda in https://github.com/skypilot-org/skypilot/pull/9829 * [Dashboard] Complete external-jobs rendering in the managed-jobs table by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10591 * [Kubernetes] Resolve a volume's default context through allowed_contexts by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10592 * [Test] Don't lose a smoke test's real failure to an unbound `proc` by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10593 * [Test] task_labels_kubernetes: scope the label check to the cluster's own pod by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10597 * [Core] Add register_slurm_property for plugin schema extension by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10600 * [Catalog] Skip Nebius presets with unresolved billing SKUs by @Michaelvll in https://github.com/skypilot-org/skypilot/pull/10400 * [Metrics] Slurm: scope and route /federate for a shared Prometheus (filter, via) by @Michaelvll in https://github.com/skypilot-org/skypilot/pull/10603 * [Kubernetes] Add kubernetes.rdma.mode for SR-IOV RDMA delivery by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10554 * [Core] Don't abort graceful teardown when command runners are unavailable by @claude[bot] in https://github.com/skypilot-org/skypilot/pull/10604 * [Slurm] Run credential check as the SSH user; keep infra partition toggle collapsible by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10610 * [Dashboard] Rename "Requested Resources" column to "Resources" on managed jobs page by @Michaelvll in https://github.com/skypilot-org/skypilot/pull/10609 * [Nebius] Support InfiniBand on Blackwell (B200/B300) platforms by @himanshu1573 in https://github.com/skypilot-org/skypilot/pull/10369 * [Nebius] Add optional boot disk encryption by @typesafeainate in https://github.com/skypilot-org/skypilot/pull/10172 * [k8s] Apply Ray patches from the pod spec, not from the installed skypilot by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10582 * [Server] Rebuild the state DB engine once the connection budget is known by @aylei in https://github.com/skypilot-org/skypilot/pull/10611 * [API] Run async continue-condition waits on a shared event loop by @aylei in https://github.com/skypilot-org/skypilot/pull/10584 * [Core] Record who requested a cluster stop/termination in its event log by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10621 * [Docs] Add resilient Ray training example by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10602 * [Jobs] Record who requested a job cancellation in the event log by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10619 * [Dashboard] Add jobs.detail.gpu-metrics plugin slot by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10620 * [Client] sky api login: follow an HTTP->HTTPS redirect by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10624 * [Kubernetes] Reuse ephemeral volume pod injection by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10622 * [Dashboard] Give a log stream's last line the same treatment as the rest, and bound the partial-line carry by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10615 * [Agent] Add Slurm migration reference to the SkyPilot skill by @Michaelvll in https://github.com/skypilot-org/skypilot/pull/10541 * [Jobs] Scope managed-job log reads to the caller's workspaces by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10614 * [Server] Mark request FAILED when enqueue fails instead of leaving it PENDING forever by @aylei in https://github.com/skypilot-org/skypilot/pull/10617 * [Dashboard] Slurm Infra: per-cluster progressive loading, node-derived GPU columns, bounded inventory SSH by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10629 * [Dashboard] Plugin-registered table filter properties by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10618 * [Slurm] Apply workspace config to sbatch options by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10631 * [Docs] Shrink the header search button on mobile by @stephenkevn in https://github.com/skypilot-org/skypilot/pull/10632 * Revert "[Server] Rebuild the state DB engine once the connection budget is known" by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10633 * [Examples] Add Hugging Face storage batch transcription example by @davanstrien in https://github.com/skypilot-org/skypilot/pull/10558 * [Slurm] Support stopping/starting container clusters using fs snapshots by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10544 * Revert "[Jobs] Scope managed-job log reads to the caller's workspaces (#10614)" by @zpoint in https://github.com/skypilot-org/skypilot/pull/10637 * Run config reload off the background event loop in cleanup_sky_logs by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10569 * [API] Add provisioning/pending/start-time metrics and availability dashboard panels by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10625 * [Dashboard] Build the app tree once, after plugin registration settles by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10643 * [Docs] Add GB300 NVL72 Ray recovery example by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10644 * [Dashboard] Let plugin routes declare the nav link they belong under by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10646 * [Dashboard] Show Started and Partition on the managed job detail page by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10647 * [Slurm] Support autostop for container clusters by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10645 * [Server] Add a second plugin install pass for innermost middleware by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10648 * Lift the typing_extensions cap by bumping selenium; drop pydantic<2.12 by @zpoint in https://github.com/skypilot-org/skypilot/pull/10649 * [k8s] Distinguish the context a cluster is submitted to from the one it runs on by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10651 * [Slurm] Fix container-cluster stop/start on NFS-backed clusters by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10652 * [AWS] Don't fail the launch when volumes can't be tagged by @zpoint in https://github.com/skypilot-org/skypilot/pull/10627 * [Core] Don't serialize an absent config path as the JSON string 'null' by @Anai-Guo in https://github.com/skypilot-org/skypilot/pull/10532 * [Slurm] Retry transiently missing jobs during status refresh by @liuxu623 in https://github.com/skypilot-org/skypilot/pull/10458 * [Server] Delete a log tail's request log once its stream ends by @aylei in https://github.com/skypilot-org/skypilot/pull/10657 * [Jobs] Filter managed jobs by infra, server-side by @zpoint in https://github.com/skypilot-org/skypilot/pull/10613 * [Jobs] Run sync state DB reads off the controller event loop by @aylei in https://github.com/skypilot-org/skypilot/pull/10654 * Stop a payload-shaped task log line from truncating the stream by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10670 * [SERVE] Load balancing least_load policy replica decrement bug and rotation issue by @JGSweets in https://github.com/skypilot-org/skypilot/pull/10445 * [Server] Keep the hook-logs request log that its client reads back by @zpoint in https://github.com/skypilot-org/skypilot/pull/10672 * Don't let one unusable apt repository block a runtime install by @zpoint in https://github.com/skypilot-org/skypilot/pull/10673 * [Server] Reclaim request logs that outlived their request row by @aylei in https://github.com/skypilot-org/skypilot/pull/10675 * [Jobs] Surface who requested a cancellation in the job's details by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10661 * [Dashboard] Sum all task durations in job group duration display by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10678 * [Dashboard] Show the owning user for pools by @claude[bot] in https://github.com/skypilot-org/skypilot/pull/10653 * [Slurm] Add quota.queue / quota.account config for sbatch --qos / --account by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10680 * [Slurm] Configure Unix submit usernames for users and service accounts by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10685 * [Slurm] Default service accounts to their creator’s Unix identity by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10686 * [Server] Publish the API server's own local disk footprint on /metrics by @aylei in https://github.com/skypilot-org/skypilot/pull/10674 * [API] Authorize download paths against authenticated user roots by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10683 * Distinguish an unsupportable request from unavailable resources by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10697 * [Slurm] Add a scope-resolved getter for slurm.quota sub-fields by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10700 * [Metrics] Track blob lookup sizes and cache hits by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10705 * [Server] Bound the auth users upsert with server-side timeouts aligned to the auth deadline by @cg505 in https://github.com/skypilot-org/skypilot/pull/10684 * [Server] Fix double close of the kubectl port-forward stdout fd under uvloop by @cg505 in https://github.com/skypilot-org/skypilot/pull/10689 * [Core] Bound idle-in-transaction time on every Postgres engine transaction by @cg505 in https://github.com/skypilot-org/skypilot/pull/10687 * [Jobs] Explain why a managed job has not started, and let a runner say more by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10692 * [Test] Guard the blob-upload route scan against FastAPI router wrappers by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10711 * [API] Count middleware rejections at the edge (no client-visible change) by @cg505 in https://github.com/skypilot-org/skypilot/pull/10691 * [Server] Count auth-path DB timeouts by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10710 * [Dashboard] Render plugin-contributed rows for contexts not enabled for compute by @aylei in https://github.com/skypilot-org/skypilot/pull/10579 * [CI] Notify Slack when the nightly run is cancelled, not only when it fails by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10719 * [CI] Let the nightly notifier depend on every other job by @zpoint in https://github.com/skypilot-org/skypilot/pull/10720 * [Test] Retry on host port conflicts when starting the remote-server container by @zpoint in https://github.com/skypilot-org/skypilot/pull/10724 * [Server] Add opt-in SO_REUSEPORT to balance connections across API server workers by @aylei in https://github.com/skypilot-org/skypilot/pull/10016 * [Pools] Make pools work under Kueue quota by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10718 * [Kubernetes] Let pods with DRA claims terminate gracefully by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10751 * Fix SLURM containerized run poisoned by exported `which` bash function by @daw3rd in https://github.com/skypilot-org/skypilot/pull/10716 * [k8s] Prefer the release's base suite when an apt install cannot fetch its packages by @MadhavJivrajani in https://github.com/skypilot-org/skypilot/pull/10755 * [Jobs] State changes/migration to support dynamic job group by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10698 * [Jobs] Attach a managed job to the job it is launched from (--job-group) by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10701 * [Jobs] Cancel the jobs launched from a job with it; the root sweeps them when it finishes by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10702 * [Jobs] Show jobs launched from a job group under it in sky jobs queue and the dashboard by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10725 * [Jobs] Number dynamic tasks like tasks; address one as <job> <task> by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10734 * [Dashboard] Dynamic tasks read as tasks of their group by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10736 * [Tests] Smoke tests for jobs launched from inside a job group by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10726 * [Docs] Launching jobs from inside a job group by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10732 * [Jobs] Let an unknown task fall through to the log reader instead of raising by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10758 * [Provision] Prefer the release's base suite when a container's apt install cannot fetch its packages by @zpoint in https://github.com/skypilot-org/skypilot/pull/10759 * [Server] Stop transient processes from growing the multiproc metrics dir by @aylei in https://github.com/skypilot-org/skypilot/pull/10575 * [Server] Refuse an upload that cannot fit on the server's local disk by @aylei in https://github.com/skypilot-org/skypilot/pull/10714 * [CI] Cancel the Buildkite build when its GitHub run is cancelled by @zpoint in https://github.com/skypilot-org/skypilot/pull/10760 * [Dashboard] Dynamic task UX nits: 'of this job' tooltip, drop (N dynamic) from group badges, Dynamic pill in the jobs list by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10761 * Add fleet GPU capacity and infra counts by @Michaelvll in https://github.com/skypilot-org/skypilot/pull/10749 * [Metrics] Measure the SSH round trip past the API server by @zpoint in https://github.com/skypilot-org/skypilot/pull/10656 * [Provision] Report the execution target in the 'Instances launched' event by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10783 * [Test] Move pinned miniconda smoke-test image off EOL Debian bullseye by @cg505 in https://github.com/skypilot-org/skypilot/pull/10784 * [Jobs] Retry transient DB errors in a job's final cleanup and bookkeeping by @aylei in https://github.com/skypilot-org/skypilot/pull/10655 * [Jobs] Restore a job's config file atomically by @aylei in https://github.com/skypilot-org/skypilot/pull/10782 * [Test] Let the pipeline generator's -k use pytest's own selection by @zpoint in https://github.com/skypilot-org/skypilot/pull/10787 * [Provision] Revert the EOL-bullseye apt workarounds now that the pinned image is on bookworm by @zpoint in https://github.com/skypilot-org/skypilot/pull/10786 * [Core] Time async and generator functions in time_me by @aylei in https://github.com/skypilot-org/skypilot/pull/10626 * [Server] Let a deployment state its state-DB connection pool size by @aylei in https://github.com/skypilot-org/skypilot/pull/10791 * [Core] Report where a managed job's start-up time goes by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10638 * Release 0.13.2rc1 by @Michaelvll in https://github.com/skypilot-org/skypilot/pull/10817 ## New Contributors * @spiffaz made their first contribution in https://github.com/skypilot-org/skypilot/pull/10156 * @edmsimplicio made their first contribution in https://github.com/skypilot-org/skypilot/pull/10230 * @yash170106065 made their first contribution in https://github.com/skypilot-org/skypilot/pull/9924 * @pavneeta made their first contribution in https://github.com/skypilot-org/skypilot/pull/10260 * @liuxu623 made their first contribution in https://github.com/skypilot-org/skypilot/pull/10284 * @complynx made their first contribution in https://github.com/skypilot-org/skypilot/pull/10345 * @Anai-Guo made their first contribution in https://github.com/skypilot-org/skypilot/pull/10356 * @Khas-Erdene-Tsogtsaikhan made their first contribution in https://github.com/skypilot-org/skypilot/pull/10286 * @ryxli made their first contribution in https://github.com/skypilot-org/skypilot/pull/10436 * @intagliated made their first contribution in https://github.com/skypilot-org/skypilot/pull/9932 * @klenovda made their first contribution in https://github.com/skypilot-org/skypilot/pull/9829 * @himanshu1573 made their first contribution in https://github.com/skypilot-org/skypilot/pull/10369 * @typesafeainate made their first contribution in https://github.com/skypilot-org/skypilot/pull/10172 * @davanstrien made their first contribution in https://github.com/skypilot-org/skypilot/pull/10558 * @daw3rd made their first contribution in https://github.com/skypilot-org/skypilot/pull/10716 **Full Changelog**: https://github.com/skypilot-org/skypilot/compare/v0.13.1rc1...v0.13.2rc1
Related Repositories
public-apis/public-apis★ 491,314
A collective list of free APIs
NousResearch/hermes-agent★ 256,162
The agent that grows with you
yt-dlp/yt-dlp★ 200,364
A feature-rich command-line audio/video downloader
microsoft/markitdown★ 191,619
Python tool for converting files and office documents to Markdown.
Significant-Gravitas/AutoGPT★ 187,709
AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.