v0.13.1rc1
skypilot-org/skypilotv0.13.1rc1Jul 24, 2026by Michaelvll
AI Summary
This release candidate focuses on stability improvements for Kubernetes and Azure cloud providers, introduces several dashboard enhancements for better job and cluster management, and fixes various bugs related to job scheduling, debugging, and resource provisioning.
Key Highlights
- Dashboard enhancements include new owner/activity toggles, a default view to 'My Clusters', and support for admin-configured URL templates.
- Kubernetes stability improvements include better EFA job co-location, fixed thread pool caps for provisioning, and enhanced handling of abnormal cluster events.
- Azure fixes resolved NIC provisioning issues with the latest azure-mgmt-network and added support for booting from private Shared Image Gallery images.
- Job management received improvements such as on-demand log streaming and emergency recovery logic for controllers.
- Debugging capabilities were enhanced with parallelized debug-dump collection and fast-fail logic for unreachable kube contexts.
New Features
- Support for `storage.existingClaim` to attach pre-provisioned PVCs in Kubernetes.
- Ability to specify a role when creating a service account token via the API.
- Configurable subPath for persisted clients directory in Helm.
- Added `--num-nodes` flag to `sky local up` for creating multi-node local clusters.
- Added a config post-save hook for the API server to carry old/new config and user context.
Full Release Notes
## SkyPilot v0.13.1rc1 Get it now with: ```bash uv pip install "skypilot>=0.13.1rc1" ``` --- ## What's Changed * [Kubernetes] Surface remediation hint on abnormal->INIT cluster events by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10060 * [Azure] Fix NIC provisioning broken by azure-mgmt-network 31.0.0 by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10062 * [Dashboard] Truncate long managed job names in the jobs table by @stephenkevn in https://github.com/skypilot-org/skypilot/pull/10011 * [Dashboard] Add recovery count to managed job detail page by @stephenkevn in https://github.com/skypilot-org/skypilot/pull/10004 * [k8s] Guard against None pod.status.conditions in termination-reason lookup by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10065 * [Debug Dump] Resolve k8s coordinates from cluster YAML so INIT clusters still get their k8s/Kueue state dumped by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10068 * [Test] Fix flaky test_helm_deploy_okta: tolerate current_url=None during dashboard redirect by @zpoint in https://github.com/skypilot-org/skypilot/pull/10073 * fix(runpod): correct and refresh region -> data center mapping by @ciprianf in https://github.com/skypilot-org/skypilot/pull/10072 * [Dashboard] Infra page: await sky check before reading enabled clouds on manual refresh by @cg505 in https://github.com/skypilot-org/skypilot/pull/9879 * [Client] Fix async get() not unwrapping the 'detail'-nested 500 payload by @cg505 in https://github.com/skypilot-org/skypilot/pull/10026 * Remove the API stream link from async request submission output by @claude[bot] in https://github.com/skypilot-org/skypilot/pull/10081 * [Jobs] Start consolidation-mode controllers only from the leader daemon by @cg505 in https://github.com/skypilot-org/skypilot/pull/10080 * [Test] chaos_proxy: jitter the kill interval to avoid phase-locked races by @cg505 in https://github.com/skypilot-org/skypilot/pull/10024 * [Jobs] Stream managed job logs on demand when a logging agent is configured by @aylei in https://github.com/skypilot-org/skypilot/pull/10069 * [k8s] Support storage.existingClaim to attach a pre-provisioned PVC by @aylei in https://github.com/skypilot-org/skypilot/pull/10085 * Add a pre-write validator hook for API server config saves by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10059 * [Dashboard] Render fenced code blocks in error messages by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10088 * [Core] Extract external links from job logs server-side by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/9934 * [API] Grant private-workspace access to users added to allowed_users pre-login by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10079 * [API] Collect debug-dump request logs via LogProvider by @cg505 in https://github.com/skypilot-org/skypilot/pull/9852 * [Jobs] Give the controller process a single class identity by @cg505 in https://github.com/skypilot-org/skypilot/pull/10083 * [Dashboard] Add owner + activity toggles to Clusters page by @hentt30 in https://github.com/skypilot-org/skypilot/pull/9884 * [Dashboard] Default Clusters to My Clusters, add scope hint, unify toggle styling by @stephenkevn in https://github.com/skypilot-org/skypilot/pull/10089 * [Dashboard] Support admin-configured URL template external links by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10090 * [Dashboard] Hint when a K8s context's nodes are filtered by allowed_nodes by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10092 * Update the config doc and default value for rbac.restrict_config_to_admins by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10042 * [Kubernetes] Co-locate multi-node EFA jobs in one AZ by @trevanth24 in https://github.com/skypilot-org/skypilot/pull/10054 * [AWS] Skip unreachable me-south-1 region in catalog fetcher by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10095 * [k8s] storage.existingClaim activates the persistent-storage layout by @aylei in https://github.com/skypilot-org/skypilot/pull/10086 * [Test] Replace retired p3.2xlarge with g4dn.xlarge in tests by @aylei in https://github.com/skypilot-org/skypilot/pull/10098 * [Core] debug-dump: fast-fail unreachable kube contexts, parallelize per-cluster collection by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10041 * [Jobs] Tolerate end-time fetch and log download failures in the controller by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10091 * [k8s] Avoid per-API-call urllib3 setLevel() to prevent logging fork deadlock on Python < 3.13 by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10104 * [Kubernetes] Derive EFA count on scale-from-zero AWS clusters by @trevanth24 in https://github.com/skypilot-org/skypilot/pull/10053 * [Kubernetes] Cap provisioning thread pools at the number of work items by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10111 * [Test] Replace retired p3/V100 AWS resources in examples by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10107 * helm: make the persisted clients directory subPath configurable by @aylei in https://github.com/skypilot-org/skypilot/pull/10101 * [Test] Move zero-quota-failover test off retired AWS p3/V100 by @zpoint in https://github.com/skypilot-org/skypilot/pull/10113 * [Helm] Add apiService.host to control API server bind interface by @nicosuter in https://github.com/skypilot-org/skypilot/pull/10050 * [Test] Mark test_debug_dump_unreachable_context_fast_fail no_remote_server by @zpoint in https://github.com/skypilot-org/skypilot/pull/10116 * [Users] Allow specifying a role when creating a service account token by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10115 * [Test] Mark test_debug_dump_unreachable_context_fast_fail no_dependency by @zpoint in https://github.com/skypilot-org/skypilot/pull/10118 * [Docs] Update Sandboxes page with new features and current SDK syntax by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10109 * [Jobs] Release the jobs-scheduler launch slot while the launch request is parked by @cg505 in https://github.com/skypilot-org/skypilot/pull/10013 * [Jobs] Emergency recovery: retry unexpected controller errors (reuse RECOVERING + recovery_source) by @cg505 in https://github.com/skypilot-org/skypilot/pull/9868 * [Azure] Support booting from private Shared Image Gallery images by @ahluwalij in https://github.com/skypilot-org/skypilot/pull/9843 * [k8s] Use `disk_size` instead of `ephemeral_storage` field by @kyuds in https://github.com/skypilot-org/skypilot/pull/9273 * [Core] Fix ray job submit failure in Ray cluster template by deconflicting dashboard agent port by @romilbhardwaj in https://github.com/skypilot-org/skypilot/pull/9856 * [k8s] Resolve urllib3 logger once at decoration time, not per API call by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10108 * Cap provisioning thread pools at the number of work items(2) by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10112 * [Metrics] Auto-detect local kubeconfig context to fix /gpu-metrics self-federation by @hentt30 in https://github.com/skypilot-org/skypilot/pull/10058 * [Test] Skip test_batch_diffusion on Azure (no L4 GPU in catalog) by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10064 * [k8s] sky local up: add --num-nodes for multi-node local clusters by @zpoint in https://github.com/skypilot-org/skypilot/pull/10138 * [API] Add config post-save hook carrying old/new config and user by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10133 * [Buildkite] Add concurrency_group marker for global per-test serialization by @zpoint in https://github.com/skypilot-org/skypilot/pull/10150 * [Core] Add SKYPILOT_DISABLE_LOCAL_API_SERVER to block local API server autostart by @Michaelvll in https://github.com/skypilot-org/skypilot/pull/10093 * [k8s] Request gzip-compressed API server responses by @aylei in https://github.com/skypilot-org/skypilot/pull/10153 * [Jobs] Clear SKYPILOT_DISABLE_LOCAL_API_SERVER when starting the controller's local API server by @zpoint in https://github.com/skypilot-org/skypilot/pull/10154 * [Kubernetes] Check ephemeral storage in check_instance_fits by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10131 * Stash the yaml into the db during launch by @SeungjinYang in https://github.com/skypilot-org/skypilot/pull/10159 * [Metrics] Take local-context detection off the scrape path by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10175 * [Docs] Update SkyPilot logo and homepage architecture image by @stephenkevn in https://github.com/skypilot-org/skypilot/pull/10179 * [Kubernetes] - Support AWS Neuron (Trainium/Inferentia) accelerators by @trevanth24 in https://github.com/skypilot-org/skypilot/pull/10169 * [Test] Fix flaky test_pool_scale_down_with_job_count_priority by @zpoint in https://github.com/skypilot-org/skypilot/pull/10182 * [GCP] Add RTX PRO 6000 Blackwell (G4) support by @pauldonzier in https://github.com/skypilot-org/skypilot/pull/9634 * [API] Close SSH proxy websocket gracefully on cluster validation failure by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10183 * [Docs] Restyle Ask AI docs widget and add dark mode support by @stephenkevn in https://github.com/skypilot-org/skypilot/pull/10180 * [Docs] Update links, styling, and page organization by @romilbhardwaj in https://github.com/skypilot-org/skypilot/pull/10187 * [Docs] Remove demo card from docs homepage and README by @claude[bot] in https://github.com/skypilot-org/skypilot/pull/10189 * [AWS] Skip default-SG pre-creation for user-specified security groups by @rohansonecha in https://github.com/skypilot-org/skypilot/pull/10178 * [Dashboard] Use quicker count for the clusters empty-state CTA by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10149 * [Dashboard] Update logo and favicon to new branding by @hopechong in https://github.com/skypilot-org/skypilot/pull/10181 * [Docs] Add Hugging Face Hub storage guide by @alex000kim in https://github.com/skypilot-org/skypilot/pull/10117 * [Dashboard] Render dynamically-registered columns on the Users table by @MadhavJivrajani in https://github.com/skypilot-org/skypilot/pull/10160 * [Test] Verify volume is actually mounted in test_volume_env_mount_kubernetes by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10190 * [CI] Tag the commit of each published nightly by @zpoint in https://github.com/skypilot-org/skypilot/pull/10192 * [Dashboard] Defer cluster fetch to background preload on the jobs page by @MadhavJivrajani in https://github.com/skypilot-org/skypilot/pull/10193 * [Nebius] Implement personalized pricing by @Jobarion in https://github.com/skypilot-org/skypilot/pull/9107 * [Dashboard] Add scope field to dashboard.external_links entries by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10194 * [Jobs] Reduce jobs queue payload for field-filtered requests by @MadhavJivrajani in https://github.com/skypilot-org/skypilot/pull/10176 * [Core] Price region-free cost estimates with the joint regional minimum by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10199 * [Docs] Update Together AI in docs by @romilbhardwaj in https://github.com/skypilot-org/skypilot/pull/10198 * [Core] Surface concise health-probe failures in unhealthy-cluster events by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10201 * [Dashboard] Show PENDING job reason in status tooltip by @lloyd-brown in https://github.com/skypilot-org/skypilot/pull/10202 * [LLM] slime on Job Groups: agentic coding-agent RL with a scalable inference fleet by @ishankaul1 in https://github.com/skypilot-org/skypilot/pull/10200 * [Volumes] Use SHORT queue for volume apply/delete to avoid LONG-pool starvation by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10195 * [Jobs] Add Prometheus gauges for managed-job recovery observability by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10186 * [API] Offload service-account auth DB lookups off the event loop by @kevinmingtarja in https://github.com/skypilot-org/skypilot/pull/10185 * [Test] Poll for async volume/PVC deletion in test_volumes_on_kubernetes by @zpoint in https://github.com/skypilot-org/skypilot/pull/10206 * [Kubernetes] OCI network_tier=best: GB200/GB300 NCCL profiles by @DanielZhangQD in https://github.com/skypilot-org/skypilot/pull/10204 * Release 0.13.1rc1 by @Michaelvll in https://github.com/skypilot-org/skypilot/pull/10219 ## New Contributors * @nicosuter made their first contribution in https://github.com/skypilot-org/skypilot/pull/10050 * @ahluwalij made their first contribution in https://github.com/skypilot-org/skypilot/pull/9843 * @MadhavJivrajani made their first contribution in https://github.com/skypilot-org/skypilot/pull/10160 **Full Changelog**: https://github.com/skypilot-org/skypilot/compare/v0.13.0rc1...v0.13.1rc1