v0.10.1

# SkyPilot v0.10.1: Improved production readiness, distributed training, cloud integrations, and more

SkyPilot v0.10.1 improves enterprise production readiness with large-scale distributed training capabilities (Llama 4 400B, OpenAI GPT-OSS), introduces/enhances integration with AMD GPUs and leading GPU clouds (CoreWeave, Nebius), and delivers enhanced reliability features for mission-critical AI workloads.

Get it now with:
```
uv pip install "skypilot>=0.10.1"
```

## Highlights

### Native Git Support

[Use your (private) git repositories](https://docs.skypilot.co/en/latest/examples/syncing-code-artifacts.html) as your SkyPilot workdir:

```yaml
# task.sky.yaml
workdir:
  url: https://github.com/my-org/my-repo.git
  ref: 1234ab # commit hash or branch name
```

Find your commit hash of your workdir in Dashboard:

<p align="center">
  <img src="https://i.imgur.com/h9U838b.png" width="80%" />
</p>

Added in (#6294, #6257, #6268).


### Distributed LLM Pretraining/Finetuning and Agentic training Examples

We released high-performance distributed training examples for large models with checkpointing support (#6525, #6551, #6242, #6273).

* Llama 4 Maverick 400B model training on more than 16 H200 GPUs: [example](https://docs.skypilot.co/en/latest/examples/training/llama-4-finetuning.html)
* OpenAI GPT-OSS model pretraining and finetuning: [example](https://docs.skypilot.co/en/latest/examples/training/gpt-oss-finetuning.html)

<p align="center">
  <img src="https://i.imgur.com/dAov9ud.png" width="80%" />
</p>

[Agentic training example](https://docs.skypilot.co/en/latest/examples/training/verl.html) with VeRL is now also available (#6443).


### Cloud and accelerator support

#### Native AMD GPU Support on Kubernetes

AMD ROCm is now supported on Kubernetes clusters with SkyPilot! 

```yaml
# task.sky.yaml
resources:
  infra: k8s/my-amd-cluster
  image_id: docker:rocm/pytorch-training:v25.6
  accelerators: MI300:4
```
#### Coreweave Integration

SkyPilot now supports CoreWeave clusters with native Infiniband and object store support (#6386, #6487, #6483).

Use your CoreWeave cluster with SkyPilot:

```yaml
resources:
  infra: k8s/my-coreweave-cluster
  network: best # Enable infiniband
```

#### Nebius cloud improvement

B200 GPUs and spot instances are now supported on Nebius cloud (#6474, #6478, #6267).

```yaml
# task.sky.yaml
resources:
  infra: nebius
  accelerators: B200:8
  use_spot: true
```

Additionally, SkyPilot now supports [``MOUNT_CACHED`` mode](https://docs.skypilot.co/en/latest/reference/storage.html#storage-modes) for Nebius cloud. (#6456)


### Autostop/Autodown based on SSH sessions

In addition to running jobs on clusters, you can also ask autostop/autodown to wait for active SSH sessions or none of them in SkyPilot YAML (#6361, #6485).

```yaml
# task.sky.yaml
resources:
  autostop:
    wait_for: jobs_and_ssh
```


### External Logging

You can now dump your SkyPilot cluster/job logs to [external logging services](https://docs.skypilot.co/en/latest/user_guides/logging.html) like AWS CloudWatch and GCP Cloud Logging (#6331, #6405, #6411, #6369). Configure it in your `~/.sky/config.yaml`:

```yaml
logs:
  store: aws  # Or 'gcp', etc.
  aws:
    ...  # Service-specific options; see below.
```


## What's New


### UX/API

- **2x Faster `sky status`**: Intelligent caching reduces status command latency by 50% (#6166)
- **Optimized Imports**: Lazy loading of dependencies for faster CLI startup (#6302)
- **Autostop/Autodown**: Improve autostop logic during cluster refresh (#6388)
- **Fast network timeout detection**: Faster timeout detection for remote API server checks (#6263)
- **Better hints for parallel launch**: Better hints for parallel launch (#6549)
- Support `sky api logout` to logout from API server (#6284, #6327, #6412)
- **Type Checking**: Enhanced mypy support with py.typed marker (#6440)

### UI

- **Interactive Tour**: Guided tour for new users to learn dashboard features (#6565, #6301)
- **Mobile Support**: Responsive design for narrow screens (#6280)
- **YAML Syntax Highlighting**: Enhanced code readability with syntax highlighting (#6404)
- **User Management**: Delete users directly from the dashboard interface (#6434)
- **Display fixes**: Cluster history tracking issues (#6512), VSCode remote connection (#6555),  workspace resources aggregation (#6537, #6283)

### Storage/Volume

- **S3 Mounting on ARM-based Instances**: Full support for S3 mounting on ARM64 instances (#6255)
- **Volume Management SDK**: Added SDK for creating, managing persistent volumes across clouds (#6439, #6170, #6383)
- **New examples for volumes**: juiceFS, NFS, etc. (#6521)

### API Server

#### API Server Deployment
- **Helm Chart on Artifact Hub**: SkyPilot Helm chart now available on Artifact Hub (#6560, #6371)
- **More robust DB**: Use Alembic for database schema migrations (#6579, #6556, #6196), thread safety improvements (#6422, #6279)
- **OAuth2 Proxy Integration**: Built-in OAuth2 proxy support (#6476)
- **Redis Secret Support**: Configuration for Redis URL via secrets (#6559)
- Deployment image size reduction (#6312, #6285)
- **GPU Detection**: Fixed GPU name matching with underscores (#6491)


#### Client Enhancements
- **Enhanced Client Reliability**: Automatic retry on transient failures (#6259, #6298, #6234)
- **Robustness enhancements**: lock timeout notifications (#6530), transient error retry (#6234), catalog refresh (#6272)
- **Service Account Token Auth Fix**: Fixed token authentication in file uploads (#6432)
- **Issues fixes**: file path escaping (#6262), 


### Managed Jobs

- **Force Disable Cloud Buckets**: Configuration option to disable cloud bucket usage for managed jobs and SkyServe (#6402, #6407)
- **Multi-Task Log Viewing**: See logs from all tasks in a managed job pipeline (#6415)


### Cloud Integrations

- **GCP N4 Instance Types**: Support for new N4 machine family (#6253)
- **GCP TPU Improvements**: Enhanced TPU provisioning logic (#6420)
- **Hyperbolic**: Marked CUSTOM_MULTI_NETWORK as unsupported (#6245)
- **RunPod**: CPU instance launching support (#6450)
- **Vast**: Port opening support (#6282)
- **Azure**: Dependency resolution fixes (#6231)

### Miscs

- **Developer Experience**: Improved format.sh for code consistency (#6580)
- **Backward Compatibility**: Fixed issues with Task field and version detection (#6548, #6564, #6485)
- **Test Stability**: Fixed various flaky tests (#6507, #6495, #6511, #6469)
- **Release Pipeline**: Fixed API version extraction and workflow issues (#6464, #6254)
- **Buildkite**: Sequential test execution to reduce resource usage (#6217)


### Production Deployment Guides

- **PostgreSQL with GCP Cloud SQL**: Complete guide for deploying API server with Cloud SQL backend (#6325)
- **Docker Deployment**: Containerized API server deployment guide (#6296)
- **AWS SSM as SSH proxy**: Setup guide for AWS Systems Manager (#6522)


## Contributors

We thank all contributors who made this release possible!

New Contributors**: @LokmaSpeedy, @jacobergzhou, @amd-pratmish, @tedspare, @jimbz, @makhalin

**All Contributors**: @alex000kim, @amd-pratmish, @andylizf, @aylei, @bikramnehra, @cblmemo, @cg505, @clayrosenthal, @concretevitamin, @DanielZhangQD, @jacobergzhou, @jimbz, @kevinmingtarja, @kyuds, @lloyd-brown, @LokmaSpeedy, @lucamanolache, @makhalin, @Maknee, @Michaelvll, @rohansonecha, @romilbhardwaj, @SalikovAlex, @SeungjinYang, @tedspare, @zhenjiasun, @zpoint


Special thanks to the community for bug reports, feature requests, and pull requests that helped improve SkyPilot!

## Full Changelog

For a complete list of changes, see the [commit history](https://github.com/skypilot-org/skypilot/compare/v0.10.0...v0.10.1).