v0.10.3

# SkyPilot v0.10.3: 2-10x Performance Improvement, Better Observability, Production-Ready Authentication and More

SkyPilot v0.10.3 delivers 2-10x performance improvement, enhanced cloud integration, improved Kubernetes integration, and strengthened production reliability features for AI/ML workloads across clouds.

## Highlights

### Massive Performance Improvements: 2-10x more efficient

- **4x faster `sky status`** ([#6883](https://github.com/skypilot-org/skypilot/pull/6883), [#6858](https://github.com/skypilot-org/skypilot/pull/6858), [#6868](https://github.com/skypilot-org/skypilot/pull/6868), [#6871](https://github.com/skypilot-org/skypilot/pull/6871), [#6882](https://github.com/skypilot-org/skypilot/pull/6882), [#6940](https://github.com/skypilot-org/skypilot/pull/6940), [#6948](https://github.com/skypilot-org/skypilot/pull/6948), [#6892](https://github.com/skypilot-org/skypilot/pull/6892), [#6908](https://github.com/skypilot-org/skypilot/pull/6908))
- **2-10x speedup for dashboard pages**, including users, infra, volumes ([#7031](https://github.com/skypilot-org/skypilot/pull/7031), [#7033](https://github.com/skypilot-org/skypilot/pull/7033), [#6959](https://github.com/skypilot-org/skypilot/pull/6959), [#7013](https://github.com/skypilot-org/skypilot/pull/7013), [#7011](https://github.com/skypilot-org/skypilot/pull/7011), [#7012](https://github.com/skypilot-org/skypilot/pull/7012))
- **2x speedup for remote PostgreSQL** environments through connection pooling ([#6998](https://github.com/skypilot-org/skypilot/pull/6998), [#6897](https://github.com/skypilot-org/skypilot/pull/6897))
- **Fixes request responsiveness and SSH lagging/disconnection issue in production** ([#6984](https://github.com/skypilot-org/skypilot/pull/6984), [#6983](https://github.com/skypilot-org/skypilot/pull/6983), , #6889, #6963, #6947, [#7015](https://github.com/skypilot-org/skypilot/pull/7015), [#6966](https://github.com/skypilot-org/skypilot/pull/6966), [#6938](https://github.com/skypilot-org/skypilot/pull/6938), [#6957](https://github.com/skypilot-org/skypilot/pull/6957))
- **Improved API server memory management** ([#7046](https://github.com/skypilot-org/skypilot/pull/7046), [#7037](https://github.com/skypilot-org/skypilot/pull/7037), [#6909](https://github.com/skypilot-org/skypilot/pull/6909))

Upgrade your SkyPilot API server to 0.10.3 and get the improvement out of the box!

### Observability & Monitoring

More SkyPilot API server and cluster metrics can now be set up for better observability, including:

- Detailed cluster events ([#6907](https://github.com/skypilot-org/skypilot/pull/6907), [#6906](https://github.com/skypilot-org/skypilot/pull/6906), [#6901](https://github.com/skypilot-org/skypilot/pull/6901), [#6899](https://github.com/skypilot-org/skypilot/pull/6899), [#6900](https://github.com/skypilot-org/skypilot/pull/6900), [#6936](https://github.com/skypilot-org/skypilot/pull/6936), #6875)
- API server detailed metrics for request latency, event loop, and process resource usage ([#7017](https://github.com/skypilot-org/skypilot/pull/7017), [#6935](https://github.com/skypilot-org/skypilot/pull/6935), [#6942](https://github.com/skypilot-org/skypilot/pull/6942), [#6968](https://github.com/skypilot-org/skypilot/pull/6968), [#7062](https://github.com/skypilot-org/skypilot/pull/7062))
  <img width="3244" height="1932" alt="image" src="https://github.com/user-attachments/assets/c6ac9302-6938-4df1-9a48-d9a4e1a4bb3b" />
- Request ID tracking for better debugging ([#6933](https://github.com/skypilot-org/skypilot/pull/6933))

### **CoreWeave Integration: Autoscaler**

CoreWeave autoscaler is now supported, with a single config field change ([#6895](https://github.com/skypilot-org/skypilot/pull/6895)):

```yaml
kubernetes:
  autoscaler: coreweave
```

### SkyPilot API Server: Authentication & Production Environment

**SkyPilot now supports Microsoft Entra ID SSO login** ([#7045](https://github.com/skypilot-org/skypilot/pull/7045), #7028), besides Okta and Google Workspace. See more docs for setting up SSO login for API server: [https://docs.skypilot.co/en/latest/reference/auth.html#sso-recommended](https://docs.skypilot.co/en/latest/reference/auth.html#sso-recommended)
<img width="60%" alt="image 1" src="https://github.com/user-attachments/assets/aeebe56c-b24a-4a70-ac37-a18597b81125" />    


## What's New

### UX Improvement

- Fix CLI auto-completion authentication issue (#6724)
- Fixed cluster ownership display ([#6989](https://github.com/skypilot-org/skypilot/pull/6989))
- Fixed API logs for daemon request ([#6841](https://github.com/skypilot-org/skypilot/pull/6841))
- SDK improvements for `tail_logs` ([#6902](https://github.com/skypilot-org/skypilot/pull/6902))
- Responsive spinner for request blocking (#6905)
- Fixed SDK authentication for download endpoints ([#6955](https://github.com/skypilot-org/skypilot/pull/6955))
- Enable log downloading in dashboard ([#6999](https://github.com/skypilot-org/skypilot/pull/6999), [#7000](https://github.com/skypilot-org/skypilot/pull/7000), [#7003](https://github.com/skypilot-org/skypilot/pull/7003), [#7010](https://github.com/skypilot-org/skypilot/pull/7010))
- Prevented clients from setting DB strings server-side ([#7042](https://github.com/skypilot-org/skypilot/pull/7042))
- Improved logout error handling ([#6874](https://github.com/skypilot-org/skypilot/pull/6874))

### Storage & File Operations

- Fixed storage mounting on ARM64 instances ([#7008](https://github.com/skypilot-org/skypilot/pull/7008))
- Improved chunk upload reliability ([#6854](https://github.com/skypilot-org/skypilot/pull/6854))
- Rsync permission fixes for Kubernetes ([#6951](https://github.com/skypilot-org/skypilot/pull/6951))

### Enhanced Cloud Platform Support

**Kubernetes Improvements**

- **Pod configuration change detection** with warnings ([#6912](https://github.com/skypilot-org/skypilot/pull/6912))
- Volume name validation ([#6863](https://github.com/skypilot-org/skypilot/pull/6863))
- Worker service cleanup on termination ([#7014](https://github.com/skypilot-org/skypilot/pull/7014))
- AMD GPU support improvements

**Nebius Cloud**

- Static IP configuration support ([#7002](https://github.com/skypilot-org/skypilot/pull/7002))
- Docker image ID fixes ([#6894](https://github.com/skypilot-org/skypilot/pull/6894))
- Pagination support for `list_instances` ([#6867](https://github.com/skypilot-org/skypilot/pull/6867))
- Customizable API domain ([#6888](https://github.com/skypilot-org/skypilot/pull/6888))
- Better network error messages ([#6971](https://github.com/skypilot-org/skypilot/pull/6971))
- Various stability improvements ([#6856](https://github.com/skypilot-org/skypilot/pull/6856), [#6896](https://github.com/skypilot-org/skypilot/pull/6896))

**AWS**

- **Elastic Fabric Adapter (EFA) automatic enablement** for improved network performance ([#6852](https://github.com/skypilot-org/skypilot/pull/6852))
- **AWS Systems Manager (SSM) support** with new `use_ssm` flag for secure connections ([#6830](https://github.com/skypilot-org/skypilot/pull/6830))
- Improved VPC error messages and handling ([#6746](https://github.com/skypilot-org/skypilot/pull/6746))
- Fixed CloudWatch region setting issues ([#6747](https://github.com/skypilot-org/skypilot/pull/6747))
- Better teardown leak prevention ([#7022](https://github.com/skypilot-org/skypilot/pull/7022))

**RunPod**

- **Volume support** for RunPod network volumes ([#6949](https://github.com/skypilot-org/skypilot/pull/6949))

**GPU Support**

- Added B200 to common accelerators in `sky show-gpus` ([#7006](https://github.com/skypilot-org/skypilot/pull/7006))

### Documentation & Examples

- **New training examples**: TorchTitan ([#6677](https://github.com/skypilot-org/skypilot/pull/6677)), distributed RL with game servers ([#6988](https://github.com/skypilot-org/skypilot/pull/6988))
- Best practices for network and storage benchmarking ([#6632](https://github.com/skypilot-org/skypilot/pull/6632))
- Clarified Ray runtime usage ([#6783](https://github.com/skypilot-org/skypilot/pull/6783))
- SSM documentation improvements ([#7050](https://github.com/skypilot-org/skypilot/pull/7050))
- Fixed broken documentation links ([#7026](https://github.com/skypilot-org/skypilot/pull/7026), [#7004](https://github.com/skypilot-org/skypilot/pull/7004))

### Testing & CI/CD & Development

- Backward compatibility tests against stable releases ([#6979](https://github.com/skypilot-org/skypilot/pull/6979))
- SSH lag unit tests ([#6968](https://github.com/skypilot-org/skypilot/pull/6968))
- Improved smoke test reliability ([#6913](https://github.com/skypilot-org/skypilot/pull/6913), [#6958](https://github.com/skypilot-org/skypilot/pull/6958), [#6967](https://github.com/skypilot-org/skypilot/pull/6967), [#7019](https://github.com/skypilot-org/skypilot/pull/7019))
- Customizable Buildkite queue support ([#7063](https://github.com/skypilot-org/skypilot/pull/7063))
- Fixed type errors in managed jobs ([#6994](https://github.com/skypilot-org/skypilot/pull/6994))

### Developer Experience

- **Python 3.12 and 3.13 support** ([#5304](https://github.com/skypilot-org/skypilot/pull/5304), [#6990](https://github.com/skypilot-org/skypilot/pull/6990))
- Performance measurement annotations ([#6943](https://github.com/skypilot-org/skypilot/pull/6943))
- Better error handling for UV-only environments ([#6893](https://github.com/skypilot-org/skypilot/pull/6893))

### API Server Improvements

- Request retention daemon fixes ([#7035](https://github.com/skypilot-org/skypilot/pull/7035))
- Better error messages for unavailable server logs ([#6855](https://github.com/skypilot-org/skypilot/pull/6855))
- Request ID header corrections ([#7025](https://github.com/skypilot-org/skypilot/pull/7025))
- Backward compatibility improvements ([#6981](https://github.com/skypilot-org/skypilot/pull/6981), [#7023](https://github.com/skypilot-org/skypilot/pull/7023), [#7058](https://github.com/skypilot-org/skypilot/pull/7058), #6987)
- Proper password hashing with bcrypt for API server ([#6806](https://github.com/skypilot-org/skypilot/pull/6806))
- More robust OAuth2 login and RBAC ([#6910](https://github.com/skypilot-org/skypilot/pull/6910), [#6974](https://github.com/skypilot-org/skypilot/pull/6974))

## Installation

To install or upgrade:

```bash
# Using pip
pip install -U "skypilot>=0.10.3"

# Using uv (recommended)
uv pip install "skypilot>=0.10.3"
```

### Upgrading the API Server

To upgrade your SkyPilot API server to version 0.10.3, use the following Helm command:

```bash
NAMESPACE=skypilot
RELEASE_NAME=skypilot
VERSION=0.10.3

helm repo update skypilot
helm upgrade -n $NAMESPACE $RELEASE_NAME skypilot/skypilot \
  --set apiService.image=berkeleyskypilot/skypilot:$VERSION \
  --version $VERSION --devel --reuse-values
```

**Note**: The `--set apiService.image=berkeleyskypilot/skypilot:$VERSION` is needed in case your previous API server was using the nightly build.

## Contributors

Thank you to all contributors who made this release possible! 🎉

### New Contributors

- @m6xwzzz made their first contribution in #6878
- @seongsukwon-moreh made their first contribution in #6994

### All Contributors

@SeungjinYang, @cg505, @SalikovAlex, @DanielZhangQD, @webconn, @m6xwzzz, @kevinmingtarja, @zpoint, @clayrosenthal, @lloyd-brown, @kyuds, @aylei, @romilbhardwaj, @Michaelvll, @andylizf, @Maknee, @alex000kim, @rohansonecha, @seongsukwon-moreh, @concretevitamin, and the entire SkyPilot community.

Special thanks to the community for bug reports, feature requests, and pull requests that helped make this release more robust and feature-rich. This release includes 100+ merged PRs from 30+ contributors! Also, thanks to @alex000kim for helping polishing this release note.

## Full Changelog

For the complete list of changes, see the [full changelog](https://github.com/skypilot-org/skypilot/compare/v0.10.2...v0.10.3).