Multi-cloud orchestration for ML workloads with automatic cost optimization. Use when you need to run training or batch jobs across multiple clouds, leverage spot instances with auto-recovery, or optimize GPU costs across providers.
npxskills add orchestra-research/ai-research-skills--skill skypilot-multi-cloud-orchestrationLoading…
Comprehensive guide to running ML workloads across clouds with automatic cost optimization using SkyPilot.
Use SkyPilot when:
Key features:
Use alternatives instead:
pip install "skypilot[aws,gcp,azure,kubernetes]"
# Verify cloud credentials
sky check
Create hello.yaml:
resources:
accelerators: T4:1
run: |
nvidia-smi
echo "Hello from SkyPilot!"
Launch:
sky launch -c hello hello.yaml
# SSH to cluster
ssh hello
# Terminate
sky down hello
# Task name (optional)
name: my-task
# Resource requirements
resources:
cloud: aws # Optional: auto-select if omitted
region: us-west-2 # Optional: auto-select if omitted
accelerators: A100:4 # GPU type and count
cpus: 8+ # Minimum CPUs
memory: 32+ # Minimum memory (GB)
use_spot: true # Use spot instances
disk_size: 256 # Disk size (GB)
# Number of nodes for distributed training
num_nodes: 2
# Working directory (synced to ~/sky_workdir)
workdir: .
# Setup commands (run once)
setup: |
pip install -r requirements.txt
# Run commands
run: |
python train.py
| Command | Purpose |
|---|---|
sky launch | Launch cluster and run task |
sky exec | Run task on existing cluster |
sky status | Show cluster status |
sky stop | Stop cluster (preserve state) |
sky down | Terminate cluster |
sky logs | View task logs |
sky queue | Show job queue |
sky jobs launch | Launch managed job |
sky serve up | Deploy serving endpoint |
# NVIDIA GPUs
accelerators: T4:1
accelerators: L4:1
accelerators: A10G:1
accelerators: L40S:1
accelerators: A100:4
accelerators: A100-80GB:8
accelerators: H100:8
# Cloud-specific
accelerators: V100:4 # AWS/GCP
accelerators: TPU-v4-8 # GCP TPUs
resources:
accelerators:
H100: 8
A100-80GB: 8
A100: 8
any_of:
- cloud: gcp
- cloud: aws
- cloud: azure
resources:
accelerators: A100:8
use_spot: true
spot_recovery: FAILOVER # Auto-recover on preemption
# Launch new cluster
sky launch -c mycluster task.yaml
# Run on existing cluster (skip setup)
sky exec mycluster another_task.yaml
# Interactive SSH
ssh mycluster
# Stream logs
sky logs mycluster
resources:
accelerators: A100:4
autostop:
idle_minutes: 30
down: true # Terminate instead of stop
# Set autostop via CLI
sky autostop mycluster -i 30 --down
# All clusters
sky status
# Detailed view
sky status -a
resources:
accelerators: A100:8
num_nodes: 4 # 4 nodes × 8 GPUs = 32 GPUs total
setup: |
pip install torch torchvision
run: |
torchrun \
--nnodes=$SKYPILOT_NUM_NODES \
--nproc_per_node=$SKYPILOT_NUM_GPUS_PER_NODE \
--node_rank=$SKYPILOT_NODE_RANK \
--master_addr=$(echo "$SKYPILOT_NODE_IPS" | head -n1) \
--master_port=12355 \
train.py
| Variable | Description |
|---|---|
SKYPILOT_NODE_RANK | Node index (0 to num_nodes-1) |
SKYPILOT_NODE_IPS | Newline-separated IP addresses |
SKYPILOT_NUM_NODES | Total number of nodes |
SKYPILOT_NUM_GPUS_PER_NODE | GPUs per node |
run: |
if [ "${SKYPILOT_NODE_RANK}" == "0" ]; then
python orchestrate.py
fi
# Launch managed job with spot recovery
sky jobs launch -n my-job train.yaml
name: training-job
file_mounts:
/checkpoints:
name: my-checkpoints
store: s3
mode: MOUNT
resources:
accelerators: A100:8
use_spot: true
run: |
python train.py \
--checkpoint-dir /checkpoints \
--resume-from-latest
# List jobs
sky jobs queue
# View logs
sky jobs logs my-job
# Cancel job
sky jobs cancel my-job
workdir: ./my-project # Synced to ~/sky_workdir
file_mounts:
/data/config.yaml: ./config.yaml
~/.vimrc: ~/.vimrc
file_mounts:
# Mount S3 bucket
/datasets:
source: s3://my-bucket/datasets
mode: MOUNT # Stream from S3
# Copy GCS bucket
/models:
source: gs://my-bucket/models
mode: COPY # Pre-fetch to disk
# Cached mount (fast writes)
/outputs:
name: my-outputs
store: s3
mode: MOUNT_CACHED
| Mode | Description | Best For |
|---|---|---|
MOUNT | Stream from cloud | Large datasets, read-heavy |
COPY | Pre-fetch to disk | Small files, random access |
MOUNT_CACHED | Cache with async upload | Checkpoints, outputs |
# service.yaml
service:
readiness_probe: /health
replica_policy:
min_replicas: 1
max_replicas: 10
target_qps_per_replica: 2.0
resources:
accelerators: A100:1
run: |
python -m vllm.entrypoints.openai.api_server \
--model meta-llama/Llama-2-7b-chat-hf \
--port 8000
# Deploy
sky serve up -n my-service service.yaml
# Check status
sky serve status
# Get endpoint
sky serve status my-service
service:
replica_policy:
min_replicas: 1
max_replicas: 10
target_qps_per_replica: 2.0
upscale_delay_seconds: 60
downscale_delay_seconds: 300
load_balancing_policy: round_robin
# SkyPilot finds cheapest option
resources:
accelerators: A100:8
# No cloud specified - auto-select cheapest
# Show optimizer decision
sky launch task.yaml --dryrun
resources:
accelerators: A100:8
any_of:
- cloud: gcp
region: us-central1
- cloud: aws
region: us-east-1
- cloud: azure
envs:
HF_TOKEN: $HF_TOKEN # Inherited from local env
WANDB_API_KEY: $WANDB_API_KEY
# Or use secrets
secrets:
- HF_TOKEN
- WANDB_API_KEY
name: llm-finetune
file_mounts:
/checkpoints:
name: finetune-checkpoints
store: s3
mode: MOUNT_CACHED
resources:
accelerators: A100:8
use_spot: true
setup: |
pip install transformers accelerate
run: |
python train.py \
--checkpoint-dir /checkpoints \
--resume
name: hp-sweep-${RUN_ID}
envs:
RUN_ID: 0
LEARNING_RATE: 1e-4
BATCH_SIZE: 32
resources:
accelerators: A100:1
use_spot: true
run: |
python train.py \
--lr $LEARNING_RATE \
--batch-size $BATCH_SIZE \
--run-id $RUN_ID
# Launch multiple jobs
for i in {1..10}; do
sky jobs launch sweep.yaml \
--env RUN_ID=$i \
--env LEARNING_RATE=$(python -c "import random; print(10**random.uniform(-5,-3))")
done
# SSH to cluster
ssh mycluster
# View logs
sky logs mycluster
# Check job queue
sky queue mycluster
# View managed job logs
sky jobs logs my-job
| Issue | Solution |
|---|---|
| Quota exceeded | Request quota increase, try different region |
| Spot preemption | Use sky jobs launch for auto-recovery |
| Slow file sync | Use MOUNT_CACHED mode for outputs |
| GPU not available | Use any_of for fallback clouds |
Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.
Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.
Set up and use 1Password CLI (op). Use when installing the CLI, enabling desktop app integration, signing in (single or multi-account), or reading/injecting/running secrets via op.
CLI to manage emails via IMAP/SMTP. Use `himalaya` to list, read, write, reply, forward, search, and organize emails from the terminal. Supports multiple accounts and message composition with MML (MIME Meta Language).
Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.
Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.
Set up and use 1Password CLI (op). Use when installing the CLI, enabling desktop app integration, signing in (single or multi-account), or reading/injecting/running secrets via op.
CLI to manage emails via IMAP/SMTP. Use `himalaya` to list, read, write, reply, forward, search, and organize emails from the terminal. Supports multiple accounts and message composition with MML (MIME Meta Language).