Implement Vast.ai reference architecture with best-practice project layout.
Use when designing new Vast.ai integrations, reviewing project structure,
or establishing architecture standards for Vast.ai applications.
Trigger with phrases like "vastai architecture", "vastai best practices",
"vastai project structure", "how to organize vastai", "vastai layout".
Production architecture for GPU compute workflows on Vast.ai. Covers the three-tier pattern (orchestrator, GPU workers, artifact storage), job queue design, and fault-tolerant training pipelines.
Prerequisites
Vast.ai account with CLI
Cloud storage (S3, GCS, or MinIO) for artifacts
Understanding of ML training pipelines
Instructions
Architecture: Three-Tier GPU Compute
┌─────────────────────────────────────────────────┐
│ ORCHESTRATOR (your server / CI / cloud function) │
│ - Job queue management │
│ - Instance provisioning via Vast.ai API │
│ - Status monitoring and auto-recovery │
│ - Cost tracking and budget enforcement │
└───────────────┬─────────────────────────────────┘
│ Vast.ai REST API
┌───────────────▼─────────────────────────────────┐
│ GPU WORKERS (Vast.ai rented instances) │
│ - Training / inference execution │
│ - Checkpoint saving to cloud storage │
│ - Health reporting back to orchestrator │
│ - Graceful shutdown on SIGTERM (spot preemption)│
└───────────────┬─────────────────────────────────┘
│ S3 / GCS / MinIO
┌───────────────▼─────────────────────────────────┐
│ ARTIFACT STORAGE (persistent) │
│ - Model checkpoints │
│ - Training logs and metrics │
│ - Dataset cache │
│ - Final model artifacts │
└─────────────────────────────────────────────────┘
Project Structure
ml-pipeline/
orchestrator/
job_queue.py # Job definition and scheduling
provisioner.py # Vast.ai instance lifecycle
monitor.py # Status polling and auto-recovery
cost_tracker.py # Budget enforcement
worker/
Dockerfile # GPU worker image
train.py # Training entry point
checkpoint.py # Cloud storage checkpoint manager
health.py # Report status back to orchestrator
config/
gpu_profiles.yaml # GPU selection criteria per job type
budgets.yaml # Cost limits per team/project
scripts/
deploy.py # CLI for launching jobs
cost_report.py # Spending analysis
For multi-environment configuration, see vastai-multi-env-setup.
Examples
Simple pipeline: Orchestrator searches for offers matching training-standard profile, provisions instance, uploads data via SCP, runs training, saves checkpoints to S3, destroys instance.
Fault-tolerant training: Worker saves checkpoint every 500 steps to S3. On preemption, orchestrator provisions replacement and worker resumes from latest checkpoint.
