Specialized skill for building production-ready serverless applications on GCP. Covers Cloud Run services (containerized), Cloud Run Functions (event-driven), cold start optimization, and event-driven architecture with Pub/Sub.
Specialized skill for building production-ready serverless applications on GCP.
Covers Cloud Run services (containerized), Cloud Run Functions (event-driven),
cold start optimization, and event-driven architecture with Pub/Sub.
Principles
Cloud Run for containers, Functions for simple event handlers
Optimize for cold starts with startup CPU boost and min instances
Set concurrency based on workload (start with 8, adjust)
Memory includes /tmp filesystem - plan accordingly
Use VPC Connector only when needed (adds latency)
Containers should start fast and be stateless
Handle signals gracefully for clean shutdown
Patterns
Cloud Run Service Pattern
Containerized web service on Cloud Run
When to use: Web applications and APIs,Need any runtime or library,Complex services with multiple endpoints,Stateless containerized workloads
# Dockerfile - Multi-stage build for smaller image
FROM node:20-slim AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
FROM node:20-slim
WORKDIR /app
# Copy only production dependencies
COPY --from=builder /app/node_modules ./node_modules
COPY src ./src
COPY package.json ./
# Cloud Run uses PORT env variable
ENV PORT=8080
EXPOSE 8080
# Run as non-root user
USER node
CMD ["node", "src/index.js"]
When to use: Latency-sensitive applications,User-facing APIs,High-traffic services
1. Enable Startup CPU Boost
gcloud run deploy my-service \
--cpu-boost \
--region us-central1
2. Set Minimum Instances
gcloud run deploy my-service \
--min-instances 1 \
--region us-central1
3. Optimize Container Image
# Use distroless for minimal image
FROM node:20-slim AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
FROM gcr.io/distroless/nodejs20-debian12
WORKDIR /app
COPY --from=builder /app/node_modules ./node_modules
COPY src ./src
CMD ["src/index.js"]
4. Lazy Initialize Heavy Dependencies
// Lazy load heavy libraries
let bigQueryClient = null;
function getBigQueryClient() {
if (!bigQueryClient) {
const { BigQuery } = require('@google-cloud/bigquery');
bigQueryClient = new BigQuery();
}
return bigQueryClient;
}
// Only initialize when needed
app.get('/api/analytics', async (req, res) => {
const client = getBigQueryClient();
const results = await client.query({...});
res.json(results);
});
5. Increase Memory (More CPU)
# Higher memory = more CPU during startup
gcloud run deploy my-service \
--memory 1Gi \
--cpu 2 \
--region us-central1
Optimization_impact
Startup_cpu_boost: 50% faster cold starts
Min_instances: Eliminates cold starts for traffic spikes
# BAD - buffers entire file in /tmp
def process_large_file(bucket_name, blob_name):
blob = bucket.blob(blob_name)
blob.download_to_filename('/tmp/large_file')
with open('/tmp/large_file', 'rb') as f:
process(f.read())
# GOOD - stream processing
def process_large_file(bucket_name, blob_name):
blob = bucket.blob(blob_name)
with blob.open('rb') as f:
for chunk in iter(lambda: f.read(8192), b''):
process_chunk(chunk)
Use Cloud Storage for large files
from google.cloud import storage
def process_with_gcs(bucket_name, input_blob, output_blob):
client = storage.Client()
bucket = client.bucket(bucket_name)
# Process directly to/from GCS
input_blob = bucket.blob(input_blob)
output_blob = bucket.blob(output_blob)
with input_blob.open('rb') as reader:
with output_blob.open('wb') as writer:
for chunk in iter(lambda: reader.read(65536), b''):
processed = transform(chunk)
writer.write(processed)
Situation: Setting concurrency to 1 for request isolation
Symptoms:
Auto-scaling creates many container instances.
High latency during traffic spikes.
Increased cold starts.
Higher costs from more instances.
Why this breaks:
Setting concurrency to 1 means each container handles only one
request at a time. During traffic spikes:
100 concurrent requests = 100 container instances
Each instance has cold start overhead
More instances = higher costs
Scaling takes time, requests queue up
This should only be used when:
Processing is truly single-threaded
Memory-heavy per-request processing
Using thread-unsafe libraries
Recommended fix:
Set appropriate concurrency
# For I/O-bound workloads (most web apps)
gcloud run deploy my-service \
--concurrency=80 \
--max-instances=100
# For CPU-bound workloads
gcloud run deploy my-service \
--concurrency=4 \
--cpu=2
# Only use 1 when absolutely necessary
gcloud run deploy my-service \
--concurrency=1 \
--max-instances=1000 # Be prepared for many instances
Node.js - use async properly
// With high concurrency, ensure async operations
const express = require('express');
const app = express();
app.get('/api/data', async (req, res) => {
// All I/O should be async
const data = await fetchFromDatabase();
const enriched = await enrichData(data);
res.json(enriched);
});
// Concurrency 80+ is safe for async I/O workloads
Python - use async framework
from fastapi import FastAPI
import asyncio
import httpx
app = FastAPI()
@app.get("/api/data")
async def get_data():
# Async I/O allows high concurrency
async with httpx.AsyncClient() as client:
response = await client.get("https://api.example.com/data")
return response.json()
# Concurrency 80+ safe with async framework
Situation: Running background tasks or processing between requests
Symptoms:
Background tasks run extremely slowly.
Scheduled work doesn't complete.
Metrics collection fails.
Connection keep-alive breaks.
Why this breaks:
By default, Cloud Run throttles CPU to near-zero when not actively
handling a request. This is "CPU only during requests" mode.
Affected operations:
Background threads
Connection pool maintenance
Metrics/telemetry emission
Scheduled tasks within container
Cleanup operations after response
Recommended fix:
Enable CPU always allocated
# CPU allocated even outside requests
gcloud run deploy my-service \
--cpu-throttling=false \
--min-instances=1
# Note: This increases costs but enables background work
Use startup CPU boost for initialization
# Boost CPU during cold start only
gcloud run deploy my-service \
--cpu-boost \
--cpu-throttling=true # Default, throttle after request
# Move heavy processing to separate service
steps:
# Main service - responds quickly
- name: 'gcr.io/cloud-builders/gcloud'
args: ['run', 'deploy', 'api-service',
'--cpu-throttling=true']
# Worker service - processes messages
- name: 'gcr.io/cloud-builders/gcloud'
args: ['run', 'deploy', 'worker-service',
'--cpu-throttling=false',
'--min-instances=1']
VPC Connector 10-Minute Idle Timeout
Severity: MEDIUM
Situation: Cloud Run service connecting to VPC resources
Symptoms:
Connection errors after period of inactivity.
"Connection reset" or "Connection refused" errors.
Sporadic failures to VPC resources.
Database connections drop unexpectedly.
Why this breaks:
Cloud Run's VPC connector has a 10-minute idle timeout on connections.
If a connection is idle for 10 minutes, it's silently closed.
Affects:
Database connection pools
Redis connections
Internal API connections
Any persistent VPC connection
Recommended fix:
Configure connection pool with keep-alive
# SQLAlchemy with connection recycling
from sqlalchemy import create_engine
engine = create_engine(
DATABASE_URL,
pool_size=5,
max_overflow=2,
pool_recycle=300, # Recycle connections every 5 minutes
pool_pre_ping=True # Validate connection before use
)
Situation: Deploying containers with slow initialization
Symptoms:
Deployment fails with "Container failed to start".
Service never becomes healthy.
"Revision failed to become ready" errors.
Works locally but fails on Cloud Run.
Why this breaks:
Cloud Run expects your container to start listening on PORT within
4 minutes (240 seconds). If it doesn't, the instance is killed.
Common causes:
Heavy framework initialization (ML models, etc.)
Waiting for external dependencies at startup
Large dependency loading
Database migrations on startup
Recommended fix:
Enable startup CPU boost
gcloud run deploy my-service \
--cpu-boost \
--startup-cpu-boost
Lazy initialization
from functools import lru_cache
from fastapi import FastAPI
app = FastAPI()
# Don't load at import time
model = None
@lru_cache()
def get_model():
global model
if model is None:
# Load on first request, not at startup
model = load_heavy_model()
return model
@app.get("/predict")
async def predict(data: dict):
model = get_model() # Loads on first call only
return model.predict(data)
# Startup is fast - model loads on first request
Start listening immediately
import asyncio
from fastapi import FastAPI
import uvicorn
app = FastAPI()
# Global state for async initialization
initialized = asyncio.Event()
@app.on_event("startup")
async def startup():
# Start background initialization
asyncio.create_task(async_init())
async def async_init():
# Heavy initialization happens after server starts
await load_models()
await warm_up_connections()
initialized.set()
@app.get("/ready")
async def ready():
if not initialized.is_set():
raise HTTPException(503, "Still initializing")
return {"status": "ready"}
@app.get("/health")
async def health():
# Always respond - health check passes
return {"status": "healthy"}
Use multi-stage builds
# Build stage - slow
FROM python:3.11 as builder
WORKDIR /app
COPY requirements.txt .
RUN pip wheel --no-cache-dir --wheel-dir /wheels -r requirements.txt
# Runtime stage - fast startup
FROM python:3.11-slim
WORKDIR /app
COPY --from=builder /wheels /wheels
RUN pip install --no-cache /wheels/* && rm -rf /wheels
COPY . .
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]
Run migrations separately
# Don't migrate on startup - use Cloud Build
steps:
# Run migrations first
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args:
- '-c'
- |
gcloud run jobs execute migrate-job --wait
# Then deploy
- name: 'gcr.io/cloud-builders/gcloud'
args: ['run', 'deploy', 'my-service', ...]
Second Generation Execution Environment Differences
Severity: MEDIUM
Situation: Migrating to or using Cloud Run second-gen execution environment
Symptoms:
Network behavior changes.
Different syscall support.
File system behavior differences.
Container behaves differently than in first-gen.
Why this breaks:
Cloud Run's second-generation execution environment uses a different
sandbox (gVisor) with different characteristics:
More Linux syscalls supported
Full /proc and /sys access
Different network stack
No automatic HTTPS redirect
Different tmp filesystem behavior
Recommended fix:
Explicitly set execution environment
# First generation (legacy)
gcloud run deploy my-service \
--execution-environment=gen1
# Second generation (recommended for most)
gcloud run deploy my-service \
--execution-environment=gen2
# GPUs only available in second-gen
gcloud run deploy ml-service \
--execution-environment=gen2 \
--gpu=1 \
--gpu-type=nvidia-l4
Check execution environment
import os
def get_execution_environment():
# Second-gen has different /proc structure
try:
with open('/proc/version', 'r') as f:
version = f.read()
if 'gVisor' in version:
return 'gen2'
except:
pass
return 'gen1'
Request Timeout Configuration Mismatch
Severity: MEDIUM
Situation: Long-running requests or background processing