performance_engineering

•

43126 months ago

When validating system performance under load, identifying bottlenecks through profiling, or optimizing application responsiveness. Covers load testing (k6, Locust), profiling (CPU, memory, I/O), and optimization strategies (caching, query optimization, Core Web Vitals). Use for capacity planning, regression detection, and establishing performance SLOs.

Quick Install

npxskills add vuralserhat86/antigravity-agentic-skills--skill performance_engineering

Instructions

Performance Engineering

Purpose

Performance engineering encompasses load testing, profiling, and optimization to deliver reliable, scalable systems. This skill provides frameworks for choosing the right performance testing approach (load, stress, soak, spike), profiling techniques to identify bottlenecks (CPU, memory, I/O), and optimization strategies for backend APIs, databases, and frontend applications.

Use this skill to validate system capacity before launch, detect performance regressions in CI/CD pipelines, identify and resolve bottlenecks through profiling, and optimize application responsiveness across the stack.

When to Use This Skill

Common Triggers:

"Validate API can handle expected traffic"
"Find maximum capacity and breaking points"
"Identify why the application is slow"
"Detect memory leaks or resource exhaustion"
"Optimize Core Web Vitals for SEO"
"Set up performance testing in CI/CD"
"Reduce cloud infrastructure costs"

Use Cases:

Pre-launch capacity planning and load validation
Post-refactor performance regression testing
Investigating slow response times or high latency
Detecting memory leaks in long-running services
Optimizing database query performance
Validating auto-scaling configuration
Establishing performance SLOs and budgets

Performance Testing Types

Load Testing

Validate system behavior under expected traffic levels.

When to use: Pre-launch capacity planning, regression testing after refactors, validating auto-scaling.

Stress Testing

Find system capacity limits and failure modes.

When to use: Capacity planning, understanding failure behavior, infrastructure sizing decisions.

Soak Testing

Identify memory leaks, resource exhaustion, and degradation over time.

When to use: Detecting memory leaks, validating connection pool cleanup, testing long-running batch jobs.

Spike Testing

Validate system response to sudden traffic spikes.

When to use: Validating auto-scaling, testing event-driven systems (product launches), ensuring rate limiting works.

Quick Decision Framework

Which test type to use?

What am I trying to learn?
├─ Can my system handle expected traffic? → LOAD TEST
├─ What's the maximum capacity? → STRESS TEST
├─ Will it stay stable over time? → SOAK TEST
└─ Can it handle traffic spikes? → SPIKE TEST

For detailed testing patterns, load scenarios, and interpreting results, see references/testing-types.md.

Load Testing Quick Starts

k6 (JavaScript)

Installation:

brew install k6  # macOS
sudo apt-get install k6  # Linux

Basic Load Test:

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '30s', target: 20 },
    { duration: '1m', target: 20 },
    { duration: '30s', target: 0 },
  ],
  thresholds: {
    http_req_duration: ['p(95)<500'],
    http_req_failed: ['rate<0.01'],
  },
};

export default function () {
  const res = http.get('https://api.example.com/products');
  check(res, {
    'status is 200': (r) => r.status === 200,
  });
  sleep(1);
}

Run: k6 run script.js

For stress, soak, and spike testing examples, see examples/k6/.

Locust (Python)

Installation:

pip install locust

Basic Load Test:

from locust import HttpUser, task, between

class WebsiteUser(HttpUser):
    wait_time = between(1, 3)
    host = "https://api.example.com"

    @task(3)
    def view_products(self):
        self.client.get("/products")

    @task(1)
    def view_product_detail(self):
        self.client.get("/products/123")

Run: locust -f locustfile.py --headless -u 100 -r 10 --run-time 10m

For REST API testing and data-driven testing, see examples/locust/.

Profiling Quick Starts

When to Profile

Symptom	Profiling Type	Tool
High CPU (>70%)	CPU Profiling	py-spy, pprof, DevTools
Memory growing	Memory Profiling	memory_profiler, pprof heap
Slow response, low CPU	I/O Profiling	Query logs, pprof block

Python Profiling

py-spy (Production-Safe):

pip install py-spy

# Profile running process
py-spy record -o profile.svg --pid <PID> --duration 30

# Top-like view
py-spy top --pid <PID>

Memory Profiling:

from memory_profiler import profile

@profile
def my_function():
    a = [1] * (10 ** 6)
    return a

# Run: python -m memory_profiler script.py

Go Profiling

pprof (Built-in):

import (
    "net/http"
    _ "net/http/pprof"
)

func main() {
    go func() {
        http.ListenAndServe("localhost:6060", nil)
    }()
    startApp()
}

Capture profile:

# CPU profile (30 seconds)
go tool pprof http://localhost:6060/debug/pprof/profile?seconds=30

# Interactive analysis
(pprof) top
(pprof) web

TypeScript/JavaScript Profiling

Chrome DevTools (Browser/Node.js):

Node.js:

node --inspect app.js
# Open chrome://inspect
# Performance tab → Record

clinic.js (Node.js):

npm install -g clinic
clinic doctor -- node app.js

For detailed profiling workflows and analysis, see references/profiling-guide.md and examples/profiling/.

Optimization Strategies

Caching

When to cache:

Data queried frequently (>100 req/min)
Data freshness tolerance (>1 minute acceptable staleness)

Redis example:

import redis
r = redis.Redis()

def get_cached_data(key, fn, ttl=300):
    cached = r.get(key)
    if cached:
        return json.loads(cached)
    data = fn()
    r.setex(key, ttl, json.dumps(data))
    return data

Database Query Optimization

N+1 prevention:

# Bad: N+1 queries
users = User.query.all()
for user in users:
    print(user.orders)  # Separate query per user

# Good: Eager loading
users = User.query.options(joinedload(User.orders)).all()

Indexing:

CREATE INDEX idx_users_email ON users(email);

API Performance

Cursor-based pagination:

app.get('/api/products', async (req, res) => {
  const { cursor, limit = 20 } = req.query;

  const products = await db.query(
    'SELECT * FROM products WHERE id > ? ORDER BY id LIMIT ?',
    [cursor || 0, limit]
  );

  res.json({
    data: products,
    next_cursor: products[products.length - 1]?.id,
  });
});

Frontend Performance (Core Web Vitals)

Key metrics:

LCP (Largest Contentful Paint): < 2.5s
INP (Interaction to Next Paint): < 200ms
CLS (Cumulative Layout Shift): < 0.1

Optimization techniques:

Code splitting (lazy loading)
Image optimization (WebP, responsive, lazy loading)
Preload critical resources
Minimize render-blocking resources

For detailed optimization strategies, see references/optimization-strategies.md and references/frontend-performance.md.

Performance SLOs

Recommended SLOs by Service Type

Service Type	p95 Latency	p99 Latency	Availability
User-Facing API	< 200ms	< 500ms	99.9%
Internal API	< 100ms	< 300ms	99.5%
Database Query	< 50ms	< 100ms	99.99%
Background Job	< 5s	< 10s	99%
Real-time API	< 50ms	< 100ms	99.95%

SLO Selection Process

Measure baseline performance
Identify user expectations
Set achievable targets (10-20% better than baseline)
Iterate as system matures

For detailed SLO framework and performance budgets, see references/slo-framework.md.

CI/CD Integration

Performance Testing in Pipelines

GitHub Actions example:

name: performance_engineering

on:
  pull_request:
    branches: [main]

jobs:
  load-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install k6
        run: |
          curl https://github.com/grafana/k6/releases/download/v0.48.0/k6-v0.48.0-linux-amd64.tar.gz -L | tar xvz
          sudo mv k6-v0.48.0-linux-amd64/k6 /usr/local/bin/

      - name: Run load test
        run: k6 run tests/load/api-test.js

Performance budgets:

// k6 test with thresholds (fail build if violated)
export const options = {
  thresholds: {
    http_req_duration: ['p(95)<500'],
    http_req_failed: ['rate<0.01'],
  },
};

Profiling Workflow

Standard process:

Observe symptoms (high CPU, memory growth, slow response)
Hypothesize bottleneck (CPU? Memory? I/O?)
Choose profiling type based on hypothesis
Run profiler under realistic load
Analyze profile (flamegraph, call tree)
Identify hot spots (top 20% functions using 80% resources)
Optimize bottlenecks
Re-profile to validate improvement

Best practices:

Profile under realistic load (not idle systems)
Use sampling profilers (py-spy, pprof) in production (low overhead)
Focus on hot paths (optimize biggest bottlenecks first)
Validate optimizations with before/after comparisons

Tool Recommendations

Load Testing

Primary: k6 (JavaScript-based, Grafana-backed)

Modern architecture, cloud-native
JavaScript DSL (ES6+)
Grafana/Prometheus integration
Multi-protocol (HTTP/1.1, HTTP/2, WebSocket, gRPC)

When to use: Modern APIs, microservices, CI/CD integration.

Alternative: Locust (Python-based)

Python-native (write tests in Python)
Web UI for real-time monitoring
Flexible for complex user scenarios

When to use: Python-heavy teams, complex user flows.

Profiling

Python:

py-spy (sampling, production-safe)
cProfile (deterministic, detailed)
memory_profiler (memory leak detection)

Go:

pprof (built-in, CPU/heap/goroutine/block profiling)

TypeScript/JavaScript:

Chrome DevTools (browser/Node.js)
clinic.js (Node.js performance suite)

For detailed tool comparisons, see references/testing-types.md and references/profiling-guide.md.

Reference Documentation

Detailed Guides:

references/testing-types.md - Load, stress, soak, spike testing patterns
references/profiling-guide.md - CPU, memory, I/O profiling across languages
references/optimization-strategies.md - Caching, database, API optimization
references/frontend-performance.md - Core Web Vitals, bundle optimization
references/slo-framework.md - Setting SLOs, performance budgets
references/benchmarking.md - Benchmarking best practices

Examples:

examples/k6/ - Load, stress, soak, spike tests
examples/locust/ - Python-based load testing
examples/profiling/ - Profiling examples (Python, Go, TypeScript)
examples/optimization/ - Caching, query, API optimization

Related Skills

For comprehensive testing strategies, see the testing-strategies skill.

For CI/CD integration patterns, see the building-ci-pipelines skill.

For infrastructure sizing based on load tests, see the infrastructure-as-code skill.

Performance Engineering v1.1 - Enhanced

🔄 Workflow

Kaynak: k6 Methodology & The Art of Capacity Planning

Aşama 1: Planning & SLOs

Goal: Testin amacı ne? (Smoke, Load, Stress, Soak?).
SLOs: Başarı kriterlerini belirle (Örn: p95 latency < 200ms, Error rate < %1).
Environment: Test ortamı Prod ile ne kadar benzer? (Scaling faktörünü belirle).

Aşama 2: Scripting & Execution

User Journey: Gerçek kullanıcı davranışını simüle et (Login -> Browse -> Buy).
Data Driven: Testi statik verilerle değil, CSV'den gelen dinamik verilerle besle (Cache'i aşmak için).
Ramp-up: Trafiği aniden değil, kademeli artır (Sistemin ısınması için).

Aşama 3: Analysis & Optimization

Correlation: Hata anında CPU/Memory/DB metrikleri ne durumdaydı?
Bottleneck: Darboğaz nerede? (App Code, DB, Network, veya Load Injector'ın kendisi?).
Report: Teknik ve yönetici özeti içeren rapor hazırla.

Kontrol Noktaları

Aşama	Doğrulama
1	Test verisi (Database seed) yeterli hacimde mi?
2	Load Generator (Test makinesi) CPU darboğazına girdi mi? (False negative riski).
3	3rd party API'lar (Stripe, Twilio) mock'landı mı? (Masraf ve ban riski).

Loading…

Related Skills

skill-creator

Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.

367,360

skill-creator

Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.

367,309

1password

Set up and use 1Password CLI (op). Use when installing the CLI, enabling desktop app integration, signing in (single or multi-account), or reading/injecting/running secrets via op.

365,915

himalaya

CLI to manage emails via IMAP/SMTP. Use `himalaya` to list, read, write, reply, forward, search, and organize emails from the terminal. Supports multiple accounts and message composition with MML (MIME Meta Language).

365,885

Performance Engineering

Purpose

When to Use This Skill

Common Triggers:

"Validate API can handle expected traffic"
"Find maximum capacity and breaking points"
"Identify why the application is slow"
"Detect memory leaks or resource exhaustion"
"Optimize Core Web Vitals for SEO"
"Set up performance testing in CI/CD"
"Reduce cloud infrastructure costs"

Use Cases:

Pre-launch capacity planning and load validation
Post-refactor performance regression testing
Investigating slow response times or high latency
Detecting memory leaks in long-running services
Optimizing database query performance
Validating auto-scaling configuration
Establishing performance SLOs and budgets

Performance Testing Types

Load Testing

Validate system behavior under expected traffic levels.

When to use: Pre-launch capacity planning, regression testing after refactors, validating auto-scaling.

Stress Testing

Find system capacity limits and failure modes.

When to use: Capacity planning, understanding failure behavior, infrastructure sizing decisions.

Soak Testing

Identify memory leaks, resource exhaustion, and degradation over time.

When to use: Detecting memory leaks, validating connection pool cleanup, testing long-running batch jobs.

Spike Testing

Validate system response to sudden traffic spikes.

When to use: Validating auto-scaling, testing event-driven systems (product launches), ensuring rate limiting works.

Quick Decision Framework

Which test type to use?

What am I trying to learn?
├─ Can my system handle expected traffic? → LOAD TEST
├─ What's the maximum capacity? → STRESS TEST
├─ Will it stay stable over time? → SOAK TEST
└─ Can it handle traffic spikes? → SPIKE TEST

For detailed testing patterns, load scenarios, and interpreting results, see references/testing-types.md.

Load Testing Quick Starts

k6 (JavaScript)

Installation:

brew install k6  # macOS
sudo apt-get install k6  # Linux

Basic Load Test:

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '30s', target: 20 },
    { duration: '1m', target: 20 },
    { duration: '30s', target: 0 },
  ],
  thresholds: {
    http_req_duration: ['p(95)<500'],
    http_req_failed: ['rate<0.01'],
  },
};

export default function () {
  const res = http.get('https://api.example.com/products');
  check(res, {
    'status is 200': (r) => r.status === 200,
  });
  sleep(1);
}

Run: k6 run script.js

For stress, soak, and spike testing examples, see examples/k6/.

Locust (Python)

Installation:

pip install locust

Basic Load Test:

from locust import HttpUser, task, between

class WebsiteUser(HttpUser):
    wait_time = between(1, 3)
    host = "https://api.example.com"

    @task(3)
    def view_products(self):
        self.client.get("/products")

    @task(1)
    def view_product_detail(self):
        self.client.get("/products/123")

Run: locust -f locustfile.py --headless -u 100 -r 10 --run-time 10m

For REST API testing and data-driven testing, see examples/locust/.

Profiling Quick Starts

When to Profile

Symptom	Profiling Type	Tool
High CPU (>70%)	CPU Profiling	py-spy, pprof, DevTools
Memory growing	Memory Profiling	memory_profiler, pprof heap
Slow response, low CPU	I/O Profiling	Query logs, pprof block

Python Profiling

py-spy (Production-Safe):

pip install py-spy

# Profile running process
py-spy record -o profile.svg --pid <PID> --duration 30

# Top-like view
py-spy top --pid <PID>

Memory Profiling:

from memory_profiler import profile

@profile
def my_function():
    a = [1] * (10 ** 6)
    return a

# Run: python -m memory_profiler script.py

Go Profiling

pprof (Built-in):

import (
    "net/http"
    _ "net/http/pprof"
)

func main() {
    go func() {
        http.ListenAndServe("localhost:6060", nil)
    }()
    startApp()
}

Capture profile:

# CPU profile (30 seconds)
go tool pprof http://localhost:6060/debug/pprof/profile?seconds=30

# Interactive analysis
(pprof) top
(pprof) web

TypeScript/JavaScript Profiling

Chrome DevTools (Browser/Node.js):

Node.js:

node --inspect app.js
# Open chrome://inspect
# Performance tab → Record

clinic.js (Node.js):

npm install -g clinic
clinic doctor -- node app.js

For detailed profiling workflows and analysis, see references/profiling-guide.md and examples/profiling/.

Optimization Strategies

Caching

When to cache:

Data queried frequently (>100 req/min)
Data freshness tolerance (>1 minute acceptable staleness)

Redis example:

import redis
r = redis.Redis()

def get_cached_data(key, fn, ttl=300):
    cached = r.get(key)
    if cached:
        return json.loads(cached)
    data = fn()
    r.setex(key, ttl, json.dumps(data))
    return data

Database Query Optimization

N+1 prevention:

# Bad: N+1 queries
users = User.query.all()
for user in users:
    print(user.orders)  # Separate query per user

# Good: Eager loading
users = User.query.options(joinedload(User.orders)).all()

Indexing:

CREATE INDEX idx_users_email ON users(email);

API Performance

Cursor-based pagination:

app.get('/api/products', async (req, res) => {
  const { cursor, limit = 20 } = req.query;

  const products = await db.query(
    'SELECT * FROM products WHERE id > ? ORDER BY id LIMIT ?',
    [cursor || 0, limit]
  );

  res.json({
    data: products,
    next_cursor: products[products.length - 1]?.id,
  });
});

Frontend Performance (Core Web Vitals)

Key metrics:

LCP (Largest Contentful Paint): < 2.5s
INP (Interaction to Next Paint): < 200ms
CLS (Cumulative Layout Shift): < 0.1

Optimization techniques:

Code splitting (lazy loading)
Image optimization (WebP, responsive, lazy loading)
Preload critical resources
Minimize render-blocking resources

For detailed optimization strategies, see references/optimization-strategies.md and references/frontend-performance.md.

Performance SLOs

Recommended SLOs by Service Type

Service Type	p95 Latency	p99 Latency	Availability
User-Facing API	< 200ms	< 500ms	99.9%
Internal API	< 100ms	< 300ms	99.5%
Database Query	< 50ms	< 100ms	99.99%
Background Job	< 5s	< 10s	99%
Real-time API	< 50ms	< 100ms	99.95%

SLO Selection Process

Measure baseline performance
Identify user expectations
Set achievable targets (10-20% better than baseline)
Iterate as system matures

For detailed SLO framework and performance budgets, see references/slo-framework.md.

CI/CD Integration

Performance Testing in Pipelines

GitHub Actions example:

name: performance_engineering

on:
  pull_request:
    branches: [main]

jobs:
  load-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install k6
        run: |
          curl https://github.com/grafana/k6/releases/download/v0.48.0/k6-v0.48.0-linux-amd64.tar.gz -L | tar xvz
          sudo mv k6-v0.48.0-linux-amd64/k6 /usr/local/bin/

      - name: Run load test
        run: k6 run tests/load/api-test.js

Performance budgets:

// k6 test with thresholds (fail build if violated)
export const options = {
  thresholds: {
    http_req_duration: ['p(95)<500'],
    http_req_failed: ['rate<0.01'],
  },
};

Profiling Workflow

Standard process:

Observe symptoms (high CPU, memory growth, slow response)
Hypothesize bottleneck (CPU? Memory? I/O?)
Choose profiling type based on hypothesis
Run profiler under realistic load
Analyze profile (flamegraph, call tree)
Identify hot spots (top 20% functions using 80% resources)
Optimize bottlenecks
Re-profile to validate improvement

Best practices:

Profile under realistic load (not idle systems)
Use sampling profilers (py-spy, pprof) in production (low overhead)
Focus on hot paths (optimize biggest bottlenecks first)
Validate optimizations with before/after comparisons

Tool Recommendations

Load Testing

Primary: k6 (JavaScript-based, Grafana-backed)

Modern architecture, cloud-native
JavaScript DSL (ES6+)
Grafana/Prometheus integration
Multi-protocol (HTTP/1.1, HTTP/2, WebSocket, gRPC)

When to use: Modern APIs, microservices, CI/CD integration.

Alternative: Locust (Python-based)

Python-native (write tests in Python)
Web UI for real-time monitoring
Flexible for complex user scenarios

When to use: Python-heavy teams, complex user flows.

Profiling

Python:

py-spy (sampling, production-safe)
cProfile (deterministic, detailed)
memory_profiler (memory leak detection)

Go:

pprof (built-in, CPU/heap/goroutine/block profiling)

TypeScript/JavaScript:

Chrome DevTools (browser/Node.js)
clinic.js (Node.js performance suite)

For detailed tool comparisons, see references/testing-types.md and references/profiling-guide.md.

Reference Documentation

Detailed Guides:

references/testing-types.md - Load, stress, soak, spike testing patterns
references/profiling-guide.md - CPU, memory, I/O profiling across languages
references/optimization-strategies.md - Caching, database, API optimization
references/frontend-performance.md - Core Web Vitals, bundle optimization
references/slo-framework.md - Setting SLOs, performance budgets
references/benchmarking.md - Benchmarking best practices

Examples:

examples/k6/ - Load, stress, soak, spike tests
examples/locust/ - Python-based load testing
examples/profiling/ - Profiling examples (Python, Go, TypeScript)
examples/optimization/ - Caching, query, API optimization

Related Skills

For comprehensive testing strategies, see the testing-strategies skill.

For CI/CD integration patterns, see the building-ci-pipelines skill.

For infrastructure sizing based on load tests, see the infrastructure-as-code skill.

Performance Engineering v1.1 - Enhanced

🔄 Workflow

Kaynak: k6 Methodology & The Art of Capacity Planning

Aşama 1: Planning & SLOs

Goal: Testin amacı ne? (Smoke, Load, Stress, Soak?).
SLOs: Başarı kriterlerini belirle (Örn: p95 latency < 200ms, Error rate < %1).
Environment: Test ortamı Prod ile ne kadar benzer? (Scaling faktörünü belirle).

Aşama 2: Scripting & Execution

User Journey: Gerçek kullanıcı davranışını simüle et (Login -> Browse -> Buy).
Data Driven: Testi statik verilerle değil, CSV'den gelen dinamik verilerle besle (Cache'i aşmak için).
Ramp-up: Trafiği aniden değil, kademeli artır (Sistemin ısınması için).

Aşama 3: Analysis & Optimization

Correlation: Hata anında CPU/Memory/DB metrikleri ne durumdaydı?
Bottleneck: Darboğaz nerede? (App Code, DB, Network, veya Load Injector'ın kendisi?).
Report: Teknik ve yönetici özeti içeren rapor hazırla.

Kontrol Noktaları

Aşama	Doğrulama
1	Test verisi (Database seed) yeterli hacimde mi?
2	Load Generator (Test makinesi) CPU darboğazına girdi mi? (False negative riski).
3	3rd party API'lar (Stripe, Twilio) mock'landı mı? (Masraf ve ban riski).