twinmind-observability

jeremylongshore

Set up comprehensive observability for TwinMind integrations with metrics, traces, and alerts. Use when implementing monitoring for TwinMind operations, setting up dashboards, or configuring alerting for meeting AI integration health. Trigger with phrases like "twinmind monitoring", "twinmind metrics", "twinmind observability", "monitor twinmind", "twinmind alerts", "twinmind tracing".

Quick Install

bunx add-skill jeremylongshore/claude-code-plugins-plus-skills -s twinmind-observability

aiautomationclaude-codedevopsmarketplacemcp

Instructions

TwinMind Observability

Overview

Set up comprehensive observability for TwinMind integrations including metrics, distributed tracing, logging, and alerting.

Prerequisites

Prometheus or compatible metrics backend
OpenTelemetry SDK installed
Grafana or similar dashboarding tool
AlertManager configured

Metrics Collection

Key Metrics

Metric	Type	Description
`twinmind_transcriptions_total`	Counter	Total transcription requests
`twinmind_transcription_duration_seconds`	Histogram	Transcription processing time
`twinmind_transcription_audio_hours`	Counter	Total audio hours processed
`twinmind_errors_total`

Loading…

Related Skills

flow

Use when you need to run Flow type checking, or when seeing Flow type errors in React code.

242,585

verify

Use when you want to validate changes before committing, or when you need to check all React contribution requirements.

242,585

feature-flags

Use when feature flag tests fail, flags need updating, understanding @gate pragmas, debugging channel-specific test failures, or adding new flags to React.

242,585

flags

Use when you need to check feature flag states, compare channels, or debug why a feature behaves differently across release channels.

242,582

// src/observability/metrics.ts
import { Registry, Counter, Histogram, Gauge, Summary } from 'prom-client';

const registry = new Registry();

// Transcription metrics
export const transcriptionCounter = new Counter({
  name: 'twinmind_transcriptions_total',
  help: 'Total TwinMind transcription requests',
  labelNames: ['status', 'model', 'language'],
  registers: [registry],
});

export const transcriptionDuration = new Histogram({
  name: 'twinmind_transcription_duration_seconds',
  help: 'TwinMind transcription processing duration',
  labelNames: ['model'],
  buckets: [1, 5, 10, 30, 60, 120, 300, 600],  // Up to 10 minutes
  registers: [registry],
});

export const audioHoursProcessed = new Counter({
  name: 'twinmind_transcription_audio_hours',
  help: 'Total audio hours processed',
  labelNames: ['model'],
  registers: [registry],
});

// API metrics
export const apiRequestCounter = new Counter({
  name: 'twinmind_api_requests_total',
  help: 'Total TwinMind API requests',
  labelNames: ['method', 'endpoint', 'status'],
  registers: [registry],
});

export const apiLatency = new Histogram({
  name: 'twinmind_api_latency_seconds',
  help: 'TwinMind API request latency',
  labelNames: ['method', 'endpoint'],
  buckets: [0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10],
  registers: [registry],
});

// Error metrics
export const errorCounter = new Counter({
  name: 'twinmind_errors_total',
  help: 'TwinMind errors by type',
  labelNames: ['error_type', 'operation'],
  registers: [registry],
});

// Rate limit metrics
export const rateLimitRemaining = new Gauge({
  name: 'twinmind_rate_limit_remaining',
  help: 'Remaining rate limit quota',
  labelNames: ['endpoint'],
  registers: [registry],
});

// AI usage metrics
export const aiTokensUsed = new Counter({
  name: 'twinmind_ai_tokens_used',
  help: 'AI tokens consumed',
  labelNames: ['operation'],
  registers: [registry],
});

// Summary generation
export const summaryCounter = new Counter({
  name: 'twinmind_summary_requests_total',
  help: 'Summary generation requests',
  labelNames: ['status', 'format'],
  registers: [registry],
});

export const actionItemsExtracted = new Counter({
  name: 'twinmind_action_items_extracted',
  help: 'Action items extracted from meetings',
  registers: [registry],
});

export { registry };

// src/twinmind/instrumented-client.ts
import {
  transcriptionCounter,
  transcriptionDuration,
  audioHoursProcessed,
  apiRequestCounter,
  apiLatency,
  errorCounter,
  rateLimitRemaining,
} from '../observability/metrics';

export class InstrumentedTwinMindClient {
  private client: TwinMindClient;

  constructor(config: TwinMindConfig) {
    this.client = new TwinMindClient(config);
  }

  async transcribe(audioUrl: string, options?: TranscriptionOptions): Promise<Transcript> {
    const timer = transcriptionDuration.startTimer({ model: options?.model || 'ear-3' });

    try {
      const result = await this.client.transcribe(audioUrl, options);

      // Record success metrics
      transcriptionCounter.inc({
        status: 'success',
        model: options?.model || 'ear-3',
        language: result.language,
      });

      // Record audio hours
      audioHoursProcessed.inc(
        { model: options?.model || 'ear-3' },
        result.duration_seconds / 3600
      );

      return result;
    } catch (error: any) {
      transcriptionCounter.inc({
        status: 'error',
        model: options?.model || 'ear-3',
        language: 'unknown',
      });

      errorCounter.inc({
        error_type: error.code || 'unknown',
        operation: 'transcribe',
      });

      throw error;
    } finally {
      timer();
    }
  }

  async request<T>(method: string, endpoint: string, data?: any): Promise<T> {
    const timer = apiLatency.startTimer({ method, endpoint });

    try {
      const response = await this.client.request(method, endpoint, data);

      apiRequestCounter.inc({ method, endpoint, status: 'success' });

      // Update rate limit gauge from response headers
      if (response.headers?.['x-ratelimit-remaining']) {
        rateLimitRemaining.set(
          { endpoint },
          parseInt(response.headers['x-ratelimit-remaining'])
        );
      }

      return response.data;
    } catch (error: any) {
      apiRequestCounter.inc({
        method,
        endpoint,
        status: error.response?.status || 'error',
      });

      throw error;
    } finally {
      timer();
    }
  }
}

// src/observability/tracing.ts
import { NodeSDK } from '@opentelemetry/sdk-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { Resource } from '@opentelemetry/resources';
import { SemanticResourceAttributes } from '@opentelemetry/semantic-conventions';
import { trace, SpanStatusCode, SpanKind } from '@opentelemetry/api';

// Initialize SDK
const sdk = new NodeSDK({
  resource: new Resource({
    [SemanticResourceAttributes.SERVICE_NAME]: 'twinmind-integration',
    [SemanticResourceAttributes.SERVICE_VERSION]: process.env.npm_package_version,
    [SemanticResourceAttributes.DEPLOYMENT_ENVIRONMENT]: process.env.NODE_ENV,
  }),
  traceExporter: new OTLPTraceExporter({
    url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT || 'http://localhost:4318/v1/traces',
  }),
  instrumentations: [getNodeAutoInstrumentations()],
});

sdk.start();

// Custom tracer for TwinMind operations
export const tracer = trace.getTracer('twinmind-client');

// Traced operation wrapper
export async function tracedOperation<T>(
  operationName: string,
  operation: () => Promise<T>,
  attributes?: Record<string, string | number>
): Promise<T> {
  return tracer.startActiveSpan(
    `twinmind.${operationName}`,
    { kind: SpanKind.CLIENT, attributes },
    async (span) => {
      try {
        const result = await operation();
        span.setStatus({ code: SpanStatusCode.OK });
        return result;
      } catch (error: any) {
        span.setStatus({
          code: SpanStatusCode.ERROR,
          message: error.message,
        });
        span.recordException(error);
        throw error;
      } finally {
        span.end();
      }
    }
  );
}

// Traced TwinMind client
export class TracedTwinMindClient {
  private client: TwinMindClient;

  async transcribe(audioUrl: string, options?: TranscriptionOptions): Promise<Transcript> {
    return tracedOperation(
      'transcribe',
      () => this.client.transcribe(audioUrl, options),
      {
        'twinmind.audio_url': audioUrl,
        'twinmind.model': options?.model || 'ear-3',
        'twinmind.diarization': options?.diarization ? 'true' : 'false',
      }
    );
  }

  async summarize(transcriptId: string): Promise<Summary> {
    return tracedOperation(
      'summarize',
      () => this.client.summarize(transcriptId),
      { 'twinmind.transcript_id': transcriptId }
    );
  }
}

// src/observability/logging.ts
import pino from 'pino';

export const logger = pino({
  name: 'twinmind',
  level: process.env.LOG_LEVEL || 'info',
  formatters: {
    level: (label) => ({ level: label }),
  },
  redact: ['apiKey', 'authorization', 'password'],
});

// Operation-specific logging
export function logTwinMindOperation(
  operation: string,
  data: Record<string, any>,
  duration?: number
): void {
  logger.info({
    service: 'twinmind',
    operation,
    duration_ms: duration,
    ...data,
  });
}

// Error logging
export function logTwinMindError(
  operation: string,
  error: Error,
  context?: Record<string, any>
): void {
  logger.error({
    service: 'twinmind',
    operation,
    error: {
      name: error.name,
      message: error.message,
      stack: error.stack,
    },
    ...context,
  });
}

# alerts/twinmind_alerts.yaml
groups:
  - name: twinmind_alerts
    rules:
      - alert: TwinMindHighErrorRate
        expr: |
          rate(twinmind_errors_total[5m]) /
          rate(twinmind_api_requests_total[5m]) > 0.05
        for: 5m
        labels:
          severity: warning
          service: twinmind
        annotations:
          summary: "TwinMind error rate > 5%"
          description: "Error rate is {{ $value | humanizePercentage }}"

      - alert: TwinMindHighLatency
        expr: |
          histogram_quantile(0.95,
            rate(twinmind_api_latency_seconds_bucket[5m])
          ) > 5
        for: 5m
        labels:
          severity: warning
          service: twinmind
        annotations:
          summary: "TwinMind P95 latency > 5s"
          description: "P95 latency is {{ $value | humanizeDuration }}"

      - alert: TwinMindTranscriptionFailures
        expr: |
          increase(twinmind_transcriptions_total{status="error"}[15m]) > 5
        for: 5m
        labels:
          severity: warning
          service: twinmind
        annotations:
          summary: "TwinMind transcription failures increasing"
          description: "{{ $value }} failures in last 15 minutes"

      - alert: TwinMindRateLimitApproaching
        expr: |
          twinmind_rate_limit_remaining < 10
        for: 1m
        labels:
          severity: warning
          service: twinmind
        annotations:
          summary: "TwinMind rate limit approaching"
          description: "Only {{ $value }} requests remaining"

      - alert: TwinMindRateLimitExceeded
        expr: |
          increase(twinmind_errors_total{error_type="RATE_LIMITED"}[5m]) > 0
        for: 1m
        labels:
          severity: critical
          service: twinmind
        annotations:
          summary: "TwinMind rate limit exceeded"
          description: "Rate limit errors detected"

      - alert: TwinMindAPIDown
        expr: |
          up{job="twinmind"} == 0
        for: 1m
        labels:
          severity: critical
          service: twinmind
        annotations:
          summary: "TwinMind integration is down"
          description: "Health check failing"

      - alert: TwinMindHighTokenUsage
        expr: |
          increase(twinmind_ai_tokens_used[24h]) > 1500000
        for: 5m
        labels:
          severity: warning
          service: twinmind
        annotations:
          summary: "High TwinMind token usage"
          description: "{{ $value }} tokens used in 24h (limit: 2M)"

{
  "dashboard": {
    "title": "TwinMind Integration",
    "panels": [
      {
        "title": "Transcription Rate",
        "type": "graph",
        "targets": [{
          "expr": "rate(twinmind_transcriptions_total[5m])",
          "legendFormat": "{{status}}"
        }]
      },
      {
        "title": "API Latency (P50/P95/P99)",
        "type": "graph",
        "targets": [
          {
            "expr": "histogram_quantile(0.5, rate(twinmind_api_latency_seconds_bucket[5m]))",
            "legendFormat": "P50"
          },
          {
            "expr": "histogram_quantile(0.95, rate(twinmind_api_latency_seconds_bucket[5m]))",
            "legendFormat": "P95"
          },
          {
            "expr": "histogram_quantile(0.99, rate(twinmind_api_latency_seconds_bucket[5m]))",
            "legendFormat": "P99"
          }
        ]
      },
      {
        "title": "Error Rate",
        "type": "graph",
        "targets": [{
          "expr": "rate(twinmind_errors_total[5m])",
          "legendFormat": "{{error_type}}"
        }]
      },
      {
        "title": "Rate Limit Remaining",
        "type": "gauge",
        "targets": [{
          "expr": "twinmind_rate_limit_remaining"
        }]
      },
      {
        "title": "Audio Hours Processed (24h)",
        "type": "stat",
        "targets": [{
          "expr": "increase(twinmind_transcription_audio_hours[24h])"
        }]
      },
      {
        "title": "AI Tokens Used (24h)",
        "type": "stat",
        "targets": [{
          "expr": "increase(twinmind_ai_tokens_used[24h])"
        }]
      }
    ]
  }
}

// src/api/metrics.ts
import express from 'express';
import { registry } from '../observability/metrics';

const router = express.Router();

router.get('/metrics', async (req, res) => {
  res.set('Content-Type', registry.contentType);
  res.send(await registry.metrics());
});

export default router;

Issue	Cause	Solution
Missing metrics	No instrumentation	Wrap client calls
Trace gaps	Missing propagation	Check context headers
Alert storms	Wrong thresholds	Tune alert rules
High cardinality	Too many labels	Reduce label values

twinmind-observability

TwinMind Observability

Overview

Prerequisites

Metrics Collection

Key Metrics

Related Skills

flow

verify

feature-flags

flags

TwinMind Observability

Overview

Prerequisites

Metrics Collection

Key Metrics

Prometheus Metrics Implementation

Instrumented Client

Distributed Tracing

OpenTelemetry Setup

Structured Logging

Alert Configuration

Prometheus AlertManager Rules

Grafana Dashboard

Metrics Endpoint

Output

Error Handling

Resources

Next Steps

Related Skills

flow

verify

feature-flags

flags