Set up comprehensive observability for CodeRabbit integrations with metrics, traces, and alerts.
Use when implementing monitoring for CodeRabbit operations, setting up dashboards,
or configuring alerting for CodeRabbit integration health.
Trigger with phrases like "coderabbit monitoring", "coderabbit metrics",
"coderabbit observability", "monitor coderabbit", "coderabbit alerts", "coderabbit tracing".
Monitor CodeRabbit AI code review effectiveness, review latency, and team adoption. Key metrics include time-to-first-review (how fast CodeRabbit posts after PR creation), comment acceptance rate (comments resolved vs dismissed), review coverage (percentage of PRs reviewed), and per-repository review volume.
Prerequisites
CodeRabbit installed on GitHub/GitLab organization
GitHub CLI (gh) authenticated with org access
Access to CodeRabbit dashboard at app.coderabbit.ai
Key Metrics
Metric
Target
Why It Matters
Review coverage
> 90%
PRs without review = blind spots
Time-to-review
< 5 min
Fast feedback keeps developers in flow
Comment acceptance
> 40%
Low acceptance = noisy reviews
Comments per PR
3-8
Too many = fatigue, too few = not useful
Review state: APPROVED
> 60%
High approval = clean code culture
Instructions
Step 1: Measure Review Coverage
#!/bin/bash
# coderabbit-coverage.sh - Review coverage for a repo
set -euo pipefail
ORG="${1:?Usage: $0 <org> <repo> [days]}"
REPO="${2:?Usage: $0 <org> <repo> [days]}"
DAYS="${3:-30}"
echo "=== CodeRabbit Review Coverage ==="
echo "Repository: $ORG/$REPO"
echo "Period: Last $DAYS days"
echo ""
TOTAL=0
REVIEWED=0
APPROVED=0
CHANGES_REQUESTED=0
SINCE=$(date -d "$DAYS days ago" +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -v-${DAYS}d +%Y-%m-%dT%H:%M:%SZ)
for PR_NUM in $(gh api "repos/$ORG/$REPO/pulls?state=all&per_page=50&sort=created&direction=desc" \
--jq ".[] | select(.created_at > \"$SINCE\") | .number"); do
TOTAL=$((TOTAL + 1))
CR_STATE=$(gh api "repos/$ORG/$REPO/pulls/$PR_NUM/reviews" \
--jq '[.[] | select(.user.login=="coderabbitai[bot]")] | last | .state // "none"' 2>/dev/null || echo "none")
if [ "$CR_STATE" != "none" ] && [ "$CR_STATE" != "null" ]; then
REVIEWED=$((REVIEWED + 1))
[ "$CR_STATE" = "APPROVED" ] && APPROVED=$((APPROVED + 1))
[ "$CR_STATE" = "CHANGES_REQUESTED" ] && CHANGES_REQUESTED=$((CHANGES_REQUESTED + 1))
fi
done
if [ "$TOTAL" -gt 0 ]; then
echo "Total PRs: $TOTAL"
echo "Reviewed by CodeRabbit: $REVIEWED ($(( REVIEWED * 100 / TOTAL ))%)"
echo " Approved: $APPROVED"
echo " Changes Requested: $CHANGES_REQUESTED"
else
echo "No PRs found in the last $DAYS days"
fi
Step 2: Track Comment Volume and Acceptance
set -euo pipefail
ORG="${1:-your-org}"
REPO="${2:-your-repo}"
echo "=== CodeRabbit Comment Analysis ==="
echo ""
TOTAL_COMMENTS=0
PR_COUNT=0
for PR_NUM in $(gh api "repos/$ORG/$REPO/pulls?state=closed&per_page=20" --jq '.[].number'); do
COMMENTS=$(gh api "repos/$ORG/$REPO/pulls/$PR_NUM/comments" \
--jq '[.[] | select(.user.login=="coderabbitai[bot]")] | length' 2>/dev/null || echo "0")
if [ "$COMMENTS" -gt 0 ]; then
TOTAL_COMMENTS=$((TOTAL_COMMENTS + COMMENTS))
PR_COUNT=$((PR_COUNT + 1))
echo "PR #$PR_NUM: $COMMENTS comments"
fi
done
if [ "$PR_COUNT" -gt 0 ]; then
echo ""
echo "Average comments per PR: $(( TOTAL_COMMENTS / PR_COUNT ))"
echo ""
echo "Healthy ranges:"
echo " 1-3 comments/PR → Profile may be too chill"
echo " 3-8 comments/PR → Good signal-to-noise ratio"
echo " 10+ comments/PR → Consider switching to chill profile"
fi
