Systematic improvement of existing agents through performance analysis, prompt engineering, and continuous iteration.
[Extended thinking: Agent optimization requires a data-driven approach combining performance metrics, user feedback analysis, and advanced prompt engineering techniques. Success depends on systematic evaluation, targeted improvements, and rigorous testing with rollback capabilities for production safety.]
Use this skill when
Improving an existing agent's performance or reliability
Analyzing failure modes, prompt quality, or tool usage
Running structured A/B tests or evaluation suites
Designing iterative optimization workflows for agents
Do not use this skill when
You are building a brand-new agent from scratch
There are no metrics, feedback, or test cases available
The task is unrelated to agent performance or prompt quality
Instructions
Establish baseline metrics and collect representative examples.
Identify failure modes and prioritize high-impact fixes.
Apply prompt and workflow improvements with measurable goals.
Validate with tests and roll out changes in controlled stages.
Safety
Avoid deploying prompt changes without regression testing.
Roll back quickly if quality or safety metrics regress.
Phase 1: Performance Analysis and Baseline Metrics
Comprehensive analysis of agent performance using context-manager for historical data collection.
Rollback Triggers:
- Success rate drops >10% from baseline
- Critical errors increase >5%
- User complaints spike
- Cost per task increases >20%
- Safety violations detected
Rollback Process:
1. Detect issue via monitoring
2. Alert team immediately
3. Switch to previous stable version
4. Analyze root cause
5. Fix and re-test before retry
4.4 Continuous Monitoring
Real-time performance tracking:
Dashboard with key metrics
Anomaly detection alerts
User feedback collection
Automated regression testing
Weekly performance reports
Success Criteria
Agent improvement is successful when:
Task success rate improves by ≥15%
User corrections decrease by ≥25%
No increase in safety violations
Response time remains within 10% of baseline
Cost per task doesn't increase >5%
Positive user feedback increases
Post-Deployment Review
After 30 days of production use:
Analyze accumulated performance data
Compare against baseline and targets
Identify new improvement opportunities
Document lessons learned
Plan next optimization cycle
Continuous Improvement Cycle
Establish regular improvement cadence:
Weekly: Monitor metrics and collect feedback
Monthly: Analyze patterns and plan improvements
Quarterly: Major version updates with new capabilities
Annually: Strategic review and architecture updates
Remember: Agent optimization is an iterative process. Each cycle builds upon previous learnings, gradually improving performance while maintaining stability and safety.
Limitations
Use this skill only when the task clearly matches the scope described above.
Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.
