Skip to main content GPT Researcher is an autonomous deep research agent that conducts web and local research, producing detailed reports with citations. Use this skill when helping developers understand, extend, debug, or integrate with GPT Researcher - including adding features, understanding the architecture, working with the API, customizing research workflows, adding new retrievers, integrating MCP data sources, or troubleshooting research pipelines.
npx skills add assafelovic/gpt-researcher --skill gpt-researcher ai python agent automation research search
GPT Researcher Development Skill
GPT Researcher is an LLM-based autonomous agent using a planner-executor-publisher pattern with parallelized agent work for speed and reliability.
Quick Start
Basic Python Usage
from gpt_researcher import GPTResearcher
import asyncio
async def main():
researcher = GPTResearcher(
query="What are the latest AI developments?",
report_type="research_report", # or detailed_report, deep, outline_report
report_source="web", # or local, hybrid
)
await researcher.conduct_research()
report = await researcher.write_report()
print(report)
asyncio.run(main())
Run Servers
# Backend
python -m uvicorn backend.server.server:app --reload --port 8000
# Frontend
cd frontend/nextjs && npm install && npm run dev
Key File Locations
Need Primary File Key Classes Main orchestrator gpt_researcher/agent.py
Research logic gpt_researcher/skills/researcher.pyResearchConductor
Report writing gpt_researcher/skills/writer.pyReportGenerator
All prompts gpt_researcher/prompts.pyPromptFamily
Configuration gpt_researcher/config/config.pyConfig
Config defaults gpt_researcher/config/variables/default.pyDEFAULT_CONFIG
API server backend/server/app.pyFastAPI app
Search engines gpt_researcher/retrievers/Various retrievers
Architecture Overview User Query → GPTResearcher.__init__()
│
▼
choose_agent() → (agent_type, role_prompt)
│
▼
ResearchConductor.conduct_research()
├── plan_research() → sub_queries
├── For each sub_query:
│ └── _process_sub_query() → context
└── Aggregate contexts
│
▼
[Optional] ImageGenerator.plan_and_generate_images()
│
▼
ReportGenerator.write_report() → Markdown report
Core Patterns
Adding a New Feature (8-Step Pattern)
Config → Add to gpt_researcher/config/variables/default.pyProvider → Create in gpt_researcher/llm_provider/my_feature/Skill → Create in gpt_researcher/skills/my_feature.pyAgent → Integrate in gpt_researcher/agent.pyPrompts → Update gpt_researcher/prompts.pyWebSocket → Events via stream_output()Frontend → Handle events in useWebSocket.tsDocs → Create docs/docs/gpt-researcher/gptr/my_feature.md
Adding a New Retriever # 1. Create: gpt_researcher/retrievers/my_retriever/my_retriever.py
class MyRetriever:
def __init__(self, query: str, headers: dict = None):
self.query = query
async def search(self, max_results: int = 10) -> list[dict]:
# Return: [{"title": str, "href": str, "body": str}]
pass
# 2. Register in gpt_researcher/actions/retriever.py
case "my_retriever":
from gpt_researcher.retrievers.my_retriever import MyRetriever
return MyRetriever
# 3. Export in gpt_researcher/retrievers/__init__.py
Configuration Config keys are lowercased when accessed:
# In default.py: "SMART_LLM": "gpt-4o"
# Access as: self.cfg.smart_llm # lowercase!
Priority: Environment Variables → JSON Config File → Default Values
Common Integration Points
WebSocket Streaming class WebSocketHandler:
async def send_json(self, data):
print(f"[{data['type']}] {data.get('output', '')}")
researcher = GPTResearcher(query="...", websocket=WebSocketHandler())
MCP Data Sources researcher = GPTResearcher(
query="Open source AI projects",
mcp_configs=[{
"name": "github",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {"GITHUB_TOKEN": os.getenv("GITHUB_TOKEN")}
}],
mcp_strategy="deep", # or "fast", "disabled"
)
Deep Research Mode researcher = GPTResearcher(
query="Comprehensive analysis of quantum computing",
report_type="deep", # Triggers recursive tree-like exploration
)
Error Handling Always use graceful degradation in skills:
async def execute(self, ...):
if not self.is_enabled():
return [] # Don't crash
try:
result = await self.provider.execute(...)
return result
except Exception as e:
await stream_output("logs", "error", f"⚠️ {e}", self.websocket)
return [] # Graceful degradation
Critical Gotchas ❌ Mistake ✅ Correct config.MY_VARconfig.my_var (lowercased)Editing pip-installed package pip install -e .Forgetting async/await All research methods are async websocket.send_json() on NoneCheck if websocket: first Not registering retriever Add to retriever.py match statement
Reference Documentation Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.
Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.
Set up and use 1Password CLI (op). Use when installing the CLI, enabling desktop app integration, signing in (single or multi-account), or reading/injecting/running secrets via op.
CLI to manage emails via IMAP/SMTP. Use `himalaya` to list, read, write, reply, forward, search, and organize emails from the terminal. Supports multiple accounts and message composition with MML (MIME Meta Language).
Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.
Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.
Set up and use 1Password CLI (op). Use when installing the CLI, enabling desktop app integration, signing in (single or multi-account), or reading/injecting/running secrets via op.
CLI to manage emails via IMAP/SMTP. Use `himalaya` to list, read, write, reply, forward, search, and organize emails from the terminal. Supports multiple accounts and message composition with MML (MIME Meta Language).
