Optimize context window usage and token efficiency. Use when managing costs or hitting context limits. Trigger with phrases like 'openrouter context', 'openrouter tokens', 'reduce tokens', 'context window'.
npxskills add jeremylongshore/claude-code-plugins-plus-skills--skill openrouter-context-optimizationLoading…
OpenRouter models have varying context windows (4K to 1M+ tokens). Since pricing is per-token, stuffing unnecessary context wastes money and can degrade output quality. This skill covers context window lookup, token estimation, conversation trimming, chunking strategies, and Anthropic prompt caching for large contexts.
sk-or-v1-...) exported as OPENROUTER_API_KEY — see the openrouter-install-auth skill for setuprequests for model-metadata lookup; tiktoken for exact token counting per the referencescurl and jq to query context windows and pricing from /api/v1/modelscontext_length and prompt price per 1M tokens for each candidate model, so you know the real budget before writing code.tiktoken per the references) and pick a model with select_model_for_context() from Context-Aware Model Selection — it applies an 80% safety margin and falls back through gpt-4o-mini (128K) → Claude 3.5 Sonnet (200K) → Gemini 2.0 Flash (1M).trim_conversation() per Conversation Trimming: system prompt plus the last N messages, with a trim-marker note injected where history was dropped.chunk_and_process() per Chunking for Large Documents — 8,000-char chunks with 500-char overlap, analyzed independently at temperature=0 and then synthesized.cache_control: {"type": "ephemeral"} per Prompt Caching for Repeated Context to cut repeated input cost by 90% on Anthropic models.prompt_tokens on every response (Enterprise Considerations) to catch context bloat before it becomes a 400 context_length_exceeded.# Check context window for specific models
curl -s https://openrouter.ai/api/v1/models | jq '[.data[] | select(
.id == "anthropic/claude-3.5-sonnet" or
.id == "openai/gpt-4o" or
.id == "google/gemini-2.0-flash-001" or
.id == "meta-llama/llama-3.1-70b-instruct"
) | {id, context_length, prompt_per_M: ((.pricing.prompt|tonumber)*1000000)}]'
import os, requests
from openai import OpenAI
client = OpenAI(
base_url="https://openrouter.ai/api/v1",
api_key=os.environ["OPENROUTER_API_KEY"],
default_headers={"HTTP-Referer": "https://my-app.com", "X-Title": "my-app"},
)
# Cache model metadata at startup
MODELS = {m["id"]: m for m in requests.get("https://openrouter.ai/api/v1/models").json()["data"]}
def estimate_tokens(text: str) -> int:
"""Rough estimate: 1 token ~ 4 characters for English text."""
return len(text) // 4
def select_model_for_context(messages: list, preferred: str = "anthropic/claude-3.5-sonnet") -> str:
"""Pick a model that fits the context, falling back to larger windows."""
estimated_tokens = sum(len(m.get("content", "")) for m in messages) // 4
FALLBACK_CHAIN = [
("openai/gpt-4o-mini", 128_000),
("anthropic/claude-3.5-sonnet", 200_000),
("google/gemini-2.0-flash-001", 1_000_000),
]
# Try preferred model first
preferred_ctx = MODELS.get(preferred, {}).get("context_length", 0)
if estimated_tokens < preferred_ctx * 0.8: # 80% safety margin
return preferred
for model_id, ctx in FALLBACK_CHAIN:
if estimated_tokens < ctx * 0.8:
return model_id
raise ValueError(f"Content too large ({estimated_tokens} est. tokens)")
def trim_conversation(
messages: list[dict],
max_tokens: int = 100_000,
keep_system: bool = True,
keep_last_n: int = 4,
) -> list[dict]:
"""Trim conversation history to fit context window.
Strategy: Keep system prompt + last N messages.
If still too large, reduce to last 2 messages.
"""
system = [m for m in messages if m["role"] == "system"] if keep_system else []
non_system = [m for m in messages if m["role"] != "system"]
kept = non_system[-keep_last_n:]
trimmed = non_system[:-keep_last_n] if len(non_system) > keep_last_n else []
total_est = sum(estimate_tokens(m.get("content", "")) for m in system + kept)
if total_est > max_tokens and keep_last_n > 2:
kept = non_system[-2:]
result = system + kept
if trimmed:
summary_note = {
"role": "system",
"content": f"[Previous {len(trimmed)} messages trimmed for context limits]",
}
result = system + [summary_note] + kept
return result
def chunk_and_process(document: str, question: str, model: str = "openai/gpt-4o-mini",
chunk_size: int = 8000, overlap: int = 500) -> str:
"""Process a large document in overlapping chunks, then synthesize."""
chunks = []
start = 0
while start < len(document):
chunks.append(document[start:start + chunk_size])
start += chunk_size - overlap
results = []
for i, chunk in enumerate(chunks):
response = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": f"Analyzing chunk {i+1}/{len(chunks)}."},
{"role": "user", "content": f"Document:\n{chunk}\n\nQuestion: {question}"},
],
max_tokens=1024, temperature=0,
)
results.append(response.choices[0].message.content)
# Synthesize
synthesis = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": "Synthesize these partial analyses."},
{"role": "user", "content": f"Question: {question}\n\nResults:\n" + "\n---\n".join(results)},
],
max_tokens=2048, temperature=0,
)
return synthesis.choices[0].message.content
# Anthropic models support prompt caching -- mark large static blocks
# Subsequent requests with same cached block cost 90% less for input tokens
response = client.chat.completions.create(
model="anthropic/claude-3.5-sonnet",
messages=[
{
"role": "system",
"content": [
{
"type": "text",
"text": large_reference_document, # 50K+ tokens
"cache_control": {"type": "ephemeral"},
}
],
},
{"role": "user", "content": "Summarize section 3."},
],
max_tokens=1024,
)
# First request: cache_creation_input_tokens at 1.25x rate
# Subsequent: cache_read_input_tokens at 0.1x rate (90% savings)
context_length and per-1M prompt pricing from /api/v1/modelsValueError when nothing fits[Previous N messages trimmed for context limits] note, and the most recent turnsMulti-turn chat with the references' prune_conversation() holding a 2,000-token budget — oldest messages drop as the conversation grows:
[Pruned] 9 -> 7 messages (1876 tokens)
Q: What about class-based decorators?...
Tokens: 412
The pruner always keeps the system message and removes the oldest non-system turns first. More worked examples: references/examples.md.
| Error | Cause | Fix |
|---|---|---|
400 context_length_exceeded | Input + max_tokens > model limit | Trim messages or use larger-context model |
400 max_tokens too large | max_tokens alone exceeds limit | Reduce max_tokens |
| Slow responses | Very large context | Use streaming; consider chunking |
| Degraded quality | Too much irrelevant context | Trim to relevant content only |
/api/v1/models at startup to cache context limits -- don't hardcode (they change)max_tokens on every request to prevent runaway completion costs on large contextsprompt_tokens in responses to detect context bloat before it hits limits
Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.
Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.
Set up and use 1Password CLI (op). Use when installing the CLI, enabling desktop app integration, signing in (single or multi-account), or reading/injecting/running secrets via op.
CLI to manage emails via IMAP/SMTP. Use `himalaya` to list, read, write, reply, forward, search, and organize emails from the terminal. Supports multiple accounts and message composition with MML (MIME Meta Language).
Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.
Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.
Set up and use 1Password CLI (op). Use when installing the CLI, enabling desktop app integration, signing in (single or multi-account), or reading/injecting/running secrets via op.
CLI to manage emails via IMAP/SMTP. Use `himalaya` to list, read, write, reply, forward, search, and organize emails from the terminal. Supports multiple accounts and message composition with MML (MIME Meta Language).