llm-serving-patterns by benchflow-ai

llm-serving-patterns

benchflow-ai

9372423 months ago

LLM inference infrastructure, serving frameworks (vLLM, TGI, TensorRT-LLM), quantization techniques, batching strategies, and streaming response patterns. Use when designing LLM serving infrastructure, optimizing inference latency, or scaling LLM deployments.

>_
Quick Install
npxskills add benchflow-ai/skillsbench--skill llm-serving-patterns
Instructions

No instructions available

Tags & Topics
Read,Glob,Grep