Get Started

Prompt Caching and Context Caching

Your system prompt is 4,000 tokens. Your RAG context is 20,000 tokens. You send both with every request. You also pay for both, every time. Prompt caching lets the provider keep that prefix warm on their side and bill you 10% of the normal rate on reuse. Used correctly, it cuts inference cost by 50 to 90% and first-token latency by 40 to 85%.

60 minutesBuildPythonMembers