Token EconomicsIntermediate
Context Caching vs Prompt Caching: Long Context Storage in Gemini and Claude
Direct Answer & Overview
A technical comparison between Anthropic's dynamic prefix Prompt Caching and Google Gemini's explicit Context Caching, detailing TTL storage fees, minimum token thresholds, and architectural integration.
1.Anthropic Prompt Caching: Dynamic Automatic Invalidation
Anthropic's prompt caching relies on prefix matching. Developers place a `cache_control: {"type": "ephemeral"}` breakpoint in system messages or tools. When subsequent requests share that exact prefix, Anthropic charges 90% less for input tokens. The cache lasts for 5 minutes and is refreshed automatically with every cache hit, requiring zero upfront storage commitment.
2.Google Gemini Context Caching: Explicit TTL Storage
Google Gemini takes an explicit database-style approach. Developers call a dedicated `/v1beta/cachedContents` endpoint to pre-store a large dataset (minimum 32,768 tokens) with a specified Time-To-Live (e.g., 2 hours). Queries referencing that cache URI pay 75% lower input fees, plus an hourly storage charge (e.g. $1.00 per 1M tokens per hour).
3.Which Caching Model Suits Your Workload?
• Dynamic Multi-Turn Chat (Claude): Ideal when conversations have frequent continuous interaction, resetting after user disconnects.
• Heavy Document / Video Analysis (Gemini): Ideal when multiple independent users will query a 500-page enterprise handbook or 1-hour video over an entire working day.
Configuring Anthropic Ephemeral Prompt Cache Breakpointpython
import anthropic
client = anthropic.Anthropic()
response = client.messages.create(
model="claude-3-5-sonnet-20241022",
max_tokens=1024,
system=[
{
"type": "text",
"text": "Here is the 100-page enterprise compliance manual: ... [90k tokens of text]",
# Mark for 90% discount on subsequent requests
"cache_control": {"type": "ephemeral"}
}
],
messages=[{"role": "user", "content": "What is the policy for overseas data transfers?"}]
)
print("Cached Tokens Read:", response.usage.cache_read_input_tokens)Frequently Asked Questions
What is the minimum token count for prompt caching?
Anthropic requires a minimum of 1,024 tokens for Claude 3.5 Sonnet (2,048 for Haiku). Gemini requires 32,768 tokens for explicit context caching.
How much money does prompt caching save?
Prompt caching reduces cached input token costs by up to 90% ($0.30 per 1M tokens on Sonnet compared to $3.00 standard).
Does changing a single word invalidate the cache?
Yes. Because caching relies on prefix hash matching, altering any token before the cache breakpoint invalidates the entire downstream cache.
A100
API100 Engineering Team
Infrastructure & Latency Research