Prompt Caching in the API

OpenAI Blog · 1 year ago

Anthropic added prompt caching to its API, which reduces costs for repeated input sections by storing them in the model's cache. Cached tokens cost 90% less than standard tokens after a 1,024-token minimum, and cached content remains available for five minutes. Users can lower expenses for applications that process the same documents, codebases, or instructions repeatedly.