Cost optimisation — caching, batching, routing

Three ways to see it

1. 1

   Lever one: prompt caching. Anthropic and OpenAI both let you mark large stable parts of a prompt (a long system instruction, a policy manual, a glossary) as cached. The first request pays the full price to load it. Every subsequent request that reuses the cache pays only 10% for that portion. For a customer-service bot whose system prompt is 8000 tokens and is reused on every call, caching brings the input bill down by roughly 70-80%. The cache lives for five minutes by default and refreshes on each use. Setup is one extra parameter in the API call.

2. 2

   Lever two: batch API. If your work is not time-sensitive — overnight processing of a thousand FBR returns, monthly summarisation of all PIA flight reports, end-of-quarter audit document review — submit it through the batch endpoint. Anthropic and OpenAI both offer batch processing at 50% off, with results returned within 24 hours. The discount is exactly the same content. The only thing you give up is the right to see the answer in 5 seconds. For any pipeline where humans review results the next morning, this is free money.

3. 3

   Lever three: model routing. Not every question needs your most expensive model. Route easy, structured queries (classification, extraction, simple Q&A) to Claude Haiku or GPT-4o-mini at one-tenth the cost. Route complex reasoning to Sonnet or Opus only when the cheaper model fails or below a confidence threshold. A simple if-else in your code or a tiny router model in front of the request chain can shift 60-70% of traffic to the cheap lane. JazzCash and Easypaisa both run pyramidal routing internally for exactly this reason.

The why-tree

Try this with Claude

AI-edge prompt: 'Here is my current prompt: [paste]. It runs 1000 times a day. Rewrite it so the stable parts go in a cache block and the variable parts stay outside. Mark the cache_control breakpoints. Estimate input-token savings as a percentage.' Compare what the model returns with the Anthropic prompt-caching documentation; the structure should match.