Edge Caching

Cache Eligibility & TTL

Responses are eligible for edge caching when the originating request carries a deterministic input hash and the serving model’s temperature parameter is set to 0. Non-deterministic requests — those with temperature above 0 or streaming enabled — bypass the cache entirely and are routed directly to the inference data plane. For eligible requests, a configurable TTL between 60 and 86,400 seconds governs cache lifetime, settable per API surface via the /v1/cache/policy endpoint or through the Meridian dashboard.

Cache Key Construction

The cache key is a SHA-256 hash composed of the canonicalized request body, the model identifier, the inference surface ID, and the tenant’s namespace prefix. This construction ensures complete isolation between tenants sharing the same underlying model replica. Requests that differ only in metadata fields — such as user-defined request IDs or tracing headers — resolve to the same cache key, maximizing hit rates without compromising response integrity. The full key schema is retrievable via GET /v1/cache/policy and auditable in the per-request trace log.

Invalidation & Purge API

Cache entries can be invalidated programmatically via DELETE /v1/cache/entries, accepting either a single key hash, a surface-scoped wildcard, or a full namespace flush. Purge operations propagate to all 47 edge nodes within 500 milliseconds using Meridian’s invalidation fanout bus — a purpose-built pub/sub layer separate from the data path to prevent purge storms from impacting cache-hit latency. Enterprise tenants on dedicated node allocations receive synchronous invalidation confirmation; shared-tier tenants receive eventual consistency guarantees with a documented maximum propagation bound.