Skip to main content
The Gateway is the zero-downtime query entry point. Size its replica count and memory for steady-state traffic plus retries during Engine cutovers.

​
Memory budget

The Gateway can buffer up to 2 MiB per connection so it can replay eligible requests. Estimate peak buffering per Gateway pod as:
Add Envoy process overhead and operational headroom to this estimate. Increase spec.gateway.replicas and the Envoy container memory limit together when concurrency grows:
An out-of-memory Gateway restart can cause client-visible failures. Monitor container memory, restarts, and Envoy response flags under peak load. Requests larger than 2 MiB are not buffered for retry. Split large requests client-side when they must tolerate cutovers.

​
Per-Engine circuit breakers

Each Gateway pod applies these limits independently to each Engine: Total capacity grows with Gateway replicas. If expected concurrency for one Engine approaches 1024 requests per Gateway pod, add Gateway replicas and verify Engine capacity as well. A tripped circuit breaker appears as an Envoy-generated 503 with response flag UO in the access log. URX means Envoy exhausted the configured retry limit and can accompany repeated upstream failures, but does not by itself identify circuit-breaker saturation. Check the target Engine, Gateway CPU and memory, upstream latency, request concurrency, and the surrounding response flags before adding capacity.