Load proof: Redis-CAS + horizontal scale¶

This page turns ARP's Redis-CAS circuit-breaker and horizontal-scale engineering from described in code into proven with real numbers. The proof runs locally and free (the grafana/k6 image hammering a multi-replica docker compose stack with AGENTIC_NO_LLM=1); only the resulting JSON is published here.

Topology¶

flowchart TD
    K6["k6 load
(grafana/k6 container)"] --> LB["nginx
round-robin lb"]
    subgraph replicas["Horizontally-scaled ARP replicas"]
        R1["ARP replica 1
FastAPI · AGENTIC_NO_LLM=1"]
        R2["ARP replica 2
FastAPI · AGENTIC_NO_LLM=1"]
        R3["ARP replica 3
FastAPI · AGENTIC_NO_LLM=1"]
    end
    LB --> R1
    LB --> R2
    LB --> R3
    R1 -.CAS.-> REDIS
    R2 -.CAS.-> REDIS
    R3 -.CAS.-> REDIS
    REDIS[("Redis
shared circuit-breaker state")]

Redis is external shared state on purpose. Each ARP replica keeps an in-process circuit-breaker tally, but the authoritative per-model counters live in Redis. When a replica records a success/failure it persists only its delta via an atomic Compare-And-Swap (read-modify-write) keyed on the current value. If two replicas write concurrently, the CAS loser re-reads and re-merges its delta on top of the winner's value, so the persisted counter reflects every replica's contribution. That is why the shared counter stays exactly consistent under concurrent multi-replica load — no double-count, no lost update — which a per-replica in-memory tally could never guarantee.

Scale / throughput: 1 vs N replicas¶

Both runs use the identical ramping-VU k6 profile against the same nginx round-robin; the only variable is the replica count, so the throughput delta is attributable to horizontal scale.

Redis-CAS shared-counter consistency¶

Live exact-sum CAS experiment — 4 independent workers (each its own router + Redis store, all sharing the one Redis the replicas use) each recorded 50 failures on the same model and persisted concurrently through the production CAS read-modify-write path.

The CAS-pressure k6 scenario issued 601 orchestration requests across the replicas (601 accepted), generating the concurrent multi-replica write pressure that the experiment runs under.

How to reproduce¶

# from the repo root (Docker required; no host k6 install needed)
bash load/run_load.sh 3                 # 1-replica + 3-replica + CAS run
python scripts/build_load_report.py     # regenerate THIS page from JSON
mkdocs build                            # render the docs site

The load run is local and free; CI only renders the committed load/results/*.json, so GitHub Pages stays $0 and deterministic.