ADR index — Agentic Runtime Platform¶
Architecture Decision Records capture the load-bearing choices behind the Agentic Runtime Platform — what was decided, what alternatives were rejected, and what it costs. Every decision is enumerated in the deck below, each with its own status (004–006 and 013 are intentionally unused number gaps). If you read only five:
- ADR-001 — the dual execution engine (LangGraph adapter alongside the native Kahn's-DAG executor).
- ADR-002 — SmartModelRouter circuit-breaker hardening (three-state breaker, adaptive cooldowns, bulkheads).
- ADR-014 — the Pydantic discriminated-union wire format that every event contract and the drift CI gate build on.
- ADR-023 — how the runtime relates to the external ExecutionKit kernel (single
executionkitpackage). - ADR-042 — the accepted sliced migration from in-tree
agentic-v2-evalto the externalagentic-evalkitframework; dependency, bridge, and scoring-cutover slices (A-C) are implemented.
Quick-access deck¶
| ADR | Title | Status | File |
|---|---|---|---|
| 001 | Dual Execution Engine (LangGraph vs. Kahn's DAG) | Accepted | ADR-001-002-003 |
| 002 | SmartModelRouter Circuit-Breaker Hardening | Accepted | ADR-001-002-003 |
| 003 | Deep Research Supervisor / CI Gating | Superseded → 007 | ADR-001-002-003 |
| 007 | Multidimensional Classification Matrix & Stop Policy | Proposed | ADR-007 |
| 008 | Testing Approach Overhaul (Value Taxonomy) | Accepted | ADR-008 |
| 009 | Scoring Enhancements (Exponential Decay, Lexicographic) | Accepted | ADR-009 |
| 010 | Commit-Driven A/B Eval Harness Methodology | Proposed | ADR-010 |
| 011 | Eval Harness API & Interface Design | Proposed | ADR-011 |
| 012 | UI Evaluation Hub & A/B Comparison | Proposed | ADR-012 |
| 014 | Pydantic Discriminated Union as Execution Event Wire Format | Accepted | ADR-014 |
| 015 | SLO Rolling Window Stored in Git | Accepted | ADR-015 |
| 016 | GitHub Models via GITHUB_TOKEN as Default E2E LLM Provider (retired upstream 2026-07-30; no CI path still defaults to it) |
Superseded | ADR-016 |
| 017 | Dataset Identifiers as Query Parameters, Not Path Segments | Accepted | ADR-017 |
| 018 | API Rate Limiting and Per-IP Auth Throttle | Accepted | ADR-018 |
| 019 | DAG Executor Top-Level Timeout Watchdog | Accepted | ADR-019 |
| 020 | LangChain Adapter Eager Validation at FastAPI Startup | Accepted | ADR-020 |
| 021 | JWT OIDC Authentication for API Routes | Accepted | ADR-021 |
| 022 | Tenant-Scoped Run and Dataset Isolation | Accepted | ADR-022 |
| 023 | ExecutionKit ↔ Runtime Execution-Contract Relationship (Option A′: single executionkit package) |
Accepted | ADR-023 · working notes |
| 024 | Expression Evaluation via AST Interpreter (eliminate eval()) |
Accepted | ADR-024 |
| 025 | SDK Task-Tool Orchestration (counterpart to the asyncio orchestrator) |
Accepted | ADR-025 |
| 026 | --resume / fork_session and the summary-seeded-session tradeoff |
Accepted | ADR-026 |
| 027 | Forced / any / auto tool_choice and cross-role verify_fact |
Accepted | ADR-027 |
| 028 | Tool descriptions are the primary tool-selection mechanism | Accepted | ADR-028 |
| 029 | Adaptive decomposition (investigate → per-file → cross-file) | Accepted | ADR-029 |
| 030 | SSRF guard pins the connection target in both modes (opt-out too) | Accepted | ADR-030 |
| 031 | Native DAG as Single Supported Execution Engine (salvaged from agentic-systems-lab; single-engine proposal NOT adopted — the LangGraph adapter is retained) |
Superseded | ADR-031 |
| 032 | Extract Scoring/Judge Domain into agentic_v2.scoring (break the server "god package", fix dependency direction) |
Accepted | ADR-032 |
| 033 | Defensive Import-Time Project-Root Resolution in Scoring Eval-Config Loader | Accepted | ADR-033 |
| 034 | Path-First File I/O Contracts for Multi-Step Workflows | Proposed | ADR-034 |
| 035 | RAG Pipeline Architecture (LanceDB + Voyage 4 Hybrid Search) | Superseded → 057 | ADR-035 |
| 036 | OllamaBackend uses the official ollama client (contained SDK swap) |
Accepted | ADR-036 |
| 037 | Live Ollama model discovery via the raw REST API (/api/tags+/api/ps, cloud via remote_host) |
Accepted | ADR-037 |
| 038 | Live discovery for LM Studio (native /api/v1/models, legacy fallbacks, multi-port) and ONNX (multi-root genai_config.json) local models |
Accepted | ADR-038 |
| 039 | Live model discovery for keyed cloud providers (OpenAI / Anthropic / Gemini / GitHub Models) | Accepted | ADR-039 |
| 040 | Curated single-source model registry (one YAML + loader feeds both engines; reconciles divergent tier chains) | Accepted | ADR-040 |
| 041 | Bounded human-approval gate timeout — a hung ApprovalProvider fails closed (DENIED) within a configurable bound (default 30 min) instead of blocking the gated tool indefinitely | Accepted | ADR-041 |
| 042 | Adopt agentic-evalkit as ARP's evaluation framework via a sliced migration; dependency, bridge, and scoring-cutover slices (A-C) shipped, in-tree package removal (D/E) sequenced behind repointing scripts/eval_gate.py off its required-check dependency on the legacy package |
Accepted | ADR-042 |
| 043 | Configurable workflow UI: per-step node config (model params / persona / observers) in the YAML schema, a JSON UI-settings store for provider endpoints + tier reranks (below env-var pins in routing precedence), self-describing DAG edges, and replay-based run comparison (POST /api/eval/compare) |
Accepted | ADR-043 |
| 044 | Evaluation scoring visibility (issue #172): loader-side golden_output_path resolution inlined as golden_output_text, loud+typed judge skips (judge_skipped/judge_skip_reason/judge_skip_code, expected_text_present, opt-in judge_required → JudgeRequiredError scoped to the evaluation), replay sample rehydration with task_id verification, and a config-backed efficiency SLO band |
Accepted | ADR-044 |
| 045 | BuildAppTool requires human approval — the build_app install/build/test shell tool joins the fail-closed HITL gate (it was ungated, contradicting the shell-approval claim) |
Accepted | ADR-045 |
| 046 | Response-path masking of high-entropy-only secrets — an opaque token matching no named pattern is now masked + classified REDACTED outbound (was CLEAN + unmasked); measured FP surface keeps structured hex/UUIDs below threshold | Accepted | ADR-046 |
| 047 | Tool approval gate enforced structurally in BaseTool.execute via __init_subclass__ (was per-dispatch-site convention, bypassable by a direct .execute()/__call__/registry call); 4 pre-gates removed for a single gate; the separate LangChain @tool system keeps its manual gate |
Accepted | ADR-047 |
| 048 | Process-wide token budget on the shared LLM client — AGENTIC_TOKEN_BUDGET now arms LLMClientWrapper.set_budget() on get_client()'s singleton (previously a dead setter with zero production callers); cumulative across the process, not per-run; skipped under AGENTIC_NO_LLM |
Accepted | ADR-048 |
| 050 | SSE chat playground endpoint (POST /api/chat, discriminated-union ChatStreamEvent) and OpenRouter as a first-class provider — joins the ADR-014 drift pipeline; rides StreamingResponse not /ws; always HTTP 200 with in-stream classified errors; bypasses SmartModelRouter/tier selection by design; OpenRouter gets cached (300s TTL) full-catalog discovery with no model_registry.yaml tier-chain entry |
Accepted | ADR-050 |
| 051 | Local-first Ollama cloud execution routing — build_ollama_model serves models present in the local daemon's /api/tags from OLLAMA_BASE_URL and routes models absent locally to ollama.com with Bearer OLLAMA_API_KEY (no key → unchanged local-only), closing ADR-037's listing-vs-execution gap; registry rungs repointed at ids providers actually list (openai:gpt-4o* → nvidia:* since OPENAI_BASE_URL is NIM-aliased in this deployment; dated sonnet → undated alias; local tails → models actually pulled) |
Accepted | ADR-051 |
| 052 | Optional typed step artifact contracts shared by both engines; code_artifact validates file maps/sentinel payloads, promotes only independently valid legacy aliases, and makes fullstack backend handoffs fail closed without changing coalesce() |
Accepted | ADR-052 |
| 054 | Evidence Ledger UI and immutable versioned model packs — warm-paper owned shadcn foundation, route splitting, Model Router consolidation, context-local pack routing, and recorded pack/model/provider provenance | Accepted | ADR-054 |
| 055 | File-size exception register — extends ADR-032's documented-exception precedent to the 12 other files over the 800-line guide (dated 2026-07-17 line counts + per-file cohesion rationale + split trigger), after the 2026-07-01 mechanical-split attempt (ARP-7, revert recorded in PR #145) caused regressions; docs/CODING_STANDARDS.md now points here instead of silently tolerating the gap |
Accepted | ADR-055 |
| 056 | Prompt-versioning registry — PromptRecord/PromptRegistry (LF-normalize then SHA-256, {declared_version}@{short_hash} fingerprint) backs load_prompt (back-compat), the engine's role-based persona lookup, and the judge's default prompt_version (was a hardcoded "judge-v1" literal that never changed); drift caught by a golden fingerprint test, not runtime warn/raise |
Accepted | ADR-056 |
| 057 | Remove the agentic_v2.rag package outright (17 modules, the agentic rag CLI group, 14 dedicated test files, the [rag] extra, docs/rag/, the non-blocking rag-extra-tests CI job) — zero non-test consumers, no deprecation shim; superseded by the standalone groundkit repo (groundkit ADR-0001, no cherry-pick back) |
Accepted | ADR-057 |
| 058 | Claude subscription backend — ClaudeSubscriptionBackend behind a new claude: prefix, registered by capability (claude-agent-sdk importable) rather than by API key, so a subscription-only operator finally gets a Claude backend instead of an Ollama-only fallback; subscription_env() blanks ANTHROPIC_API_KEY/ANTHROPIC_AUTH_TOKEN in the CLI subprocess because agentic_v2.models.secrets puts them in os.environ and the child would otherwise silently bill the API account; also repairs ClaudeSDKAgent, which raised AttributeError on every run (message.type on dataclasses that have no .type) |
Accepted | ADR-058 |
| 059 | Curated cost lane + fail-closed failover ceiling — cost_lane (local/free/paid) added to every registry entry; AGENTIC_MAX_COST_LANE filters (not reorders) get_model_candidates_for_tier's full candidate list including a pinned model_override, raising CostLaneCeilingExceededError rather than falling through unfiltered when every candidate is filtered out; _invoke_with_failover logs + records a lane_crossings run event whenever failover moves into a strictly more expensive lane. Closes the gap the SWE-fix A/B campaign hit reaching Anthropic on a run meant to spend nothing |
Accepted | ADR-059 |
| 060 | One set of scheduling rules, proved in Lean 4 — DAGExecutor and the orchestrator's _execute_plan share one rule set: a parallel limit of at least 1, plans validated before anything runs (LLM-written plans included), a failed step's dependents skipped with a recorded reason, and success only when nothing failed or was dropped (the deadlock branch now fails the run). A Lean model of DAGExecutor in proofs/ (CI-only) with a replay test tying it to the code; bounded parallelism and honest status proved, safety, no duplicate starts and completeness open (see proofs/README.md) |
Accepted | ADR-060 |
| 061 | JSON-safe secret masks — a SecretDetector mask ends before the first unescaped " when the text up to it still fully matches the pattern, so masking a secret inside a JSON string no longer eats the string's terminator and comma (the cause of lost review_reports on 2026-10-11); detection unchanged; connection_string widened to postgresql, +driver, mongodb+srv, rediss |
Accepted | ADR-061 |
| 062 | The LangGraph adapter runs each step once, after all its dependencies — a multi-dependency step gets one waiting edge (add_edge([deps], step)) instead of one edge per dependency, which ran joins once per superstep a dependency finished in (fullstack_generation's review_code ran twice, first with tests=None); a loop step with dependents exits through a synthetic __loop_exit__<step> node so they run once after the last iteration, not every iteration; WorkflowRunner hides that node; failure propagation still differs from native |
Accepted | ADR-062 |
Note: ADRs 004-006 and 013 were never created or were withdrawn; those numbering gaps are intentional and should not be reclaimed. ADR-049 and ADR-053 are claimed by parked or in-flight work (ADR-049 by a noncompensatory-gates draft in a stashed working set; ADR-053 by the memoryctl branch behind PR #205) — do not reclaim those numbers either. (The decision once numbered ADR-013 in the agentic-systems-lab fork was salvaged into this repo as ADR-031, not as 013.) ADR-023 working notes (migration plan, phase tracker, finish plan, divergence audit, preservation matrix) were moved to drafts/ on 2026-06-17 to resolve a naming collision; the canonical decision record remains at ADR-023. |
Lineage chains¶
Engine Domain:
ADR-001 (Dual Engine) ─── standalone
Models Domain:
ADR-002 (Circuit Breaker) ─── standalone
ADR-016 (GitHub Models default) ─── Superseded 2026-07-29 (upstream retirement; judge default moved to nvidia: in PR #240)
Research Domain:
ADR-003 (CI Gating) ──superseded-by──> ADR-007 (Classification Matrix)
└──extended-by──> ADR-009 (Scoring Enhancements)
Testing Domain:
ADR-008 (Test Value Taxonomy) ─── standalone
Evaluation Domain:
ADR-010 (Harness Methodology) ──extended-by──> ADR-011 (API Interface)
└──extended-by──> ADR-012 (UI Hub)
Observability Domain:
ADR-014 (Event Wire Format) ─── standalone
ADR-015 (SLO Rolling Window) ─── standalone
Evaluation Surface Domain:
ADR-017 (Dataset ID Query Params) ─── standalone (ratification)
Security / Reliability Domain (Sprint 1):
ADR-018 (Rate Limiting + Auth Throttle) ─── standalone; cluster mode deferred to Sprint 2
ADR-019 (DAG Top-Level Timeout) ─── standalone; composes additively with step-level timeouts
ADR-020 (Adapter Eager Validation) ─── standalone; narrows ADR-001 startup behavior
ADR-021 (JWT OIDC Auth) ─── extends ADR-018 auth boundary; preserves API-key fallback
ADR-022 (Tenant Isolation) ─── extends ADR-021 identity boundary with data scoping
ADR-024 (AST Expression Interpreter) ─── standalone; eliminates eval() in the engine condition evaluator
Execution Kernel Domain:
ADR-023 (EK as OpenAI-message-format kernel, Option A′) ─── builds on ADR-002 (router) + ADR-014 (additive-only wire format); single `executionkit` package; `executionkit-contracts` retired; aligns runtime LLMBackend onto EK LLMProvider via ek_adapters bridge
Workflow I/O Domain:
ADR-034 (Path-First I/O Contracts) ─── extends ADR-014 wire-format discipline to file artifact handoffs
RAG / Retrieval Domain:
ADR-035 (RAG Pipeline Architecture) ──superseded-by──> ADR-057 (Remove agentic_v2.rag Package)
(originally built on ADR-014 (Pydantic contracts) + ADR-002 (SmartModelRouter) +
ADR-019 (DAG timeout); promoted from RAG-pipeline-blueprint.md)
ADR-057 (Remove agentic_v2.rag Package) ─── zero non-test consumers, no shim;
supersedes ADR-035; superseded-by relationship is one-way — RAG functionality
moves to the standalone groundkit repo (groundkit ADR-0001), not to a new ARP ADR
Eval / Scoring Domain:
ADR-032 (Extract Scoring Package) ─── standalone; removes server "god package" coupling
ADR-033 (Eval-Config Project-Root Resolution) ─── extends ADR-032; import-time path fix
ADR-042 (Adopt agentic-evalkit, sliced migration) ─── supersedes agentic-v2-eval;
builds on ADR-032's scoring package as the landing site for the new bridge module;
follows the external-package precedent set by ADR-023 (ExecutionKit)
Implementation status¶
Scope note: This table covers the ADRs that have received a formal implementation audit: 001–024, 032–038, and 042. ADRs 025–031 and 039–041 have not yet been audited and are intentionally absent — their absence is not a statement about implementation state.
| ADR | Decision | Implemented | Tests | Last Audit |
|---|---|---|---|---|
| 001 | Yes | ~65% | Protocol + adapter tests | 2026-03-17 |
| 002 | Yes | ~80% | Extensive router/rate-limit tests | 2026-03-17 |
| 003 | Superseded | Legacy fragments only | Legacy scoring tests | 2026-03-17 |
| 007 | Yes | ~50% | Unit + wiring tests | 2026-03-17 |
| 008 | Yes | ~90% | Phase 0-3 complete: cleanup (-23), +539 new tests | 2026-03-17 |
| 009 | Yes | ~85% | CI + multidimensional scoring tests | 2026-03-17 |
| 010 | Proposed | ~10% (primitives only) | Reused primitives only | 2026-03-17 |
| 011 | Proposed | ~15% (partial eval infra) | Adjacent route/UI helpers only | 2026-03-17 |
| 012 | Proposed | ~10% (existing evaluations table only) | None specific | 2026-03-17 |
| 014 | Yes | 100% (contracts + schema-drift gate) | test_schemas.py, golden output | 2026-04-22 |
| 015 | Yes | 100% (rolling windows, nightly gate) | slo measurement tests | 2026-04-22 |
| 016 | Superseded | GITHUB_TOKEN wiring/fork-skip guards remain in code but no CI-executed path defaults to gh:*; upstream retired 2026-07-30 |
CI workflow invariants (stale) | 2026-07-29 |
| 017 | Yes | 100% (shape already live; ADR ratifies) | Sample-list route tests in tests/server/ |
2026-04-22 |
| 018 | Yes | 100% (slowapi middleware + AuthThrottle) | Rate-limit + throttle unit tests | 2026-05-14 |
| 019 | Yes | 100% (asyncio.timeout + BFS cascade) | DAG executor timeout tests | 2026-05-14 |
| 020 | Yes | 100% (validate_selected + lifespan hook) | Adapter registry startup tests | 2026-05-14 |
| 021 | Yes | 100% (OIDC JWT middleware + API-key fallback) | OIDC auth middleware tests | 2026-05-18 |
| 022 | Yes | 100% (tenant context + scoped runs/datasets) | Tenant isolation + OIDC claim tests | 2026-05-18 |
| 023 | Yes | F0–F5 landed (Option A′: single executionkit package; executionkit-contracts retired; B-1/B-2 fixed; AGENTIC_EK_PROVIDER opt-in) |
81 EK bridge tests green; flag-OFF floor held | 2026-06-01 |
| 024 | Yes | 100% (AST interpreter replaces eval() in the engine condition evaluator) | test_expressions.py (128 cases incl. dunder/DoS vectors) | 2026-06-13 |
| 032 | Yes | 100% (agentic_v2/scoring/ package live) |
scoring unit tests | 2026-06-17 |
| 033 | Yes | 100% (import-time root resolution in eval_config.py) | scoring eval-config tests | 2026-06-17 |
| 034 | Proposed | 0% (pilot on fullstack_generation workflow pending) | — | 2026-06-17 |
| 035 | Superseded | agentic_v2/rag/ removed 2026-08-10 (ADR-057); package no longer exists in the tree |
rag/ tests removed with the package | 2026-08-10 |
| 036 | Yes | 100% (OllamaBackend on ollama.AsyncClient; ollama promoted to core dep) |
test_ollama_canonical.py (SDK-stubbed) | 2026-06-21 |
| 037 | Yes | 100% (models/ollama_discovery.py raw probe; merged into enumerate_known_models; UI badges) |
test_ollama_discovery.py, test_langchain_models_unit.py, ModelFinderPage.test.tsx | 2026-06-21 |
| 038 | Yes | 100% (models/local_discovery.py LM Studio /api/v1/models + compatibility fallbacks and ONNX genai_config.json; merged into enumerate_known_models) |
test_local_discovery.py, test_langchain_models_unit.py | 2026-07-14 |
| 042 | Yes | Slices A-C: scoring/step_scoring.py scores through scoring/evalkit_bridge.py against runtime-owned rubrics in scoring/rubrics/; no legacy imports remain under agentic_v2/scoring/; unavailable evidence carries no numeric score. D: scripts/eval_gate.py scores through scoring/criterion_aggregation.py and no longer imports agentic_v2_eval. E pending — models/llm.py and the bridge's parity tests still import it; the legacy package is removed only after eval-golden-gate passes on merged main. |
test_evalkit_bridge.py; test_step_scoring_additional.py (fresh-interpreter legacy-import block + package AST scan) | 2026-09-10 |
Supporting documents¶
| Document | Description |
|---|---|
| RAG-pipeline-blueprint.md | Research blueprint and full implementation rationale for ADR-035. Retained alongside the formal ADR for deep reference. |
| drafts/ | Superseded ADR-023 working notes (migration plan, phase tracker, finish plan, divergence audit, preservation matrix). Not formal ADRs; retained for historical reference. |
Previously listed supporting audit files were removed during the 2026-04-22 docs cleanup because they had fallen out of sync with the ADRs themselves. This index is now the canonical source.