Integration architecture — Agentic Runtime Platform¶

This document describes every cross-package communication boundary, the data contracts at each boundary, port allocations, shared dependencies, and the end-to-end data flow from workflow execution to UI visualization to offline evaluation.

Package dependency map¶

agentic-tools  ──────────────────────────────┐
    │                                         │
    │  imports (LLMClient, benchmarks)        │  imports (LLMClient, benchmarks, lazy)
    ▼                                         ▼
agentic-workflows-v2                  agentic-v2-eval
    │
    │  REST API + WebSocket
    ▼
agentic-workflows-v2/ui

Dependency direction is always downward or lateral. agentic-workflows-v2 has no runtime import of agentic-v2-eval: the one source-level reference is agentic_v2/models/llm.py, an unreferenced adapter (omitted from coverage) that imports agentic_v2_eval.interfaces.LLMClientProtocol, and devex/workspace_test_runner.py mentions the package by name. The eval package does not import from the runtime. The UI has no Python imports; it communicates only over HTTP and WebSocket.

Runtime ↔ UI¶

Transport layer¶

Channel	Protocol	Path	Port
API requests	HTTP/JSON REST	`/api/*`	8010 (backend), proxied from 5173 in dev
Run streaming	HTTP/SSE	`/api/runs/{run_id}/stream`	8010
Execution streaming	WebSocket	`/ws/execution/{run_id}`	8010

Development proxy¶

The Vite development server (vite.config.ts) proxies all /api/ and /ws/ requests to the backend:

proxy: {
  "/api": "http://localhost:8010",
  "/ws": {
    target: "ws://localhost:8010",
    ws: true,
  },
}

The proxy target is overridden by the VITE_API_PROXY_TARGET environment variable, allowing the frontend to point at a remote or Docker-hosted backend during development.

Production static serving¶

When a production build of the frontend exists under agentic-workflows-v2/ui/dist/, the FastAPI server mounts it directly:

GET /assets/* — served from ui/dist/assets/ as static files
All other non-/api/ paths — fall through to ui/dist/index.html (SPA client-side routing)

This means a single uvicorn process can serve both the API and the UI, with no separate web server required.

REST API endpoint reference¶

All endpoints are prefixed with /api/. Authentication is controlled by AGENTIC_API_KEY (see Deployment Guide). This table is the integration-relevant subset; the full route list, including chat, settings, model finder, catalog and evaluation routes, is in Runtime API contracts.

Method	Path	Description
`GET`	`/health`	Server health check (public, no auth required)
`GET`	`/agents`	List all registered agents and their capabilities
`GET`	`/workflows`	List available workflow definitions
`GET`	`/adapters`	List registered execution engine adapters
`GET`	`/workflows/{name}/dag`	DAG nodes and edges for @xyflow/react visualization
`GET`	`/workflows/{name}/capabilities`	Workflow I/O declarations (input/output schema)
`GET`	`/workflows/{name}/editor`	Full workflow document for the visual editor
`PUT`	`/workflows/{name}`	Save an edited workflow document
`POST`	`/workflows/validate`	Validate a workflow document
`POST`	`/run`	Execute a workflow asynchronously; returns `run_id`
`GET`	`/runs`	List recent run summaries
`GET`	`/runs/summary`	Aggregate statistics across all runs
`GET`	`/runs/{filename}`	Full result for a specific run (by filename)
`GET`	`/runs/{filename}/evaluation`	In-server evaluation detail for a run
`GET`	`/runs/{run_id}/stream`	SSE stream of events for a completed or in-progress run
`GET`	`/eval/datasets`	List available evaluation datasets
`GET`	`/workflows/{name}/preview-dataset-inputs`	Preview dataset inputs for a workflow

WebSocket protocol¶

POST /api/run starts asynchronous execution and returns a run_id. The UI connects to ws://host/ws/execution/{run_id} to receive step lifecycle events in real-time.

Every frame is one event from the Pydantic discriminated union in agentic_v2/contracts/events.py — the single source of truth for the wire format (ADR-014). The type discriminator takes one of twelve values:

workflow_start · step_start · step_end · token_delta · step_complete · step_error
workflow_end · error · evaluation_start · evaluation_complete
approval_required · approval_decision

All events carry run_id and timestamp; step-scoped events add step, and the step result events (step_end, step_complete, step_error) add status, duration_ms, and optional model_used/tokens_used/tier/input/output/error fields. Example:

{
  "type": "step_end",
  "run_id": "…",
  "step": "analyze",
  "status": "success",
  "duration_ms": 1234.5,
  "timestamp": "2026-07-05T12:00:00Z"
}

The TypeScript mirror (ui/src/api/events.generated.ts) is generated from this contract; the wire-format-drift CI job blocks divergence. Client-only transport events (keepalive, connection_established) are defined by hand in ui/src/api/types.ts and are not part of the Python contract.

The useWorkflowStream hook in ui/src/hooks/useWorkflowStream.ts implements the client-side state machine that maps these events to React state updates.

Runtime ← Tools¶

agentic-workflows-v2 imports from agentic-tools in three places.

LLM client¶

Import location: agentic_v2/models/llm.py

from tools.llm.llm_client import LLMClient as LegacyClient

LegacyClient is the alias this module uses for the shared LLMClient. Nothing else in the runtime imports models/llm.py today, so neither ModelRouter nor SmartModelRouter routes through it; their provider backends are the runtime's own (models/backends_*.py). Runtime provider credentials are resolved through the SecretProvider abstraction in models/secrets.py.

LangGraph model builders¶

Import location: agentic_v2/langchain/model_builders.py

from tools.llm.llm_client import LLMClient

The LangGraph engine's model builders import the shared LLMClient lazily to back the legacy local: provider path. The import is deferred and error-guarded, so the runtime works without the shared package importable as long as no local: model is requested. (onnx:<folder> ids use the runtime's own OnnxBackend instead.)

Benchmarks and datasets¶

Import location: agentic_v2/server/datasets.py

from tools.agents.benchmarks.datasets import BENCHMARK_DEFINITIONS
from tools.agents.benchmarks.loader import load_benchmark

The server exposes evaluation datasets (listed at GET /api/eval/datasets) that are drawn from the shared benchmark definitions in tools/agents/benchmarks/. The dataset_matching.py module uses heuristics to match a named workflow to the most relevant dataset for automated evaluation.

Eval ← Tools¶

agentic-v2-eval uses lazy imports for all agentic-tools dependencies to keep the eval package installable in environments where the full tools package is not present.

LLM client (lazy import)¶

Import location: agentic_v2_eval/adapters/llm_client.py

def get_llm_client():
    from tools.llm.llm_client import LLMClient
    return LLMClient

The LLM evaluator (evaluators/llm.py) obtains the client from this adapter at evaluation time. agentic-tools is a hard dependency of the package (dependencies = ["pyyaml", "agentic-tools"]); if the import fails anyway, the adapter logs a warning and the evaluator needs an injected client.

Benchmarks (lazy import)¶

Import location: agentic_v2_eval/datasets.py

def load_benchmark_datasets():
    from tools.agents.benchmarks.datasets import get_all_datasets
    return get_all_datasets()

The eval package can reference the same benchmark datasets as the runtime server, enabling apples-to-apples comparison of model outputs against ground truth.

Eval ↔ Runtime¶

The evaluation package and the runtime package have no direct Python import relationship. They are integrated at the data level: the runtime writes run results as JSON files (default location: runs/ in the project root). The eval CLI does not read those run files directly: agentic-v2-eval evaluate takes a JSON list of per-criterion score objects (see Evaluation architecture), so a run file has to be converted into that shape first.

This separation is intentional. It allows the eval framework to be used for offline analysis, CI batch scoring, or evaluation of outputs from any source, not just the runtime.

[Runtime] → writes → runs/{run_id}.json
                              │
                              ▼
[Converter] → reads runs/{run_id}.json → writes results.json (list of score dicts)
[Eval CLI] → reads → agentic-v2-eval evaluate results.json
                              │
                              ▼
                    reports/{run_id}.{json|md|html}

The runtime's in-server scoring lives in agentic_v2/server/evaluation.py (orchestration) and the transport-free agentic_v2/scoring/ package (evaluation_scoring.py, judge.py, and friends — extracted per ADR-032). It implements scoring logic independently from the eval package and is used when the POST /api/run request includes evaluation parameters — it does not depend on agentic-v2-eval.

Shared dependencies¶

The following dependencies are declared across multiple packages and must remain version-compatible.

Dependency	Runtime (`agentic-workflows-v2`)	Eval (`agentic-v2-eval`)	Tools (`agentic-tools`, repo-root `pyproject.toml`)
`pydantic`	`>=2.0,<3`	(via `agentic-tools`)	`>=2.14.0`
`pyyaml`	`>=6.0,<7`	`>=6.0`	`>=6.0.3`
`aiohttp`	`>=3.14.3,<4`	No	`>=3.14.4`
`openai`	No (only `langchain-openai` via the `[langchain]` extra)	No	`>=3.27.0,<4`
`anthropic`	`>=0.40,<2` (optional `[claude]` extra)	No	`>=1.13.0,<2`
`numpy`	No	No	`>=2.4.6,<3`

The ranges mirror each pyproject.toml; those files and uv.lock are the source of truth.

All packages use Pydantic v2 APIs exclusively (model_dump(), model_validate(), model_fields). The legacy Pydantic v1 .dict() and .parse_obj() methods are not used anywhere.

End-to-end data flow¶

The following describes the complete lifecycle of a workflow execution request, from browser click to evaluated result.

Browser (React UI)
    │
    │  POST /api/run { workflow: "code_review", input_data: {...}, adapter: "native" }
    ▼
Sanitization ASGI middleware (server/middleware/ + agentic_v2/middleware/)
    │     ├── secrets detector
    │     ├── PII detector
    │     ├── injection detector
    │     └── unicode anomaly detector
    │     (BLOCKED requests are rejected with 422 before any route handler runs)
    ▼
FastAPI /api/run route (server/routes/workflows.py)
    │
    ├── Validates request against the workflow input schema (422 on failure)
    ├── Starts background execution task (server/execution.py)
    └── Returns { run_id: "uuid" }
    │
    │  (Browser connects to ws://.../ws/execution/{run_id})
    ▼
Background Execution (server/execution.py)
    │
    ├── Adapter dispatch (adapters/registry.py)
    │     └── native → adapters/native (DAGExecutor / PipelineExecutor)  OR  langchain → langchain/runner.py
    │
    ├── DAG step scheduling (engine/dag_executor.py — Kahn's algorithm)
    │     For each step in topological order:
    │       ├── Agent resolution (engine/agent_resolver.py)
    │       ├── Context assembly (engine/context.py)
    │       ├── Prompt assembly (engine/prompt_assembly.py)
    │       ├── LLM call (models/smart_router.py → tools/llm/llm_client.py)
    │       ├── Tool execution (engine/tool_execution.py → tools/builtin/*.py)
    │       ├── Output verification (engine/verification.py)
    │       └── WebSocket event broadcast (server/websocket.py)
    │
    ├── Result serialization → runs/{run_id}.json
    │
    └── Optional inline scoring (agentic_v2/scoring/evaluation_scoring.py)
          └── LLM judge (agentic_v2/scoring/judge.py → tools/llm/llm_client.py)
    │
    ▼
Browser receives real-time step events via WebSocket
    │
    └── React state updates (useWorkflowStream.ts)
          └── DAG node status coloring (@xyflow/react)
    │
    ▼ (Optional offline step)
Evaluation Framework (agentic-v2-eval)
    │
    ├── Load run result (JSON) and rubric (YAML)
    ├── LLM evaluator or pattern evaluator
    │     └── LLMClient (tools/llm/llm_client.py)
    └── Reporter (JSON / Markdown / HTML)

OpenTelemetry Tracing¶

When AGENTIC_TRACING=1, the native path creates these spans:

Span	Module	Description
`engine.execute`	`engine/dag_executor.py`	Top-level workflow span on `DAGExecutor.execute()`
`agent.<name>`	`agents/base.py`, `engine/agent_resolver.py`	One span per agent run

The LangGraph path emits workflow_start/workflow_end and step_start/step_complete trace events through the TraceAdapter; the OpenTelemetry adapter (integrations/tracing.py) turns them into workflow and step spans named after the event type, with step spans parented to the workflow span. TraceparentMiddleware adds W3C traceparent response headers for browser correlation. There are no dedicated llm.call or tool.execute spans.

Spans are exported via OTLP to OTEL_EXPORTER_OTLP_ENDPOINT (default: http://localhost:4317). The otel/ directory contains a Docker Compose configuration for running a local OpenTelemetry Collector.

Sensitive data (prompt text, LLM outputs, tool arguments) is excluded from spans by default. Set AGENTIC_TRACE_SENSITIVE=1 to include it.

Port allocation¶

Service	Port	Protocol	Notes
FastAPI backend	8010	HTTP / WebSocket	`--port 8010` default
Vite dev server	5173	HTTP	Frontend development server
Vite hot-module reload	5183	WebSocket	Frontend development only
OTLP gRPC collector	4317	gRPC	Default OTLP endpoint
OTLP HTTP collector	4318	HTTP	`OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf`

Default CORS origins allowed by the server (configurable via AGENTIC_CORS_ORIGINS):

http://localhost:5173
http://127.0.0.1:5173
http://localhost:8000
http://127.0.0.1:8000
http://localhost:8010
http://127.0.0.1:8010