Execution helper API¶
Session¶
executionkit.kit.Kit ¶
Kit(provider: LLMProvider, *, track_cost: bool = True, messages: Sequence[dict[str, Any]] | None = None, rate_limiter: TokenBucket | None = None)
Session that holds a :class:~executionkit.provider.Provider and
tracks cumulative token usage across all pattern calls.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
provider
|
LLMProvider
|
The LLM provider to use for all calls. |
required |
track_cost
|
bool
|
When |
True
|
messages
|
Sequence[dict[str, Any]] | None
|
Optional seed conversation (OpenAI-format message dicts) for
multi-turn use via :meth: |
None
|
rate_limiter
|
TokenBucket | None
|
Optional :class: |
None
|
Conversation state
:attr:messages holds the running transcript across :meth:turn calls.
The single-shot pattern methods (:meth:react, :meth:consensus, …) do
not read or write it — use :meth:turn for stateful conversations.
Source code in executionkit/kit.py
consensus
async
¶
Run the :func:~executionkit.patterns.consensus.consensus pattern.
All keyword arguments are forwarded unchanged to :func:consensus.
Source code in executionkit/kit.py
map_reduce
async
¶
Run the :func:~executionkit.patterns.map_reduce.map_reduce pattern.
All keyword arguments are forwarded unchanged to :func:map_reduce.
map_prompt_template and reduce_prompt_template are required
keyword arguments.
Source code in executionkit/kit.py
pipe
async
¶
Run :func:~executionkit.compose.pipe with this Kit's provider.
All keyword arguments are forwarded unchanged to :func:pipe.
Source code in executionkit/kit.py
react
async
¶
Run the :func:~executionkit.patterns.react_loop.react_loop pattern.
All keyword arguments are forwarded unchanged to :func:react_loop.
The provider must satisfy :class:~executionkit.provider.ToolCallingProvider;
a :exc:TypeError is raised if it does not.
Source code in executionkit/kit.py
refine
async
¶
Run the :func:~executionkit.patterns.refine_loop.refine_loop pattern.
All keyword arguments are forwarded unchanged to :func:refine_loop.
Source code in executionkit/kit.py
stream_consensus
async
¶
stream_consensus(prompt: str, *, temperature: float = _STREAM_CONSENSUS_TEMPERATURE, max_tokens: int = DEFAULT_MAX_TOKENS, max_cost: TokenUsage | None = None, trace: TraceCallback | None = None) -> StreamingPatternResult
Stream a single live completion of prompt.
Consensus voting needs complete responses to compare, so the full
pattern has no coherent token stream; this convenience method streams
one generation (no voting). Token deltas arrive live and
result.cost becomes accurate once the stream is drained, at which
point the spend is folded into this Kit's cumulative :attr:usage.
Source code in executionkit/kit.py
stream_react_loop
async
¶
stream_react_loop(prompt: str, tools: Sequence[Tool] = (), *, temperature: float = _STREAM_REACT_TEMPERATURE, max_tokens: int = DEFAULT_MAX_TOKENS, max_cost: TokenUsage | None = None, trace: TraceCallback | None = None) -> StreamingPatternResult
Stream a single live model turn for prompt.
The full ReAct loop runs tools across multiple rounds and cannot be
expressed as one token stream, so this convenience method streams a
single model generation. tools is accepted for parity with
:meth:react but is not executed (tool-call deltas carry no
message content). result.cost is accurate after the stream drains
and folds into this Kit's :attr:usage.
Source code in executionkit/kit.py
turn
async
¶
Run one conversational turn, carrying history across calls.
Appends user_text to :attr:messages, runs
:func:~executionkit.patterns.react_loop.react_loop over the full
history, then replaces :attr:messages with the returned transcript so
the next turn continues the conversation. Cost folds into
:attr:usage, exactly like :meth:react.
The provider must satisfy
:class:~executionkit.provider.ToolCallingProvider; a :exc:TypeError
is raised otherwise. If the turn fails (e.g. budget exhausted),
:attr:messages is left unchanged so the turn can be retried.
All other keyword arguments are forwarded unchanged to react_loop.
Source code in executionkit/kit.py
Kit tracks cumulative usage for calls made through its methods. Direct calls
through kit.provider are not included.
Retry, pacing, and cost¶
executionkit.engine.retry.RetryConfig
dataclass
¶
RetryConfig(max_retries: int = 3, base_delay: float = 1.0, max_delay: float = 60.0, exponential_base: float = 2.0, retryable: tuple[type[Exception], ...] = (RateLimitError, ProviderError), rate_limit_strategy: TokenBucket | None = None)
Immutable retry configuration with exponential backoff.
Attributes:
| Name | Type | Description |
|---|---|---|
max_retries |
int
|
Maximum number of retry attempts after the initial call,
so a call makes at most |
base_delay |
float
|
Base delay in seconds before first retry. |
max_delay |
float
|
Maximum delay cap in seconds. |
exponential_base |
float
|
Multiplier for exponential backoff. |
retryable |
tuple[type[Exception], ...]
|
Tuple of exception types that trigger retries. |
rate_limit_strategy |
TokenBucket | None
|
Optional |
__post_init__ ¶
Reject a negative retry budget at construction.
A negative value yields an empty attempt range, so with_retry never
calls fn at all and falls through to its unreachable guard —
a RuntimeError about the library's internals in answer to a bad
argument. structured() already rejects the same input with a
ValueError; this makes the two agree.
Source code in executionkit/engine/retry.py
get_delay ¶
Calculate jittered backoff delay for the given attempt (1-indexed).
Uses full jitter (random value in [0, capped_exponential]) to prevent thundering-herd effects when multiple coroutines retry simultaneously.
Source code in executionkit/engine/retry.py
DEFAULT_RETRY is:
max_retries counts calls after the initial attempt. The default therefore
allows at most four dispatched attempts for a retryable failure.
executionkit.engine.rate_bucket.TokenBucket
dataclass
¶
Coroutine-paced token bucket with Retry-After penalty support.
Attributes:
| Name | Type | Description |
|---|---|---|
rate |
float
|
Tokens refilled per second (sustained throughput). Must be > 0. |
capacity |
float
|
Maximum tokens held — the burst size. Must be >= 1.0 so a
single-token :meth: |
acquire
async
¶
Block until one token is available, then consume it.
Honours any active penalty window (see :meth:drain) before refilling,
so a prior Retry-After cooldown is always respected first.
Source code in executionkit/engine/rate_bucket.py
drain ¶
Force-drain the bucket as if the provider rate-limited us for seconds.
Called immediately after receiving a
:class:~executionkit.errors.RateLimitError with retry_after.
Removes seconds * rate tokens (floored at zero) and arms a penalty
deadline seconds into the future so the next :meth:acquire waits out
the full cooldown. last_refill is advanced so the refill calculation
on the next :meth:acquire does not immediately undo the drain.
Source code in executionkit/engine/rate_bucket.py
executionkit.cost.CostTracker ¶
Mutable accumulator for token and call counts.
Source code in executionkit/cost.py
add_usage ¶
Add pre-computed usage to the tracker (e.g. from a pattern result).
Use this instead of accessing private fields directly.
Source code in executionkit/cost.py
record ¶
record_without_call ¶
Record token usage from a response without incrementing the call counter.
Used by :func:~executionkit.patterns.base.checked_complete because
:meth:reserve_call (called from _before_attempt) pre-increments
_calls before the await to uphold the asyncio concurrency
contract described in the module-level docstring. Token totals are
therefore recorded here, after the provider returns, without a second
increment to the call counter.
Source code in executionkit/cost.py
reserve_call ¶
Reserve a call slot before dispatching (for budget-safe accounting).
Called by :func:~executionkit.patterns.base.checked_complete before
awaiting the provider call. Reservations intentionally count every
wire attempt, including failed attempts, so call budgets cap retries
as well as successes.
Concurrency contract: this method must be called in the same
synchronous run-segment as the preceding budget check — with no
await between them — to prevent concurrent asyncio coroutines
from racing past the check. See module-level docstring for details.
This guarantee does NOT hold under threading.
Source code in executionkit/cost.py
snapshot ¶
Return current totals as an immutable snapshot without mutating state.
Equivalent to :meth:to_usage; provided as the public, intention-revealing
name for callers that capture point-in-time deltas (e.g. per-step cost
attribution or loop checkpoint state). Calling it never advances any
counter, so two consecutive snapshots with no intervening record are
equal.
Source code in executionkit/cost.py
to_usage ¶
Return an immutable snapshot of accumulated usage.
executionkit.cost.estimate_cost ¶
Estimate the dollar cost of a :class:~executionkit.types.TokenUsage snapshot.
ExecutionKit ships no price table — rates change and differ by account tier. Pass per-token rates from your provider's pricing page.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
usage
|
TokenUsage
|
Token-usage snapshot (e.g. from :meth: |
required |
input_rate
|
float
|
Cost per input token (e.g. |
required |
output_rate
|
float
|
Cost per output token (e.g. |
required |
Returns:
| Type | Description |
|---|---|
float
|
Estimated cost in the same currency unit as the supplied rates. |
Example::
tracker = CostTracker()
# … run some patterns …
cost = estimate_cost(tracker.to_usage(), input_rate=3e-6, output_rate=15e-6)
print(f"Estimated cost: ${cost:.4f}")
Source code in executionkit/cost.py
executionkit.engine.convergence.ConvergenceDetector
dataclass
¶
ConvergenceDetector(delta_threshold: float = 0.01, patience: int = 3, score_threshold: float | None = None)
Tracks score history and detects convergence via delta + patience.
Convergence is declared when either:
- score_threshold is set and the current score meets or exceeds it, or
- The score delta has been below delta_threshold for patience
consecutive iterations.
Attributes:
| Name | Type | Description |
|---|---|---|
delta_threshold |
float
|
Minimum meaningful score improvement. |
patience |
int
|
How many consecutive stale iterations before stopping. |
score_threshold |
float | None
|
Optional absolute score target for early exit. |
reset ¶
should_stop ¶
Record a score and return whether convergence is reached.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
score
|
float
|
Evaluator score, must be in [0.0, 1.0] and not NaN. |
required |
Returns:
| Type | Description |
|---|---|
bool
|
True if the loop should stop (converged or threshold met). |
Raises:
| Type | Description |
|---|---|
ValueError
|
If score is NaN or outside [0.0, 1.0]. |
Source code in executionkit/engine/convergence.py
CostTracker is mutable and is safe for the package's asyncio accounting
sequence. It is not thread-safe.
Routing¶
executionkit.routing.RouteRule
dataclass
¶
RouteRule(name: str, provider: LLMProvider, predicate: RoutePredicate, metadata: Mapping[str, Any] = dict())
A named provider selection rule.
executionkit.routing.Router ¶
Select a provider by evaluating rules before a pattern call.
Source code in executionkit/routing.py
run
async
¶
run(pattern: RoutedPattern, prompt: str, *, context: Mapping[str, Any] | None = None, **kwargs: Any) -> PatternResult[Any]
Select a provider from context, then call pattern with it.
Routing inputs are passed explicitly via context and forwarded only to
the route predicates; **kwargs are forwarded only to pattern.
Keeping the two disjoint stops routing keys (e.g. tier) from leaking
into the pattern call — which would raise TypeError for any pattern
that does not declare them.
Source code in executionkit/routing.py
Workflow¶
executionkit.workflow.Step
dataclass
¶
Step(name: str, run: WorkflowRun, depends_on: tuple[str, ...] = (), metadata: Mapping[str, Any] = dict())
A named workflow step with optional dependencies.
executionkit.workflow.Workflow ¶
Run named steps once their dependencies are available.
Source code in executionkit/workflow.py
run
async
¶
run(initial_context: Mapping[str, Any] | None = None, *, trace: TraceCallback | None = None, approval_gate: ApprovalGate | None = None, checkpoint_fn: CheckpointFn | None = None, resume_from: WorkflowCheckpoint | None = None) -> WorkflowResult
Execute the workflow, optionally checkpointing after each batch.
Parameters¶
initial_context:
Key/value pairs added to the step context before execution. Keys
must not match workflow step names.
trace:
Optional async or sync callback receiving
:class:~executionkit.observability.TraceEvent objects.
approval_gate:
Optional gate consulted before each step runs.
checkpoint_fn:
Called with a :class:WorkflowCheckpoint after each batch of
completed steps. The caller is responsible for persisting the
checkpoint; this library imposes no storage requirement.
resume_from:
A previously saved :class:WorkflowCheckpoint. Steps whose names
appear in resume_from.outputs are skipped. Accumulated outputs
and token usage are restored. When None, execution starts from
the beginning.
Raises¶
ValueError
If a key in initial_context collides with the name of one of
this workflow's steps.
Source code in executionkit/workflow.py
177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 | |
executionkit.workflow.WorkflowCheckpoint
dataclass
¶
Snapshot of workflow state after a completed step batch.
Captures the step index (number of completed steps), accumulated
outputs, and token budget consumed so far. Both to_dict and
from_dict use only plain Python types so the caller can serialise
with any backend (JSON, pickle, database, etc.) without a dependency
on this module.
outputs
instance-attribute
¶
Accumulated step outputs, keyed by step name.
from_dict
classmethod
¶
Restore a :class:WorkflowCheckpoint from a plain-Python dict.
Raises¶
ValueError If the dict is missing a required key or contains a value of the wrong type. The message names the exact missing or malformed field so that callers can produce actionable diagnostics.
Source code in executionkit/workflow.py
to_dict ¶
Serialise to a plain-Python dict.
JSON-serializability depends on the caller ensuring that all step
outputs stored in this checkpoint are themselves JSON-safe primitives.
Non-primitive output values (e.g. custom objects, bytes) will cause
json.dumps to raise when applied to the returned dict.
Source code in executionkit/workflow.py
executionkit.workflow.WorkflowResult
dataclass
¶
Outputs and aggregate cost from a workflow run.
Planning¶
executionkit.planning.PlanStep
dataclass
¶
A human-readable executable plan step.
executionkit.planning.Plan ¶
executionkit.planning.PlanResult
dataclass
¶
Outputs and aggregate cost from plan execution.
Approval¶
executionkit.approval.ApprovalRequest
dataclass
¶
ApprovalRequest(action: str, subject: str, metadata: MappingProxyType[str, Any] = (lambda: MappingProxyType({}))())
A request to approve an operation before it runs.
executionkit.approval.ApprovalDecision
dataclass
¶
Approval result.
executionkit.approval.ApprovalGate ¶
ApprovalGate(callback: ApprovalCallback, *, timeout_seconds: float | None = None, on_timeout: Literal['approve', 'deny', 'raise'] = 'raise')
Async-compatible approval callback wrapper.
Parameters¶
callback:
Synchronous or async callable that receives an :class:ApprovalRequest
and returns an :class:ApprovalDecision (or a truthy/falsy value).
timeout_seconds:
If given, the callback must resolve within this many seconds. When it
does not, on_timeout controls the fallback behaviour. None
(default) preserves the original blocking behaviour.
on_timeout:
What to do when timeout_seconds elapses before the callback resolves.
* ``"raise"`` (default) — raise :class:`ApprovalTimeoutError`.
* ``"approve"`` — treat the timed-out request as approved.
* ``"deny"`` — treat the timed-out request as denied.
.. warning::
Setting on_timeout="approve" creates a fail-open gate: if the
approval callback does not respond in time, the operation is
automatically permitted. This is a potential privilege-escalation
vector. Only use this option when you have deliberately decided that
availability is more important than security for the guarded operation.
Source code in executionkit/approval.py
executionkit.approval.ApprovalDeniedError ¶
ApprovalDeniedError(message: str, *, cost: TokenUsage | None = None, metadata: dict[str, Any] | None = None)
Bases: ExecutionKitError
Raised when an approval-gated operation is denied.
Source code in executionkit/errors.py
executionkit.approval.ApprovalTimeoutError ¶
ApprovalTimeoutError(message: str, *, cost: TokenUsage | None = None, metadata: dict[str, Any] | None = None)
Bases: ExecutionKitError
Raised when an approval gate times out and on_timeout is "raise".
Source code in executionkit/errors.py
Tracing¶
executionkit.observability.TraceEvent
dataclass
¶
A structured event emitted during pattern and coordination calls.
TraceCallback accepts a TraceEvent and may return None or an awaitable.
executionkit.observability.emit_trace
async
¶
Emit event to an optional callback.
Source code in executionkit/observability.py
The lower-level OpenTelemetry helpers live in
executionkit.observability. They are not package-root exports:
executionkit.observability.llm_span ¶
Context manager that wraps a single LLM call in an OTel span.
When opentelemetry-api is not installed the context manager is a
no-op — the yielded value is None and no tracing infrastructure is
touched. The call path is identical in both cases so code that checks
for OTel availability lives only here.
Usage::
with llm_span(model) as span:
response = await provider.complete(messages)
record_llm_span_attributes(span, model, response)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
str
|
The model identifier string (e.g. |
required |
Yields:
| Type | Description |
|---|---|
Any
|
An OTel |
Source code in executionkit/observability.py
executionkit.observability.record_llm_span_attributes ¶
record_llm_span_attributes(span: Any, model: str, input_tokens: int, output_tokens: int, cost_usd: float | None = None) -> None
Set standard LLM attributes on an active OTel span.
This is a no-op when span is None (OTel absent) or when the span
is non-recording (e.g. a sampled-out or no-op span from the SDK).
Attributes set:
llm.model— already set by :func:llm_span; repeated here for completeness if callers construct spans themselves.llm.input_tokens— prompt / input token count.llm.output_tokens— completion / output token count.cost_usd— estimated dollar cost; omitted whenNoneso we never emit a fabricated value when no rate table is available.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
span
|
Any
|
The span returned by the :func: |
required |
model
|
str
|
Model identifier string. |
required |
input_tokens
|
int
|
Number of input (prompt) tokens consumed. |
required |
output_tokens
|
int
|
Number of output (completion) tokens produced. |
required |
cost_usd
|
float | None
|
Optional pre-computed cost in USD. Pass |
None
|
Source code in executionkit/observability.py
Exceptions¶
All package exceptions derive from ExecutionKitError. Each instance carries
cost and metadata.
ExecutionKitError
├── LLMError
│ ├── RateLimitError
│ ├── PermanentError
│ └── ProviderError
└── PatternError
├── BudgetExhaustedError
├── ConsensusFailedError
└── MaxIterationsError
ApprovalDeniedError and ApprovalTimeoutError also derive from
ExecutionKitError.
executionkit.errors.ExecutionKitError ¶
ExecutionKitError(message: str, *, cost: TokenUsage | None = None, metadata: dict[str, Any] | None = None)
Bases: Exception
Base exception for all ExecutionKit errors.
Source code in executionkit/errors.py
executionkit.errors.LLMError ¶
Bases: ExecutionKitError
Errors originating from LLM provider communication.
Source code in executionkit/errors.py
executionkit.errors.RateLimitError ¶
RateLimitError(message: str, *, retry_after: float = 1.0, cost: TokenUsage | None = None, metadata: dict[str, Any] | None = None)
Bases: LLMError
Provider returned HTTP 429 — retryable after retry_after seconds.
Source code in executionkit/errors.py
executionkit.errors.PermanentError ¶
PermanentError(message: str, *, cost: TokenUsage | None = None, metadata: dict[str, Any] | None = None)
Bases: LLMError
Non-retryable provider error (e.g. 401 authentication failure).
Source code in executionkit/errors.py
executionkit.errors.ProviderError ¶
ProviderError(message: str, *, cost: TokenUsage | None = None, metadata: dict[str, Any] | None = None)
Bases: LLMError
Catch-all retryable provider error for unexpected HTTP failures.
Source code in executionkit/errors.py
executionkit.errors.PatternError ¶
PatternError(message: str, *, cost: TokenUsage | None = None, metadata: dict[str, Any] | None = None)
Bases: ExecutionKitError
Errors raised by reasoning pattern logic.
Source code in executionkit/errors.py
executionkit.errors.BudgetExhaustedError ¶
BudgetExhaustedError(message: str, *, cost: TokenUsage | None = None, metadata: dict[str, Any] | None = None)
executionkit.errors.ConsensusFailedError ¶
ConsensusFailedError(message: str, *, cost: TokenUsage | None = None, metadata: dict[str, Any] | None = None)
Bases: PatternError
Consensus pattern could not reach agreement.
Source code in executionkit/errors.py
executionkit.errors.MaxIterationsError ¶
MaxIterationsError(message: str, *, cost: TokenUsage | None = None, metadata: dict[str, Any] | None = None)
Bases: PatternError
Loop pattern exceeded its iteration limit.