Answer¶
The composition root behind grk answer (ADR-0019; Phase 5 of SPEC.md §9):
an optional query rewrite, a retrieval search, cited synthesis, and an
optional advisory faithfulness judge, wired into one call and one report.
Every collaborator instance is injected rather than constructed here —
resolving a chat config, constructing a concrete chat provider, opening a
Retriever over a persisted collection is the caller's job, so
AnswerPipeline itself never imports groundkit.config or
groundkit.providers.llm. The types are concrete rather than abstract, and
this page does not claim otherwise: only the search collaborator is a
structural Protocol, while the synthesizer, rewriter and judge are the named
classes, which the module therefore imports. All three are ChatProtocol
consumers living under providers, so the substitution a caller wants is
reached by injecting a different chat provider into them — and composing them
here costs the module no dependency on the eval harness.
grk answer deliberately does not reach the service
surface (ADR-0019) — the read-only REST API and MCP server never run
synthesis, so this module's contract has no counterpart there.
Abstention has one representation, and this module doesn't add a second:
AnswerReport carries no separate abstained field. An empty citations
tuple on a synthesized answer already is abstention, and a second boolean
here would only be a second, potentially disagreeing, way to say the same
thing.
When a faithfulness judge is injected, it runs after synthesis on
everything retrieval returned, not only the subset the answer ended up
citing, and its verdict is recorded unconditionally — including when the
answer abstained. If the judge itself raises, that error propagates out of
AnswerPipeline.answer uncaught: a requested component that breaks is a
typed failure, not a silently-skipped opinion (ADR-0018 decision 5).
groundkit.answer ¶
Phase 5 answer composition: rewrite, retrieve, synthesize, judge (SPEC.md §9).
This is the composition root behind the future grk answer CLI verb
(ADR-0019): the code that wires an optional query rewrite, a retrieval
search, a cited synthesis, and an optional advisory faithfulness judge into
one call and one report. Building the collaborators it composes — resolving
a chat config, constructing a concrete chat provider, opening a
Retriever over a persisted collection — is deliberately not this
module's job: this module never imports groundkit.config or
groundkit.providers.llm, and constructs no provider itself.
What "injected" does and does not mean here¶
Every collaborator instance is supplied by the caller — none is
constructed, discovered, or defaulted here. The types are not
correspondingly abstract, and nothing below claims they are: only
search is a structural Protocol (:class:SearchCallable, and even that
exists so this module's own tests need not build a real Retriever,
not as a declared extension point). synthesizer, rewriter and
judge are annotated as the concrete
:class:~groundkit.providers.synthesis.Synthesizer,
:class:~groundkit.providers.query_rewrite.QueryRewriter and
:class:~groundkit.providers.judge.FaithfulnessJudge classes, which this
module therefore imports; a substitute must be an instance of the class,
not merely something with a matching shape.
That is a deliberate trade rather than an oversight. All three are
themselves :class:~groundkit.providers.protocols.ChatProtocol consumers
whose own seam is the chat provider, so the substitution a caller actually
wants — a different model, a scripted fake, a redacting wrapper — is
reached by injecting a different ChatProtocol into them, and a second
Protocol layer here would only restate that seam one level out. All three
also now live together under groundkit.providers (GK-021 moved the
judge there from groundkit.evals), so importing them commits this
module to no dependency on the eval harness.
Abstention has one representation¶
:class:~groundkit.providers.synthesis.SynthesizedAnswer has no
abstained field — an empty citations tuple is abstention
(docs/specs/phase-5-boundary-features.md §6, left open there as Q2 and
settled here for this pipeline). :class:AnswerReport does not add a
second, redundant boolean: it exposes exactly the same rule its
citations field already carries, unchanged, all the way through.
The judge is advisory, never a gate, and never special-cases abstention¶
When a :class:~groundkit.providers.judge.FaithfulnessJudge is injected, it
runs after synthesis and its verdict is recorded — unconditionally, with no
attempt to detect or skip judging an abstained answer, because "abstained"
is not a branch this module takes, only a shape citations happens to
have. If judging is requested and the judge raises
:class:~groundkit.errors.JudgeError (or anything else), that error
propagates out of :meth:AnswerPipeline.answer uncaught: a requested
component that breaks is a typed failure, not a silently-skipped opinion
(ADR-0018 decision 5). "Advisory" describes what a successful verdict is
permitted to influence — nothing here inspects verdict.faithful to
change the answer or the citations — not whether a broken judge is allowed
to fail quietly.
What the judge is shown¶
The judge is given every retrieved result's content, in the same rank order the synthesizer was shown them — not only the subset of results the answer ended up citing. ADR-0018 frames the judge's question as "whether every claim in the Answer is supported by at least one of the numbered Sources", and the synthesis prompt already presented the full retrieved set as those sources. Narrowing the judge to only the cited subset would judge the answer against a smaller context than the one it was actually generated from, and would make "was this text in front of the model when it wrote that claim" a question the verdict cannot actually answer.
Query rewrite makes two strings, and both survive¶
When a :class:~groundkit.providers.query_rewrite.QueryRewriter is
injected, retrieval runs against its rewritten output, but synthesis and
judging run against the caller's original query text — the rewrite exists
to help retrieval match, not to replace the question the answer must
actually address. :class:AnswerReport records both strings explicitly
(query and rewritten_query) rather than picking one, which is
exactly the ambiguity ADR-0019 §2 says a single-query-field model like
SearchResponse cannot express.
Fail-closed, single source of truth¶
An empty or whitespace-only query is rejected before any collaborator
runs — no rewrite, no search, no synthesis call is ever made for it. A
search that comes back with zero results is not separately checked
here: :class:~groundkit.providers.synthesis.Synthesizer already raises
:class:~groundkit.errors.SynthesisError on an empty result set, and
duplicating that check here would create two sources of truth for one
rule. Every other collaborator failure — a rewrite error, a search error, a
synthesis error, a judge error — propagates unmodified: this module neither
catches nor wraps any of them.
No exception raised directly by this module interpolates query, answer, or retrieved chunk text.
AnswerPipeline ¶
AnswerPipeline(
search: SearchCallable,
synthesizer: Synthesizer,
*,
rewriter: QueryRewriter | None = None,
judge: FaithfulnessJudge | None = None,
)
Composes rewrite, retrieval, synthesis, and an advisory judge into one call.
Every collaborator instance is supplied by the caller — this class
builds none of them, imports no configuration, and constructs no
provider. Their types are concrete rather than abstract, deliberately;
see the module docstring for what that does and does not buy. A class
(rather than a bare function) because every optional collaborator it
holds — rewriter, judge — is fixed for the lifetime of a caller's
usage (one grk answer invocation, one eval run) and this repo's
existing Phase 5 collaborators (Synthesizer, QueryRewriter,
FaithfulnessJudge) are themselves constructor-injected classes; a
function would need every collaborator re-passed on each call instead of
bound once. See the module docstring for the rewrite/synthesis query
split, the judge's input set, and the fail-closed precondition this
class itself enforces.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
search
|
SearchCallable
|
The retrieval collaborator, e.g. a
:class: |
required |
synthesizer
|
Synthesizer
|
Produces the cited answer from the query and the search results. |
required |
rewriter
|
QueryRewriter | None
|
Optional. When supplied, every :meth: |
None
|
judge
|
FaithfulnessJudge | None
|
Optional. When supplied, every :meth: |
None
|
Source code in src/groundkit/answer.py
answer
async
¶
Compose one answer for query.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
query
|
str
|
The caller's question. Rejected before any collaborator runs if empty or whitespace-only. |
required |
top_k
|
int | None
|
Result cap forwarded to |
None
|
mode
|
SearchMode
|
Search mode forwarded to |
'bm25'
|
Returns:
| Type | Description |
|---|---|
AnswerReport
|
The composed :class: |
Raises:
| Type | Description |
|---|---|
RetrievalError
|
|
QueryRewriteError
|
Propagated unmodified from |
ChatError
|
Propagated unmodified from any collaborator's underlying chat provider call. |
ChatProviderNotConfiguredError
|
Propagated unmodified, as above
(a |
SynthesisError
|
Propagated unmodified from |
JudgeError
|
Propagated unmodified from |
Source code in src/groundkit/answer.py
AnswerReport ¶
Bases: BaseModel
The full record of one grk answer composition (ADR-0019).
Attributes:
| Name | Type | Description |
|---|---|---|
query |
str
|
The caller's original query text, recorded verbatim regardless of whether a rewrite ran. |
rewritten_query |
str | None
|
The rewritten query text, when a
:class: |
answer |
str
|
The synthesized answer text, citation markers intact. |
citations |
tuple[Citation, ...]
|
The cited results' verifiable
:class: |
results |
tuple[RetrievalResult, ...]
|
Every result that was retrieved and therefore available to synthesis (and, if judged, to the judge), in the order retrieval returned them — what was retrieved, so the answer is auditable against its actual inputs rather than only against the subset it happened to cite. |
verdict |
FaithfulnessVerdict | None
|
The advisory faithfulness verdict, when a
:class: |
SearchCallable ¶
Bases: Protocol
Structural shape of the retrieval collaborator :class:AnswerPipeline composes.
Matches Retriever.search's exact positional/keyword shape, so a
bound method reference (retriever.search) satisfies this Protocol
with no adapter, and a plain async def function with the same
shape satisfies it too — "a Retriever-like search callable or
instance" resolves to one requirement, callability with this
signature, regardless of which of the two a caller hands in.
This is a private, orchestration-level structural type local to this
module — distinct from ChatProtocol, the one provider-boundary
Protocol Phase 5 formally introduces in providers/protocols.py
(docs/specs/phase-5-boundary-features.md §3). It exists only so
this module's own tests can supply a minimal scripted double instead of
constructing a real Retriever — real stores, a real BM25 index —
for every case; it is not a declared multi-implementation extension
point the way ChatProtocol is.