solvi.core.specialist¶
The specialist contract (stable since 1.1; until then solvi.experimental.specialist): a model proposes a typed spec,
code checks it against the source, code renders it; every run replays to identical bytes from its own record or from
its record of kind "specialist" in a TraceStorage (replay_runs). align matches two readers' readings by the overlap
of their spans (time, boxes, character offsets).
Verifiable specialists: a model proposes, code checks against the source, code renders, the store keeps the run (stable since 1.1; until then solvi.experimental.specialist, which re-exports every name for one release).
One contract for every specialist (charts first; slides, tables, speech later):
- propose — a model (small and fast, an LLM, or a rule-based extractor) writes a typed intermediate spec (a pydantic model), never the result itself;
- check — deterministic code verifies the spec against the source and the specialist's rules; what does not verify is dropped or changed, each with a reason (an Issue) — it is marked, never invented or repaired by a guess;
- render — deterministic code builds the result from the verified spec only (same verified spec → same bytes);
- trace — every step goes into the run's own hash chain (
Run.trace): the source's hash, the proposal, the check, the output's hash; with a TraceStorage (Specialist(proposer, storage=...)) the run is also one record of kind "specialist" in the store's chain — the chain the decisions use, sostore.verify()covers it.Specialist.replayre-checks the recorded proposal and re-renders it: identical issues and identical bytes, or it says what differs.
The determinism contract of replay: replay re-executes code and re-renders from recorded model outputs; it does not
re-call a model. A model proposer need not be deterministic (a remote LLM at temperature 0 returns other outputs across
identical runs), so the record keeps what the proposer returned — the parsed spec, and, when the proposer returned
text (a model's reply), that text as it came — and replay parses that output again (it must give the
recorded spec), checks and renders it, and compares the bytes. Only a record that holds no output at all (a record
written without one) makes replay call the proposer — Replay.from_record is then False and Replay.note says so, and
the bytes are equal only if the proposer happens to repeat itself. Two runs of the same source give the same bytes
only with a deterministic proposer; solvi.testing.conformance.check_specialist(..., deterministic=False) checks the
replay alone.
from solvi.experimental.charts import ChartSpecialist, RuleProposer
sp = ChartSpecialist(RuleProposer(), storage="runs.jsonl")
run = sp.run(text, "revenue by region")
run.output # the SVG, or None when nothing verified
print(run.report()) # what was kept, dropped, changed and why
sp.replay(run.stored_id, text).ok # True: same checks, same bytes, from the stored record
ChartSpecialist().replay(run.to_dict(), text).ok # the same from the run's own record (no storage needed)
replay_runs(store, ChartSpecialist(), texts) # every stored run of that specialist → [] when all replay
The stored record ("specialist" kind): specialist, version, question, source_sha256 and source_chars (the source itself
is not stored unless store_source=True: pass it to replay), proposer, proposal (the spec; for a blocked run the raw proposal and
proposal_error), proposal_raw (the proposer's output when it was text — a model's reply, as it came), issues, kept,
verified, meta, renderer, output_sha256, output_bytes, trace_head (the head of the run's own chain) and — when the
specialist reports them (fingerprints()) — fingerprints. Run.to_dict() keeps its shape;
Run.stored_id names the record. The in-memory Trace stays the run's own summary: the record holds the same data,
flattened, under the store's hash.
A subclass defines spec_type (the pydantic model of the proposal), check(spec, source) -> Checked and
render(checked) -> str | None; the proposer is any callable (source, question) -> spec | dict (an id attribute,
when it has one, names it in the trace). A proposer is not replayed — its output is recorded; the parse, the check and
the render are, from that record.
Issue
dataclass
¶
One finding of the check: where in the spec (path, e.g. "series[0].points[2]"), what (code, stable,
machine-readable), how bad (severity) and why (message, for people).
Checked
dataclass
¶
The check's result: the verified spec (only what passed; None when nothing did) and every issue.
Trace ¶
A hash chain of steps: entry i stores its step, its data, the previous entry's hash and its own hash = sha256(prev + canonical(step, data)). Editing any entry breaks every hash after it.
Not solvi.core.runtime.Trace: a run's own chain, kept with the run (to_dict / replay). With a storage the run is
also a record of kind "specialist" in the store's chain (see the module docs), which verify() covers.
Run
dataclass
¶
Run(specialist: str, version: str, source: str, question: str | None, proposal: Any, checked: Checked, output: str | None, trace: Trace, stored_id: str | None = None)
One run of a specialist: the proposal, the check, the output (None when nothing verified) and the trace.
to_dict ¶
The record to store: everything replay needs except the source (pass with_source=True to include it).
Replay
dataclass
¶
Replay(ok: bool, problems: list, from_record: bool = True, note: str = '', output_sha256: str | None = None)
The result of replaying a specialist's recorded run: ok (true in a boolean context) and the problems found;
from_record — False when the record held no proposer output and the proposer was called (note says so);
output_sha256 — the hash of the re-rendered output (None when nothing was rendered or nothing re-ran).
Specialist ¶
The contract (see the module docs). Subclasses set name, version, spec_type and implement check and
render; proposer is the default proposer for run; storage (a TraceStorage or a path) keeps every run as a
record of kind "specialist" (see the module docs). A subclass may report fingerprints of what made a run (a model,
a renderer) by fingerprints(). store_source=True keeps the source text in each stored record ("source"), so
replay(stored_id) needs no text passed — the record then grows by the source's size, and the store holds the text
(mind what it may contain); the default keeps only its hash and length.
fingerprints ¶
What made a run, as fingerprints the stored record keeps (e.g. {"proposer": ..., "model": ...}); None: none reported.
applicable ¶
Does this specialist apply to the source and the question? Checked before the proposer is called: a False (or an exception) blocks the run with the issue "not_applicable", the proposer is not called. Override it to state the precondition — a specialist is called because its task applies, not because the input has its form. Default: always.
parse ¶
A proposer's output (a spec_type, a dict or a JSON string) → spec_type; ValidationError / ValueError.
run ¶
source (a text) → Run. proposer overrides the default; proposal= skips proposing (a recorded or hand-written spec). With a storage the run is stored (Run.stored_id).
replay ¶
Re-parse the recorded proposer output, re-check it against the source and re-render it: the same source, the
same spec, the same issues, identical output bytes — without calling the proposer (see the determinism contract
in the module docs). record: a Run, its to_dict() (the run's own chain is verified, and the record's copies
against it), a stored record of kind "specialist" (its hash is checked) or a stored id (read from storage,
default the specialist's own); source: the text (or in the record). A run that was blocked before any spec (a
failed proposer, an invalid proposal) has nothing to re-check: its record is verified, and a recorded invalid
output must still be invalid. Only a record with no proposer output at all calls proposer (default the
specialist's own): Replay.from_record is then False and Replay.note says so.
Reading
dataclass
¶
One reader's reading of a span of the source: its value, where it is (span: (start, end) for an interval —
seconds, characters — or (x0, y0, x1, y1) for a box), the reader's confidence (None: not given) and what it read
before any normalising (raw).
Agreed
dataclass
¶
A value both readers gave on overlapping spans: the value (a's), both readings — with their spans and confidences, for the provenance record ("quoted by two readers") — and the overlap.
Alignment
dataclass
¶
Alignment(agreed: list = list(), only_a: list = list(), only_b: list = list(), conflicts: list = list())
align's result: agreed (the readings a check may accept), only_a / only_b (readings without a counterpart
at min_overlap or above), conflicts [(a, b, overlap)] (overlapping spans, different values: material for a
conflict or a review).
canonical ¶
Canonical JSON bytes (sorted keys, no spaces, UTF-8): the input of every hash in the trace.
replay_runs ¶
Replay every stored run of specialist (its records of kind "specialist" with its name) → the ones that do not
replay: [{"id", "seq", "problems"}] ([] — all replay). sources: the texts the runs read — a list of texts or {sha256:
text}; a record that kept its source (store_source=True) uses it when the texts do not hold it; a run whose source
is not among them is listed ("no source given ...").
interval_overlap ¶
Two intervals (start, end) → the length they share over the shorter one's length, in [0, 1] (0 for an empty span).
box_iou ¶
Two boxes (x0, y0, x1, y1) → intersection over union, in [0, 1] (touching boxes: 0).
align ¶
align(a, b, overlap=interval_overlap, min_overlap=0.5, same=None, min_a=0.0, min_b=0.0) -> Alignment
Two readers' readings of one source (two speech models' words with their times, two OCR engines' words with their boxes, two extractors' quotes with their character offsets) → Alignment.
a, b: Readings. Readings whose confidence is below min_a / min_b are dropped before matching (None passes; a
dropped reading is in no list). Then pairs are matched one-to-one, greedily by descending overlap(span_a, span_b)
(interval_overlap, box_iou or your own, in [0, 1]), ties by (a's order, b's order); a pair below min_overlap is
not matched, and a reading matched once is not reused. A matched pair is agreed when same(value_a, value_b)
(default: equality — a normaliser goes here), else a conflict. agreed, conflicts and only_a come in a's order,
only_b in b's.
Nothing here decides acceptance: a specialist's check takes agreed as what it may accept and conflicts as its
conflict / review material; the thresholds are the task's (calibrated), not this function's.
The old path, until 1.2¶
solvi.experimental.specialist re-exports every name above, as the same objects, with an ExperimentalWarning.
Graduated in 1.1: the specialist contract is the stable solvi.core.specialist (its runs on the core TraceStorage). This path re-exports every name, as the same objects, with the experimental warning (its removal, planned for 1.2, is now in a future release). Import solvi.core.specialist.