Skip to content

Asking: System and Response

from solvi import System

system = System(cat, questions)
res = system.ask(init_state)                   # all questions
res = system.ask(init_state, ["ship"])         # a subset: questions=["ship"] (or "ship")

system.ask(init_state, questions=None, *, workers=None, order=None, store=True, early_exit=None, check_loss=None); aask takes the same questions and keyword-only order, store, timeout, speculate, early_exit, check_loss. (names=, the 0.7 spelling of questions=, was removed in 0.9.)

System(catalog, questions, *, workers=1, order="default", producers="declared", learn=None, input_model=None, strategist=None, storage=None, timeout=None, cost_policy="declared", lang="en", early_exit=True, cache=None, allow_unreachable_hard_checks=False, check_models=None, check_intervals=None, check_policies=None); cache=N: the opt-in pure-decision cache (see Compact records); learn: after every ask, update the parts' measured costs and the learned order / producer policies from what happened (default: on when order or producers is "learned"); the others are described where they matter. storage (a TraceStorage or a path) saves every response with its whole trace, hash-chained across responses (see Storing decisions). Every option after questions is keyword-only. With a JSON-lines store every ask appends one line with the hash of init_state, the answers, the flow and the hash of every trace record, plus the whole response and the chain fields (journal="file.jsonl", the older spelling of storage="file.jsonl", was removed in 0.9). ask(..., store=False) skips saving one response. input_model: a pydantic model of init_state (see Types); ask also takes a BaseModel instance.

Response

Field Content
res["q"] the Result for question q: its answer is res["q"].answer (there is no res.answers)
res.results dict of all results
res.flow the planned flow; print(res.flow) lists the steps and why each was taken
res.flow.per_question question -> names of the parts in its flow
res.flow.skipped catalog part -> why it was not executed (inputs unavailable, or not needed)
res.flow.unresolved question -> facts that nothing can compute from this init_state
res.state_text() a printable listing (computed_state in 0.7): each computed fact, its value, quote offsets, extraction confidence, errors
res.values dict of all fact values (inputs and computed)
res.trace the hash-chained trace (see The trace)
res.audit(question=None) what each answer rests on and which safeguards fired (see Grounded decisions)
res.safeguards the safeguard events of this response
res.checks every check of the flow as data, in flow order: CheckResult(name, questions, status, hard, reason, then) — status "passed", "failed", "skipped" (a failed hard check closed every question that needed it; or no data: a hard check that could not be planned because inputs it needs are not given, listed after the flow's checks with missing, the given inputs it lacks) or "error" (it ran and could not be evaluated); reason is what Fail(...) said (else the docstring's first line, else " is false"; for an error the error); then the answers a failed hard check set. Also in to_dict() / to_json() and so in a stored decision ("checks", derived from the trace; ignored on load and recomputed)
res.ms decision time in milliseconds
res.confidence overall confidence: the probability that every answered question is right (product of the answers' confidences, errors taken as independent — conservative when constraints tie answers together); abstained questions are left out
res.complete, res.weakest did every question get an answer; (question, confidence) of the least confident answer
res.overall all of it as data: {confidence, weakest, answered, abstained, complete, feasible, not_stated, by_kind} (by_kind: per answer kind the answered count and the product of their confidences); also in to_json() and the first line of print(res.audit())
res.not_stated the questions answered "not stated" (solvi.Unknown)
res.model_dump(), res.to_json() the response as data / JSON; Response.from_json(text, catalog=...) loads it back (see Types)

Result

Field Content
.answer one of the options (a tuple for multi-label and rank, a number for an estimate, the coerced text for a span, solvi.Unknown for "not stated"), or None when abstaining
.confidence float in [0, 1]
.why the reason: rule inputs and their values, or the top feature contributions of a learned head, or why it abstained or was forced
.status "ok", "forced" (a hard check decided) or "abstain"
.probs class probabilities for answers from a learned head or a model decision (empty for plain rules)
.provenance, .source where the answer came from: computed (a rule, a hard check), learned (a head, a learned rule), decided (a model-backed rule); and which rule / check / head
.guard, .repaired the safeguard that settled it (hard_check, no_data, outside_options, low_confidence, grounding, type_rejected, evidence_missing, ...), and (previous answer, constraints) if joint decoding changed it
.kind the answer type's kind (yes_no, choice, ordinal, multi, span, rank, estimate)
.evidence, .span the supporting quotes [Quote(text, start, end, source)]; a span answer's Quote
.not_stated, .interval, .scores, .extra the answer is solvi.Unknown; an estimate's interval; a ranking's scores; the details as data

A question abstains when:

  • a fact it needs cannot be computed from init_state (nothing in the catalog produces it); when that fact is a hard check that governs the question, the guard is no_data — "no data for hard check cap: missing limit", the check is in res.checks as "skipped" with missing=["limit"] and res["q"].extra["no_data"] == {"cap": ["limit"]} — told apart from a hard check that ran and could not be evaluated (guard hard_check, check "error");
  • a part it depends on raised an exception (the error is kept in the trace);
  • its rule returned a value outside the options, or returned None to abstain;
  • a hard check failed and has no then entry for it;
  • it has no rule and no trained head.