solvi.core.deciders.model¶
DecideModel: a decider checkpoint (or any scorer) — load, score, decide, adapt, make decision parts.
Part of solvi.core.deciders, which re-exports every name; import from there.
DecideModel: a decider checkpoint (or any scorer) — load, score, decide, adapt, and make decision parts. (Part of solvi.core.deciders, which re-exports every name.)
DecideModel ¶
DecideModel(scorer, meta=None, model_id=None, path=None, backend=None, cache_size=4096, multi_question=None, act=None, max_len_long=None, identity_safe=False, identity_lease=None)
A decider: an input (a text or a state) + a question (task, options, kind) → a probability per option.
DecideModel.load(path_or_hf_id, device=None, backend="auto") loads a solvi-decide checkpoint (a folder with config.json,
tokenizer.json, solvi_decide.json, model.safetensors and/or onnx/model_fp16.onnx; or a Hugging Face id). backend:
"torch", "onnx" or "auto" (ONNX when the file and onnxruntime are there, else torch). solvi_decide.json declares what the
checkpoint can do (modes, act head, several questions per pass, temperatures, thresholds: docs/decide_format.md).
DecideModel(scorer, meta=None, model_id=...) wraps any object with logits(items) → per Item an array [K] or [K, C] (a
column per mode: [:, 0] choose-one, [:, 1] multi-label, more as meta["columns"] says) or {"logits": array, "act":
logit}; optionally logits_pass(passes) → per Pass a list of those (one per question) and fingerprint().
batchable
property
¶
Can several questions about one input share a forward pass (declared by the checkpoint, and the scorer has logits_pass)?
has_not_stated
property
¶
Does the checkpoint give a "not stated" output (an answer-primitives checkpoint's unknown)?
has_pointer
property
¶
Can the checkpoint point at a piece of its input (span answers, evidence quotes)?
state_format
property
¶
The serialization of states for this checkpoint: the first of its declared ones that solvi writes ("paths" by default; a text-only checkpoint reads the "paths" lines as text).
max_len
property
¶
The tokens this checkpoint reads in one sequence (question and input): the encoder's, else the checkpoint's
max_len, else 512.
max_len_long
property
¶
The tokens long="full" reads whole (question and input): the load(max_len_long=...) override, else the
checkpoint's max_len_long, else None (the checkpoint was not trained on long inputs).
long_declared
property
¶
Does the checkpoint itself declare a long-input length (max_len_long in solvi_decide.json)?
block
property
¶
Does this model score in the block layout (declared by the checkpoint, and the scorer has logits_pass)? Then every question — alone or with others — is scored in that layout: its answer does not depend on the other questions of its pass, and adapt / fit / teach see the same logits as the runtime.
load
classmethod
¶
load(path_or_id, device=None, backend='auto', max_len=None, bs=16, multi_question=None, act=None, max_len_long=None)
path_or_id: a checkpoint folder (~ is expanded) or a Hugging Face id (downloaded once, then read from the
cache). multi_question / act: override what solvi_decide.json declares (for experiments, e.g. testing a checkpoint
in multi-question passes); both are part of the fingerprint. max_len: the tokens of one ordinary pass (default:
the checkpoint's max_len); it also sets long="retrieve"'s budget. max_len_long: the length long="full" reads
whole (default: the checkpoint's max_len_long; a checkpoint that declares none refuses long="full" unless it is
given here — with a warning: it was not trained on long inputs). Part of the fingerprint.
act_threshold_for ¶
The act threshold the checkpoint ships for a target error rate (its act.threshold_for_error table: the entry
with the largest error ≤ the target). ValueError when it ships none — use calibrate_for on your examples.
wire ¶
How a question kind is asked: natively when the checkpoint was trained on it, else as a single choice (score: the levels; noul: the options "yes" / "no"; rank: the options, ordered by probability; number: the bins, as a score when the checkpoint has scores). A span is native only.
weights_fingerprint ¶
A hash of the checkpoint (files, or the scorer's own fingerprint), the backend, the default calibration and (for checkpoints in the v2 format, or with overrides) the declared capabilities.
fingerprint ¶
The checkpoint plus every adaptation (so a replay knows the calibration a decision used).
metadata ¶
The checkpoint's metadata (without the training history), capabilities, the default calibration and every adaptation.
on_cpu ¶
Does the network run on a CPU (a torch scorer on "cpu", an ONNX session without CUDA)? None when unknown (a stand-in or remote scorer).
count_tokens ¶
Tokens of a text for this checkpoint: its tokenizer when it has one, else solvi.core.deciders.longdoc.approx_tokens.
truncation ¶
What one ordinary pass leaves unread of an input → None when it reads all of it, else {"input_tokens",
"read_tokens", "question_tokens", "max_len"}. specs: the question, or the questions of a shared pass. None for
a scorer that reads the text as it is (an LLM, a hosted decision model) and in the block layout when the input
fits its budget.
clear_cache ¶
Forget scored outputs after a provider change. Adaptations are kept; recalibrate them on fresh outputs. A forward pass already in flight may finish, but cannot put its old outputs back in the cache.
logits ¶
Raw logits of the scored options ("other" excluded): {option: logit}, or a list of them for a list of inputs.
spec: as decide (not_stated=, evidence=, k=, ...: the question is scored as decide scores it).
act_probability ¶
The probability that the model's answer is right, from its act logit: sigmoid(logit / T_act), or the checkpoint's act calibrator — a logistic regression over ACT_FEATURES of the calibrated decision (see docs/decide_format.md).
decide ¶
decide(text, task, options, descriptions=None, multi=False, other=None, kind=None, min_confidence=None, **spec)
→ Decision(value, probs) (a list of them for a list of inputs). The value is always one of the options; a text
or a state (dict, list, pydantic model: see state_text). spec: not_stated=, k=, bins=, unit=, coverage=,
evidence= (see decision). min_confidence: escalate below this confidence (escalate_below= in 0.7).
score ¶
→ {option: probability} (a list of them for a list of inputs): softmax over the options for a single choice, a
score or yes/no, a sigmoid per option with multi=True; the adaptation of this question applied, "other" by its
threshold. spec: as decide (not_stated=, evidence=, ...).
decide_pass ¶
Several decision parts about one input, scored together → [Decision] (one forward pass when the checkpoint
supports it, else one per question). What the runtime does for parts grouped in flow.batches.
adaptation ¶
adaptation(task, options, descriptions=None, multi=False, other=None, create=False, kind=None, **spec)
The Adaptation of one question (None when it has none; create=True makes an empty one). spec as for
adapt / fit / teach (not_stated=, evidence=, ...): a question asking for evidence has its own key, so its
adaptation is found only with the same spec.
adapt ¶
adapt(texts, task, options, descriptions=None, multi=False, other=None, kind=None, logits=None, **spec)
Label-bias correction without labels: the mean logit of each option over unlabelled inputs of the domain (centered over the options) is subtracted before the softmax / sigmoid. → the Adaptation. A later fit is kept consistent. logits: the inputs' logits already computed (one per text, in the question's option order) — what a DecisionPart passes, so the correction is fitted on the signal it decides on (option_order="average", long).
fit ¶
fit(examples, task, options, descriptions=None, multi=False, other=None, lam=1.0, folds=4, kind=None, logits=None, **spec)
Few-shot adaptation "S" from labelled examples [(input, correct)]: a shift and a shared scale on the (bias-corrected) logits by L-BFGS (per option; a score: a tilt and a spread over the levels; yes/no: one bias), a temperature on out-of-fold predictions, and — when some examples are labelled "other" — the "other" threshold that maximizes out-of-fold accuracy. Replaces earlier examples. Examples labelled "not stated" are left out (the "not stated" logit is not adapted). logits: the examples' logits already computed (one per example; see adapt). → the Adaptation.
teach ¶
teach(text, correct, task, options, descriptions=None, multi=False, other=None, lam=1.0, kind=None, logits=None, **spec)
One labelled example, absorbed at once: the shift / scale is refitted from the kept examples (warm start, K + 1 parameters or fewer — about a millisecond or two); the temperature and the "other" threshold stay until the next fit. logits: the input's logits already computed (see adapt). → the update time in ms (the model's forward pass, if the input was not scored before, is not included).
reset ¶
Forget the adaptation of one question, or every adaptation.
save_adaptations ¶
Write every adaptation (with its kept examples' logits) to a JSON file, with the checkpoint's fingerprint.
load_adaptations ¶
Read adaptations written by save_adaptations. strict: refuse them if the checkpoint differs (the bias and shift were fitted on another model's logits).
decision ¶
decision(name, task, text_fact='doc', options=(), descriptions=None, multi=False, other=None, *, kind=None, type=None, min_confidence=None, min_act=None, use_act=None, max_error=None, score_value=None, not_stated=False, k=None, bins=None, unit=None, coverage=None, evidence=False, option_order='canonical', permutations=4, min_margin=None, long=None, top_k=None, rerank=False, perturb=0, retrieve_query=None, state_keys=(), _shared=frozenset())
A catalog part: text_fact (a fact name, or a list of them) → Decision(value, probs).
The question: options (a list, or {option: description}) and kind ("choice", "multi", "score", "noul"; default
choice, or multi with multi=True) — or a Python type, as type= or in place of the options: Literal[...] / an Enum
(choice), list[Literal[...]] (multi), Scale[...] (score), bool (noul: the value is True / False). The input: a
text fact is read as it is (several joined by new lines); a state (dict, list, pydantic model) by state_text
(several facts: {fact: value}).
Escalation: the model's act signal when it has one (use_act=False ignores it; min_act overrides the checkpoint's threshold; max_error=0.1 takes the checkpoint's threshold for that error rate), and a calibrated confidence below min_confidence (see calibrate_for); an escalated decision is rejected — the fact is missing, the answer abstains ("model escalated" / "low confidence" in the audit and stats). min_margin=0.1: also escalate when the two most probable answers are closer than that (a near tie is where a misleading text flips the choice).
Option order (choice and multi questions): "canonical" (the default) asks in sorted order, so how a caller lists
the options cannot change the answer (the part's options are then in that order); "given" asks as listed (0.5.0);
"average" averages the model's logits over permutations rotations of the list (each costs a forward pass) —
against a model's preference for positions.
perturb=k: ask again on up to k variants of the input without its instruction-like sentences ("ignore the rules and answer X", "SYSTEM: ...", "the correct answer is X" — deterministic rules, solvi.core.deciders.perturb) and escalate when the answer changes ("answer depends on an instruction-like sentence: ..."). An input without such sentences costs nothing extra; one with them costs up to k forward passes.
Register with cat.fn(part) (a fact other parts read) or make it a question's answer with part.question(cat).
The value is one of the options by construction; the options are the part's closed set; provenance decided; the
trace records the model, whose fingerprint covers the checkpoint and this part's adaptation and thresholds.
Answer primitives (an answer-primitives checkpoint, docs/decide_format.md §9): Maybe[T] or not_stated=True — "not stated" is an
answer (solvi.Unknown); Rank[Literal[...], k] or kind="rank", k= — the options best first; Estimate[edges] or
kind="number", bins=, unit=, coverage= — a number over bins; Span[T] or kind="span" — a piece of the (one, given)
text fact, coerced to T when it is a question's answer; evidence=True (or a number) — supporting quotes from the
pointer. A checkpoint that cannot give what is asked raises here.
Long texts: long=None cuts a text beyond max_len (the tokenizer truncates it, as before); long="retrieve" splits
it into sections, selects the top_k that bear on the question by BM25 (rerank=True: re-ordered by the decider's own
relevance, one yes / no pass per candidate section) and decides on them; spans and evidence point into the whole
text, and the sections read are in the decision's extra["long"] (solvi.core.deciders.longdoc). top_k=None (the default): sections
of about 170 tokens — budget / 170, at least 3 (3 at max_len 512, 12 at 2048). retrieve_query: the words the
sections are searched by, in place of the question's own (its task, options and descriptions) — the labels the
document writes next to the value ("Invoice No Contract No Ref"), or the document's language when the question
is asked in another one; the decider still reads the question as it is. long="full" (a checkpoint trained
on long inputs: max_len_long in its solvi_decide.json) reads a text that does not fit max_len whole, up to
max_len_long tokens, and retrieves within max_len_long beyond that (recorded in extra["long"]); a GPU mode — on a
CPU a whole 8k-token text takes seconds per question.
An option the question's kind does not use is a ValueError, not ignored: score_value= (score questions; default "median"), k= (rank), bins= / unit= / coverage= (number; coverage default 0.8), other= (choice and multi), min_margin= (not multi), top_k= / rerank= (with long=), min_act= / max_error= (a checkpoint with an act head), and kind= that contradicts multi=True.
Names (0.8): min_confidence= (0.7: escalate_below=), min_act= (act_threshold=), max_error= (target_error=),
not_stated= (unknown=) — the old ones were removed in 0.9 (a TypeError names the new one); the part keeps the thresholds as
part.min_confidence / part.min_act.
decisions ¶
One decision part per field of a pydantic model class: the field's type is the question (bool, Literal[...],
an Enum, Scale[...], list[Literal[...]]), its description the task (else its title, else its name), and
json_schema_extra may carry "options" ({option: description}), "min_confidence", "min_act", "use_act",
"max_error", "other", "score_value" (the 0.7 keys "escalate_below", "act_threshold", "target_error" were removed
in 0.9: a field with one raises). → {field: DecisionPart} in field order.
questions ¶
decisions(...) registered as the answers of questions named after the fields → [Question].