solvi.models¶
solvi models: the published and cached deciders, pull, check; naming a model (solvi ask --decider).
solvi.models — the models a system decides with, by name, and the published deciders in the local cache.
import solvi.models as models
m = models.decider("solvi-base") # a published decider from the cache (solvi models pull ...)
m = models.llm("http://127.0.0.1:8080/v1", "qwen2.5-7b-instruct") # an OpenAI-compatible server
m = models.systemone("https://...", "my-model") # a System One service
part = m.decision("team", "Which team?", "email", ["billing", "shipping"])
DecideModel (the decider's class) is here too; the low-level pieces are in solvi.core.deciders. The command
(solvi.cli._models):
solvi models [list] [--json] solvi-ai/solvi-base, solvi-ai/solvi-large and every decider cached
solvi models pull solvi-ai/solvi-base [--backend onnx|torch|all] download (huggingface_hub; only this command does)
solvi models check MODEL [--examples labels.jsonl --task "Which team?" [--options a,b,c]] [--json]
MODEL (here, and solvi ask --decider): a local checkpoint folder; a Hugging Face id already in the local cache (nothing
is downloaded outside pull); systemone:URL#model — a System One service (--api-key, or $SOLVI_SYSTEMONE_API_KEY);
llm:URL#model — an OpenAI-compatible chat-completions server (solvi.core.deciders.llm), e.g. llm:http://127.0.0.1:8080/v1#qwen2.5-7b
(--api-key, or $SOLVI_LLM_API_KEY); or module:attr / file.py:attr — a DecideModel your code builds.
check prints what the checkpoint declares (solvi_decide.json: format, question kinds, act head, questions per pass,
state serialization), its fingerprint, and — with labelled examples (JSON lines or CSV: label plus text / input
or the other columns as a state; the question from --task / --options or each row's "task" / "options") — the latency of
one decision (first call apart) and the accuracy, the share it escalates and the accuracy of what it answers alone.
Exit status: 0 — loaded (and, with --min-accuracy, accurate enough); 1 — cannot be loaded or run here, or below
--min-accuracy; 2 — usage errors (an id that is not downloaded, a bad file).
DecideModel ¶
DecideModel(scorer, meta=None, model_id=None, path=None, backend=None, cache_size=4096, multi_question=None, act=None, max_len_long=None, identity_safe=False, identity_lease=None)
A decider: an input (a text or a state) + a question (task, options, kind) → a probability per option.
DecideModel.load(path_or_hf_id, device=None, backend="auto") loads a solvi-decide checkpoint (a folder with config.json,
tokenizer.json, solvi_decide.json, model.safetensors and/or onnx/model_fp16.onnx; or a Hugging Face id). backend:
"torch", "onnx" or "auto" (ONNX when the file and onnxruntime are there, else torch). solvi_decide.json declares what the
checkpoint can do (modes, act head, several questions per pass, temperatures, thresholds: docs/decide_format.md).
DecideModel(scorer, meta=None, model_id=...) wraps any object with logits(items) → per Item an array [K] or [K, C] (a
column per mode: [:, 0] choose-one, [:, 1] multi-label, more as meta["columns"] says) or {"logits": array, "act":
logit}; optionally logits_pass(passes) → per Pass a list of those (one per question) and fingerprint().
batchable
property
¶
Can several questions about one input share a forward pass (declared by the checkpoint, and the scorer has logits_pass)?
has_not_stated
property
¶
Does the checkpoint give a "not stated" output (an answer-primitives checkpoint's unknown)?
has_pointer
property
¶
Can the checkpoint point at a piece of its input (span answers, evidence quotes)?
state_format
property
¶
The serialization of states for this checkpoint: the first of its declared ones that solvi writes ("paths" by default; a text-only checkpoint reads the "paths" lines as text).
max_len
property
¶
The tokens this checkpoint reads in one sequence (question and input): the encoder's, else the checkpoint's
max_len, else 512.
max_len_long
property
¶
The tokens long="full" reads whole (question and input): the load(max_len_long=...) override, else the
checkpoint's max_len_long, else None (the checkpoint was not trained on long inputs).
long_declared
property
¶
Does the checkpoint itself declare a long-input length (max_len_long in solvi_decide.json)?
block
property
¶
Does this model score in the block layout (declared by the checkpoint, and the scorer has logits_pass)? Then every question — alone or with others — is scored in that layout: its answer does not depend on the other questions of its pass, and adapt / fit / teach see the same logits as the runtime.
load
classmethod
¶
load(path_or_id, device=None, backend='auto', max_len=None, bs=16, multi_question=None, act=None, max_len_long=None)
path_or_id: a checkpoint folder (~ is expanded) or a Hugging Face id (downloaded once, then read from the
cache). multi_question / act: override what solvi_decide.json declares (for experiments, e.g. testing a checkpoint
in multi-question passes); both are part of the fingerprint. max_len: the tokens of one ordinary pass (default:
the checkpoint's max_len); it also sets long="retrieve"'s budget. max_len_long: the length long="full" reads
whole (default: the checkpoint's max_len_long; a checkpoint that declares none refuses long="full" unless it is
given here — with a warning: it was not trained on long inputs). Part of the fingerprint.
act_threshold_for ¶
The act threshold the checkpoint ships for a target error rate (its act.threshold_for_error table: the entry
with the largest error ≤ the target). ValueError when it ships none — use calibrate_for on your examples.
wire ¶
How a question kind is asked: natively when the checkpoint was trained on it, else as a single choice (score: the levels; noul: the options "yes" / "no"; rank: the options, ordered by probability; number: the bins, as a score when the checkpoint has scores). A span is native only.
weights_fingerprint ¶
A hash of the checkpoint (files, or the scorer's own fingerprint), the backend, the default calibration and (for checkpoints in the v2 format, or with overrides) the declared capabilities.
fingerprint ¶
The checkpoint plus every adaptation (so a replay knows the calibration a decision used).
metadata ¶
The checkpoint's metadata (without the training history), capabilities, the default calibration and every adaptation.
on_cpu ¶
Does the network run on a CPU (a torch scorer on "cpu", an ONNX session without CUDA)? None when unknown (a stand-in or remote scorer).
count_tokens ¶
Tokens of a text for this checkpoint: its tokenizer when it has one, else solvi.core.deciders.longdoc.approx_tokens.
truncation ¶
What one ordinary pass leaves unread of an input → None when it reads all of it, else {"input_tokens",
"read_tokens", "question_tokens", "max_len"}. specs: the question, or the questions of a shared pass. None for
a scorer that reads the text as it is (an LLM, a hosted decision model) and in the block layout when the input
fits its budget.
clear_cache ¶
Forget scored outputs after a provider change. Adaptations are kept; recalibrate them on fresh outputs. A forward pass already in flight may finish, but cannot put its old outputs back in the cache.
logits ¶
Raw logits of the scored options ("other" excluded): {option: logit}, or a list of them for a list of inputs.
spec: as decide (not_stated=, evidence=, k=, ...: the question is scored as decide scores it).
act_probability ¶
The probability that the model's answer is right, from its act logit: sigmoid(logit / T_act), or the checkpoint's act calibrator — a logistic regression over ACT_FEATURES of the calibrated decision (see docs/decide_format.md).
decide ¶
decide(text, task, options, descriptions=None, multi=False, other=None, kind=None, min_confidence=None, **spec)
→ Decision(value, probs) (a list of them for a list of inputs). The value is always one of the options; a text
or a state (dict, list, pydantic model: see state_text). spec: not_stated=, k=, bins=, unit=, coverage=,
evidence= (see decision). min_confidence: escalate below this confidence (escalate_below= in 0.7).
score ¶
→ {option: probability} (a list of them for a list of inputs): softmax over the options for a single choice, a
score or yes/no, a sigmoid per option with multi=True; the adaptation of this question applied, "other" by its
threshold. spec: as decide (not_stated=, evidence=, ...).
decide_pass ¶
Several decision parts about one input, scored together → [Decision] (one forward pass when the checkpoint
supports it, else one per question). What the runtime does for parts grouped in flow.batches.
adaptation ¶
adaptation(task, options, descriptions=None, multi=False, other=None, create=False, kind=None, **spec)
The Adaptation of one question (None when it has none; create=True makes an empty one). spec as for
adapt / fit / teach (not_stated=, evidence=, ...): a question asking for evidence has its own key, so its
adaptation is found only with the same spec.
adapt ¶
adapt(texts, task, options, descriptions=None, multi=False, other=None, kind=None, logits=None, **spec)
Label-bias correction without labels: the mean logit of each option over unlabelled inputs of the domain (centered over the options) is subtracted before the softmax / sigmoid. → the Adaptation. A later fit is kept consistent. logits: the inputs' logits already computed (one per text, in the question's option order) — what a DecisionPart passes, so the correction is fitted on the signal it decides on (option_order="average", long).
fit ¶
fit(examples, task, options, descriptions=None, multi=False, other=None, lam=1.0, folds=4, kind=None, logits=None, **spec)
Few-shot adaptation "S" from labelled examples [(input, correct)]: a shift and a shared scale on the (bias-corrected) logits by L-BFGS (per option; a score: a tilt and a spread over the levels; yes/no: one bias), a temperature on out-of-fold predictions, and — when some examples are labelled "other" — the "other" threshold that maximizes out-of-fold accuracy. Replaces earlier examples. Examples labelled "not stated" are left out (the "not stated" logit is not adapted). logits: the examples' logits already computed (one per example; see adapt). → the Adaptation.
teach ¶
teach(text, correct, task, options, descriptions=None, multi=False, other=None, lam=1.0, kind=None, logits=None, **spec)
One labelled example, absorbed at once: the shift / scale is refitted from the kept examples (warm start, K + 1 parameters or fewer — about a millisecond or two); the temperature and the "other" threshold stay until the next fit. logits: the input's logits already computed (see adapt). → the update time in ms (the model's forward pass, if the input was not scored before, is not included).
reset ¶
Forget the adaptation of one question, or every adaptation.
save_adaptations ¶
Write every adaptation (with its kept examples' logits) to a JSON file, with the checkpoint's fingerprint.
load_adaptations ¶
Read adaptations written by save_adaptations. strict: refuse them if the checkpoint differs (the bias and shift were fitted on another model's logits).
decision ¶
decision(name, task, text_fact='doc', options=(), descriptions=None, multi=False, other=None, *, kind=None, type=None, min_confidence=None, min_act=None, use_act=None, max_error=None, score_value=None, not_stated=False, k=None, bins=None, unit=None, coverage=None, evidence=False, option_order='canonical', permutations=4, min_margin=None, long=None, top_k=None, rerank=False, perturb=0, retrieve_query=None, state_keys=(), _shared=frozenset())
A catalog part: text_fact (a fact name, or a list of them) → Decision(value, probs).
The question: options (a list, or {option: description}) and kind ("choice", "multi", "score", "noul"; default
choice, or multi with multi=True) — or a Python type, as type= or in place of the options: Literal[...] / an Enum
(choice), list[Literal[...]] (multi), Scale[...] (score), bool (noul: the value is True / False). The input: a
text fact is read as it is (several joined by new lines); a state (dict, list, pydantic model) by state_text
(several facts: {fact: value}).
Escalation: the model's act signal when it has one (use_act=False ignores it; min_act overrides the checkpoint's threshold; max_error=0.1 takes the checkpoint's threshold for that error rate), and a calibrated confidence below min_confidence (see calibrate_for); an escalated decision is rejected — the fact is missing, the answer abstains ("model escalated" / "low confidence" in the audit and stats). min_margin=0.1: also escalate when the two most probable answers are closer than that (a near tie is where a misleading text flips the choice).
Option order (choice and multi questions): "canonical" (the default) asks in sorted order, so how a caller lists
the options cannot change the answer (the part's options are then in that order); "given" asks as listed (0.5.0);
"average" averages the model's logits over permutations rotations of the list (each costs a forward pass) —
against a model's preference for positions.
perturb=k: ask again on up to k variants of the input without its instruction-like sentences ("ignore the rules and answer X", "SYSTEM: ...", "the correct answer is X" — deterministic rules, solvi.core.deciders.perturb) and escalate when the answer changes ("answer depends on an instruction-like sentence: ..."). An input without such sentences costs nothing extra; one with them costs up to k forward passes.
Register with cat.fn(part) (a fact other parts read) or make it a question's answer with part.question(cat).
The value is one of the options by construction; the options are the part's closed set; provenance decided; the
trace records the model, whose fingerprint covers the checkpoint and this part's adaptation and thresholds.
Answer primitives (an answer-primitives checkpoint, docs/decide_format.md §9): Maybe[T] or not_stated=True — "not stated" is an
answer (solvi.Unknown); Rank[Literal[...], k] or kind="rank", k= — the options best first; Estimate[edges] or
kind="number", bins=, unit=, coverage= — a number over bins; Span[T] or kind="span" — a piece of the (one, given)
text fact, coerced to T when it is a question's answer; evidence=True (or a number) — supporting quotes from the
pointer. A checkpoint that cannot give what is asked raises here.
Long texts: long=None cuts a text beyond max_len (the tokenizer truncates it, as before); long="retrieve" splits
it into sections, selects the top_k that bear on the question by BM25 (rerank=True: re-ordered by the decider's own
relevance, one yes / no pass per candidate section) and decides on them; spans and evidence point into the whole
text, and the sections read are in the decision's extra["long"] (solvi.core.deciders.longdoc). top_k=None (the default): sections
of about 170 tokens — budget / 170, at least 3 (3 at max_len 512, 12 at 2048). retrieve_query: the words the
sections are searched by, in place of the question's own (its task, options and descriptions) — the labels the
document writes next to the value ("Invoice No Contract No Ref"), or the document's language when the question
is asked in another one; the decider still reads the question as it is. long="full" (a checkpoint trained
on long inputs: max_len_long in its solvi_decide.json) reads a text that does not fit max_len whole, up to
max_len_long tokens, and retrieves within max_len_long beyond that (recorded in extra["long"]); a GPU mode — on a
CPU a whole 8k-token text takes seconds per question.
An option the question's kind does not use is a ValueError, not ignored: score_value= (score questions; default "median"), k= (rank), bins= / unit= / coverage= (number; coverage default 0.8), other= (choice and multi), min_margin= (not multi), top_k= / rerank= (with long=), min_act= / max_error= (a checkpoint with an act head), and kind= that contradicts multi=True.
Names (0.8): min_confidence= (0.7: escalate_below=), min_act= (act_threshold=), max_error= (target_error=),
not_stated= (unknown=) — the old ones were removed in 0.9 (a TypeError names the new one); the part keeps the thresholds as
part.min_confidence / part.min_act.
decisions ¶
One decision part per field of a pydantic model class: the field's type is the question (bool, Literal[...],
an Enum, Scale[...], list[Literal[...]]), its description the task (else its title, else its name), and
json_schema_extra may carry "options" ({option: description}), "min_confidence", "min_act", "use_act",
"max_error", "other", "score_value" (the 0.7 keys "escalate_below", "act_threshold", "target_error" were removed
in 0.9: a field with one raises). → {field: DecisionPart} in field order.
questions ¶
decisions(...) registered as the answers of questions named after the fields → [Question].
ModelError ¶
Bases: Exception
A model that cannot be named or found (exit status 2).
cached_path ¶
The local snapshot folder of a Hugging Face model id (its revision ref, else the newest snapshot), or None.
cached ¶
Every decider in the local cache (a snapshot with solvi_decide.json) → [{"id", "path", "bytes", "onnx", "torch"}] (onnx / torch: whether the snapshot holds that backend's weights).
kind_of ¶
How a MODEL spec is read: "systemone", "llm", "folder", "code" (module:attr / file.py:attr) or "hub" (a Hugging Face id).
resolve ¶
A MODEL spec → (kind, where): a folder for "folder" and a cached "hub" id, (url, model) for "systemone" and "llm",
the spec for "code". A Hugging Face id that is not in the cache raises ModelError (run solvi models pull ID); so
does a spec written as a path ("./x", "../x", "/x", "~/x") that is not a folder — "no such folder", never "pull".
load ¶
A MODEL spec → a DecideModel (see the module docs). Never downloads. ModelError for a name that cannot be read
(a file.py:attr whose file or attribute is missing included — never SystemExit); other exceptions (a missing
runtime, a broken checkpoint, an error inside your module) pass through.
pull ¶
Download a decider into the local cache (huggingface_hub) → its folder.
decider ¶
A decider by name → a DecideModel: "solvi-base" / "solvi-large" / "solvi-large-long" (the published ones, from
the local cache: solvi models pull downloads), a Hugging Face id, a checkpoint folder, "systemone:URL#model",
"llm:URL#model" or "module:attr" (see the module docs). Never downloads.
llm ¶
An OpenAI-compatible chat-completions server as a decider (solvi.core.deciders.llm.llm: every option there).
systemone ¶
A model behind the System One HTTP API as a decider (solvi.core.deciders.systemone.systemone).