Skip to content

solvi.models

solvi models: the published and cached deciders, pull, check; naming a model (solvi ask --decider).

solvi.models — the models a system decides with, by name, and the published deciders in the local cache.

import solvi.models as models
m = models.decider("solvi-base")                     # a published decider from the cache (solvi models pull ...)
m = models.llm("http://127.0.0.1:8080/v1", "qwen2.5-7b-instruct")   # an OpenAI-compatible server
m = models.systemone("https://...", "my-model")      # a System One service
part = m.decision("team", "Which team?", "email", ["billing", "shipping"])

DecideModel (the decider's class) is here too; the low-level pieces are in solvi.core.deciders. The command (solvi.cli._models):

solvi models [list] [--json]                       solvi-ai/solvi-base, solvi-ai/solvi-large and every decider cached
solvi models pull solvi-ai/solvi-base [--backend onnx|torch|all]      download (huggingface_hub; only this command does)
solvi models check MODEL [--examples labels.jsonl --task "Which team?" [--options a,b,c]] [--json]

MODEL (here, and solvi ask --decider): a local checkpoint folder; a Hugging Face id already in the local cache (nothing is downloaded outside pull); systemone:URL#model — a System One service (--api-key, or $SOLVI_SYSTEMONE_API_KEY); llm:URL#model — an OpenAI-compatible chat-completions server (solvi.core.deciders.llm), e.g. llm:http://127.0.0.1:8080/v1#qwen2.5-7b (--api-key, or $SOLVI_LLM_API_KEY); or module:attr / file.py:attr — a DecideModel your code builds.

check prints what the checkpoint declares (solvi_decide.json: format, question kinds, act head, questions per pass, state serialization), its fingerprint, and — with labelled examples (JSON lines or CSV: label plus text / input or the other columns as a state; the question from --task / --options or each row's "task" / "options") — the latency of one decision (first call apart) and the accuracy, the share it escalates and the accuracy of what it answers alone. Exit status: 0 — loaded (and, with --min-accuracy, accurate enough); 1 — cannot be loaded or run here, or below --min-accuracy; 2 — usage errors (an id that is not downloaded, a bad file).

DecideModel

DecideModel(scorer, meta=None, model_id=None, path=None, backend=None, cache_size=4096, multi_question=None, act=None, max_len_long=None, identity_safe=False, identity_lease=None)

A decider: an input (a text or a state) + a question (task, options, kind) → a probability per option.

DecideModel.load(path_or_hf_id, device=None, backend="auto") loads a solvi-decide checkpoint (a folder with config.json, tokenizer.json, solvi_decide.json, model.safetensors and/or onnx/model_fp16.onnx; or a Hugging Face id). backend: "torch", "onnx" or "auto" (ONNX when the file and onnxruntime are there, else torch). solvi_decide.json declares what the checkpoint can do (modes, act head, several questions per pass, temperatures, thresholds: docs/decide_format.md).

DecideModel(scorer, meta=None, model_id=...) wraps any object with logits(items) → per Item an array [K] or [K, C] (a column per mode: [:, 0] choose-one, [:, 1] multi-label, more as meta["columns"] says) or {"logits": array, "act": logit}; optionally logits_pass(passes) → per Pass a list of those (one per question) and fingerprint().

batchable property

batchable

Can several questions about one input share a forward pass (declared by the checkpoint, and the scorer has logits_pass)?

has_act property

has_act

Does the checkpoint give an act / escalate signal per question?

has_not_stated property

has_not_stated

Does the checkpoint give a "not stated" output (an answer-primitives checkpoint's unknown)?

has_pointer property

has_pointer

Can the checkpoint point at a piece of its input (span answers, evidence quotes)?

state_format property

state_format

The serialization of states for this checkpoint: the first of its declared ones that solvi writes ("paths" by default; a text-only checkpoint reads the "paths" lines as text).

max_len property

max_len

The tokens this checkpoint reads in one sequence (question and input): the encoder's, else the checkpoint's max_len, else 512.

max_len_long property

max_len_long

The tokens long="full" reads whole (question and input): the load(max_len_long=...) override, else the checkpoint's max_len_long, else None (the checkpoint was not trained on long inputs).

long_declared property

long_declared

Does the checkpoint itself declare a long-input length (max_len_long in solvi_decide.json)?

block property

block

Does this model score in the block layout (declared by the checkpoint, and the scorer has logits_pass)? Then every question — alone or with others — is scored in that layout: its answer does not depend on the other questions of its pass, and adapt / fit / teach see the same logits as the runtime.

load classmethod

load(path_or_id, device=None, backend='auto', max_len=None, bs=16, multi_question=None, act=None, max_len_long=None)

path_or_id: a checkpoint folder (~ is expanded) or a Hugging Face id (downloaded once, then read from the cache). multi_question / act: override what solvi_decide.json declares (for experiments, e.g. testing a checkpoint in multi-question passes); both are part of the fingerprint. max_len: the tokens of one ordinary pass (default: the checkpoint's max_len); it also sets long="retrieve"'s budget. max_len_long: the length long="full" reads whole (default: the checkpoint's max_len_long; a checkpoint that declares none refuses long="full" unless it is given here — with a warning: it was not trained on long inputs). Part of the fingerprint.

act_threshold_for

act_threshold_for(error)

The act threshold the checkpoint ships for a target error rate (its act.threshold_for_error table: the entry with the largest error ≤ the target). ValueError when it ships none — use calibrate_for on your examples.

wire

wire(kind)

How a question kind is asked: natively when the checkpoint was trained on it, else as a single choice (score: the levels; noul: the options "yes" / "no"; rank: the options, ordered by probability; number: the bins, as a score when the checkpoint has scores). A span is native only.

weights_fingerprint

weights_fingerprint()

A hash of the checkpoint (files, or the scorer's own fingerprint), the backend, the default calibration and (for checkpoints in the v2 format, or with overrides) the declared capabilities.

fingerprint

fingerprint()

The checkpoint plus every adaptation (so a replay knows the calibration a decision used).

metadata

metadata()

The checkpoint's metadata (without the training history), capabilities, the default calibration and every adaptation.

on_cpu

on_cpu()

Does the network run on a CPU (a torch scorer on "cpu", an ONNX session without CUDA)? None when unknown (a stand-in or remote scorer).

count_tokens

count_tokens(text)

Tokens of a text for this checkpoint: its tokenizer when it has one, else solvi.core.deciders.longdoc.approx_tokens.

text

text(v)

An input (a text, a Quote, a scalar or a state) → the text this checkpoint reads.

truncation

truncation(specs, text, read_len=0)

What one ordinary pass leaves unread of an input → None when it reads all of it, else {"input_tokens", "read_tokens", "question_tokens", "max_len"}. specs: the question, or the questions of a shared pass. None for a scorer that reads the text as it is (an LLM, a hosted decision model) and in the block layout when the input fits its budget.

clear_cache

clear_cache()

Forget scored outputs after a provider change. Adaptations are kept; recalibrate them on fresh outputs. A forward pass already in flight may finish, but cannot put its old outputs back in the cache.

logits

logits(text, task, options, descriptions=None, multi=False, other=None, kind=None, **spec)

Raw logits of the scored options ("other" excluded): {option: logit}, or a list of them for a list of inputs. spec: as decide (not_stated=, evidence=, k=, ...: the question is scored as decide scores it).

act_probability

act_probability(sp, d, act)

The probability that the model's answer is right, from its act logit: sigmoid(logit / T_act), or the checkpoint's act calibrator — a logistic regression over ACT_FEATURES of the calibrated decision (see docs/decide_format.md).

decide

decide(text, task, options, descriptions=None, multi=False, other=None, kind=None, min_confidence=None, **spec)

→ Decision(value, probs) (a list of them for a list of inputs). The value is always one of the options; a text or a state (dict, list, pydantic model: see state_text). spec: not_stated=, k=, bins=, unit=, coverage=, evidence= (see decision). min_confidence: escalate below this confidence (escalate_below= in 0.7).

score

score(text, task, options, descriptions=None, multi=False, other=None, kind=None, **spec)

→ {option: probability} (a list of them for a list of inputs): softmax over the options for a single choice, a score or yes/no, a sigmoid per option with multi=True; the adaptation of this question applied, "other" by its threshold. spec: as decide (not_stated=, evidence=, ...).

decide_pass

decide_pass(text, parts, names=None)

Several decision parts about one input, scored together → [Decision] (one forward pass when the checkpoint supports it, else one per question). What the runtime does for parts grouped in flow.batches.

adaptation

adaptation(task, options, descriptions=None, multi=False, other=None, create=False, kind=None, **spec)

The Adaptation of one question (None when it has none; create=True makes an empty one). spec as for adapt / fit / teach (not_stated=, evidence=, ...): a question asking for evidence has its own key, so its adaptation is found only with the same spec.

adapt

adapt(texts, task, options, descriptions=None, multi=False, other=None, kind=None, logits=None, **spec)

Label-bias correction without labels: the mean logit of each option over unlabelled inputs of the domain (centered over the options) is subtracted before the softmax / sigmoid. → the Adaptation. A later fit is kept consistent. logits: the inputs' logits already computed (one per text, in the question's option order) — what a DecisionPart passes, so the correction is fitted on the signal it decides on (option_order="average", long).

fit

fit(examples, task, options, descriptions=None, multi=False, other=None, lam=1.0, folds=4, kind=None, logits=None, **spec)

Few-shot adaptation "S" from labelled examples [(input, correct)]: a shift and a shared scale on the (bias-corrected) logits by L-BFGS (per option; a score: a tilt and a spread over the levels; yes/no: one bias), a temperature on out-of-fold predictions, and — when some examples are labelled "other" — the "other" threshold that maximizes out-of-fold accuracy. Replaces earlier examples. Examples labelled "not stated" are left out (the "not stated" logit is not adapted). logits: the examples' logits already computed (one per example; see adapt). → the Adaptation.

teach

teach(text, correct, task, options, descriptions=None, multi=False, other=None, lam=1.0, kind=None, logits=None, **spec)

One labelled example, absorbed at once: the shift / scale is refitted from the kept examples (warm start, K + 1 parameters or fewer — about a millisecond or two); the temperature and the "other" threshold stay until the next fit. logits: the input's logits already computed (see adapt). → the update time in ms (the model's forward pass, if the input was not scored before, is not included).

reset

reset(task=None, options=None, descriptions=None, multi=False, other=None, kind=None, **spec)

Forget the adaptation of one question, or every adaptation.

save_adaptations

save_adaptations(path)

Write every adaptation (with its kept examples' logits) to a JSON file, with the checkpoint's fingerprint.

load_adaptations

load_adaptations(path, strict=True)

Read adaptations written by save_adaptations. strict: refuse them if the checkpoint differs (the bias and shift were fitted on another model's logits).

decision

decision(name, task, text_fact='doc', options=(), descriptions=None, multi=False, other=None, *, kind=None, type=None, min_confidence=None, min_act=None, use_act=None, max_error=None, score_value=None, not_stated=False, k=None, bins=None, unit=None, coverage=None, evidence=False, option_order='canonical', permutations=4, min_margin=None, long=None, top_k=None, rerank=False, perturb=0, retrieve_query=None, state_keys=(), _shared=frozenset())

A catalog part: text_fact (a fact name, or a list of them) → Decision(value, probs).

The question: options (a list, or {option: description}) and kind ("choice", "multi", "score", "noul"; default choice, or multi with multi=True) — or a Python type, as type= or in place of the options: Literal[...] / an Enum (choice), list[Literal[...]] (multi), Scale[...] (score), bool (noul: the value is True / False). The input: a text fact is read as it is (several joined by new lines); a state (dict, list, pydantic model) by state_text (several facts: {fact: value}).

Escalation: the model's act signal when it has one (use_act=False ignores it; min_act overrides the checkpoint's threshold; max_error=0.1 takes the checkpoint's threshold for that error rate), and a calibrated confidence below min_confidence (see calibrate_for); an escalated decision is rejected — the fact is missing, the answer abstains ("model escalated" / "low confidence" in the audit and stats). min_margin=0.1: also escalate when the two most probable answers are closer than that (a near tie is where a misleading text flips the choice).

Option order (choice and multi questions): "canonical" (the default) asks in sorted order, so how a caller lists the options cannot change the answer (the part's options are then in that order); "given" asks as listed (0.5.0); "average" averages the model's logits over permutations rotations of the list (each costs a forward pass) — against a model's preference for positions.

perturb=k: ask again on up to k variants of the input without its instruction-like sentences ("ignore the rules and answer X", "SYSTEM: ...", "the correct answer is X" — deterministic rules, solvi.core.deciders.perturb) and escalate when the answer changes ("answer depends on an instruction-like sentence: ..."). An input without such sentences costs nothing extra; one with them costs up to k forward passes.

Register with cat.fn(part) (a fact other parts read) or make it a question's answer with part.question(cat). The value is one of the options by construction; the options are the part's closed set; provenance decided; the trace records the model, whose fingerprint covers the checkpoint and this part's adaptation and thresholds.

Answer primitives (an answer-primitives checkpoint, docs/decide_format.md §9): Maybe[T] or not_stated=True — "not stated" is an answer (solvi.Unknown); Rank[Literal[...], k] or kind="rank", k= — the options best first; Estimate[edges] or kind="number", bins=, unit=, coverage= — a number over bins; Span[T] or kind="span" — a piece of the (one, given) text fact, coerced to T when it is a question's answer; evidence=True (or a number) — supporting quotes from the pointer. A checkpoint that cannot give what is asked raises here.

Long texts: long=None cuts a text beyond max_len (the tokenizer truncates it, as before); long="retrieve" splits it into sections, selects the top_k that bear on the question by BM25 (rerank=True: re-ordered by the decider's own relevance, one yes / no pass per candidate section) and decides on them; spans and evidence point into the whole text, and the sections read are in the decision's extra["long"] (solvi.core.deciders.longdoc). top_k=None (the default): sections of about 170 tokens — budget / 170, at least 3 (3 at max_len 512, 12 at 2048). retrieve_query: the words the sections are searched by, in place of the question's own (its task, options and descriptions) — the labels the document writes next to the value ("Invoice No Contract No Ref"), or the document's language when the question is asked in another one; the decider still reads the question as it is. long="full" (a checkpoint trained on long inputs: max_len_long in its solvi_decide.json) reads a text that does not fit max_len whole, up to max_len_long tokens, and retrieves within max_len_long beyond that (recorded in extra["long"]); a GPU mode — on a CPU a whole 8k-token text takes seconds per question.

An option the question's kind does not use is a ValueError, not ignored: score_value= (score questions; default "median"), k= (rank), bins= / unit= / coverage= (number; coverage default 0.8), other= (choice and multi), min_margin= (not multi), top_k= / rerank= (with long=), min_act= / max_error= (a checkpoint with an act head), and kind= that contradicts multi=True.

Names (0.8): min_confidence= (0.7: escalate_below=), min_act= (act_threshold=), max_error= (target_error=), not_stated= (unknown=) — the old ones were removed in 0.9 (a TypeError names the new one); the part keeps the thresholds as part.min_confidence / part.min_act.

decisions

decisions(schema, text_fact='doc', fields=None, **kw)

One decision part per field of a pydantic model class: the field's type is the question (bool, Literal[...], an Enum, Scale[...], list[Literal[...]]), its description the task (else its title, else its name), and json_schema_extra may carry "options" ({option: description}), "min_confidence", "min_act", "use_act", "max_error", "other", "score_value" (the 0.7 keys "escalate_below", "act_threshold", "target_error" were removed in 0.9: a field with one raises). → {field: DecisionPart} in field order.

questions

questions(cat, schema, text_fact='doc', fields=None, min_confidence=None, **kw)

decisions(...) registered as the answers of questions named after the fields → [Question].

ModelError

Bases: Exception

A model that cannot be named or found (exit status 2).

cached_path

cached_path(repo_id, revision='main')

The local snapshot folder of a Hugging Face model id (its revision ref, else the newest snapshot), or None.

cached

cached()

Every decider in the local cache (a snapshot with solvi_decide.json) → [{"id", "path", "bytes", "onnx", "torch"}] (onnx / torch: whether the snapshot holds that backend's weights).

kind_of

kind_of(spec)

How a MODEL spec is read: "systemone", "llm", "folder", "code" (module:attr / file.py:attr) or "hub" (a Hugging Face id).

resolve

resolve(spec)

A MODEL spec → (kind, where): a folder for "folder" and a cached "hub" id, (url, model) for "systemone" and "llm", the spec for "code". A Hugging Face id that is not in the cache raises ModelError (run solvi models pull ID); so does a spec written as a path ("./x", "../x", "/x", "~/x") that is not a folder — "no such folder", never "pull".

load

load(spec, backend='auto', api_key=None)

A MODEL spec → a DecideModel (see the module docs). Never downloads. ModelError for a name that cannot be read (a file.py:attr whose file or attribute is missing included — never SystemExit); other exceptions (a missing runtime, a broken checkpoint, an error inside your module) pass through.

pull

pull(repo_id, backend='onnx', revision=None)

Download a decider into the local cache (huggingface_hub) → its folder.

decider

decider(spec='solvi-base', *, backend='auto', api_key=None)

A decider by name → a DecideModel: "solvi-base" / "solvi-large" / "solvi-large-long" (the published ones, from the local cache: solvi models pull downloads), a Hugging Face id, a checkpoint folder, "systemone:URL#model", "llm:URL#model" or "module:attr" (see the module docs). Never downloads.

llm

llm(base_url, model, api_key=None, **kw)

An OpenAI-compatible chat-completions server as a decider (solvi.core.deciders.llm.llm: every option there).

systemone

systemone(base_url, model, api_key=None, **kw)

A model behind the System One HTTP API as a decider (solvi.core.deciders.systemone.systemone).