Questions without a rule: fit, learn_rule, teach¶
Some answers are hard to write as a rule (a risk level, a region from a messy address). solvi offers two ways to learn
them from labeled examples: an answer head (fit) and a readable rule list (learn_rule). Both use the facts that your catalog computes, not raw text.
fit: a learned answer head, corrected instantly¶
history = [(init_state_1, "low"), (init_state_2, "high"), ...] # labeled examples
head = system.fit("risk", history)
print(head.features, head.selection) # the facts it kept, and the leave-one-out error after adding each
print(head.loo_acc) # exact leave-one-out accuracy
system.fit(question, examples, features=None, *, select=None, min_gain=0.0) fits a closed-form ridge head
(solvi.core.deciders.heads.FastHead): one matrix decomposition per ridge strength, so it takes milliseconds to a few seconds, and the
ridge strength is chosen by exact leave-one-out accuracy (head.loo_acc).
- Features are all facts computable from the examples'
init_statekeys, the given keys themselves included (a given number is a feature as it is, without a function around it; a given value that cannot be encoded — a long text, a dict — is left out and named inhead.dropped). Numbers are encoded as a value plus thresholds at training quantiles, booleans as +/-1, strings with at most 20 distinct values as categories, and numeric vectors (a document embedding fromLongSpanExtractor.embedder()) per dimension; when there are few of them, their pairwise products are added so middle classes and interactions can be expressed. - Without
features=, the facts are selected greedily: start from the answers' shares and add the fact that lowers the exact leave-one-out squared error most, while it lowers it by more thanmin_gain× the error of the shares (min_gain=0.0: any decrease). The selected facts become the question's flow, so later requests compute only what the head reads. The squared error is a proper score: a rare answer counts.features=[...]keeps every fact listed;select=Falsekeeps every computable fact;select=Trueselects among the facts listed. - The answer's
whylists the largest feature contributions,probsgives all class probabilities. - A head left with no feature answers the same for every input;
fitwarns when that happens and says why: no fact lowered the leave-one-out error (the facts do not tell the answers apart on these examples), or no fact could be computed from the examples' inputs — every parameter of a part is a fact it reads, one with a default value too (def fn(facts, _nm=nm)waits for a fact_nm).
Before 0.8 fit was a logistic regression whose features were chosen by cross-validated accuracy, and fit_fast was
the ridge head on every feature. Accuracy is a poor guide when one answer is most of the examples: it can keep almost
no fact. The selection by squared error keeps the facts that matter for a rare answer too. On very few examples (a few
dozen) choosing among many facts overfits: pass select=False there. benchmarks/fast_head.py compares the selection
with every fact on the example tasks — accuracy on fresh examples and the fitting time; re-run it on your own data.
fit_fast(...) was fit(..., select=False) in 0.8 and was removed in 0.9.
Its other property is online learning: system.teach(question, init_state, correct) updates the head immediately with
a rank-one Sherman–Morrison step (about 0.1–0.2 ms: benchmarks/fast_head.py, examples/10_learn_in_milliseconds.py) and returns the time in ms. Other questions, rules and hard checks
do not change, and the example still goes to the journal.
head = system.fit("suspicious", history[:10], select=False) # start small: every fact
for state, label in reviewer_corrections:
system.teach("suspicious", state, label) # each one is absorbed at once
Refits as examples accumulate. A rank-one step keeps what the first fit chose: the ridge strength, the featurizer
(number scales and the known values of each category — a value first seen later counts as none of them) and whether
pairwise products are used. Chosen on 10 examples these are often wrong for 300. So the head keeps its examples and,
each time teach doubles their number (at 20, 40, 80, 160, … after a start on 10), fits again on all of them — exactly
a fresh fit on those examples and the facts it reads (the selection is not made again: the flow stays) — then goes on
with rank-one steps. Without refits a head started on a few examples and taught up to a few hundred stays behind a
fit on all of them; with refits it catches up. The cost:
- the update that triggers a refit takes as long as a fit on that many examples, and an early refit that switches pairwise products on (hundreds of columns from few examples) is the slowest — as long as the first fit on those examples would take. The other updates are unchanged, and a refit usually drops the pairwise products a start on few examples switched on, so later updates are often faster;
- the kept examples: their fact rows, about 1 KB each for 14 plain facts (vector facts such as embeddings cost their
length), up to
refit_untilexamples (2000). Past that no refit is due, the rows are dropped and the head goes on with rank-one steps only; - a refit is a change like any update:
teachmakes it at once and it is not gated. An update target of the change pipeline (LearningTarget, below) manages decision parts, not fitted heads: while it is attached withgate_teach=True,teachonly stores the correction and the head (and its refit schedule) does not move. The head depends only on its first fit and the sequence of corrections, so replaying them gives the same head (the same fingerprint); keep acopy.deepcopy(head)to go back.
fit(..., refit=None) turns it off (rank-one steps only, no examples kept); refit=1.5 refits more often.
learn_rule: a readable rule list¶
rules = system.learn_rule("zone", examples, features=["address_upper"], min_support=3, min_precision=0.8, max_rules=40)
print(rules)
This learns an ordered decision list ("if feature then answer", with a default at the end) and installs it in the
catalog as the question's rule. From then on it behaves like a hand-written rule: deterministic, explained by its inputs,
and re-checkable by replay.
facts: the computed facts to build literals from. Booleans givefact is True/False, numbersfact ≈ rounded value, strings give one literal per upper-cased word or number (in any script:"ул. Северная, 12"givesУЛ,СЕВЕРНАЯ,12), plushas number starting 'NN'for numbers of 5 or more digits (postcodes, codes). Other values are compared for equality. A name that is neither a part of the catalog nor a given fact of the examples raisesValueError, and so does an empty list of examples; nothing is installed then.- Each step adds the literal with the best smoothed precision on still-uncovered examples, with at least
min_supportexamples, while precision stays at or abovemin_precision; at mostmax_rulesrules. system.learned_rules[question]keeps theRuleList;printshows each rule with its support.
A learned rule replaces any rule previously registered for that question, and takes precedence over a trained head. Because the list is readable, you can spot rules that only memorized a few examples, or that reproduce a labeling error, and fix the labels or the catalog.
teach: record corrections¶
teach appends the correction to the journal or storage (it does nothing without one). It does not retrain by itself:
read the corrections back (system.storage.corrections(), or the {"teach": ..., "init": ..., "answer": ...} lines) and
include them in the next fit or learn_rule call. Non-JSON values in init_state are stored as JSON (dates as ISO
strings; 0.5 wrote their repr), other objects as their repr.
Each correction keeps where it came from: system.teach("risk", state, "high", label_source="outcome", by="ledger",
of=res.stored_id) — source is "human" (the default: a person corrected or confirmed the answer), "outcome" (what
really happened) or "rule" (your code rejected a model's proposal and decided instead); anything else raises
UntrustedLabel. of is the stored id of the decision it corrects; corrections() returns all three.
Outcomes as labels: system.outcome¶
An agent often sees what its decision led to a few steps later — the parcel was lost, the loan defaulted, Link lost a
heart after the move. outcome turns that into a label of the decision, without a separate labelling pass:
res = system.ask(state) # stored: System(storage=...)
...
system.outcome(res, "west", note="Link lost a heart going east", by="env") # → the label's stored id
system.outcome(res.stored_id, "west") # by id, from another process on the same store
system.guarantee("move", examples, max_risk=0.05, corrections=True) # recalibrate, explicitly, with them
outcome(decision, value, *, question=None, note=None, by=None): decision is the Response saved by the system's
storage, or its stored id (also of a compact record); value the correct answer the outcome shows (as for teach);
question which of the decision's questions (default: its only one); note what happened, in words — kept with the
label (corrections() gives it as "note"; redact erases it with the content); by who or what saw it. It is
stored as a correction with source "outcome" through the same source check as every label, with of = the decision
and the decision's recorded input, so the system report counts it against the decision it labels.
Nothing learns from an outcome and no promise moves by itself. Use the labels explicitly: recalibrate a guarantee on a
schedule or after a drift flag (System.guarantee(..., corrections=True) — the report's "labels" counts them by
source), refit (fit on storage.corrections()), or keep them as knowledge. Recalibrating on every outcome as it
arrives is not in the stable path: measured, it broke the promises — the outcomes an agent sees are not a random sample
of its decisions (a decision that abstained has none), and a threshold moved after every label is no longer the one the
promise was calibrated for. The experimental module that did it was removed in 1.1 for that reason: recalibrate on a
schedule (say, every 500 new outcomes) or when a drift monitor flags the stream.
System 2's answers as labels: label_source="verified"¶
When a slower, stronger solver (an LLM, a person-in-the-loop service — "System 2") handles what your fast system escalates, some of its answers are worth learning from. Its own unchecked answers are not labels — learning from everything it says failed before — but an answer that passed the checks and a guarantee can be one, if you say so and only where it was measured to help:
import random
from solvi import Answer, Catalog, Question, System
from solvi.core.store import JSONLStorage
from solvi.core.store import TRUSTED_SOURCES, VERIFIED
cat = Catalog()
@cat.fn
def score(x: float) -> float:
return x
rng = random.Random(0)
def draw(n): # (input, correct answer)
return [({"x": (x := rng.random())}, x + rng.gauss(0, 0.15) > 0.5) for _ in range(n)]
store = JSONLStorage("decisions.jsonl")
slow = System(cat, [Question("label", "label?", Answer.yes_no())], storage=store) # System 2, sharing the store
slow.fit("label", draw(300), features=["score"])
slow.guarantee("label", draw(300), max_risk=0.05)
res = slow.ask({"x": 0.97}) # answered alone, under its guarantee → can be a verified label
slow.teach("label", {"x": 0.97}, res["label"].answer, label_source="verified", by="system-2", of=res.stored_id)
report = slow.guarantee("label", draw(300), max_risk=0.05, # recalibrate on human + verified labels
corrections=store, sources=TRUSTED_SOURCES + (VERIFIED,))
report["labels"] # {"examples": 300, "corrections": {"verified": 1}, "skipped": 0, "ids": [...]}
What "verified" means is checked when the label is stored: of= must name a decision in the same store that answered
the question alone (status "ok": it passed the hard checks and was not escalated) under a guarantee
(System.guarantee or a part's act_guard), with that very answer — otherwise UntrustedLabel says which condition
failed, and nothing is stored. Human labels, outcomes and rule rejections stay the stronger sources; a verified label
never replaces them.
Where it goes, on measurement (three stand tasks replayed as a stream — product matching, contract clauses, bank requests with new intents — System 2 = an LLM's recorded answers; the details in the best practices):
| channel | takes "verified"? |
why |
|---|---|---|
a guarantee's calibration: System.guarantee(..., corrections=store, sources=...) |
yes, when sources names it |
fed every verified answer, System 1 answered more within its promise on two tasks of three |
a head (fit / teach) |
no — teach(label_source="verified") only stores the label |
gained on one task of three |
| a memory of corrections | no — UntrustedLabel |
broke System 1's promise on the contract task |
a labelled update (LearningTarget of solvi.experimental.changes) |
no — listed in labels()["rejected"] |
its ladder is a head and a memory |
Feed the calibration every answer System 2 vouched for, not only the cases where it disagreed with System 1: a
threshold calibrated on disagreements alone sees only System 1's mistakes, and the verified disagreements were too few
to move anything. The report's labels lists the stored ids it read; calibrating again without them undoes it.
Attaching a learner (experimental, unmeasured)¶
A learner can be attached to a running system and replace the model it decides with only when a measured comparison
says the new version is better (solvi.experimental.stream; no gain is claimed: no stand has run). On a Dispatcher
every decision goes through the loop and each version serves under its own stream certificate; on a System alone the
corrections are the loop's labels, only the pre-freeze gates decide (they cannot be switched off there), and a watch
rolls a promoted version back when it measures worse than the one before it.
from solvi.experimental.stream import AdapterLoop, DefaultPolicy
loop = AdapterLoop.attach(dispatcher, dir="loop/", max_risk=0.05, secret=key, # a LoRA learner on the part
policy=DefaultPolicy(s2_labels=True)) # the slow path's labels count
dispatcher.ask(x).learning # the version that decided, its fingerprint, the loop's coin and events
loop.pause(); loop.resume(); loop.train_now(); loop.rollback(0); loop.status()
loop.save("ckpt/") # AdapterLoop.load("ckpt/", learner, system=..., secret=key).attach_to(dispatcher)
dispatcher.replay_all() # each decision replayed under the version it was made with
loop.detach() # version 0 back, hooks removed
loop = AdapterLoop.attach(system, learner, initial=v0) # a System alone: system.teach(...) feeds it (label mode)
loop.status()["watch"] # label mode after a promotion: the served version against the one before it
DefaultPolicy trains when the labels grew and the served version is short on its last held-out labels: it errs
alone above α, or answers alone on less than target_share (0.5 by default) of them. The share is there because a
certified version keeps its error alone at most α: on error alone a weak but certified version, short in coverage and
not in error, is never retrained. The share is read on held-out labels, a selection-biased sample that is mostly
deferred decisions, so it reads lower than the share on the whole stream, and a strong version will usually be
retrained too; the promotion gates (the novel-only paired comparison, the memorisation probe, the novel-risk bar) are
what keep a worse version from being served. DefaultPolicy(target_share=None) trains on the error alone only. The
default is unmeasured.
The control is a LoopPolicy subclass: give each hook, and the loop calls them.
from solvi.experimental.stream import LoopPolicy
class OnlyHuman(LoopPolicy):
def accept(self, st, x, y, source): return source == "human"
def split(self, st, key): return ("holdout", "calib", "train")[hash(key) % 3] # use a stable hash in practice
def want_train(self, st): return st.labels["train"] >= 100 and st.served == 0
def select(self, st, rows): return list(rows)
def hyper(self, st): return {"seed": st.next_version}
def freeze(self, st, report): return report.right - report.wrong > report.served_right - report.served_wrong
def promote(self, st, paired): return True
def discard(self, st, paired): return None
def alarm(self, st, who, report): return "recertify"
# optional: def watch(self, st, w) — label mode's rollback after a promotion (default: False, never;
# DefaultPolicy's rule is watch_rollback(w), an anytime-valid e-process and the margin)
A version is promoted only when it is certified (label mode: a threshold found on the calib labels), whatever
promote returns.
A learner is any object with update, seen and either snapshot (online) or train(job) (batch); its versions
score (score(x) → (answer, signal), fingerprint()) and, to serve, install(system) / uninstall(system). Refused:
a question answered by a Cascade, Vote or Route, or by a part that is a member of one. With a loop attached
System.teach changes nothing at once (the loop owns the part). The labels the default policy takes are a person's,
an outcome's, a rule's or an audit's; the slow path's only with s2_labels=True. A loop on a Dispatcher needs a label
source: an attach whose policy takes no label from the slow path is refused (give s2_labels=True, a trusted oracle as
the slow path whose Thoughts carry {"oracle": {"label_source": "human"}}, or use label mode on the System); refused
labels are counted in status()["labels"]["refused"] and 200 in a row with none accepted warn once. With a loop, a Dispatcher serves one decision at a time, the slow
path's call included (measured once on a stand-in, the script is not in the repository: 3.7 × slower at four threads and a 50 ms slow path). A Cascade, Vote
or Route over a part a loop serves is refused until detach(). Unmeasured: nothing here says the loop improves a
system. See experimental.