solvi.memory¶
A memory of corrected cases next to a decider: nearest neighbours with an abstain threshold, trusted sources only.
A memory of corrected cases: the inputs people (or outcomes, or your rules) corrected, and at decision time the nearest of them — a second signal next to the decider, with an abstain threshold, recorded in the trace.
mem = team.memory() # a CorrectionMemory bound to the decision part `team`
mem.add(email, "billing", source="human", by="ann", stored_id=res.stored_id)
mem.learn_from(store) # or every trusted correction of the question in a TraceStorage
mem.calibrate(max_risk=0.05) # the abstain threshold, leave-one-out over the stored corrections
team(email=...) # extra["memory"]: its proposal, the cases it rests on, its fingerprint
What a case is: the decider's probabilities over the options for the input (from the raw logits at the checkpoint's
temperature, before any adapt / fit / teach, so a later fit does not move the stored cases), optionally the input's words
(text=True: a set of hashed words, no embedding model), the label (the decision's label: an option, "yes" / "no", a tuple
for multi-label, "
The proposal: the k nearest cases within radius (total-variation distance between the probability vectors, averaged with
the words' Jaccard distance when text=True), each weighted 1 − distance / radius; the label with the most weight is
proposed when it leads the others by at least min_strength (its weight minus theirs) and holds at least
min_agreement of the weight — otherwise the memory abstains and says why. Ties are broken by case id: the same memory
gives the same answer every time.
What it does with a proposal (mode):
"check" (default) it never answers: when the decider would answer alone and the memory proposes another label, the
decision escalates ("memory of corrections disagrees: …"); agreement is recorded. It can only make
more decisions escalate, so a guarantee from act_guard still holds.
"answer" as "check", and when the decider escalated by its own threshold (act, confidence, margin) and the
memory proposes a label, the memory answers with it; the trace says so (action "answered", the
escalation it replaced) and the part's act_guard promise is not claimed for that answer (the
memory's own promise, from calibrate, is recorded instead). The decision's confidence is then
the model's own probability of that answer — its probs still describe the model — so a
min_confidence meant for the model is not passed on the neighbours' word; their agreement
is in extra["memory"]. The answer is listed among the conformal candidates.
Inside a Cascade / Vote / Route the memory only checks.
Every decision records extra["memory"]: {"fp", "n", "mode", "proposal", "strength", "agreement", "abstain", "neighbours": [{"id", "label", "distance", "weight", "source", "by", "time", "stored_id"}], "action"}; the memory's fingerprint is part of the decision part's fingerprint, so a replay tells a decision made with another memory state.
UntrustedLabel ¶
Bases: ValueError
A label from a source outside TRUSTED_SOURCES (the model, the system itself, an unknown process).
Case
dataclass
¶
Case(id: str, features: tuple, label: object, source: str, by: str | None = None, time: float | None = None, stored_id: str | None = None, words: tuple | None = None)
One corrected case (see the module docstring).
Proposal
dataclass
¶
Proposal(label: object = None, strength: float = 0.0, agreement: float = 0.0, neighbours: list = list(), abstain: str | None = None)
What the memory proposes for one input: a label (None: it abstains, abstain says why), how strongly (the label's
weight minus the others'), the share of the weight it holds, and the cases it rests on.
CorrectionMemory ¶
CorrectionMemory(part, k=7, radius=0.15, min_strength=1.0, min_agreement=0.8, text=False, text_weight=0.5, mode='check')
Corrected cases of one decision part and their nearest neighbours (see the module docstring). Usually made by
part.memory(...), which also attaches it to the part.
features ¶
The decider's probabilities over the options for a text (raw logits, checkpoint temperature), rounded. learning: the features are for a case to store — a model that gave no usable output raises (see decide._usable).
add ¶
Store one corrected case: an input (a text, a state, or Facts by name) and its right answer. source: "human", "outcome" or "rule" — anything else raises UntrustedLabel (the system's own answers are never labels). → the Case (an identical case already stored is not added twice).
learn_from ¶
Add every trusted correction of question (default: the part's name) stored in a TraceStorage (its teach
records: System.teach, save_correction). An input is the correction's state: the part's facts are read from it,
or computed by system (System.facts_for) when the state holds only the inputs they are computed from. →
{"added", "skipped": [(id, why)]} — a correction from an untrusted source or with an answer this decision
cannot give is skipped, never added.
remove ¶
Forget cases by id (e.g. a correction found to be wrong) → how many were removed.
propose ¶
The memory's proposal for an input (a text, a state or Facts) → Proposal.
calibrate ¶
Choose min_strength by conformal risk control, leave-one-out over the stored cases: each case is proposed for by
the others, and the lowest strength is taken at which P(the memory proposes AND is wrong) ≤ max_risk, for inputs like
the stored corrections (inf: it never proposes). A case's twins — cases with the same features (and words) —
are left out with it: a correction stored twice would otherwise vouch for itself. Recorded as the memory's
promise; changes its fingerprint.
→ {"min_strength", "proposed" (share of the cases it would propose for), "error" (among them), "risk", "n",
"guarantee", "radius", "nearest": {"min", "median", "max"} (each case's distance to its nearest other case)} and,
when no case has another within the radius, "note": the memory then proposes for none of them, nothing can be
learned about its proposals, and min_strength is inf (it does not propose) — compare nearest with radius
(its values are None, and the note says so, when every stored case is a twin of every other).
fingerprint ¶
A hash of everything that changes a proposal: the settings, the checkpoint, and every case.
to_dict ¶
The memory as JSON-able data: the checkpoint's hash, the question, the settings and every case (see load_dict).
load_dict ¶
Replace the settings and cases with a to_dict() (strict: refuse cases from another checkpoint or of another question). A label this question cannot give, or cases whose features differ in length, are refused always.
load ¶
Replace the settings and cases with a file written by save (strict: refuse a memory of another checkpoint or question, as load_dict). → self.
apply ¶
Record the memory's proposal on a finished decision and act on it by the mode (see the module docstring). alone: the part decides by its own thresholds (False inside a combination: the memory only checks).
words ¶
The words of a text as a sorted tuple of 32-bit hashes (lower case, 3+ letters) — the cheap text representation.