Skip to content

solvi.experimental.changes

One change pipeline for four sources of change (experimental): solvi.experimental.changes.Pipeline(stages, storage, store=...).

One change pipeline for four sources of change (experimental): a document version, verified experience, a person's correction, a search proposal — every one a candidate that passes the same pluggable stages, is journaled, promoted into what already holds that kind of knowledge, and can be rolled back.

from solvi.experimental.changes import Change, Pipeline, Static, Consistency, CleanCases, FreshEnvironments, hazard_body

ch = Change("outcome", payload={"kind": "rule", "body": hazard_body("lava", {"next": "lava"}, ["step"])},
            proposer="model:x", reason="died on lava 3 times", evidence=["dec-17", "dec-42"],
            expected={"failures": "-1 per 10 episodes"}, change_type="rule")
pipe = Pipeline([KnownFailures(), Static(), Consistency(), CleanCases(successes), FreshEnvironments(...)],
                storage="changes.jsonl", store=ks, approval={("rule", "outcome"): "gate"})
res = pipe.propose(ch)              # → Result: "promoted" (with a version) | "rejected" (the stage) | "held" | "error"
pipe.rule(res.change_id, True, by="ann")    # a person's ruling for a held change (or a compile Ruling keep / drop)
pipe.versions(); pipe.current; pipe.rollback(0); pipe.lineage(res.change_id); pipe.observe(res.change_id, {...})
pipe.revert(res.version)            # undo this version alone (rollback(v) undoes every version after v)
w = pipe.watch(res.version, promise={"north": 0.10, "rare": 0.10})   # ShipWatch: per-slice e-process after ship
w.observe(ok, slice="rare")         # → None | {"event": "reverted" | "graduated" | "ended", ...}

What it is: a generalisation of the learning loop of 0.7–1.0 (a candidate, gates with recorded numbers, promote / rollback, versions, a changelog) from one source (labels) to four, with the stages as objects like the knowledge store's write gates; the loop itself was removed in 1.1 and its ladder and gates are the "update" target's. Nothing here stores knowledge: an admitted change goes into what already holds it (_targets.py) — a KnowledgeStore item ("item": fact, rule, skill, action, with its sub-kind in body["kind"], e.g. "hazard", and the change type in body["change"]), a compiled catalog version ("compiled": solvi.core.versions.Versions), or a labelled update ("update": LearningTarget(system, ...) runs the update ladder — fit, a memory of the corrected cases, the adapter hook — and its gates (consistency, heldout, honesty, act_guard, size) on the stored labels; each gate's numbers are a stage verdict of the change, "update:", and the first that fails refuses it).

The candidate. Change(source, artifact="item", payload=...): source is "document", "outcome", "person" or "search"; it maps onto the knowledge store's SOURCES as document → "spec", outcome → "outcome", person → "person", and search → "outcome" with by = "search:", and only with evidence (Static refuses a search proposal without it) that the confirmation stage confirmed: a search proposal is promotable only when Confirmation.on_evidence ran and admitted it (else rejected: "search proposal: confirmation on its evidence did not run"), so the store's SourceGate keeps refusing the system's own unverified answers. Also: the proposer and the reason, the evidence, the proposer's expected effect (recorded; observe later records what happened next to it), the change type (fact / rule / meaning) when known, the parent version, rule code to sandbox-check (code), and the rule's executable form (check, not journaled; its code fingerprint is).

The stages (_stages.py; each run(change, ctx) → StageVerdict admit / refuse / hold with measurements): KnownFailures (a candidate rejected before is refused — new), Static (wraps SourceGate, KnowledgeStore.proposal and the compiler's sandbox check), Consistency (wraps ConsistencyGate; records key conflicts and narrows / widens), Confirmation (wraps a compilation's acceptance or a replay on the evidence), CleanCases (wraps Agenda.dry_run), Impact (wraps decision_diff / solvi.core.store.diff.diff), FreshEnvironments (ported: paired selection, deterministic replay, confirmation with a bootstrap lower bound > 0, rules never crash, progress not lower), Approval (new: a table (kind, source) → "gate" | "gate+person" | "person"), Canaries (wraps solvi.testing.honesty; a suspiciously perfect score holds), RefusalRegression (new, not switchable: the share refused on a held should-abstain set before and after the change; a change that answers what was refused before is refused — refusal is the first casualty of self-change), SliceGuard (new: per-slice paired losses against the version in force and against the original version — harm in a small slice under an overall gain, and a chain of small changes each harmless on its own, are refused).

Refusal regression cannot be switched off. For every change whose target can answer (Pipeline(answer_of=...), or a target with answerer(change or None, ctx): VersionsTarget, LearningTarget, a target of yours — an answerer, a threshold, a head, a gate) the pipeline runs RefusalRegression — inserted before Approval when it is not among the stages, a configured one of yours in its place — and journals its measurement; its refusal is final under every approval level. The held set is Pipeline(refusals=), the refusals other stages keep (CleanCases / Confirmation (refusals=)) and the stage's own; without any, the change is held ("no should-abstain set declared for a target that can answer") until a person rules — so every pipeline whose target can answer declares a set. The set follows the policy version (a document version may rightly change what goes to a person). An item change (KnowledgeTarget) is checked only with answer_of=: an item answers nothing by itself.

Decision on approval "person": the automated stages still run and are journaled, but their refusals are advisory (recorded with "advisory": true; the person decides) — except the static stage's, which stays final (a malformed change cannot be promoted by anyone).

Hazard rules: the stored artifact is a rule item with body["kind"] == "hazard" (hazard_body); no second item is written through Agenda.gate. gates_of(store, scope=None, parse=...) is a read-only view of the active hazard items as (check, blocks) gate checks, by the same code path the dry run uses (gate_of), so an agent installs them live. The condition language is supplied by the caller through parse= (lawlang + a macro catalog is the decided form); equality of state keys (hazard_gate) is the placeholder.

The journal (_journal.py): records of kind "change" in a TraceStorage — "propose" (the change, its fingerprint, parent), one "verdict" per stage, then "promoted" (version, target reference), "rejected" (the stage and reason), "held", "error" (a stage raised: the stage and the exception), "ruling", "rollback" and "observed"; promoted / rejected / held carry the approval level ("level"). The storage's hash chain covers them (a tampered record fails verify()); with the decisions' store they share one chain. Versions: version 0 is the state before the first change; each promotion is the next number with the version in force as its parent; rollback(v) undoes every version after v through its target (KnowledgeStore.rollback for items) and is itself a record; a rollback with a reason (why=) records the undone changes as failed, and KnownFailures refuses them as it refuses a rejected one (failed=False: a rollback that is not a verdict on the change, e.g. moving to another branch). A stage that raises is not a verdict on the change either: the proposal ends with status "error" (an "error" record with the stage and the exception), the failure memory does not keep it, and the same candidate runs again once the stage is fixed (Pipeline(raise_stage_errors=True): journaled, then raised). EvaluatorLock (ported): held-out seeds pinned by SHA-256, the evaluator's code pinned, an access log, lock_out() for proposer processes, leak_scan.

What is checked: tests/test_changes.py — the chain verifies and a tampered record is caught; a hazard rule from "outcome" passes static → consistency → clean cases → fresh environments (a toy corridor) → approval and is an active rule item, listed in versions and undone by rollback; a rejected candidate is recorded with its stage and refused when proposed again; approval "person" holds until a ruling; a search proposal needs confirmation on its evidence; gates_of reads the active hazard items without writing; Static refuses a candidate catalog with a part that reads a sensitive field (Static(sensitive=), Catalog(sensitive=)) or a citation not in the text (Static(text=, spec=)); Impact refuses an exception whose diff moves decisions outside expected["only"] and admits the exact one.

Measured (2026-10, two setups; the credit one is benchmarks/changes_credit.py, the bank-request stream has no script in the repository): on a credit policy the pipeline carried a document version (144 of 144 changed decisions traced to their clause), admitted 12 of 12 exact exceptions, held every widening one for a person and refused 8 of 8 exceptions that touched other decisions, 0 extraneous changes, 14 versions replayed; on a bank-request stream with a person's labels every one of 22 candidates passed every stage — the run with the gates equals the run without them — so the stages added nothing there. The module is journal, impact analysis, approval levels and rollback — infrastructure, not a claim that its gates admit fewer harmful changes.

What is not done: - no rewind over an Environment (Confirmation.on_evidence takes the replay as a callback); - narrows / widens is proved over the rule's text only for a subset of Python (comparisons, and / or / not, arithmetic on numbers, membership in literal sets; Consistency(prove=True), the smt extra); outside it, it is measured on probe inputs through the answerers (Consistency(probes=), opt-in) or taken from a callback; - no regression-scenario stage (solvi test cases.json); - the should-abstain set is declared by the caller: none is derived (stored abstentions are not one — a correction may rightly make the system answer them), and an empty set holds the change for a person; - no scope flag on a world change for rules from experience; - no validity periods in compile Versions (nor the change-type classifier in the compiler); - Static only checks code; IsolatedTests (opt-in, _isolated.py) runs it on test cases in the compiler's process sandbox (solvi.experimental.compile.sandbox.run), the expected answers kept in the parent; - a held change lives only in this process's memory until it is ruled on (its journal records survive, the Change object does not); Pipeline.expire_held(max_age) rules on the ones that waited too long.

IsolatedTests

IsolatedTests(cases, *, entry: str = 'main', source: Callable | None = None, compare: Callable | None = None, max_failed: int = 0, mem_mb: int = 1024, cpu_s: int = 60, wall_s: int = 120, item_s: int = 5, allowed=(), runner: Callable | None = None, cost: float = 100.0)

The candidate's code (change.code, or source(change, ctx)) run on test cases in an isolated process (new; opt-in).

cases: [{"id"?, "input", "expected"}] or [(input, expected)] — an input is the entry function's argument list (a list or tuple) or its keyword arguments (a dict); values are JSON — or cases(change, ctx) → such a list. entry: the function of the module that is called. Only the inputs are sent to the child; the expected answers are compared in this process by compare(got, expected) → bool (default strict_equal: the same type and value). Limits: mem_mb, cpu_s and wall_s for the whole run, item_s per input (an input that runs longer raises in the child and fails). allowed: more modules the code may import (as for the sandbox's check).

Refused when the code does not load (the sandbox's check refuses it, or it raises on import), the run crashes or exceeds a limit, or more than max_failed cases fail (a wrong answer, an exception, a value that is not JSON); a change with no code is refused ("no code to test"). A failure of the sandbox itself (the interpreter, the driver) is not a verdict on the change: the stage raises, and the pipeline ends the proposal with status "error". The verdict's numbers: n, passed, failed, and the first failing ids with what came back. runner: the sandbox's run (replaceable for tests). cost: the stage's cost for Pipeline(order="cheap_first").

ChangeJournal

ChangeJournal(storage)

Records of kind "change" in a TraceStorage (a path, a TraceStorage, or None / ":memory:" for a MemoryStorage; see the module docstring).

write

write(action, **data)

Append one record → the stored record (with its id, seq and hash).

records

records(action=None)

The journal's records in order (dicts), optionally only those of one action.

fold

fold(key, init, step)

A reader's state over the records, kept up to date: init() → the state, step(state, record) for each record in order; later calls apply only the records written since (all again when the storage was re-read). → the state.

verify

verify()

The storage's chain check (TraceStorage.verify) → {"ok", "problems", ...}.

EvalLocked

Bases: Exception

The evaluator refuses: a pinned file or code changed, or this process is locked out.

EvaluatorLock

EvaluatorLock(seeds_file, *, pinned=None, code=None, pins=None, access_log=None, env=LOCK_ENV)

Held-out seeds pinned by hash, the evaluator's code pinned by hash, an access log, a lock-out for candidates.

seeds_file: a JSON file {"seeds": [...], ...} written once (EvaluatorLock.create); pinned: its SHA-256 (fixed at pre-registration); code: {name: a path or a function} that make up the evaluator; pins: {name: SHA-256} for them; access_log: a TraceStorage or path for the access records (kind "eval_access", hash-chained); env: the variable lock_out() sets (inherited by child processes).

create staticmethod

create(path, n=None, *, seeds=None, low=1000000000, high=2 ** 31 - 1, purpose='held-out evaluation')

Write n random seeds (os.urandom, not a range: knowing the rule does not reveal them; default 500) once, read-only → the file's SHA-256 (the pin). Refuses when the file exists.

seeds: your own held-out seeds instead (distinct ints; also accepted in place of n: EvaluatorLock.create("eval_seeds.json", [11, 12, 13])) — then keep them out of every file a proposer reads.

manifest

manifest()

The current hashes: {"seeds": sha or None, "code": {name: sha or None}} — pin these at pre-registration.

verify

verify()

Raise EvalLocked unless the seeds file and every pinned piece of code match their pins → True.

lock_out

lock_out()

Call at the start of every proposer / search process: it and its children cannot read the seeds.

seeds

seeds(purpose, *, by=None)

The held-out seeds — for the pre-registered evaluation only. Refused in a locked-out process; verified; Granted reads and locked-out attempts are access records. Verification errors before a read are not counted as completed accesses.

access_count

access_count()

How many times the seeds were handed out (the access log's records; "evaluate once" = 1).

leak_scan

leak_scan(paths, max_bytes=200 * 2 ** 20)

Files under paths that mention a held-out seed (journals, prompt caches, candidate sources) → [(path, [seeds...])].

MemoryStorage

MemoryStorage(system=None, clock=None, *, record='full')

Bases: TraceStorage

A hash-chained TraceStorage kept in a Python list: the change journal of a run that needs no file (many short pipelines, tests). Nothing persists past the process; verify(), redact and the rest work as on disk (it passes solvi.testing.conformance.check_storage).

Approval

Approval(table: dict | None = None, *, default: str = 'gate+person', widen: str = 'gate+person')

Who approves: table {(artifact kind, source): "gate" | "gate+person" | "person"} (the artifact kind is the item's kind for an item — "rule", "fact", ... — else "compiled" / "update"; a key may also be the kind alone); default for a pair not in the table. "gate": admitted. "gate+person" and "person": held until a person's ruling (Pipeline.rule), then admitted or refused. A change the consistency stage classified as widening behaviour is raised from "gate" to widen. Decision: under "person", the automated stages still run, and their refusals are advisory (recorded; the person decides), except the static stage's.

Canaries

Canaries(run: Callable, *, baseline: dict | None = None, tolerance: float = 0.02, perfect: Callable | None = _perfect)

A honesty run of the candidate: run(change, ctx) → metrics (solvi.testing.honesty.metrics's dict, or your own numbers in the same direction). Refused when a gated number is worse than baseline by more than tolerance (solvi.testing.honesty.compare); held — a flag, not a success — when perfect(metrics) says the score is suspiciously perfect (default: every case answered and none wrong, on a set built so that a perfect score proves access to the answers: a removed test, a reference the candidate should not see). Ready-made: Canaries.honesty(system_of, cases) asks the candidate system the cases (honesty.load_set(path)["cases"]).

Canaries.drawn(run_cases, pool, k) (opt-in): a fixed set is one a proposer that sees it can tune to; this one draws k cases per change from a pool, seeded by the stage's secret (random per stage unless secret= is given) and the change's journal id, so neither the caller nor the proposer picks or predicts them; run_cases(change, ctx, cases) → metrics. Recorded: "drawn", "pool", and "commit" — a hash of the drawn ids (with the secret the draw can be re-made and checked; the ids themselves are not journaled). It helps only while the proposer does not see the pool.

CleanCases

CleanCases(records, *, gate: Callable | None = None, parse: Callable = hazard_gate, max_share: float = 0.0, refusals=())

How many recorded successes the change would block: the change installed as a gate of an empty solvi.core.knowledge.Agenda and dry_run(records) over (state, action) pairs that worked (windows without a failure, accepted calls). Refused when the share of records it blocks exceeds max_share (default 0: a hazard rule may not block anything that worked). records: a list or records(change, ctx); gate: change → (check, blocks) (default gate_of with parse); parse: the condition language, body → (check, blocks) (default hazard_gate). refusals: the should-abstain inputs kept next to the clean cases (RefusalRegression reads them).

Confirmation

Confirmation(check: Callable, *, refusals=())

Independent confirmation by a callback check(change, ctx) → a StageVerdict, bool or dict. Two ready-made ones:

Confirmation.compiled() a "compiled" artifact is confirmed when the compilation was accepted (two drafts agree on the pool, separate tests pass, the person's rulings hold — solvi.experimental.compile); Confirmation.on_evidence(replay) every evidence id must be confirmed by replay(eid, change) → bool (an outcome rule must fire on what it was learned from, e.g. rewound k steps before the failure). The ids are the proposer's: with exists(eid, ctx) → bool (opt-in; e.g. a lookup in the decisions store) every id must be in the record — an id that is not refuses the change — and each counts once (a repeated id does not raise the share); min_evidence (with exists=): the fewest distinct ids.

refusals: should-abstain inputs kept with the confirmation's examples; RefusalRegression reads them.

Consistency

Consistency(*, classify: Callable | None = None, probes=None, refused: Callable | None = None, prove=False, before=None, types: dict | None = None, outcome=None)

The knowledge store's ConsistencyGate on the proposed item (premises exist and are present; not self-defeating), plus what it measures: the active items on the same key holding another value (conflicts: the store resolves them by source rank or asks a person — not a refusal here) and, with classify(change, ctx) → "narrows" | "widens", the direction of the behaviour change (Approval upgrades "gate" to "gate+person" for a widening change); classify may also return a dict {"direction": ..., other measurements} — all of it is recorded. An artifact other than an item is admitted with only the classification.

probes (opt-in): the library measures the direction instead of taking the caller's word. Each probe input (a list, or probes(change, ctx) → inputs) is asked with the answerer in force and with the candidate's (the pipeline's answer_of= or the target's answerer, as RefusalRegression asks them); refused(answer) (default is_refusal) decides. "widens" when the candidate answers a probe the version in force refused, else "narrows" when it refuses one that was answered, else "same"; recorded with the counts ("widened", "narrowed", "probes"). A classify given too is a claim: recorded as "claimed", and the direction is "widens" when either says so. When the answerers cannot be had the direction is "widens" ("unmeasured": a person looks).

prove (opt-in, needs the smt extra: pip install "solvi[smt]"): the direction is proved over the rule's text (prove_direction: z3 over a subset of Python — comparisons, and / or / not, arithmetic on numbers, membership in literal sets). The candidate's rule is change.code (else the source of change.check), the rule in force is before= (its text or function); prove may instead be prove(change, ctx) → (rule in force, candidate rule). A proof is recorded as "proof" (proved, witness inputs, sorts, ms); a decided proof sets the direction ("both" → "widens") and the probes are not asked; "unknown" (a construct outside the subset, no rule in force, the solver gave up) falls back to the probes when given, else to classify. types=, outcome=: as in prove_direction.

Context dataclass

Context(pipeline: Any, store: Any, journal: Any, change_id: str, verdicts: dict = dict(), level: str | None = None, ruling: tuple | None = None)

What a stage sees besides the change: the pipeline, its KnowledgeStore (or None), its journal, the change's journal id, the verdicts so far (stage name → StageVerdict), the approval level, and the ruling (ok, by, note) once a person gave one.

FreshEnvironments

FreshEnvironments(env: Callable, agent: Callable, *, select, confirm, score: Callable | None = None, progress: Callable | None = None, max_steps: int = 200, boot: int = 2000, seed: int = 0, min_lower: float = 0.0, progress_tolerance: float = 0.0, replay: int = 2, on_episode_start: Callable | None = None, handed_over: Callable | None = None, failure_policy: str | None = None, max_failure_rate: float | None = None, world_registry=None, candidate_ids=None, candidate_id=None, world_id=None, alpha=0.05)

The candidate against the current agent on fresh worlds of an Environment, paired by seed (ported from a gated evolution harness).

env: a function → a new Environment; agent: a function (change or None, ctx) → a policy (state, actions) → action (None: the current agent); select / confirm: seeds, or functions (change, ctx) → seeds — two disjoint sets of worlds never used before (draw them fresh per candidate; never the held-out evaluation seeds, see EvaluatorLock); score: row → number, higher is better (default: 1 when the episode did not fail); progress: row → number (default the episode's info["progress"]). on_episode_start: a function (policy, seed) called before every episode, with the policy about to play and the world's seed — to reset a policy's memory between worlds, seed its own randomness or log; None (default): nothing.

The bar, in order: (1) selection — the mean paired gain on select is > 0; (2) the candidate's rules never crash (a policy that raises); (3) replay — the candidate replays its first replay selection worlds with the same actions; (4) confirmation — on confirm, the bootstrap lower bound of the paired gain is > min_lower; (5) progress not lower — the upper bound of the paired progress change on confirm is ≥ −progress_tolerance (a rule that makes the agent overcautious is refused). Everything measured is recorded.

world_registry (opt-in): an owner-declared world namespace in the Pipeline journal. Requires a frozen finite candidate_ids list, candidate_id(change, ctx), world_id(seed), and alpha. Both sets are reserved durably before agent evaluation, never refunded, and refused on reuse. JSONL must be nonrotating with fsync=True; SQLite uses its write transaction. The nominal bootstrap interval uses level=1-alpha/len(candidate_ids), not a distribution-free FWER certificate. Identities and unpredictable fresh sampling remain the owner's responsibility.

failure_policy (opt-in, default None): an independent veto on BOTH sets, using the environment's info["failed"] convention even with a custom score. "zero_failures" refuses any candidate failure; "no_new_failures" refuses any paired safe-baseline → failed-candidate world; "max_failure_rate" refuses an observed candidate failure share above the explicitly required max_failure_rate in [0, 1]. The latter is a sample cap, not a population or research guarantee. Counts, policy and veto reasons are recorded per set. A cap without "max_failure_rate" is an error. Defaults and score/progress/bootstrap bars are unchanged.

handed_over (opt-in): handed_over(row) → bool, a world the candidate handed over (abstained, escalated). When progress is the share answered, an honest change that hands over more is "lower progress" and was refused: with handed_over, such a world takes the version in force's progress in the progress comparison (a refusal is neither progress nor regress; its score — a failure avoided — still counts). Recorded as "handed_over" per set.

Impact

Impact(measure: Callable, *, max_share: float | None = None)

The share of past decisions the change would change. measure(change, ctx) → {"changed", "total"} (or a DiffReport / DecisionDiff); refused when changed / total > max_share (None: only recorded), or when the measure's dict says "ok": False (the caller's own rule; its "reason" is the refusal's).

Exact exceptions: a change whose expected["only"] lists the decisions it may change (stored decision ids; with a DecisionDiff over inputs, the inputs' ids) is refused when the diff moves any other — reason "changes N decision(s) other than the allowed ones: [the first ten ids]" — and the measurement records extraneous (the ids outside only). A dict measure gives the ids it changed as "ids" (or its own "extraneous"). Ready-made:

Impact.of_system(storage, system_of) solvi.core.store.diff.diff(storage, system_of(change, ctx)) — the stored decisions re-run with the candidate system; Impact.of_compiled(storage, old_of) solvi.core.versions.decision_diff(old_of(ctx), change.payload, store=storage) — with the clauses that cause each change.

KnownFailures

KnownFailures(*, rollbacks=True, key=None, context=None, ttl=None, clock=None)

A candidate whose identity (identity(): Change.fingerprint's source, artifact, content and code, with a keyed item's body read as its subject, relation, value and kind only) was rejected before, or was rolled back with a reason (Pipeline.rollback(why=...), failed=True), is refused with the earlier stage and reason — a search does not walk in circles, and an extra key in a rule's body does not make a rejected rule new. A stage that raised (status "error") is not a failure: the candidate runs again. Reads the change journal (kept up to date incrementally).

By default a failure is remembered for ever and everywhere. context(change) → a JSON value (the tests' version, a season, a regime): a failure counts only for a candidate proposed in the same context — a change rejected under old tests is retried under new ones. ttl (seconds): a failure older than that no longer counts (as solvi.core.knowledge.FailureMemory unblocks a plan after a window); the age is clock() (default time.time) minus the journal record's time, so give the journal's storage the same clock. key(change) → a JSON value replaces identity() (two candidates with the same key are one). The key and context are journaled in this stage's verdict (measured["memory"]), so the memory survives the process. A refusal by this stage does not renew a failure.

rollbacks: do rollbacks with a reason count as failures (default True; False: only rejections do).

RefusalRegression

RefusalRegression(*, refusals=(), refused: Callable | None = None, question: str | None = None, tolerance: float = 0.0, count_tolerance: int = 0, cumulative: bool = False, original=None, coverage: str = 'declared', sample: int = 50, pool=None, seed: int | None = None, missing_answer: str = 'skip')

Refusal is the first casualty of self-change, so every change whose target can answer — an answerer, a threshold, a head, a gate — is checked on a held should-abstain set: outside or unknown inputs the system must refuse or hand over.

The set: the pipeline's refusals= (a list of inputs, or refusals(change, ctx) → inputs), the refusals kept by any other stage (CleanCases(refusals=), Confirmation(refusals=), a stage of yours with a refusals attribute), and this stage's own refusals=. Each input is asked with the answerer in force and with the candidate's — answer_of(change or None, ctx) (the pipeline's answer_of=, else the target's answerer(change or None, ctx)) — and refused(answer) (default is_refusal; question= picks one question of a Response) decides.

It records: n, refused before / after, the shares, lost (inputs refused before and answered after — a change that adds refusals elsewhere does not hide them), their first ids, and the count per source of the set. Refused when lost > max(count_tolerance, floor(tolerance · n)); default tolerance 0.0 and count_tolerance 0: one input that was refused and is now answered refuses the change (raise count_tolerance — e.g. to 1 — for an answerer that flips a borderline input between runs). Refused too when the candidate's answerer cannot be had ("cannot measure"). With nothing in force before (no answerer for None), nothing can regress: admitted, recorded. With an empty set the change is HELD ("no should-abstain set declared for a target that can answer") until a person rules: a check that cannot be switched off must measure something, and a vacuous pass would be a switch in disguise — so every pipeline whose target can answer declares a set. The set follows the policy version: a document version may rightly change what goes to a person, so declare the set of the version in force.

cumulative (opt-in): the version in force moves with every promotion, so a chain of changes each losing up to the allowance adds up ("salami"). With cumulative=True the inputs are also compared with the original version: which of them it refused is pinned (by example id) the first time this stage measures — or given as original, {id: refused} (a dict, or original(change, ctx) → one; pass it to keep the pin across processes) — and a change is refused too when the inputs refused originally and answered by the candidate are more than the allowance (recorded as "lost_original"; inputs the original never saw are compared with the version in force only).

coverage (opt-in): "declared" (default) measures the declared set only, so a proposer who sees it can tune a change to keep exactly those inputs refused. "sampled" also draws sample inputs at random from pool — a list of past inputs, pool(change, ctx) → inputs, or a TraceStorage (the inputs of its recorded decisions) — for every change; those the version in force refuses join the set (source "sampled"). The draw uses seed (None: a fresh seed per change, recorded with the measurement so it can be repeated) and is recorded: drawn, refused, their ids.

missing_answer (opt-in): the pipeline inserts this stage only for a change whose target can answer, so a change to a knowledge item with no answer_of= passes unmeasured. "skip" (default) keeps that: a stage of yours among the stages admits such a change unmeasured. "refuse": a stage of yours among the stages runs on every change and refuses one whose refusals cannot be measured (no answer_of= and no answerer on its target) — declare answer_of= for the pipeline, or a target with an answerer, for every change you want through.

Not switchable: Pipeline inserts this stage (before Approval) for every change whose target can answer, when it is not among the stages; a stage of yours (other tolerances) is used in its place. Its refusal is final under every approval level (like the static stage's: a person's "person" level does not make it advisory).

held_set

held_set(change, ctx)

The should-abstain inputs for this change and how many each source gave → (inputs, {source: count}).

SliceGuard

SliceGuard(measure: Callable, *, original=None, tol: float = 0.02, cumulative_tol: float | None = None, alpha: float = 0.05, min_n: int = 30, on_small: str = 'hold', pin_file=None, pinned: str | None = None)

Per-slice non-regression, against the version in force AND against the original version (new).

An aggregate gate passes harm in a small slice under cover of an improvement elsewhere; a gate that compares only with the version in force passes a chain of small changes ("salami") each under the tolerance and not significant on its own. This stage compares every slice, both ways.

measure(change, ctx) → {slice: (in_force_losses, candidate_losses)}: per-case losses (lower is better) of the version in force and of the candidate on the SAME cases, in the same order (paired). original: the original version's losses on those cases, {slice: losses} — a dict, or original(change, ctx) → one; None (default): pinned from the in-force losses the first time this guard runs (so "original" is the version in force then; the pin lives in this object — pass the dict, or a pin_file, to keep it across processes); False: no comparison with the original. pin_file: a path (opt-in, with original=None) — the pin kept in a file instead of this object: written once, read-only, at the first run in any process ({"slices": {slice: losses}}, slice names as strings), read by every later run, so a new process (a new guard) compares with the same original. Its SHA-256 is in the verdict's numbers ("original_file"); pinned: that SHA-256 fixed elsewhere (next to EvaluatorLock's pins) — a file that differs, or is missing, refuses the change.

A slice is refused against a reference when the mean excess d = mean(candidate − reference) is above the tolerance (tol against the version in force, cumulative_tol (default tol) against the original), or when d > 0 is significant: one-sided z at alpha divided by the number of comparisons made (Bonferroni — valid however the slices depend on each other; within a slice the cases are taken as independent for the standard error). An excess with no spread at all (every case worse by the same amount) counts as significant. A slice with fewer than min_n cases, or one the original has and the measurement lacks: on_small = "hold" (default: a person decides) | "refuse" | "admit". A slice the original lacks is compared with the version in force only (listed as "new"). The verdict's numbers: per slice {"n", "in_force": {"d", "se"}, "original": {"d", "se"}}, the z used, "small", "new".

z

z(m)

The one-sided critical z for m comparisons (Bonferroni).

Stage

Bases: Protocol

A stage of the change pipeline.

You implement: name and run(change, ctx) → StageVerdict (admit / refuse / hold, with measurements).

You get for free: it runs in the pipeline's order; its verdict is a journal record (kind "change"); a refusal rejects the change (recorded with the stage, so the same candidate is refused next time); a hold keeps it until Pipeline.rule(...).

Stability: experimental.

Static

Static(*, parse: Callable | None = None, evidence_exists: Callable | None = None, sandbox: bool = True, sensitive=None, catalog_of: Callable | None = None, text: str | None = None, spec=None)

Well-formedness before anything is run.

Checks: the source is one of the four; a "search" proposal carries evidence (it enters the store as an observation, so it must point at what was observed); an item's kind and body are accepted by KnowledgeStore.proposal and its source by the store's SourceGate; change.code (rule code) passes the compiler's sandbox check (solvi.experimental.compile.sandbox.check: pure functions, standard-library imports); each evidence id exists (evidence_exists(eid) → bool, e.g. a TraceStorage's record or an experience log); parse(change, ctx) → a list of problems (e.g. a lawlang parse).

A candidate that carries a catalog (catalog_of: a Catalog or a System as the payload, or as its "catalog" / "system" key; a catalog version — solvi.core.versions.CatalogVersion, a Compiled too — when sensitive or text is given) is refused when solvi.check.lint flags a part that reads a sensitive field (reads_sensitive, reads_sensitive_via): the catalog's own Catalog(sensitive=...) and sensitive here; and, with text (the policy its parts cite), when solvi.check.citations finds an error — a quote not in the text, and with spec (a Spec) a clause that does not exist or does not hold the quote. catalog_of(payload) → a Catalog / System or None replaces how the catalog is found.

StageVerdict dataclass

StageVerdict(decision: str, reason: str = '', measured: dict | None = None, stage: str = '')

A stage's answer about one change: "admit", "refuse" or "hold" (wait for a person), why, and what it measured.

of classmethod

of(x, stage='')

A callback's answer as a StageVerdict: a StageVerdict, a knowledge-store gate Verdict (admit: bool), a bool, a dict {"ok": bool, "reason"?, ...measurements}, or None (admitted, nothing measured).

LearningTarget

LearningTarget(system, storage=None, *, parts=None, ladder=None, gates=None, changelog=None, holdout=0.3, calibration=0.2, gate_teach=True)

Labelled updates as candidate changes: LearningTarget(system, storage=None, *, parts=None, ladder=None, gates=None, changelog=None, holdout=0.3, calibration=0.2, gate_teach=True) — the arguments of the ladder and gates (_updates.Updates; their defaults in _updates.LADDER / GATES). While a target is attached, System.teach only stores a correction (gate_teach=False keeps the instant update; detach() ends it) and a Change(artifact="update") learns from the stored labels: promoted with the version of the update, or refused at the first gate that failed.

parts property

parts

{question: the decision part an update changes}.

current property

current

The update version in force (None when the live state is no recorded version).

answerer

answerer(change, ctx=None)

input → Response: the live system for None; for a change, the candidate the ladder would build from the stored labels on a shadow (nothing promoted or recorded) — the system as it is when there is nothing new.

restore

restore(version)

Restore any recorded version of the update (also one recorded by another process on the same store) → it.

detach

detach()

Stop gating System.teach (the update records stay).

labels

labels()

The trusted labels → {"labels": [Label], "rejected": [(id, why)]}.

history

history()

Every update record (kind "update"), in order.

versions

versions()

The promoted update versions → [{"version", "fp", "time", "action", "id"}].

fingerprint

fingerprint()

The fingerprint of the updated parts' state.

version_of

version_of(part_fp)

The update versions in which a part had this fingerprint (as a decision's trace records it).

Promoted dataclass

Promoted(ref: Any, verdicts: list = list())

A target's promotion with the verdicts of its own gates: ref is journaled as the change's reference, each verdict as a stage of the change.

Refused

Refused(reason, *, verdicts=(), stage=None)

Bases: RuntimeError

The target's own gates refused the change at promotion: the reason is recorded with stage (default "promote"), and verdicts (StageVerdicts of the target's gates, when it has them) are journaled before it.

VersionsTarget

VersionsTarget(versions)

system

system(**kw)

The System of the version in force.

answerer

answerer(change, ctx=None)

input → Response: the version in force (None before any), or the candidate's catalog version (its system(allow_unaccepted=True)); None when it cannot be loaded.

ShipWatch

ShipWatch(pipe, version: int, promise, *, slices=None, alpha: float = 0.05, horizon: int | None = None, by: str = 'ship_watch', act: bool = True, outcome: str = 'failure')

ShipWatch(pipe, version, promise, *, slices=None, alpha=0.05, horizon=None, by="ship_watch", act=True).

promise: a failure rate in (0, 1) for every slice (then slices= names them) or {slice: rate}; alpha: the probability bound of reverting a change that keeps its promise in every slice; horizon: alone outcomes after which the watch graduates (None: never); act=False decides without reverting or journaling (a simulation); outcome="loss": observe(x, ...) takes a numeric loss x in [0, 1] and each slice runs a bounded betting e-process (experimental._evalue.MeanBound) against "the slice's mean loss is at most its promise" ("failures" then counts losses above the promise). observe(ok, slice=None, *, alone=True) → None | {"event": "reverted" | "graduated" | "ended" | "superseded", ...}; Only an effective active item or the latest snapshot of an artifact can be watched. A newer snapshot or an inactive item ends the watch as "superseded" before its next outcome is counted; external rollback ends it as "ended". Neither path rolls back another version. The terminal event is journaled on the next observe call. status() → per slice {promise, outcomes, failures, supervised, evidence (log K), threshold (log S/alpha)}.

bet staticmethod

bet(failures: int, outcomes: int, promise: float) -> float

The predictable bet on a failure: the KT estimate's excess over the promise, clipped to [0, 0.95/promise].

observe

observe(ok: bool, slice=None, *, alone: bool = True)

One outcome of the watched change in slice → None, or the event that ended the watch.

Economics

Economics(estimate, *, costs, units)

Refuse nonpositive expected net BEFORE later stages; all amounts share owner units.

estimate(change, ctx) returns {gain, loss}. Costs explicitly name anticipated stages, not measured runtime. Admission never bypasses any subsequent stage.

ProposerHistory

ProposerHistory(*, window, min_labels=1, max_loss=0.5, unknown='audit', clock=None)

Windowed, explicitly independently labelled outcomes; never infer truth from admission.

Unknown proposers audit-and-continue by default, or hold until external labels. Identity and independent evidence authenticity are owner responsibilities.

StagedTraffic

StagedTraffic(pipeline, version, *, rollout_id, phases)

Return a version for owner execution; never execute/deploy candidate code.

Predeclare phases: {share, units:[stable IDs], max_excess_loss}. Each unit must receive a [0,1] independent actual outcome. Missing labels HOLD. A single owner owns calls; restart reads the durable change journal.

Change dataclass

Change(source: str, artifact: str = 'item', payload: Any = None, proposer: str = '', reason: str = '', evidence: list = list(), expected: dict | None = None, change_type: str | None = None, parent: int | None = None, code: str | None = None, check: Callable | None = None)

A candidate change (see the module docstring). payload: for "item", {"kind", "body", and optionally "scope", "derived_from", "supersedes", "valid", "reconfirm_after"}; for "compiled", a solvi.core.versions.CatalogVersion (a Compiled too); for "update", None (LearningTarget reads the stored labels itself) or the updated state, whose fingerprint (an attribute or a dict key) identifies the candidate (two updates to different states are different candidates).

kind property

kind

The artifact kind the approval table reads: the item's kind for an item, else the artifact.

store_source property

store_source

The knowledge store's source for this change (None for a search proposal without evidence).

item_kwargs

item_kwargs()

The arguments of KnowledgeStore.add / proposal for an item change.

content

content()

What identifies the candidate (the failure memory compares it): source, artifact, content, code.

Result dataclass

Result(change_id: str, status: str, version: int | None = None, stage: str | None = None, reason: str = '', verdicts: list = list(), ref: Any = None)

What one proposal (or ruling) came to: status "promoted" | "rejected" | "held" | "error" (a stage raised: not a verdict on the change, which may be proposed again); the version for a promotion; the stage that refused, held or raised; every verdict.

Pipeline

Pipeline(stages, storage, *, store=None, approval=None, targets=(), refusals=(), answer_of=None, raise_stage_errors=False, order='given', costs=None)

The change pipeline (see the module docstring). stages: Stage objects run in order; storage: the journal's TraceStorage (a path or a store; None or ":memory:": a MemoryStorage, kept in this process only); store: the KnowledgeStore items go into; approval: an Approval, or its table (appended as the last stage when no Approval is among the stages); targets: extra targets (VersionsTarget, LearningTarget) — a KnowledgeTarget on store is added by itself; refusals: the held should-abstain inputs (a list, or refusals(change, ctx) → inputs) RefusalRegression checks every answering change on; answer_of: answer_of(change or None, ctx) → a callable input → answer (None: the answerer in force), which makes every change answering (else a change answers when its target has answerer(change or None, ctx)); raise_stage_errors: a stage that raises ends the proposal with status "error" (journaled; not a rejection, so the failure memory does not keep it) — True: journaled, then the exception is raised. order: "given" (default: the stages run as listed) or "cheap_first" (opt-in: sorted by cost, ties in the given order, Approval last — the pipeline stops at the first refusal, so evidence that is dear to collect is not paid for a change a cheap stage refuses; the order is journaled with the proposal). A stage's cost: costs={name: cost}, else its cost attribute, else STAGE_COSTS for a built-in, else DEFAULT_COST (see stage_cost). A stage that reads another's verdict (ctx.verdicts) must not cost less than that one.

RefusalRegression is not switchable: for every change whose target can answer it runs (inserted before Approval when it is not among the stages; yours is used in its place) and is journaled; there is no argument to remove it.

current property

current

The version in force: the last promoted one not rolled back (0: before the first change).

answers

answers(change) -> bool

Can this change's target answer (so RefusalRegression must run on it)? With answer_of= every change can; else when its target has answerer(change or None, ctx).

answerer

answerer(which, ctx, change)

The answerer (input → answer) of which — the candidate change, or None for the one in force — through answer_of= or the change's target; None when there is none.

stages_for

stages_for(change)

The stages a change runs through, in order: the pipeline's, with RefusalRegression inserted before Approval (else last) for an answering change when none is among them.

propose

propose(change: Change) -> Result

Run the stages on a candidate, record every verdict, then promote it, reject it, or hold it → Result.

rule

rule(change_id, ruling, *, by) -> Result

A person's ruling on a held change: approve (True, "approve", a Ruling "keep") or reject (False, "reject", a Ruling "drop"). The stages after the one that held it run next.

history

history(action=None)

The journal's change records in order (dicts).

versions

versions()

Every promoted version, with effective in_force. Knowledge items must also be active in their store; superseded items become in force again if reverting their successor restores them. Snapshot versions retain their journal ancestry; only the latest snapshot of an artifact can be watched or individually reverted.

rollback

rollback(version, *, by=None, why=None, failed=None)

Undo every version in force after version (newest first) through its target; recorded. → the undone version numbers. failed: are the undone changes failures (KnownFailures refuses them when proposed again)? None (default): when a reason is given (why=).

revert

revert(version, *, by=None, why=None, failed=None)

Undo one version in force — and only it — through its target; recorded (a "rollback" record with revert). Later versions stay in force (rollback(v) undoes every version after v). A target whose versions are snapshots (not independent: a compiled catalog, a labelled update) can revert only its newest version in force: undoing one out of the middle would restore a state that drops the later ones. failed: as in rollback. → [version].

watch

watch(version, promise, **kw)

A ShipWatch on a shipped version (see ShipWatch): per slice an e-process on its alone outcomes against its promise; reverts it alone when a slice breaks the promise.

lineage

lineage(change_id)

The change and its ancestors: [{"change", "version", "parent"}], following each one's parent version back to version 0.

observe

observe(change_id, effect: dict, *, by=None)

Record what the change did next to what its proposer expected → {"expected", "observed"}.

held

held()

The ids of the changes held in this pipeline, waiting for a ruling.

expire_held

expire_held(max_age, *, approve=False, by='expiry', now=None)

Rule on every change held for max_age seconds or longer (since its "held" record; now default: the journal storage's clock) — rejected by default (approve=True: approved), the ruling journaled as by by; a held change without a person no longer waits for ever. → the Results.

strict_equal

strict_equal(a, b)

The same JSON type and value, recursively (1 is not True, 5.0 is not 5; NaN equals nothing).

paired_bootstrap

paired_bootstrap(base, cand, *, boot=2000, seed=0, level=0.95)

The mean of cand[i] − base[i] and its bootstrap interval at level → (mean, low, high). Same worlds, same order.

run_episode

run_episode(env, policy, seed, *, max_steps=200)

Play one episode: policy(state, actions) → an action, until done or max_steps → the row (module docstring).

prove_direction

prove_direction(before, after, *, types: dict | None = None, outcome=_ANY, refused: Callable | None = None, timeout_ms: int = 2000) -> dict

Does the rule after accept more inputs than before (widens), fewer (narrows), both, or exactly the same? Proved over the two rules' text with z3; outside the supported subset (module docstring) → "unknown".

before, after: a rule's text (a def, a lambda, or an expression over the facts) or its function. types: {fact: int | float | bool | str} for a fact whose sort the use does not tell (a number is real by default). outcome: the rule accepts an input when it returns this value (default: when it returns a value that is not a refusal — None, False, or what refused says; default is_refusal). → {"proved": "widens" | "narrows" | "both" | "same" | "unknown", "reason", "witness_widens" / "witness_narrows" (an input that shows it), "sorts", "ms"}. Raises ImportError (how to install it) when z3 is missing.

catalog_of

catalog_of(payload, *, compiled=True)

The catalog a change carries → a Catalog, a System or None: the payload itself (a Catalog or a System), a dict payload's "system" / "catalog", or — with compiled=True — a catalog version's (CatalogVersion.catalog(), checked and loaded as pure functions).

identity

identity(content)

What the failure memory compares: Change.content() (or a journaled change dict) without the body keys of a keyed item (a body with a subject "s" and a relation "r") other than IDENTITY_KEYS — an added "nonce" or note does not make a rejected rule new. Other candidates: their whole content, as Change.fingerprint.

allowed

allowed(state, action, gates)

Does any (check, blocks) gate forbid action in state? (a policy's helper) → bool.

gate_of

gate_of(change, parse: Callable = hazard_gate)

The (check, blocks) a change installs as an agenda gate: change.check when given (with blocks from its body), else parse(body) (default hazard_gate).

gates_of

gates_of(store, scope=None, *, parse: Callable = hazard_gate)

A read-only view: the (check, blocks) of every active, usable hazard rule item in a KnowledgeStore (under scope when given), by the same parse the dry run uses → [(check, blocks)]. An agent installs them live (Agenda.gate(name, check, blocks=blocks) on an Agenda without a store, or allowed(state, action, gates)); nothing is written, so no item is duplicated.

hazard_body

hazard_body(name, when, actions, **extra)

A hazard rule as a knowledge item body: {"s": "hazard:", "r": "forbid", "o": {"when", "actions"}, "kind": "hazard"} — in state when, the actions matching actions (glob patterns) are forbidden. This rule item is the stored artifact; no second item is written through Agenda.gate (gates_of gives a read-only view instead).

The condition language is the caller's: pass parse= (body → (check, blocks)) to gate_of / gates_of / CleanCases, e.g. a lawlang condition over a macro catalog. The built-in hazard_gate reads when as equality of state keys only — a placeholder, not the decided rule form.

hazard_gate

hazard_gate(body)

A hazard body → (check, blocks) for Agenda.gate: check(state) is True (allowed) unless every when key matches (equality only: the placeholder condition language; see hazard_body).

is_refusal

is_refusal(answer, question=None)

Did an answer refuse or hand over? None; a solvi Response (the question's result, or any question when question is None, has status "abstain"); a result or a dict whose status is "abstain" / "escalate"; a Decision whose escalate is set; a string such as "abstain", "escalate", "hand over", "unknown" (REFUSAL_WORDS, any case). A domain answer like "refuse" (a loan refused) or "no" is an answer, not a refusal: pass your own refused= to RefusalRegression when your system hands over with another word (e.g. "review").

stage_cost

stage_cost(stage, costs=None)

A stage's cost: costs[name] (Pipeline(costs=)), else its cost attribute (set it on any stage: st.cost = 5), else STAGE_COSTS for a built-in, else DEFAULT_COST.