solvi.experimental.laws¶
Laws of an environment from prediction mismatches (experimental): solvi.experimental.laws.LawSet(grammar, base=...),
accepted only by a held-out test on other groups; a probe planner for live systems; export into the knowledge store.
Where each check is expensive (a live probe, a planner call), RankedProposer(inner, scorer, buffer) reorders another
proposer's candidates — JointRanker (ranks candidate conditions by the joint statistic of the whole condition over the
logged rows, no training; with no rows it is the blind order), BagOfAtomsRanker (trained on synthetic tasks drawn from
your grammar), PriorRanker (a prior order over the condition's own features — its number of atoms, its atoms' kinds,
their prior usefulness and the co-occurrence of their features in true conditions — learned from synthetic tasks of the
grammar; it reads no rows and falls back to the blind order when it learned nothing), HandoverRanker(prior, joint) (by default a blend: the joint Jaccard plus 20 / (20 + rows) times the prior's rank
among the candidates; mode="switch": the prior below min_rows logged rows of the
action, the joint statistic from there on; last says which scored) or the control FeatureRanker (a raw per-atom
statistic, no training); the candidates themselves never change.
BudgetedProposer(inner, candidates, neg_budget=, mismatches=) cuts another proposer's candidates to a task budget
in its order; with Grammar(negations=...) the candidates with a negated literal get their own budget (neg_budget,
default the same size; "shared": one budget), so they never crowd out a later mismatch's positive candidates. Both are experimental and off by default; an
admitted negated law can steer the search away from the true law (numbers in the best practices).
Scope: environments whose behaviour follows discrete rules — the features were measured on grid games and relational laws over generated graphs. On a real agent tool-call log (write calls of the public AgentDojo benchmark, laws learned as exceptions to a provenance guard's verdicts) the opt-in 1.3.0 features showed no measured gain over the default (coverage ratio 0.981, soundness inconclusive) and harmful writes let through stayed above 0.15 (0.177 with every feature on, 0.218 with the default, at about 0.10 false interruptions), about a tuned gradient-boosting model on the same atoms (0.188). For noisy behavioural logs use a calibrated scorer.
Numeric laws: the held-out test accepts row by row. LawSet.test counts a held-out row as a hit when the law's
prediction is within tol / rtol / same of the observed value, and accepts on that count. Nothing reads the
residuals together, so two cases go wrong. A law with a systematic bias — every residual inside the tolerance but on one
side, or growing with an input — passes as easily as the true law. When the law's constants are fitted from the data, a
tolerance set from the measurement error alone ignores the error of the fitted constants, and the true law can fail
rows it should pass. Before trusting an accepted numeric law, check its residuals as a whole: a confirm= closure over
the held-out rows that refits the constants and runs, for example, a χ² or a sign test on the residuals. Widen the
tolerance by the fitted constants' uncertainty. The χ² after a refit is built in as an opt-in,
residual_check(refit, level=0.995, dof_sigma=None), used with tol=math.inf (see its docstring and
best practices for the measurement and its cautions); the widened tolerance and the sign test
are not shipped.
For an independently estimated measurement variance, dof_sigma=nu uses the upper bound
(n-k) * F.ppf(level, n-k, nu) instead of the known-variance chi-square bound. Its exact reference requires both
RSS / sigma**2 ~ chi2(n-k) and an independent nu * sigma_hat**2 / sigma**2 ~ chi2(nu); for example, a
correctly specified Gaussian linear refit with an appropriate fixed variance model and an independent variance
estimate. Five independent groups of three repeated measurements provide ten variance degrees of freedom.
The caller supplies and justifies this model: nu alone does not establish independence. In particular,
estimating sigma_hat**2 = RSS/(n-k) from the identical residuals makes their standardized sum of squares
identically n-k, so the F reference is invalid. Shared rows require a justified independent decomposition.
Nonlinear fits, random relative-error normalization, non-Gaussian noise and selecting a model on the same data
do not automatically give an exact F test. This option does not supply a model-selection correction or a
guarantee against accepting a wrong formula. Threshold formulas and runtime behavior are unchanged.
Exceptions tied to a base revision¶
Absolute laws remain independent of the fallback by default. For exceptions to a published tariff, policy
or other explicitly versioned base, declare LawSet(grammar, base=pricing_v1,
lapse_on_base_change=True, base_version="tariff-1"). Every admitted law then records base_version.
set_base(pricing_v2, version="tariff-2") retires active and suspended exceptions before the new fallback
can answer. Old evidence stays in retired; admitting the exception for the new revision requires new
validation. Existing acceptance accounting is kept.
Use a new, never-recycled revision token even if a mutable callable keeps its Python identity. This is an owner declaration, not automatic inspection of changing tables or closures. Direct base assignment and per-prediction base overrides are refused for bound sets; changing revision inside admission or a dry run is also refused. A stored hypothesis from the old revision refuses to run. Operations remain owner-serialized.
Compiled code and Knowledge export currently refuse bound laws because those formats do not enforce the
revision dependency. Use a separate default (absolute) LawSet for rules whose meaning does not depend on
an edition of the base. version also changes when the fallback changes, so callers can invalidate caches.
Laws of an environment from prediction mismatches (experimental): the environment is the judge, never a model.
from solvi.experimental.laws import Atom, Grammar, LawSet, Buffer, Transition, discriminate
g = Grammar([Atom("front=tree", lambda s, a: s["front"] == "tree", feature="front"),
Atom("front=stone", lambda s, a: s["front"] == "stone", feature="front"),
Atom("wood>=1", lambda s: s["wood"] >= 1, feature="wood")], k=3)
held_out = Buffer(g)
held_out.extend(Transition(s, a, seen, group=world) for s, a, seen, world in other_worlds)
laws = LawSet(g, base=my_map) # base: what you predict without laws
for t in stream: # each transition is predicted before it is learned from
laws.step(t, held_out) # mismatch → candidates → the held-out test → accept / retire
laws.predict(state, "do") # the most specific matching law, else the base
laws.to_knowledge(ks, {"env": "grid"}) # "rule" items of kind "law", sources and evidence; retired → retire()
Where it fits. The module is for environments whose behaviour follows discrete rules. Its features were measured on rule-governed worlds: grid games and relational laws over generated graphs. On a real agent tool-call log (write calls of the public AgentDojo benchmark, 1,213 held-out rows from 30 tasks; laws learned as exceptions to a provenance guard's verdicts) the opt-in 1.3.0 features showed no measured gain over the default: coverage the same (650 against 663 freed rows, ratio 0.981, CI 0.896–1.051), soundness inconclusive (false-law share 0.157 against 0.073, the difference's CI includes 0), no pre-registered gain criterion met. Harmful writes let through stayed above 0.15 — 0.177 with every feature on, 0.218 with the default, at about 0.10 false interruptions — about the level of a tuned gradient-boosting model on the same atoms (0.188 at 0.089 false interruptions). For noisy behavioural logs use a calibrated scorer.
The loop. Before every transition the set predicts (most specific matching law first, ties to the earlier law — or,
ties="newer", the later — else the base predictor). A wrong prediction — a mismatch — asks the proposer for candidates "for this action, under this
condition, the effect is this". A candidate is accepted only by a test on held-out transitions of other groups than the
one it was proposed from (another world, another session, another path): support ≥ min_support, precision ≥
min_precision, and it removes more errors than it adds where it would be the applied law (net > 0); of the passing
candidates the one with the largest net is accepted (ties: fewer atoms, higher precision). A law whose running held-out
precision (the buffer rows where it applies, plus later observations from other groups) falls below retire_below is
retired. At most cap laws are active.
The grammar is yours: boolean atoms over the state (fn(state) or fn(state, action)), grouped by feature, and
conjunctions of at most k of them (one atom per feature). Effects: by default the observation itself; apply(effect,
state) and effects(state, observed) let an effect be a change ("+1 wood") that predicts different observations in
different states.
Proposers (Proposer: propose(lawset, state, action, observed) → candidates). Enumerate (default): every condition
true in the mismatched state. RandomProposer: conditions drawn from the whole grammar — the control. BudgetedProposer(inner,
candidates, neg_budget=): another proposer cut to a task budget; with negated literals they get their own budget, so they
never crowd out the positive candidates of a later mismatch. CallableProposer:
any function, an LLM among them; it proposes, the test decides.
Several law sets predicting one step together (experimental: on MiniGrid more accurate than one joint LawSet, 0.999
against 0.984, but no saving in transitions): solvi.experimental.laws.factored
— FactoredModel(channels, extract, delta=) keeps one LawSet and held-out buffer per channel, splits δ over the
channels (joint false-accept bound Σ δ_c by the union bound), drops duplicate rows within a group, voids rows whose
label relied on a law of another channel once that law falls (Row(depends=), Buffer.void), and reports per channel.
The turn skeleton and the row extraction stay yours.
The probe planner. Where each test is a call into a live system, split(hypotheses, safe, probes, read_only=, cost=)
picks the safe probe whose predicted outcomes split the hypotheses most — read-only actions before writes — and
discriminate(laws_a, laws_b, safe, probes) the cheapest safe probe where two law sets differ; lawset.hypothesis(c)
is the set with one candidate added, so a mismatch's candidates become hypotheses to tell apart.
Ranking candidates where each check is expensive (a live probe, a planner call). RankedProposer(inner, scorer,
buffer) reorders another proposer's candidates and never adds, drops or changes one: the order is what a probe budget
spends (the hypotheses split / discriminate receive first). Features come from the buffer's bitsets
(atom_features: per atom, its precision, recall, Jaccard and anti-match against the effect on the logged rows, its
rate, the base rate, and the best of the pairs among the top atoms — a fixed length; candidate_features sums them per
condition). BagOfAtomsRanker(seed).fit(grammar, sample) is a numpy logistic regression from an atom's features to
"this atom is in the true condition", trained on synthetic tasks drawn from the grammar (synthetic_task: a random
condition labels sampled states); a condition's score is the sum of its atoms' log-odds, so no position matters.
FeatureRanker(stat) is the control: the raw statistic, no training. On synthetic tasks of one grammar (7 categorical
and 3 counter features, 30 atoms, k = 3; 60 fresh tasks with 2% label noise, trained on 150) the checks to the true
condition were 69.7 in the blind order (by length, then name), 2.1 with the trained ranker, and 24.6 / 30.4 / 42.8
with the raw Jaccard / recall / precision (the best raw threshold picked after the fact: 12.8; tests/test_laws_rank.py).
In distribution only on the synthetic tasks. JointRanker() ranks candidate conditions by the joint statistic of the
whole condition over the logged rows — the Jaccard between the rows the whole condition matches and the rows where the
effect predicts right, equal Jaccard → fewer atoms — with no training: a sum over atoms cannot see two atoms that pick
out the effect only together, or a correlated twin of a true atom. With no logged rows every score is equal and the
order is the blind one — the statistic needs rows. On three synthetic settings of the library's own grammars (five seeds
× 60 tasks each; benchmarks/laws_rank_joint.py runs one seed) — the grammar above; deep conditions of 3–5 atoms of k = 5
with about 6,000 candidates per task and 5% label noise; correlated features (three feature pairs copied with p = 0.8) —
it needed 1.00 / 17 / 1.01 checks to the true condition (medians 1 / 1 / 1) against 85 / 2,182 / 126 blind, 27 / 1,310 /
47 for the raw Jaccard and 2.1 / 591 / 7.7 for the trained bag of atoms; a control scoring against the effect's rows
shuffled was worse than the blind order (112 / 2,434 / 166). On the deep setting the gain shows from about 20 logged
rows and is full by 300. The same law was accepted under every order in 894 of 900 tasks (by construction: the
acceptance scores every candidate; six exact ties); nothing was accepted in 180 of 300 deep noisy tasks at 300 rows —
acceptance needs more rows than ranking. The trained scorer it beats is the library's linear bag of atoms; in
distribution only.
Before the buffer has rows: PriorRanker(seed).fit(grammar, sample, tasks=, rows=, sizes=) scores a condition by its
own features — its number of atoms, its atoms' kinds (feature, comparison type read off the name: atom_kind), each
atom's prior usefulness (how often it is in the true condition when it is true in the mismatched state) and the
co-occurrence of its features in true conditions — a linear softmax over each synthetic task's candidates; it reads no
rows (shuffle_labels=True is the control). Unless its held-out checks on its own training tasks are 5% below both the
blind order's and a shuffled-label fit's, it records learned=False and scores as the blind order: a prior that learned
nothing scores as blind (the held-out check covers the training family only). HandoverRanker(prior, JointRanker()) hands over gradually:
score = the joint Jaccard + n0 / (n0 + rows) × the prior's rank among the candidates in [0, 1] (n0 = 20): the prior's order
with no rows, the joint statistic's as rows grow; a prior with learned=False leaves the joint statistic alone;
mode="switch" is the fixed switch at min_rows; last says what scored (RankedProposer.scored_by too). The prior
learns what the task family favours: on a planted grammar whose sampler draws three features 30 times as often it needed
10.3 checks at 0 rows against 43.1 blind (mean over 40 tasks, one run; tests/test_laws_rank_prior.py); on a skewed family of deep conditions (the
second setting's grammar with a chosen skew: three features drawn 8 times and one comparison type 4 times as often;
5,112 candidates per task; five seeds, 300 tasks) 208 against 1,245 blind, and the blend 184 / 98.6 / 8.9 at 20 / 100 /
300 rows against 1,019 / 361 / 16.8 for the joint statistic (the fixed switch at 20 rows: the joint statistic's); on
the three settings above, drawn uniformly, the prior fell back in 2 of 5 seeds on the first and needed 0.93 / 0.91 /
0.83 of the blind checks, and on the deep one the blend at 300 rows needed 27.2 against 17.2 (111 of 300 tasks worse,
31 better, medians 2 and 1; benchmarks/laws_rank_joint.py, five seeds, one run).
Measured once on a live file server (8 mismatches; candidates contradicted by logged rows skipped free): every order
spent 29 probes in all, because 1–5 candidates per mismatch were left to probe and the true conditions were single atoms
— the shortest hypotheses, which the length-first order meets first. Ranking can pay only where many consistent
hypotheses remain after the free checks; the control first.
Knowledge. to_knowledge(store, scope) writes each law as a KnowledgeStore item of kind "rule" with body {"kind":
"law", "action", "if", "then"} (the sub-kind in the body, as the change pipeline's hazard rules), source "outcome",
evidence = support / hits / precision / net / the birth group / the ids of the held-out transitions it was accepted on;
a retired law is retire()d with its reason; writing again writes nothing new; the store's journal verifies and replays.
Measured (2026-10, two setups, one run each, fixed before the run): on a game simulator with written mechanics (20 fresh
worlds, 4,731 transitions) the laws closed 94% of the gap between a learned map and the written mechanics (0.892 → 0.988
against a ceiling of 0.994); random candidates from the same grammar with the same held-out test reached 0.977 — the
laws' own gain over them is 1.1 points, and the pre-registered bar for that control (random candidates at most +2 points
over the base) was missed by +8.4: the test does the work, the mismatch is a compact way to propose (61 laws against
103); an LLM proposer under the same test ended 0.95 points below enumeration. On a real MCP filesystem server with a
path policy the agent was not told, probes planned from each mismatch learned every denial class in one session:
refusals 6 → 0 in the second session (200 calls per session), no allowed call wrongly skipped, against 48 → 29 with
random probes at the same budget. benchmarks/laws_toy.py reproduces both findings in miniature (tests/test_laws_toy.py,
pytest -m slow).
Compiled laws. An atom may carry its code — Atom(name, fn, feature=, code="s['front'] == 'tree'", defs=...), a
Python expression over s and a that equals fn, with the module-level source it needs. law.to_code(grammar) is a
pure function for one law; lawset.to_module() one module that imports nothing from solvi (the laws in the set's own
order, most specific first; predict(s, a, base=None, apply=None) with your base predictor as the fallback). A law
whose atoms have no code is refused, naming them; so is an action or effect that is not a literal. Compiled code is
accepted only after the differential check: check_compiled(module, buffer, lawset) runs the module against
lawset.predict (and the applied law's id) on every row of the buffer, and the bar is 0 differences. A grammar may grow
(grammar.add(atom), e.g. a relational atom a proposer invents); a buffer evaluates the new atom on its older rows the
first time a condition names it.
Sized acceptance. Fixed thresholds do not see how many candidates were tried on one buffer (a setup of ours tried about
52,000 at support ≥ 5). By default (sized=True) a candidate passes only when its hits would be unlikely if it were no
better than its null: P[Binomial(support, null) ≥ hits] ≤ δ_N, N = the candidates scored on this buffer so far
(buffer.tested), δ_N = δ / (N·(N + 1)) — summing to below δ however long the stream runs — or δ / space when you
declare space= (Bonferroni). The null (null=): "base", the default — "no better than chance": the effect's base rate
among the buffer's rows of that action (the proposing group left out), floored at 0.05 and capped at min_precision — or
a fixed precision (null=0.9: the law must beat 0.9). min_precision stays a separate filter on the observed precision.
A fixed null close to 1 is the wrong question in a deterministic world: a law right on hundreds of rows can still fail
to beat 0.9 at tens of thousands of candidates. The need: a deterministic law needs ⌈ln(1/δ_N) / ln(1/null)⌉ rows, with δ_N = δ / (N·(N + 1)): at 52,000 candidates and δ = 0.1, 9 rows at a base rate of 3% (floored to 5%), 68 at 70%, and 228 against a fixed null of 0.9 (at 821 candidates: 6, 45, 150); errors ask for more. On buffers with no law
(2,000 rows; an effect independent of 40 sparse atoms; the 821 conditions of at most two atoms) the fixed thresholds
(support ≥ 5, precision ≥ 0.9) accepted a law in 100 of 100 buffers at an effect rate of 70%, the sized rule with the
base null in 0 of 100 (at 3%: 0 of 100 for both); a deterministic two-atom law at a base rate of 3% (1,000 rows) was
accepted in 100 of 100 buffers with the base null on a median of 29 rows, in 0 of 100 with a fixed null of 0.9
(tests/test_laws_sized_probes.py). A buffer too small for its null accepts nothing — the rule working, not failing: declare
space= or pass sized=False knowingly (the fixed thresholds; the measurements above ran so, and their false
accepts are the known price). Each Law records tested, null, need, level and admitted ("test", "probes",
"hand").
Admission by probes. Where the world (or an exact model of it) answers questions, lawset.admit_by_probes(candidates,
ask, probes=, safe=, budget=) asks where the surviving candidates disagree, CEGIS-style: candidates refuted by logged
rows (buffer=) go first; the survivors are grouped into classes by their predictions on the probe pool (the set without
a new law is a hypothesis too); split picks the safe probe that splits the classes most; the refuted are dropped; the
shortest candidate is accepted only when one class survives (identifiability) — else an Admission "undetermined"
with the classes, never a guess. On a synthetic world (a hidden two-atom law, 176 candidates, 300 trials) at 8 checks a
random held-out sample with the shortest survivor accepted a wrong law in 0.593 of the trials; the active admission in
none (0.897 right, 0.103 undetermined, 7.56 asks; at 10 checks 1.000 right). Where more than one law is missing the
"too narrow" refutation is not sound: the candidates it refuted that were never wrong where they applied are kept as
witnesses, and nothing is accepted while the set with all of them added still fits every row and predicts otherwise
than the surviving class where it would apply (x0 ∨ x1 with a base saying False: wrong admissions 29 → 6 of 100;
assume_one_law=True keeps the old rule). The probes are of the world you may ask: a law set whose other laws
or base are wrong where it probes may refute a right candidate (a candidate is refuted only where it is the applied law
and wrong, or where it does not hold, the set is wrong and its effect is what happened).
Interventions. intervention(state_before, state_after, actions) → one Delta per action: the prediction before and
after with the law behind each (explain), the change, and covered — False where the base predictor, not a law,
speaks on either side, so an effect the laws never tested is marked a guess.
Abstention. predict(state, action, disagree="abstain") (or LawSet(..., disagree="abstain")) returns Undetermined
where the most specific matching laws — those no other matching law's condition strictly contains — predict different
outcomes: both passed their test, neither is an exception of the other, and the order between them would only be a
tie-break. It is counted in abstained; the default stays most specific first, the learning loop always predicts so,
and a compiled module dispatches most specific first. The search cost: lawset.cost (candidates proposed, tests,
accepted, probes, simulated) and Step.tested — report it next to accuracy. lawset.report(buffer, steps) sums a
run up and states per action the rows the buffer has and the support the sized acceptance needs at this N
(sized_need against the action's least base null; need_at_precision for a law right on min_precision of its rows):
an action with fewer rows than that cannot get a law on this buffer — «too few rows», not «no law». For one candidate,
lawset.why_not(candidate, buffer) names the first test it fails, in test's order (active, readmit, cap, net,
min_support, min_precision, exact, sized_need, sized_bound), and by how many rows — without touching the buffer's N.
Coarse outcomes and a backing-off base (opt-in). LawSet(grammar, **coarse(lambda seen: seen["kind"])): a law predicts
the observation's class (an OutcomeClass), right when the observation falls in it — details no law can know (a random
amount) no longer make every row a mismatch; report counts outcomes by class. BackoffBase(levels, min_rows=,
outcome=) is a base predictor fitted on transitions: the most frequent outcome for the state's key at the most specific
level ([("front", "wood"), ("front",), ()], the action always part of the key) with at least min_rows rows, backing
off to more general levels; explain names the level used. Fit it before the set uses it, on rows the laws are not
tested on.
History atoms (opt-in). History(grammar, depth=1, actions=..., observed=..., same=True) lifts a grammar to contexts —
the present state plus the last depth steps — with atoms "t-1:Same(1) («what was observed one step
back»: one law for a value that persists instead of one per value). LawSet(h.grammar, **h.kwargs(base=...)); a
Tracker (h.tracker()) builds each step's Context along a stream and reset()s per episode; predict on
tracker.context(state). The extra atoms enlarge the space the sized acceptance divides its error over.
Per-law confidence (opt-in). LawConfidence(lawset, buffer): each active law's posterior precision on its held-out
rows, exact only with no miss on ≥ 60 rows, and at most one tail split — the atom where its misses concentrate, kept
when Fisher's exact test passes at alpha / atoms tested — with a shrunk confidence on each side. of(state, action) →
(value, law, confidence); predict(state, action, below=t) returns Abstained where the applied law's confidence is
below t. A tail no atom tracks gives per-law, not per-state, abstention. The confidence is an estimate — a Laplace
posterior mean plus one Fisher-exact split on an outside atom with m-estimate shrinkage, "exact" meaning no miss on
≥ 60 rows — not a bound: there is no risk guarantee, and the threshold t is yours to pick on your own data. Without
it the law set implicitly treats every law as exact.
Noisy probes (opt-in). admit_by_probes(..., noise=ε, level=L): where an answer of the world may be wrong with a known
probability ε, one answer refutes nothing — every hypothesis keeps a likelihood, the next probe is the one with the
largest expected information gain under that noise (split(score="eig", noise=ε), a repeat when that is best), and a
law is accepted only when one class of hypotheses holds at least L of the weight. On a synthetic single-law benchmark,
given the same number of answers as the best repetition scheme (a majority vote per probe or a per-probe sequential
test, each tuned under the same wrong-accept cap), it accepted the right law more often when answers were scarce and as
often with fewer answers when they were not; the default admission is often wrong under such noise. A known method:
noisy generalized binary search over classes of hypotheses, stopped at a posterior threshold.
Suspended laws (opt-in). LawSet(..., on_fall="suspend", reactivate_min=m): a law whose running precision falls leaves
the active set but stays a candidate (suspended); it is re-checked on the next m fresh observations where it would
apply and comes back when they bear it out. doubt_probe(probes) names a probe where a suspended law and the set
disagree. The default still retires.
What is not done:
- no grammar that cannot be enumerated (where the mismatch should matter most) has been measured;
- the sized rule, admission by probes and abstention are measured on synthetic worlds only: not yet on the game
simulator or the file server, and a setup that tests tens of thousands of candidates needs a buffer of a few
hundred rows per accepted law (a buffer of tens of rows needs sized=False, and then the fixed thresholds' false
accepts are its price);
- retirement is by precision only; a weak law no observation contradicts stays;
- the store item's id is its content: a law retired and accepted again is the same item, asserted again;
- no wiring into a System's decisions or an Agenda gate (a law set predicts; using the prediction is yours);
- noisy probes (noise=) assume symmetric flip noise with ε known — to the method and, through their tuning, to the
repetition schemes it was compared with — and one missing law, on a synthetic single-law benchmark at three noise
levels and two answer budgets; at the highest noise level with the small budget it did not hold the wrong-accept
cap; a misspecified ε, state-dependent noise, numeric noise and several missing laws are not measured;
- suspension does not handle the "dead end": a law whose states the agent stops visiting is never re-checked and
stays suspended;
- history atoms are measured at depth=1 only;
- LawConfidence is measured with ≥ 191 rows per law; at the default exact_rows=60 a law right on 97 % of its rows
is flagged exact about 1 time in 5 (0.97⁶⁰); it does not retire laws and gives no risk guarantee.
BackoffBase ¶
BackoffBase(levels: Sequence = ((),), *, min_rows: int = 5, outcome: Callable[[Any], Any] | None = None)
A base predictor that backs off from specific to general: levels, most specific first — each a tuple of state
keys (("front", "wood")) or a function (state, action) → key; the action is always part of the key, and the
action alone is the last level unless () is given. base(state, action) is the most frequent observation among
the fitted transitions with the same key at the first level that has at least min_rows of them (ties: the first
seen); None where no level has enough. outcome: a coarse key — the base then counts and answers classes
(OutcomeClass), as the laws of a coarse set do.
base = BackoffBase([("front", "wood"), ("front",), ()], min_rows=5).fit(transitions)
laws = LawSet(grammar, base=base) # what is predicted where no law applies
Fit it once, on transitions the laws are not tested on, before the set uses it: a LawSet caches where its base is
right per buffer row (a base refitted while in use gives stale counts — give the set a new BackoffBase).
explain(state, action) → {"value", "level", "key", "rows", "share"} (level None: no level had enough rows).
fit ¶
Count transitions (Transition objects, a Buffer, or (state, action, observed) triples) → self.
OutcomeClass
dataclass
¶
A predicted outcome class (the observation read through a coarse key): what a law of a coarse set predicts,
and what BackoffBase(outcome=) answers. value: the class.
CompiledCheck
dataclass
¶
The differential check: rows compared, how many differed (the bar is 0), and up to 20 differing rows as {"id", "action", "compiled", "interpreted"} (each side: (law id, prediction)).
Abstained
dataclass
¶
The answer of LawConfidence.predict where the applied law's confidence is below the threshold. Never equal to
an observation.
LawConfidence ¶
LawConfidence(lawset: LawSet, buffer: Buffer, *, prior: tuple = (1.0, 1.0), shrink: float = 10.0, alpha: float = 0.01, min_rows: int = 20, exact_rows: int = 60, atoms=None)
Per-law confidence (an estimate, not a bound) and tail abstention over a law set and its held-out buffer (see the module
docstring). prior: (a, b) of the Beta prior; shrink: m, the pull of a tail side towards the law's confidence;
alpha: the family-wise level of the tail test; min_rows: least rows on each side of a split; exact_rows: least
rows for exact; atoms: the atom names a split may use (default: every atom of the grammar).
fit ¶
Count every active law's held-out rows and find its tail → {law id: LawStats}.
of ¶
(value, law, confidence): the applied law's prediction and its confidence in this state (its tail side where it has one); where no law applies, the set's prediction, None and None.
predict ¶
The value, or Abstained where a law speaks with confidence < below.
LawStats
dataclass
¶
LawStats(law: Law, hits: int, n: int, confidence: float, exact: bool, atom: str | None = None, p: float | None = None, when_true: float | None = None, when_false: float | None = None, n_true: int = 0, n_false: int = 0)
One law's counts and confidence: hits / n on its held-out rows, confidence (the posterior mean), exact, and the tail split (atom, its p-value, confidence where the atom is true / false, rows on each side) or atom None.
Context
dataclass
¶
What a history-lifted law set sees: the present state and the past steps, the most recent first (fewer than
depth at the start of an episode).
back ¶
The step d back (1 = the last one), or None before the episode had d steps.
History ¶
History(grammar: Grammar, depth: int = 1, *, actions: Iterable = (), observed: Iterable = (), past_atoms: bool = True, start: bool = True, same: bool = False, k: int | None = None)
Lift grammar to contexts with atoms over the last depth steps (see the module docstring).
actions / observed: the values the "t-d:action=" / "t-d:observed=" atoms name (none listed: no such atoms);
past_atoms: lift the grammar's own atoms to the past steps (default True); start: add "t-d:start"; same: add the
effect variant Same(d); k: the largest conjunction (default: the grammar's).
base ¶
fn(state, action) → the same on a context's present state.
apply ¶
The law set's apply: Same(d) → the observation d steps back; any other effect → fn(effect, state)
on the present state (default: the effect itself).
effects ¶
The law set's effects: fn(state, observed) on the present state (default: [observed]), plus Same(d)
for each d whose past observation equals this one (same=True).
kwargs ¶
kwargs(*, base: Callable | None = None, apply: Callable | None = None, effects: Callable | None = None) -> dict
base / apply / effects for LawSet(h.grammar, **h.kwargs(...)).
Past
dataclass
¶
One step back: the state, the action taken in it, what was observed after.
Same
dataclass
¶
The effect «the observation is what was observed d steps back». Predicts Same.MISSING (never an
observation) where the context has no such step.
Tracker ¶
Builds contexts along a stream: context(state) → the present state with the last depth steps; push
records a step; reset starts an episode (no past). transition does both for one step.
transition ¶
The transition on this step's context (built before the step is recorded), then the step is recorded.
Admission
dataclass
¶
Admission(status: str, law: Law | None, classes: list, asked: list, refuted: int, null: bool, why: str, witnesses: list = list(), refuted_list: list = list(), untested: list = list(), laws: list = list(), unknown: list = list())
What admit_by_probes decided. status: "accepted" (one equivalence class of candidates survived and it is
not the set without a law) or "undetermined"; law: the accepted law (the shortest candidate of the surviving
class; under max_new=2 with two laws accepted, the first of laws) or None; classes: the surviving candidates grouped by what they predict on every probe offered (shortest
first in each, classes by their shortest); asked: the probes asked, as Transitions (group "probe"); refuted: the
candidates the logged rows and the probes refuted (a count — the candidates themselves are refuted_list); null:
the set without any new law still survives; why: the reason; witnesses: the candidates refuted only as too narrow
that are right wherever they were applied and predict otherwise than the surviving class where they apply — while
any is left nothing is accepted (several laws may be missing; empty under max_new=2, which does not use this
guard); refuted_list: the refuted candidates, in the order they fell; laws: every law accepted (one, or two with
max_new=2; empty when undetermined); untested: the atoms of a longer candidate in the surviving class that the
shortest one drops and that never varied on the pool and the rows seen where the shortest one holds — the
evidence cannot tell the shorter law from the longer, so the accepted law may be too general there (see
coverage of admit_by_probes; empty under max_new=2, where coverage is not checked); unknown: the probes whose
answer was Unknown, as Transitions (spent, used for nothing else).
Atom ¶
Atom(name: str, fn: Callable[..., Any], feature: str | None = None, *, code: str | None = None, defs: str | Iterable[str] = ())
A named boolean feature of a state: fn(state) or fn(state, action) → bool. feature groups atoms that
describe the same thing (thresholds of one counter, the material of one cell): a condition holds at most one atom
per feature. Default: the atom's own name.
code: optional, a Python expression over s (the state) and a (the action) that equals fn — what a compiled
law set (LawSet.to_module) runs in its place; defs: the module-level source the expression needs (a string or a
tuple of strings; identical ones are written once). An atom without code cannot be compiled.
Buffer ¶
Held-out transitions, grouped. Per action: one bitset per atom over its rows and one per group, so a law's
support and precision on the rows of every group but one are a few AND operations. The proposing transition's group
is left out of every test (exclude=). tested: how many distinct candidates (action, condition, effect) have been
scored on this buffer, by every law set that tested on it — the N the sized acceptance divides its error by; scoring
the same candidate again does not raise it.
window: keep only the last window rows (all actions together, in arrival order); older rows leave as new ones
arrive and every bitset is re-based (bit 0 is always the oldest row kept). left counts the rows that left;
version changes with every row added or dropped. The sized rule's N after rows leave: it never decreases (the
bound is over every candidate tested on the stream, not on the rows kept), and a candidate scored again counts
again once the window has turned over since it was last counted — window rows added since then, so it is judged
on other rows: a new test. Within one turnover a re-score is not counted, as on a buffer without a window. Without
a window (the default) nothing leaves and N counts distinct candidates as before.
dedup (experimental, opt-in): a row whose situation is already in the buffer is not added (add returns False,
duplicates counts it). The situation: dedup=True — (group, action, the atoms true, the observation); a function
dedup(t) → a key — (group, action, that key), e.g. an object id to count support by distinct object. The sized
test treats every held-out row as an independent draw; an agent that replays the same situation (a random walk
revisiting a few objects) adds the same draw many times and inflates support. Measured on a synthetic world with no
law (10 worlds of 3 objects with 8 persistent atoms and a persistent coin outcome, 20 random-walk rows per world,
the 72 candidates of up to two atoms; 1,000 buffers): a law was accepted in 872 by default (support about 20–26
rows from a few objects) and in 0 with dedup=True; where a law exists (pass exactly where a0 & a1, 20 worlds) the
right law was the first accepted in 591 of 1,000 with dedup and 473 without (tests/test_laws_dedup_heldout.py).
The bound then assumes distinct situations are independent draws; rows of one situation with different
observations (a stochastic outcome) are kept. With a window, a situation that leaves may be added again.
void(row_ids) (experimental): rows whose label is no longer trusted (it was read off a law that has since fallen;
solvi.experimental.laws.factored calls it for row provenance) leave every test: allowed, group_mask and
rows skip them, so support, hits, nulls, check and an admission's logged rows no longer see them. They stay in
the buffer: len, n and tested (the N of the sized bound, which counts candidates tested, not rows kept) do
not decrease, and their atoms still count as evidence that an atom varies (match, coverage="varied").
voided counts them.
situation ¶
The key dedup compares rows by: (group, action, the atoms true, the observation) for dedup=True, else
(group, action, dedup(t)). None without dedup.
add ¶
Add a row → True, or False when dedup is on and its situation is already in the buffer (duplicates).
__getstate__ ¶
Pickled without _cache: its keys hold id()s of the comparison functions (apply, base, same),
which name other objects in another process. The cache is rebuilt from the rows on first use.
count_tested ¶
Add candidate keys (action, condition, effect) to the tested ones → how many were new (N grows by these;
with a window, a key counted window or more rows ago counts again).
void ¶
Take the rows with these ids out of every test (see the class docstring) → how many were newly voided.
Unknown ids are ignored; voiding twice is a no-op. version changes when any row was voided.
rows ¶
The rows of one action (or every row), without the group exclude.
where ¶
The rows of action where test(row) holds, cached under (what, key) and extended as rows arrive.
BudgetedProposer ¶
BudgetedProposer(inner, candidates: int, *, neg_budget: 'int | str | None' = None, mismatches: int | None = None)
Another proposer's candidates cut to a budget, in the inner order: at most candidates candidates over all
mismatches, and proposals for at most mismatches mismatches (None: no cap). used, used_neg, mismatches_seen
and last (the candidates of the last call) after a call. Wrap the ranked proposer (RankedProposer) so the cut
keeps the best-ranked candidates.
neg_budget (with Grammar(negations=...)): a candidate whose condition holds a negated literal is charged to its
own budget of neg_budget candidates, the others to candidates; proposing stops when the positive budget is
spent (or the mismatch cap is reached), so the positive candidates of every mismatch are exactly those the same
proposer gives without negations. Default (None): neg_budget = candidates — a separate budget of the same size;
0 drops the negated candidates; "shared": one budget of candidates for both, cut in the inner order (the
behaviour before the option). Under a shared budget negated literals use up the budget before the mismatch that
proposes the true positive law is reached: on a second task (relational laws over generated graphs, the program size growing with each
mismatch, 2,000 candidates) found fell from 0.64 to 0.55 and 0.25 to 0.11 on two families that need no negation,
and a separate budget restored all 21 lost tasks in a replay. The cost: up to candidates + neg_budget candidates
per task, and every one tested counts in the sized bound's N (Buffer.tested); in that replay 1.5–1.9× the shared
budget's candidates per task. Without negations (or with no negated candidate) it is a plain budget.
CallableProposer ¶
Any proposer as a function — an LLM, a person's list: fn(state, action, observed, true_atoms, grammar) →
conditions (iterables of atom names). A condition the grammar refuses is dropped and counted (dropped). On a
game simulator with written mechanics, an LLM proposer under the same test ended 0.95 points below enumeration:
use one where the grammar is too large to enumerate, never as the judge.
Candidate
dataclass
¶
A proposed law: for action under condition (atom names), the effect is effect.
Delta
dataclass
¶
Delta(action: Any, before: Any, after: Any, law_before: Any, law_after: Any, change: Any, changed: bool | None, covered: bool)
One action's predicted effect of an intervention (LawSet.intervention): the prediction before and after, the law behind each (None: no law, the base predictor's value), change (after − before for numbers, else None), changed (None where a side is Undetermined) and covered (a law speaks on both sides).
Enumerate ¶
The default: every condition the grammar builds from the atoms true in the mismatched state, with every effect
variant of the observation (lawset.effects). Under Grammar(negations="refine") also the grammar's refinements
on the buffer the call gives mismatch(…, buffer=…, exclude=…) — step passes its own (without the mismatch's
group). Without that buffer refinement enumeration raises before evaluating atoms or effects: pass the same
buffer to mismatch and admit_by_probes. For intentional positive-only proposals use negations=False.
Grammar ¶
Grammar(atoms: Iterable[Atom], k: int = 3, *, min_k: int = 0, implies: dict | Callable | None = None, negations: bool | str = False)
The hypothesis space, written by you: the atoms and the largest conjunction k. A condition is a sorted tuple of
atom names, at most k, at most one per feature; the empty condition () holds everywhere. min_k (default 0): the
fewest atoms a condition has — min_k=1 forbids the empty condition (a law must name at least one atom), so
conditions, random_condition and check never give one and LawSet.mismatch, test and admit_by_probes
drop a shorter candidate.
implies (opt-in, default None): atom implications by construction — {atom name: [the atom names it implies]} or a
function name → the names it implies; closed transitively (a ⇒ b, b ⇒ c gives a ⇒ c), also over atoms added
later (add(atom, implies=...)). An implied atom adds nothing to a condition, so conditions never gives a
condition holding an atom and one it implies (x>=2 ∧ x>=1 is x>=2: fewer candidates, a smaller N), and
admit_by_probes never lists an atom the accepted condition implies as untested (without a declaration a
property atom that implies an object atom is "untested" for ever: it never varies where the property holds, and
coverage="undetermined" refuses every such law). A wrong declaration would hide a genuinely untested atom, so a
Buffer warns on the first row that breaks one (a true, b false; check_implies lists them all).
negations (experimental, opt-in, default False): a literal "not:a and not:a never share a condition); true(state, action) then gives one literal per
atom (the atom or its negation), check / allows / Law.holds / Buffer.match read literals (the buffer
computes a literal's rows as the complement of the atom's: no new storage), the compiled code is not (<expr>).
"refine" (the documented mode): at most one negated literal per condition, and only where the atom varies —
Enumerate adds P ∧ not:a to a positive condition P that holds in the mismatched state only when a took both
values on the held-out rows P matches (an exception to cut out), and the lone not:a when a took both values on
the held-out rows of the action (one extra candidate per atom: "transform unless R" is the law not:R). The
rows are the buffer the mismatch is proposed on: step passes its own, and a direct mismatch call must be
given it (mismatch(…, buffer=…) — admit_by_probes(buffer=…) does not reach the refinements, its candidates
are computed before it sees the buffer); refinements gives both; min_k counts positive atoms, so min_k ≥ 1
keeps lone negations out. "all": every
literal is an atom like any other (the full literal grammar: 1 + n + C(n, 2) conditions per state at k = 2, n
atoms) — for small grammars and for measurement. The
sized bound is unchanged in form: every literal candidate scored is counted in N (Buffer.tested) and tested at
δ_N. Under "refine" which candidates are proposed depends on the buffer (which atoms vary under P) — as Enumerate
already depends on the state; every proposed candidate is counted, so the per-candidate level holds and the sum
still bounds. Candidates per mismatch, k = 2, one atom per feature, measured in tests/test_laws_negations.py:
"refine" 2.1–3.9×, "all" 7.2–22.5× the positive grammar's on grammars shaped like a grid game's effect channels; null buffers (40
sparse atoms, 2,000 rows, the effect independent at 70 % and 3 %) accepted no law in 100 under either mode.
add ¶
Append an atom (a proposer may invent one, e.g. a relational atom). A buffer built before evaluates it on
its older rows when a condition first names it. ValueError if the name is taken. implies: the atom names it
implies (added to the grammar's declarations, see Grammar).
is_negated ¶
name is a negated literal not:<atom> of this grammar (False when negations are off).
positive ¶
The positive atoms of a condition (all of it when negations are off).
literals ¶
Every literal a condition can hold: the atoms in grammar order, then (negations on) their negations.
implies_of ¶
The atoms name implies, transitively (without itself).
implied ¶
Every atom the condition implies and does not hold itself (empty without implies); with negations also
the contrapositive: not:b implies not:a for every declared a ⇒ b.
redundant ¶
The condition holds an atom and one it implies (conditions never gives one); with negations also
not:b ∧ not:a for a ⇒ b (the second adds nothing) and the contradiction a ∧ not:b (it never holds).
check_implies ¶
The declared implications a ⇒ b (transitive ones too) that logged rows of buffer contradict — a true
and b false on the row → [(a, b, the row ids)], over every action. Empty without implies.
true ¶
The names of the atoms true for action in state, sorted; with negations the literals true there, one per
atom (the atom or not:<atom>).
check ¶
A condition in its one spelling (sorted) → tuple; ValueError for an unknown atom, two atoms of one feature or more than k atoms.
allows ¶
The condition has at least min_k (positive) atoms and, under negations="refine", at most one negated
literal (the other rules are check's).
conditions ¶
Every condition built from the atoms true (in size order, then by name): what holds in one state. With
implies, none that holds an atom and one it implies. negations="all": from every literal true;
"refine": from the positive atoms only (the negated ones come from refinements, which reads the buffer).
refinements ¶
refinements(true: Iterable[str], buffer: 'Buffer', action, exclude=None, conditions: Iterable[tuple] | None = None) -> list[tuple[str, ...]]
negations="refine": the conditions P ∧ not:a for each positive condition P of conditions (default
conditions(true)) with 0 ≤ |P| < k and each negated literal not:a true in the state, of another feature,
where a took both values on the rows of action that P matches in buffer (without the group exclude) — an
exception exists there to cut out; P = () (the lone not:a, where a varies on the action's rows) when the
empty condition is in conditions or min_k is 0. [] in the other modes or without a buffer.
candidates ¶
candidates(true: Iterable[str], buffer: 'Buffer | None' = None, action=None, exclude=None) -> list[tuple[str, ...]]
The conditions Enumerate proposes for one state: conditions(true), plus refinements under
negations="refine" when a buffer is given. Their number times the effect variants is what a mismatch adds
to N (the sized bound's candidate count) at most — fewer when some were scored before.
sizes ¶
How many conditions of each size 0..k conditions(true) gives.
random_condition ¶
A condition of size atoms drawn from the whole grammar (one per feature); with negations from the full
literal grammar, a negated literal turned positive where min_k or "refine" needs it.
Law
dataclass
¶
Law(id: str, action: Any, condition: tuple, effect: Any, support: int = 0, hits: int = 0, precision: float | None = None, net: int = 0, born: Any = None, status: str = 'active', proposer: str = 'enumerate', sources: tuple = (), seq: int = 0, n: int = 0, hits_after: int = 0, held: tuple = (0, 0), why: str | None = None, tested: int = 0, need: int = 0, level: float | None = None, null: float | None = None, admitted: str = 'test', aliases: tuple = (), base_version: str | None = None, recent: Any = None, doubt: Any = None, consolidation: dict | None = None)
An accepted law (see the package docstring). support / hits / precision / net: on the held-out buffer at
acceptance (rows of every group but born), counted on the rows where it would be the applied law (the set's
support_on); sources: the ids of the rows it was accepted on (the same rows); n / hits_after:
observations from other groups since, while it was the law applied; held: (hits, n) on the buffer rows where it is
the applied law, from the last test; status: "active" | "retired" | "suspended" | "proposed" (a dry run's,
admit_by_probes(commit=False)); why: the retirement's reason. tested: the candidates scored on the buffer
when it was accepted (with the sized acceptance, the N of its bound; admitted by probes: the candidates it was
told apart from); null: the null precision it was tested against (the effect's base rate, or the set's fixed
null); need: the clean support the sized rule asked for at that N; level:
the per-candidate error level δ_N it passed (None: fixed thresholds, sized=False, or a law added by hand);
admitted: "test" (the held-out buffer), "probes" (admit_by_probes), "consolidation" (consolidate) or
"hand" (add). consolidation: an opt-in replacement's proof (replaced IDs, evidence IDs, original and
counterfactual net, held-out correct counts); None on ordinary admissions. aliases: the effects
declared one with effect under the law's condition (LawSet(equivalent=)) and merged into it; effect (the
first candidate given) is the one applied.
running ¶
(hits, n): held-out rows where it applies plus observations from other groups since its acceptance.
to_code ¶
The source of a pure function def law_<n>(s, a) → (True, effect) where this law applies, else (False,
None), from the atoms' code. ValueError naming the atoms without code, or for an action / effect that is not a
literal.
LawSet ¶
LawSet(grammar: Grammar, *, base: Callable | None = None, apply: Callable | None = None, effects: Callable | None = None, proposer: Proposer | None = None, min_support: int = 5, min_precision: float = 0.9, retire_below: float = 0.8, retire_min: int = 5, cap: int = 300, sized: bool = True, delta: float = 0.1, space: int | None = None, null: str | float = 'base', disagree: str = 'first', support_on: str = 'matched', readmit: str = 'any', acceptance: str | None = None, confirm: Callable | None = None, ties: str = 'earlier', tol: float | None = None, rtol: float | None = None, same: Callable[[Any, Any], bool] | None = None, recent: int | None = None, memo: bool = False, on_fall: str = 'retire', reactivate_min: int = 10, suspended_cap: int = 50, rivals: bool = False, equivalent: Iterable[tuple] | Callable | None = None, lapse_on_base_change: bool = False, base_version: str | None = None)
Laws over one grammar, most specific first (see the package docstring).
base: the predictor when no law applies, base(state, action) (default: None). apply: apply(effect, state) → the
predicted observation (default: the effect is the observation). effects: effects(state, observed) → the effect
variants an observation supports (default: [observed]). proposer: default Enumerate(). The acceptance: support ≥
min_support, precision ≥ min_precision and a net error reduction > 0 on the held-out rows, and — sized=True, the
default — the sized rule below; retirement: running held-out precision < retire_below over at least retire_min
cases; at most cap active laws. min_precision below retire_below is warned at construction: a law that passes
with a precision between the two is retired by the check its own test runs, and the next mismatch accepts it again.
support_on: the rows a candidate's support, hits and precision are counted on — "applied" (opt-in): where it
would be the applied law, its condition's rows less those a more specific (or, at equal length, one ties puts
first) active law already claims, the rows its net is counted on too; "matched" (the default): every row its
condition matches. This checks the full observed matching domain, including rows currently claimed by narrower
laws; it does not certify unseen inputs. Under "applied", a default whose exceptions are already laws is judged
on the rows they leave it: a default right on
45 of its 45 rows with 10 rows of a known exception under it is 45 / 45, not 45 / 55 (refused at min_precision 0.9
under "matched"), and its precision at acceptance is the precision its first check recounts.
readmit: "any" (the default) — a retired law may be accepted again by the next test; "grown" — not on a buffer
that has not grown since its retirement (the same evidence that retired it; refusals counted in
refused_readmit); on a Buffer(window=W) not before W rows were added since (the rows it was accepted on may
still be in the window: retired by recent observations, it would pass again on the stale rows). A retirement from observe outside step does not see the buffer: it is stamped with the
buffer's size at the next test, which errs toward waiting.
The sized acceptance (sized=True). Fixed thresholds do not see how many candidates were tried on one buffer: on a
buffer with no law at all (2,000 rows, an effect at 70% independent of 40 sparse atoms) the 821 conditions of at
most two atoms passed support ≥ 5 and precision ≥ 0.9 in 100 of 100 buffers. The sized rule tests each candidate
against a null at the level δ_N = sized_level(N, delta, space), N the candidates scored on this buffer so far
(buffer.tested): it passes when P[Binomial(support, null) ≥ hits] ≤ δ_N. The null (null=): "base" (the
default) — "the condition is no better than chance": the effect's base rate, the share of the buffer's rows of
that action (the proposing group left out) whose observation the candidate's effect predicts, floored at 0.05 and
capped at min_precision (0.95 when that is 1) — or a fixed float (the precision the law must beat). min_precision
stays a separate filter on the observed precision. A law right on every row needs support ≥ sized_need(N, δ,
null) = ⌈ln(1/δ_N) / ln(1/null)⌉: at 52,000 candidates and δ = 0.1, 9 rows at a base rate of 3% (floored to 5%),
68 at 70%, and 228 against a fixed null of 0.9; every row it gets wrong asks for more. Then a candidate no better
than its null passes with probability ≤ δ_N, and over every candidate tested on the buffer the chance of any such
acceptance is ≤ delta — on the same null buffers: 0 of 100. With the base null that statement is "better than
chance", not "precision above min_precision" (the filter adds that on the observed rows). The bound assumes the
held-out rows are independent draws and the candidates were not chosen by looking at them. sized=False: the
fixed thresholds (the measurements ran so). Each accepted Law records tested (N), null, need and
level. N counts distinct candidates: the same (action, condition, effect) scored again on the buffer — the next
mismatch proposing it again — does not raise it (it did before 1.2: the same 56 candidates scored three times on an
unchanged buffer made N 168).
rivals (opt-in, default False): a candidate that passes is refused while an indistinguishable rival exists —
another offered candidate or an active law with the same action and effect, a condition neither wider nor
narrower, matching exactly the same held-out rows (at least one; rival_of); refused_rival counts the refusals.
What it ensures: no law is accepted on a buffer that cannot tell its condition from another offered one — the
choice is never made by the order of the candidates. What it does not: a rival that was not offered is not seen,
and a law with no rival is judged by the bound above as before. What it costs: a true law waits until the data
tell it from its rivals (a held-out row where they differ). Measured on a grid-game-shaped world (a key pushed into
a STOP door passes, into a STOP wall blocks; the right mover OPEN & cell SHUT -> pass, the wrong
mover OPEN & cell STOP -> pass right on every row but key-into-wall ones; levels arrive one at a time, up to 20,
the first accepted law counts; 1,000 streams): with a key-into-wall row on 10% of levels the wrong law was first
in 198 streams by default and in 0 with rivals, the right one in 763 and 816, and in 65 streams nothing was
accepted within 20 levels (0 by default); at 30%, false accepts (a law below 0.9 precision on a 3,000-level
reference) 17 → 15. Noise in an overridden law's rows with no exception (78 candidates on 80 rows): 0 accepts in
715, 1,000 and 905 buffers at δ = 0.1, with and without rivals (tests/test_laws_rivals.py).
acceptance: "sized" (the default with sized=True), "fixed" (sized=False) or "exact" — for few-shot buffers, where
the sized rule cannot pass anything (3 groups × 6 rows, a deterministic one-atom law: 0 of 20 learned): a candidate
passes when it is right on every held-out row where it applies, with support ≥ min_support, and net > 0; no
binomial bound, so nothing bounds the false accepts over many candidates (with min_support=2 the same few-shot task
learned the law in 20 of 20) — give it an independent check through confirm. Overrides sized.
confirm: confirm(candidate) → bool, an external check (a probe, a person, a second data set) asked about each
candidate that passed, best first, before it is accepted; a refusal sets the candidate aside for this test. For a
fitted numeric law, residual_check(refit) builds one: the χ² of the residuals after a refit on all rows (opt-in,
used with tol=math.inf; its docstring has the measurement and the cautions).
ties: the order between laws of equal length — "earlier" (the default: the earlier accepted law speaks) or "newer"
(the later one does: a law learned after the world changed is not hidden behind an older one of the same length).
The order is used everywhere a law is chosen: applied/predict, the rows a candidate would take in the test,
hypothesis and admit_by_probes; more specific laws always come first.
disagree: "first" (default: the most specific matching law, ties by ties) or "abstain" (predict returns
Undetermined where laws none of which is an exception of another disagree; see predict).
tol / rtol / same: how a prediction is compared with an observation — and two predictions with each other — in
step, mismatch, test (support, hits and net), observe, check, conflict and, unless the call gives its
own, admit_by_probes: exact equality by default; numbers within an absolute tol and a relative rtol
(|a − b| ≤ tol + rtol·max(|a|, |b|); tuples and lists element by element), or same(a, b) → bool. For a numeric
effect measured with noise: y = 2.5·x over x from 1 to 1e5 with a relative noise of 1e-7 is a mismatch on every
row under exact equality (no law learned in 60 steps); with rtol=1e-5 the law is learned and no later row is a
mismatch (tests/test_laws_rtol.py).
cost: the search-cost record — candidates (proposed for mismatches), tests (candidates scored on a buffer),
accepted, probes (asked by admit_by_probes), simulated (0 here; a model-based learner adds its steps) — report it
next to accuracy; proposals is the same count as solvi.core.slow.Proposals (proposed = dropped + refuted +
admitted).
recent: None (the default) — a law's running precision counts every observation since its acceptance (plus its
held-out rows), so a law right on 1,000 rows needs about 250 wrong ones before it falls below 0.8 after the world
changes; a number M — once a law has M observations since its acceptance, only the last M count (pair it with
Buffer(window=) on a stream whose laws change). memo: keep scores until the buffer or the active laws change
(memo_hits).
on_fall: what happens to a law whose running precision falls — "retire" (the default) or "suspend" (opt-in): it
leaves the active laws but stays a candidate in suspended, status "suspended". Each later observation where its
condition holds (no more specific active law applying, not from its birth group) goes into its own window of the
last reactivate_min (default 10) such observations; when the window is full, its precision is at least
min_precision and P[Binomial(m, null) ≥ hits] ≤ delta (null: the law's own null from its test, else 0.5), it is
reactivated (reactivated counts) with its running counts reset — judged on fresh observations only, not on the
buffer that retired it. doubt_probe(probes) names a probe where a suspended law and the set disagree (to ask
the world, then step it). At most suspended_cap (default 50) are kept; the oldest is then retired. The test may
also accept the same law again (it leaves suspended then). suspend(law) / reactivate(law) by hand.
equivalent (opt-in, experimental, default None): declared effect equivalences — [(e1, e2, guard_atom), ...]: "under
the atom guard_atom, the effects e1 and e2 are one effect" (a relative +1 and an absolute =1 under
count=0; a move along the facing and in the action direction under "facing = action"); or a function
equivalent(e1, e2, condition) → bool, a declaration about a whole condition for effects that are not literals.
Two candidates of one action and one condition whose effects are declared one under an atom the condition holds
are congruent: the same hypothesis wherever the condition holds, so no state can tell them apart. mismatch
proposes one of them (the first; fewer candidates, a smaller N), test and admit_by_probes merge them before
scoring / splitting, and the accepted law lists the others in Law.aliases — under siblings="undetermined" they
no longer count as conditions the pool cannot tell apart. Candidates whose condition does not hold the guard are
not congruent (the probes must separate them, as without a declaration). A guard is required (None: ValueError).
A declaration that is false where the guard holds is caught on rows only: check_equivalent(buffer) lists the
rows that contradict it, and a law that is wrong where its alias would be right falls by its running precision.
Use effects= variants when you do not know which encoding the world uses; equivalent when you know where
they coincide.
lapse_on_base_change=True, base_version="revision": every law in this set is an exception of one owner-declared base revision. set_base(base, version="new-revision") retires active/suspended exceptions; the default False keeps absolute laws independent of the fallback. Tokens are explicit declarations: changes inside a mutable callable are not detected automatically. Do not change the base during admission or a dry run; such attempts are refused. Bound laws cannot be compiled or exported to Knowledge without preserving the dependency, so those exports currently refuse. Operations on a LawSet are owner-serialized; this option does not add thread safety.
lapse_on_base_change
property
¶
Whether every law in this set is an exception bound to the declared base revision.
version
property
¶
A counter that goes up whenever the set's laws change: a law admitted (test, admit_by_probes, add,
reactivate), retired, suspended or withdrawn from suspended. A planner that caches answers recomputes when
it moved. Not changed by a dry run (admit_by_probes(..., commit=False)), by predictions or by observe
when no law falls. Kept across pickling.
proposals
property
¶
What the proposer offered and what the test did with it (solvi.core.slow.Proposals): dropped by the
grammar, refuted (proposed for a mismatch and not accepted), admitted.
set_base ¶
Replace the fallback; opt-in exceptions lapse on a new declared revision.
Default sets contain absolute laws: changing their base preserves laws. Every explicit unbound call refreshes cached fallback scores, including the same callable after owner-declared closure/table changes. With lapse_on_base_change=True every active/suspended law belongs to base_version; a new, never-reused version retires it before the new base can serve. The old laws and evidence remain in retired. Re-admit a new Candidate using fresh evidence; reactivation is not re-validation.
Tokens are owner declarations, not automatic detection of mutable code, closures or tables. Update the version even when the callable is the same object. A repeated current version is a no-op only for the same callable. Direct base assignment and per-prediction overrides are refused for bound sets. Admission/error budgets are never reset here.
__getstate__ ¶
Pickled without the score memo or its weak buffer identity reference; fallback cache identity is refreshed on unpickle, independently of other owners sharing a buffer.
agrees ¶
A prediction is the observation: == by default, else within the set's tol / rtol or by its same.
applied ¶
The law that speaks for action in state: the first matching active law (most specific first, ties by the
set's ties), or None.
matching ¶
Every active law of action whose condition holds in state, in the set's order (most specific first).
maximal ¶
Matching active laws with no strictly more specific matching exception.
Equal-length siblings are all retained, in serving order. A matching default remains only when no stricter matching condition overrides it.
predictions ¶
All (Law, predicted observation) pairs, in serving order; no base fallback.
For checking observations, use maximal=True (exceptions override their ancestors, while equal-specificity siblings must all agree), or violations(). This reader does not change predict's serving tie policy.
violations ¶
Every maximal matching law disagreeing with observed, using agrees().
Unlike applied()/predict(), this checks every applicable sibling. Retired and suspended laws are excluded; no law means no checked constraint.
conflict ¶
Where maximal matching laws disagree, return Undetermined, else None.
A strictly more specific matching exception overrides its ancestor; equal-specificity siblings all count.
predict ¶
The first matching law's effect applied to the state, or the base predictor's answer. disagree="abstain"
(or the set's own disagree): where the maximal matching laws disagree (conflict), an Undetermined instead
— the abstention is counted in abstained. The default "first" keeps most specific first, ties by the set's
ties.
A version space in miniature: two laws that both passed their test and neither of which is an exception of the other cover this state and say different things — the order between them is a tie-break, not evidence.
explain ¶
predict with its ground → (value, law): the law that gave the value, or None where no law applies and the
value is the base predictor's (None without one) — so "a law says 0" and "no law, the base says 0" are told
apart. Where disagree="abstain" abstains, (the Undetermined, None).
intervention ¶
intervention(state_before, state_after, actions, base: Callable | None = None, *, disagree: str | None = None) -> list[Delta]
The predicted effect of an intervention that turns state_before into state_after, per action → [Delta]
in the order of actions (a string action may be given alone): the prediction before and after with the law behind
each (explain; None: no law applies, the value is the base predictor's), change (after − before when both
are numbers, else None), changed (the two disagree by the set's agrees; None where either side is
Undetermined) and covered — a law, not the base, speaks on both sides. An uncovered effect is a guess: the
laws say nothing about the intervention there. disagree: as for predict.
hypothesis ¶
The predictor this set would be with candidate accepted (nothing is changed): for the probe planner.
congruent ¶
The same action and condition, and effects declared one under an atom the condition holds (equivalent):
the two candidates predict alike wherever the condition holds. False without declarations.
check_equivalent ¶
The declared equivalences that logged rows contradict → [(e1, e2, where, row ids)]: a declaration
(e1, e2, guard) on rows where the guard holds and apply(e1, s) and apply(e2, s) disagree (where = the
guard); and every active law's aliases on rows where its condition holds (where = the condition; this also
covers a declaring function). warn: one UserWarning naming them. Rows of every action and group.
mismatch ¶
No mismatch (the prediction is the observation) → []; else the proposer's candidates, without the laws
already active. buffer (with its group exclude left out, as step leaves the transition's group): the rows
the proposer may need — under Grammar(negations="refine") default enumeration requires it and raises if
absent on a real mismatch. Matching predictions still return [] without a buffer. Custom proposers may
supply their own negated candidates without one. admit_by_probes(buffer=…) does not reach them: its candidates are
computed here, before it sees the buffer, so pass the same buffer to this call. A candidate a leftover probe
falsified on a buffer that has not grown since is not re-proposed (see admit_by_probes, falsify="leftover";
the refusals are counted in refused_falsified).
score ¶
(support, hits, net) of a candidate on the held-out rows (every group but exclude): the rows it is counted
on — where it would be the applied law (support_on="applied", opt-in), or every row its condition matches
("matched") — those of them its effect predicts right, and the errors it removes minus the errors it adds where
it would be the applied law. memo=True: kept until the buffer (a row added or dropped) or the active laws
change; a repeated score on the same state is a lookup (memo_hits).
score_many ¶
score_many(candidates: Iterable[Candidate], buffer: Buffer, exclude=None, *, stable: bool = False) -> list[tuple[int, int, int]]
Score candidates in input order, including duplicates, without admitting or counting tests.
By default this is exactly sequential score calls, including iterator/callback interleaving.
stable=True explicitly promises that callbacks, candidates, active laws and buffer stay stable
throughout this call: reuse one view per action locally, as test does. No view survives the call.
This promise includes mutable closure state and side effects that version counters cannot detect.
Existing opt-in memo=True score semantics remain unchanged; this method does not enable memo.
base_rate ¶
The base null for (action, effect): the share of the buffer's rows of action (without the group
exclude) whose observation effect predicts, floored at 0.05 and capped at min_precision (0.95 when that is
1); the floor when the buffer has no row of the action.
rival_of ¶
An indistinguishable rival of candidate on the held-out rows (every group but exclude) → the rival's
condition, or None. A rival: an active law or one of others, of the same action and effect, whose condition
is neither a subset nor a superset of the candidate's and matches exactly the same held-out rows (at least
one) — the two differ on no held-out row, so the buffer cannot say which condition is the law. A refinement
(a superset condition matching the same rows) is not a rival: the shorter one wins on length. With
rivals=True test refuses a candidate that has one (see the class docstring).
passes ¶
The acceptance on counts alone (net aside): support ≥ min_support, hits ≥ min_precision · support, and with
sized=True P[Binomial(support, null) ≥ hits] ≤ sized_level(tested, delta, space). null: the set's fixed null
when it has one; with null="base" pass the rate (base_rate). acceptance="exact": support ≥ min_support and
hits == support.
test ¶
Accept up to accept candidates (default one: the best) that pass on the held-out rows of every group but
exclude (the proposing transition's): passes(support, hits, N) — support ≥ min_support, precision ≥
min_precision and, sized, the binomial bound at N = the candidates scored on this buffer so far, this call's
included — and net > 0. Best: the largest net, then fewer atoms, then the higher precision, then the earlier
candidate. → the accepted laws. The check after each acceptance may retire laws (the accepted one too):
they are in last_retired (and in Step.retired when step ran the test).
consolidate ¶
Opt-in empirical compaction; return a replacement Law, or None without changing the book.
Replace at least two strictly narrower same-action/effect laws. Test on ALL matched held-out rows with
the existing acceptance, sized/null, rivals, readmission and confirmation gates, and positive net against
the book with those laws removed. No formerly correct held-out prediction may become wrong. Other active
laws must survive the normal check. The original book need not have positive net: this is compaction,
not evidence of error reduction. law.consolidation records both nets, replaced IDs and evidence IDs.
Like ordinary empirical admission this can answer on unseen states under the declared condition; matched
evidence is not a domain guarantee. In particular, a broader default must pass on all its matched rows.
commit=False returns a proposed law, without confirmation callbacks, book changes or spending buffer N.
Rejections and callback exceptions leave the book unchanged; attempted committed tests still spend N.
Confirmation is a read-only predicate: writes to this set's book containers, counters and existing Law
fields are rolled back even when it returns True; reentrant consolidation is refused. External effects,
grammar/buffer writes and mutation inside shared action/effect/group payload objects are outside this scope.
add ¶
Put a law in without the test (a person's law, a written rule, a test fixture) → the law.
admit_by_probes ¶
admit_by_probes(candidates: Iterable[Candidate], ask: Callable[[Any, Any], Any], *, probes: Iterable[tuple[Any, Any]], safe: Callable[[Any, Any], bool] | None = None, budget: int = 8, buffer: Buffer | None = None, exclude=None, read_only: Callable[..., bool] | None = None, cost: Callable[[Any, Any], Any] | None = None, order: str = 'split', unique: bool = True, seed: int = 0, tol: float | None = None, same: Callable[[Any, Any], bool] | None = None, assume_one_law: bool = False, rtol: float | None = None, coverage: str = 'warn', max_new: int | None = None, noise: float | None = None, level: float = 0.95, fit_level: float | None = 0.01, fit_support: int = 1, commit: bool = True, siblings: str = 'shortest', coverage_on: Buffer | Iterable[Buffer] | None = None, falsify: str | None = 'auto') -> Admission
Admit one of candidates by asking the world where they disagree (CEGIS-style), not by a random held-out
sample. ask(state, action) → the observation (the environment, or an exact model of it); probes: the (state,
action) pairs you may ask about (a finite pool; also what the equivalence classes are computed on); safe:
which of them may be asked (default: all); budget: at most this many asks.
A candidate is the set with it added (hypothesis): where it is the applied law it predicts its effect,
elsewhere what the set predicts now. It is refuted by a row (a logged one from buffer — every group but
exclude, its own actions only — or an asked probe) where it is the applied law and its effect is wrong, or
where its condition does not hold, the set is wrong and its effect is what was observed (it is too narrow).
Under Grammar(negations="refine") the negated candidates reach this path only when the buffer was given to the
mismatch that computed them (mismatch(…, buffer=…)): this method's own buffer= cannot add them, the
candidates exist before it runs. The set without a new law ("null") is a hypothesis too, refuted where the
set is wrong. Loop: group the surviving hypotheses by their predictions on every probe of the pool; while
more than one group is left and the budget lasts, ask the safe probe that splits the groups most (split:
read-only first, the larger split, the lower cost), drop the refuted. Accept the shortest candidate (fewest
atoms, then the order given) only when one group of candidates is left and the null is not in it or alive
(identifiability); otherwise "undetermined" with the surviving groups — never a guess. order="random",
unique=False: the control — budget random safe probes, the shortest survivor accepted (a held-out sample of
that size). tol / same: predictions and observations compared within an absolute tolerance (numbers; tuples
element by element) or by same(a, b), for refutation, for the classes and for the probe choice (split) —
with exact equality a measurement noise of 1e-6 refuted every candidate of y = c·x; with tol=1e-4 the right c
is admitted. rtol: a relative tolerance (|a − b| ≤ tol + rtol·max(|a|, |b|)) — over x from 1 to 1e5 with a
relative noise of 1e-6, tol=1e-4 refutes every candidate (undetermined) and rtol=1e-4 admits the right c.
None of the three: the set's own comparison (its tol / rtol / same; exact by default).
The "too narrow" refutation is sound only when exactly one law is missing. Where two are (the world x0 ∨ x1, a base that says False, 10 binary atoms, k = 2, a pool of 200 random states, 8 checks), each right law is too narrow where the other fires, both were refuted, and a wrong survivor — the empty condition, or an unrelated atom that held by chance where it was asked — was accepted as the last class: 29 of 100 trials accepted a law false on some state. So a candidate refuted only as too narrow and never wrong where it applied is a witness, and the set with every witness added is the several-laws alternative: while it is right on every row seen and predicts otherwise than the surviving class somewhere on the pool where the class would apply, the probes go on to tell the two apart (within the budget) and nothing is accepted before — on that world 6 of 100 wrong (77 accepted, 23 undetermined; the same at 20 checks: in the 6 the witnesses together cover the wrong survivor wherever it applies on the pool, so no probe tells them apart). assume_one_law=True: the narrow refutation is final (before 1.2).
max_new=2: a version space over law sets instead — the set with no new law, with one candidate, or with two
(_versions.admit_up_to_two): a hypothesis is refuted where it predicts wrong and one of its new laws spoke,
or none of them holds and one of their effects is what was observed; the probes split the classes of
hypotheses that predict alike on the pool, and the smallest hypothesis of the one surviving class is accepted —
one law, or two (Admission.laws; with two accepted, law is the first of them). It assumes at most two laws
are missing (the witness guard is not used; three or more missing laws are not measured); the classes grow
with the square of the candidates. Where one law is missing it needs more probes than the default (at a
budget of 8 it is mostly undetermined there; see docs/experimental.md, laws) — give it a budget of 12 or
more. witnesses and untested are empty in this mode and coverage is
not checked, so an over-general law can be accepted without the coverage warning. max_new=1 is
assume_one_law=True; None (the default) the witness guard above. order="split" and unique=True only.
coverage: what to do when the pool cannot tell the accepted (shortest) candidate from a longer one of its
class — an atom the longer one adds never varied on the pool or the rows seen where the shorter one holds
(a pool where every state is next to the table admits "make & wood ≥ 1 → pickaxe" without "near table";
the probes agree with both, so the shortest is taken). "warn" (the default): accept as before, list the atoms
in Admission.untested and warn; "undetermined": refuse to accept, with the atoms in untested; "ignore":
accept silently (before 1.2). Give the pool states where those atoms vary to settle it. An atom the
shortest condition implies (Grammar(implies=)) is never listed: it cannot vary where the condition holds.
"varied" (opt-in, experimental): refuse only for an untested atom that took both values on some row of
coverage_on (one Buffer or several) where the shortest condition holds — the
evidence exists elsewhere in the history and the pool missed it; an atom that never varied anywhere is listed
in untested and does not block (the shorter law is taken, Occam). Those rows are read through their atom
bitsets only (no test, no N increment); with no coverage_on, the pool and buffer= are all the
evidence and nothing blocks.
Measured on a synthetic world (10 binary features, 20 atoms, k = 3; the hidden law a random conjunction of two atoms; the 176 candidates Enumerate proposes from one state where it fires; a pool of 200 random states; 300 trials; a wrong law = one that differs from the hidden law on any of the 1,024 states): at 8 checks a random held-out sample with the shortest survivor accepted took a wrong law in 0.593 of the trials (0.407 right); the active admission asked 7.56 probes on average and accepted 0.897 right, 0.000 wrong, 0.103 undetermined (at 10 checks: 1.000 right in 7.66 asks; with assume_one_law=True at 8: 0.987 right, 0.013 undetermined, 7.45 asks — the check against the several-laws alternative costs about one probe). The uniqueness rule alone on 8 random probes: 0.000 wrong but 0.893 undetermined; 20 random probes with the shortest survivor: 0.037 wrong (tests/test_laws_sized_probes.py reruns the 8-check comparison on 150 trials). The accepted law records admitted="probes", tested = the number of candidates, support = the rows where it applied, born = "probe".
noise=ε (opt-in; 0 < ε < 0.5): an answer may be wrong — with probability ε it is not the true outcome. One
answer then refutes nothing: every hypothesis (the set without a new law, and the set with each candidate)
keeps a likelihood — 1 − ε for an answer it predicts, ε / (G − 1) otherwise, G the distinct predictions at that
probe — and each class of hypotheses that predict alike on the pool a weight (equal prior per class; within a
class the mean likelihood of its members). The next probe is the safe one with the largest expected
information gain under that noise (split(score="eig")); it may be a probe already asked (a repeat
measurement). It stops when one class holds at least level of the weight (default 0.95) or the budget of
answers is spent; the shortest candidate of that class is accepted when the class is not the set without a new
law, else "undetermined". It assumes one law is missing (the witness guard is not used), the noise is
independent between answers, and ε is known; coverage is not checked and classes lists every class (no
class is dropped). order="split", unique=True and max_new=None only. Logged buffer rows count as answers
under the same noise.
The weight compares only the offered hypotheses with each other (a closed world): it says which candidate is
least bad, not that it is right. When the true law is not among the candidates, a wrong one can hold all of the
weight — in 1.2.0, with the hidden law x0 ∧ x1 not offered, 200 logged rows gave "x0" all of it and it was
accepted with 0 probes asked, right on 56 of the 101 rows where it fired at ε = 0.05
(tests/test_laws_noise_fit.py). So the chosen law must also fit the answers (fit_level,
fit_support; the open-world guard): on the rows where it applies (logged buffer rows of its action, every
group but exclude, and the asked probes) a right law misses only by noise, at most Binomial(n, ε). With m
misses on n rows, P[Binomial(n, ε) ≥ m] < fit_level (default 0.01: a right law with independent answers and
the stated ε is refused by this check at most 1 % of the time), or n < fit_support (default 1), gives
"undetermined" with a reason that says the likely law does not fit the noise (or that there is too little
support to check the fit). Few rows give the check little power — two misses in five at ε = 0.05 still pass —
so raise fit_support when a few answers can carry the weight. fit_level=None: no fit check (as in 1.2.0).
commit=False: a dry run. The same decision is made (the probes are asked and spent, cost["probes"] counts
them), and the Admission is returned as with commit=True, but the set is left as it was: the law(s) of an
"accepted" admission are not added (their status is "proposed"), no suspended law is moved, and version,
cost["accepted"] and the next law's id are unchanged. Add the law afterwards with add (admitted="hand")
or run again with commit=True; refusing it needs no retire, so retired keeps only laws that fell.
siblings: "shortest" (the default) — the shortest candidate of the one class left is accepted, ties by the order given, even when the class holds other conditions of the same length; "undetermined" — then refuse: the pool cannot tell those conditions apart and the order would choose (two conditions of one atom each that predict alike on every probe). unique=True and max_new=None or 1; under noise= the likely class is read the same way.
ask may return Unknown (from solvi.experimental.laws) when the probe's situation turned out not to be the
one asked about: the answer refutes nothing and supports nothing, the probe counts as spent (budget,
cost["probes"]) and is listed in Admission.unknown, not in asked. Every actual oracle attempt is
charged immediately before the call, including a callback that raises; errors propagate without committing
a proposed law, and commit=False still retains that real attempt cost.
falsify="leftover" (experimental; selected by the default "auto" on supported modes; explicit None
disables it): the split phase
stops once one class is left and leaves the rest of the budget unspent, so when the grammar cannot express the
true law the accepted survivor is the shortest wrong law and no probe has tried to refute it where it fires.
With "leftover", once the admission would accept, the safe pool probes not yet asked where the accepted law
would be the applied law (its condition holds and no more specific active law shadows it — not merely where
its condition holds) are drawn uniformly at random with seed, up to the leftover budget − probes asked; the
first answer other than the law's effect makes the admission "undetermined" (reason: refuted where it applies,
with the probe) and commits nothing. Otherwise the law is committed as without the option (its support counts
the rows before the falsification probes) and the reason says how many of the eligible pool probes were asked.
The falsification probes are spent: they are in asked (ids "falsify-…"; an Unknown answer in unknown)
and in cost["probes"], never above budget. No probe is held back from the split phase (a reserve was
measured too costly at a small budget). max_new=2 checks the full chosen hypothesis before committing
either law; a failed pair does not blacklist its individual members. noise= raises ValueError with it
(under noise one answer refutes nothing).
The falsified law is remembered in the set: a candidate refuted where it applies by a leftover probe is not
re-proposed (mismatch(…, buffer=…) drops it) and not offered here again until the buffer grows past the size
it was falsified at — the readmit="grown" rule — or, when it was falsified with no buffer, while the call
gives none (a probe answer does not un-happen; without a row store nothing can change it). The refusals are
counted in refused_falsified, why_not names one ("falsified"), and when every candidate of a call was
dropped this way the admission is "undetermined" saying so, at no probe. Before this change the same mismatch
asked its probes again on every call — refuted 3× by the same probe in the handoff's repro; passing the asked
rows back as buffer= was the workaround and still is (it also refutes the candidates the split phase
refuted, which this memory does not).
Measured on a synthetic boolean world (8 bits and a counter, 2,560 states, six rules, two grammars — a benchmark re-created from a published description —, a pool of 200 states, 20 seeds, budgets 8 and 20, exact answers; a wrong law is one false on some state where it applies). On the cells whose law needs a negation the grammar lacks, at budget 20 the default admitted 131 wrong laws and "leftover" 1. That one survivor was probe-starved: a narrow law with only 3 eligible pool states for a leftover of 13, all asked, none where it is wrong — the leftover probes cannot refute a law when none of them falls in its wrong region; give the pool states where the law applies (drawing them from the whole state space where it applies let 0 of 131 through, measured once, reported, not an option). No loss in sound admissions on the cells whose law the grammar can express: 124 against 123 at budget 8, 251 against 236 at budget 20 (wrong 5 against 69 there, final accuracy 0.987 against 0.972), because a right law is never refuted. At budget 8 it does little where the law is inexpressible: the leftover was 0–1 probe in 105 of 140 rounds and 83 of 83 wrong laws survived — give a larger budget. The price is asks: a right law spends its whole leftover (about 10 of 20 probes at budget 20; 7.9 → 10.0 asks per round over the grid). n probes without a refutation bound the law's miss rate on its own domain at about 3/n at 95 % (0.24 at the 12.5 probes available on average at budget 20): the wrong laws there were wrong on about half their domain and fell after 1.9 probes on average; a law wrong on a few percent of its domain survives. Two missing laws (a disjunction) are not fixed (19 of 51 wrong against 26 of 57 at budget 8; use max_new=2). Not measured with siblings="undetermined", coverage="undetermined", noisy answers or a real environment. Two pre-registered runs of one stand: the first run's reproduction gate was sized below the ≈ 6 % wrong this method documents on a two-law cell, failed there, and was revised — disclosed — before the second run; the numbers above are the second run's, and the first run's agree (2 wrong against 120 at budget 20; sound 254 against 242). These measurements concern one synthetic world, two runs. falsify="auto" (default) uses "leftover" for exact answers, including max_new=2 (the whole selected hypothesis is checked before any law commits). noise= requires explicit falsify=None acknowledging closed-family selection without exact falsification. Explicit "leftover" rejects noise=. The default can spend the whole budget even on a right law; finite pool falsification is not a statistical guarantee.
check ¶
Recount each law's held-out rows on buffer (rows of other groups than its birth where it is the applied
law) and retire the laws whose running precision fell below retire_below → the retired laws.
running ¶
(hits, n) the retirement judges: law.running(), or with recent=M, once the law has M observations since
its acceptance, only the last M of them.
observe ¶
Count a transition against the law applied to it (unless it comes from the law's birth group) and retire the law if its running precision fell → the retired laws (on_fall="suspend": suspended). Suspended laws whose condition holds count it in their window and may be reactivated (after the active laws have counted it; a law this transition suspends does not count it).
suspend ¶
Take a law out of use but keep it as a candidate (suspended), re-checked on later observations.
reactivate ¶
Put a suspended law back in use, its running counts reset (judged from here on).
doubt_probe ¶
doubt_probe(probes: Iterable[tuple[Any, Any]], safe: Callable[[Any, Any], bool] | None = None, *, read_only: Callable[..., bool] | None = None, cost: Callable[[Any, Any], Any] | None = None)
(law, Probe) for the most recently suspended law that some safe probe tells apart from the set (split
over the set and the set with that law back), or None. Ask the world there and step the answer: it goes into
the law's window.
retire ¶
Take a law out of use (kept in retired with the reason).
step ¶
One transition of the loop: predict, count it against the applied law (retire), on a mismatch propose and
test on the buffer without this transition's group; remember: then add it to the buffer.
learn ¶
The loop over a stream, in order (not a schedule: every transition is predicted before it is learned from).
need_at ¶
The least support with which a law right on precision of its rows passes the acceptance at N = tested
candidates against null (min_support, min_precision and — sized — the binomial bound); None when no support
does: a precision at or below the null, or below min_precision. At precision 1 and sized: sized_need.
why_not ¶
why_not(candidate: Candidate, buffer: Buffer, exclude=None, others: Iterable[Candidate] = ()) -> dict
Why test would not accept candidate on buffer (the held-out rows of every group but exclude) — the
first of its tests the candidate fails, in the order test applies them, and by how much. Read-only: the
buffer's N and the set's cost are not touched. → {"passes", "failed", "reason" (in words), "support", "hits",
"net", "precision", "null", "tested", "need", "short"}.
null_warning: present for sized null="base"; describes the same-buffer estimated comparator and possible regime mixing, without declaring every such test invalid. This is a stateless diagnostic; it does not detect a historical change or change admission.
failed: None (it would pass; confirm, when the set has one, is still asked), "active" (already an active
law), "readmit" (retired on a buffer this size, readmit="grown"), "falsified" (refuted where it applies by a
leftover probe on a buffer this size, admit_by_probes(falsify='leftover'), and not re-proposed since), "cap"
(the set holds cap laws), "net"
(it removes no more errors than it adds where it would be the applied law), "min_support", "min_precision",
"exact" (acceptance="exact": wrong on some row it applies to), "sized_need" (fewer rows than the sized rule
needs at this N), "sized_bound" (enough rows, but the binomial tail against the null is above δ_N), "rival"
(rivals=True: an active law or one of others — the other candidates of the same test — is an
indistinguishable rival, rival_of; "rival" in the result names its condition). tested: the
N test would score it at (this candidate counted when it is new to the buffer); need: the support a law right
on every row needs at that N (min_support when not sized); short: rows short of need for the support tests,
hits short of min_precision · support for "min_precision", else 0. A net refusal may also give
"covered_by" (active, syntactically narrower same-action/effect law IDs matching some held-out rows)
and a "hint"; this does not establish coverage or correctness on unseen inputs.
report ¶
report(buffer: Buffer, steps: Iterable[Step] | None = None, *, precision: float | None = None) -> dict
What a learning run did, and per action whether the buffer could admit a law at all — so «0 laws accepted»
reads as «too few rows» where that is the reason. → {"accepted", "active", "retired", "tested" (N, the
candidates scored on the buffer), "cost", the run's "steps" / "mismatches" (with steps), "actions": {action:
{"rows", "outcomes", "null", "need", "need_at_precision", "precision", "enough"}}, "too_few_rows" (the actions
whose rows are fewer than "need"), "summary" (in words)}.
Per action: "rows" in the buffer; "null" the least base null among its outcomes (the rarest outcome's share,
floored at 0.05 — the set's fixed null when it has one); "need" the least support a law right on every row
needs to pass at this N against that null (sized_need; min_support with sized=False) — a law's support is at
most the action's rows, so rows < need means no law of that action can be accepted on this buffer, whatever
its condition; "need_at_precision" the same for a law right on precision of its rows (default
min_precision; None when no support suffices). More outcomes are harder: their nulls are higher.
to_module ¶
The active laws as one Python module that imports nothing from solvi: one function per law, LAWS in this
set's dispatch order (most specific first), applied(s, a) and predict(s, a, base=None, apply=None) with
the base predictor as the fallback. Check it with check_compiled(module, buffer, lawset). ValueError naming
every atom without code.
to_knowledge ¶
Write the laws into a KnowledgeStore: each a "rule" item with body {"kind": "law", "action", "if", "then"},
source "outcome" (the environment answered), by "laws:
Proposer ¶
Bases: Protocol
Proposes candidate laws for a mismatch: propose(lawset, state, action, observed) → candidates. It proposes;
the held-out test decides.
RandomProposer ¶
The control: n conditions drawn from the whole grammar (default: as many as Enumerate would propose for this
mismatch, with the same mix of sizes), each with an effect variant of the observation. Seeded.
Step
dataclass
¶
Step(transition: Transition, predicted: Any, law: Law | None, mismatch: bool, candidates: int = 0, tested: int = 0, accepted: list = list(), retired: list = list())
One transition of the loop: what was predicted (and by which law), whether it was a mismatch, how many candidates were proposed and how many of them were scored on the buffer (tested), which laws were accepted and which retired — by this transition's observation, or by the check the test runs after an acceptance (a law accepted and retired by that check is in both lists).
Transition
dataclass
¶
One observed step: the state before, the action, what was observed after, and its group (the world, the session, the path — what must not judge a law born from it). id: given, or a content hash.
Undetermined
dataclass
¶
The answer of predict(..., disagree="abstain") where laws that are not exceptions of one another disagree:
the laws (the most specific matching ones, none of whose conditions contains another's) and what each predicts.
Never equal to an observation.
GaussianProbes ¶
Finite numeric hypotheses with owner-declared independent Normal measurement noise.
sigma is a positive standard deviation in the prediction's units, or fn(state, action). Information ranking integrates the continuous Normal mixture using 64 deterministic midpoint-quantile nodes per component: approximate expected bits, not a certificate. Predictions/sigma must remain frozen during an experiment. Posterior confidence is conditional on this finite family, prior and noise model, not guaranteed coverage.
choose ¶
Read-only first, then expected bits (or bits/cost), lower cost, original order.
Unsafe probes are excluded before prediction/noise/cost callbacks. A supplied max_cost excludes unaffordable probes before selection; repeats stay eligible.
observe ¶
Update on one fresh measurement, including another at the same point.
run ¶
run(observe, safe, probes, *, max_observations, max_cost=None, confidence=None, read_only=None, cost=None, per_cost=False)
Adaptive bounded loop; observe(state, action) performs the actual measurement.
Stop on declared call/cost budget, conditional posterior confidence, or no informative affordable safe probe. Failed measurements propagate, not retry. Returned history holds posterior after each completed observation.
Probe
dataclass
¶
A chosen probe: the state and action to try, each hypothesis's prediction there, and the split score (how many
hypotheses the answer can rule out at least: those outside the largest group of equal predictions); info: the
expected information of its answer in bits (the entropy of the groups of equal predictions, every hypothesis
equally likely); cost: its cost when split was given one.
BagOfAtomsRanker ¶
BagOfAtomsRanker(seed: int = 0, l2: float = 0.001, steps: int = 400, lr: float = 1.0, top_pairs: int = 64)
Bases: _ByAtom
A numpy logistic regression from an atom's features to "this atom is in the true condition"; score(condition) =
Σ over its atoms of the atom's log-odds (under independence, the log of the odds that this is the condition, up to
a constant). Trained by fit on synthetic tasks drawn from the grammar itself: a random condition labels logged
states, the features are read off the buffer, the target is the condition. Seeded and deterministic (full-batch
gradient descent from zero weights).
fit ¶
fit(grammar: Grammar, sample: Callable[[Random], Any], *, tasks: int = 200, rows: int = 300, noise: float = 0.0) -> 'BagOfAtomsRanker'
Train on tasks synthetic tasks: sample(rng) → a state drawn as the logged states are (seeded rng).
fit_arrays ¶
The regression itself, on atom feature rows and their labels.
FeatureRanker ¶
Bases: _ByAtom
The control: the raw statistic, no training. score(condition) = Σ over its atoms of (stat − center), stat one of
STATS ("jaccard" by default), so an atom below center lowers the score.
HandoverRanker ¶
From the prior to the joint statistic as the logged rows arrive. A prior is a prior, the row statistic is evidence, and evidence takes over gradually.
mode="blend" (the default): score = the joint statistic's Jaccard (in [0, 1]) + w(rows) · the prior's score mapped
to [0, 1] by its rank among the candidates (so the two scales do not fight), w(rows) = n0 / (n0 + rows) — 1 with no
rows (the prior's order), ½ at n0 rows, → 0 as rows grow; equal blends → fewer atoms first, then the inner order (as
JointRanker's ties). A prior with learned False (PriorRanker's fallback) leaves the joint statistic alone (the
blind order with no rows). mode="switch": the prior while the action has fewer than min_rows logged rows, the
joint statistic from then on. rows: the logged rows of the candidates' action without the group exclude (several
actions → the fewest). last: "blend", "prior" or "joint" — what scored the last call; scored: a Counter.
JointRanker ¶
The joint statistic of the whole condition, no training: a candidate's score is the Jaccard between the logged
rows its whole condition matches and the rows where its effect predicts right (buffer.match(action, condition)
against the effect mask, both over the rows without the group exclude); equal Jaccard → fewer atoms first, then
the inner order. The bag-of-atoms scorers add per-atom scores and so cannot see that two atoms together pick out the
effect's rows when neither does alone, or that a correlated twin of a true atom adds nothing; this one scores the
conjunction itself. A statistic over the logged rows: with no rows every score is equal and the order is the inner
one (the blind order). effect_mask is the rows a score compares against — override it for a control.
effect_mask ¶
The rows of action (without the group exclude) where effect predicts right.
jaccard ¶
jaccard(candidates: Sequence[Candidate], lawset: LawSet | None, buffer: Buffer, exclude=None) -> list[float]
The Jaccard per candidate (0 where neither its condition nor its effect has a row).
score ¶
score(candidates: Sequence[Candidate], lawset: LawSet | None, buffer: Buffer, exclude=None) -> list[float]
A score per candidate (higher = check earlier): the Jaccard minus a tie-break by the number of atoms small enough never to reorder two different Jaccards (they differ by at least 1 / rows²).
PriorRanker ¶
PriorRanker(seed: int = 0, *, l2: float = 0.001, steps: int = 50, smoothing: float = 2.0, shuffle_labels: bool = False, fallback: bool = True, margin: float = 0.05)
A prior order for candidate conditions that reads no rows: a linear score over features of the condition itself,
learned from synthetic tasks drawn from the grammar — for a buffer with no rows yet (then the row statistics give
the blind order). The features of a condition: its number of atoms (one-hot 0..k) and the sums over its atoms of
four log-rates learned from the tasks' true conditions — the atom's prior usefulness (how often the atom is in the
true condition when it is true in the mismatched state), its feature's, its kind's (the comparison type read off the
name: atom_kind), and over its pairs of features their co-occurrence in true conditions beyond the two features'
own rates (a log ratio; 0 for independent features). The rates are smoothed towards the overall rate with
smoothing pseudo-counts; an atom, feature or kind never seen scores the overall rate.
fit draws tasks synthetic tasks (synthetic_task, the same signature as BagOfAtomsRanker.fit; sizes=(lo, hi)
draws condition sizes uniform in lo..hi as a setting does) and fits the weights by a softmax over each task's
candidates (grammar.conditions of the atoms true in the mismatched state) with the true condition as the target:
Newton's method from zero weights on standardized features (L2 l2, at most steps steps), the rates of each half of the tasks read
from the other half (so a task's own condition never feeds its features), the final rates from all tasks. Seeded
and deterministic. shuffle_labels=True is the control: each task's target replaced by one of its candidates drawn
at random (seeded) — a prior that learns nothing should do no better than the blind order. Unfitted: every score
0 (the inner order).
The fallback (fallback=True, the default): a prior that learned nothing must never be worse than the blind order.
fit measures, on its own training tasks, the mean checks to the true condition under weights fitted on the other
half of the tasks (cv["prior"]), under a fit on shuffled labels measured the same way (cv["shuffled"]) and in
the blind order (cv["blind"]: the grammar's order, by length then name); unless the first is at least margin
(5%) below both others, learned is False and every score is 0 — the inner order. learned is None with
fallback=False (the raw control).
fit ¶
fit(grammar: Grammar, sample: Callable[[Random], Any], *, tasks: int = 200, rows: int = 300, noise: float = 0.0, sizes: tuple[int, int] | None = None) -> 'PriorRanker'
Train on tasks synthetic tasks: sample(rng) → a state drawn as the logged states are (seeded rng).
fit_tasks ¶
Train on given tasks: anything with state, action and condition (a SyntheticTask, a logged mismatch
whose law is known). ValueError for none.
score ¶
score(candidates: Sequence[Candidate], lawset: LawSet | None, buffer: Buffer, exclude=None) -> list[float]
A score per candidate (higher = check earlier); reads no rows of buffer (its grammar names the features of
atoms added after the fit).
RankedProposer ¶
Another proposer's candidates, reordered by scorer.score(candidates, lawset, buffer) (highest first, ties in
the inner order). Never adds, drops or changes a candidate: only the order, which is what a probe budget spends
(the order in which split / discriminate receive hypotheses). The buffer: the logged rows the features are read
from; the held-out test stays the judge. Any scorer with that score works (FeatureRanker, BagOfAtomsRanker,
JointRanker, PriorRanker, HandoverRanker); scored_by after a call: the scorer's last when it has one (a
HandoverRanker's "prior" / "joint"), else its class name.
SyntheticTask
dataclass
¶
SyntheticTask(buffer: Buffer, lawset: LawSet, state: Any, action: Any, observed: Any, condition: tuple)
A task drawn from the grammar: a buffer of logged rows labelled by a random condition ("on" where it holds, "off"
elsewhere, flipped with probability noise), a law set predicting "off", and a mismatch — a state where the
condition holds, observed "on".
coarse ¶
Outcomes grouped coarser than the observation: LawSet(grammar, **coarse(lambda seen: seen["kind"])). A law
then predicts the observation's class key(observation) (an OutcomeClass), and a prediction is right when the
observation falls in that class — so details a law cannot know (a random amount, a timestamp) no longer make every
row a mismatch. → the keyword arguments effects, apply and same for LawSet (do not pass those three too);
LawSet.report then counts outcomes by class.
check_compiled ¶
Run the compiled module (source or module) against lawset on every row of buffer: the applied law's id and
the prediction must agree on each row.
load_module ¶
Execute generated source as a fresh module (not registered in sys.modules).
fisher_p ¶
Two-sided Fisher exact test p-value of the 2×2 table [[a, b], [c, d]] (sum of the tables at most as likely).
sized_level ¶
The error level one candidate is tested at when tested candidates have been scored on the buffer: δ / space
when you declared the number of candidates you will test (space, Bonferroni; valid while tested ≤ space), else
δ / (N·(N + 1)) with N = tested — the n-th candidate gets at most δ / (n·(n + 1)) and these sum to less than δ over
any number of candidates, so the bound holds however long the stream runs.
sized_need ¶
The smallest support with which a law that is right on every row passes the sized rule at this N: the least s with null**s ≤ level, i.e. ⌈ln(1/level) / ln(1/null)⌉ — about (2·ln N + ln 1/δ) / ln(1/null) without a declared space. At N = 52,000, δ = 0.1: 9 at null 0.05 (a rare effect's base rate, floored), 68 at 0.7, 228 at 0.9.
discriminate ¶
discriminate(laws_a, laws_b, safe: Callable[[Any, Any], bool], probes: Iterable[tuple[Any, Any]], *, read_only: Callable[..., bool] | None = None, cost: Callable[[Any, Any], Any] | None = None, tol: float | None = None, same: Callable[[Any, Any], bool] | None = None, rtol: float | None = None) -> Probe | None
The cheapest safe probe where two law sets (or predictors) predict different outcomes — read-only actions
before writes, then the lower cost, then the order given; None when they agree on every safe probe. tol / rtol /
same: as in split (different = not within the tolerance).
eig ¶
The expected information gain in bits of one answer at a probe whose hypotheses fall into groups of equal
prediction with these total weights (summing to 1), when an answer is wrong with probability noise and a wrong
answer is one of the other groups' values, uniformly: H(answer) − H(answer | hypothesis).
split ¶
split(hypotheses: Sequence, safe: Callable[[Any, Any], bool], probes: Iterable[tuple[Any, Any]], *, read_only: Callable[..., bool] | None = None, cost: Callable[[Any, Any], Any] | None = None, tol: float | None = None, same: Callable[[Any, Any], bool] | None = None, rtol: float | None = None, per_cost: bool = False, score: str = 'split', noise: float = 0.0, weights: Sequence[float] | None = None) -> Probe | None
The probe that best splits the hypotheses (law sets, or predictors fn(state, action)), among probes —
(state, action) pairs — that safe(state, action) allows. Order: read-only probes first (read_only(action), or
read_only(state, action) when the kind of call is in the state),
then the larger split, then the lower cost(state, action), then the order given. None when no safe probe tells
any two hypotheses apart. Predictions are told apart by exact equality, or within tol (numbers: |a − b| ≤ tol,
tuples element by element), within a relative rtol (|a − b| ≤ tol + rtol·max(|a|, |b|)) or by same(a, b) —
two formulas that differ by float rounding (x·0.1·10 and x) are the same hypothesis under tol=1e-9; over
predictions from 0.003 to 3,000, rtol=1e-6 keeps a law 0.1 % off apart where no single tol does both.
per_cost=True (opt-in; needs cost): the probe with the most information per unit of cost — info / cost, info
the expected bits of its answer (Probe.info) — after read-only first; so a cheap probe that halves the hypotheses
is taken before one ten times dearer that splits them a little better. A cost ≤ 0 counts as 1e-12 (a free probe
first).
score="eig" (opt-in): rank by the expected information gain of the answer under measurement noise — noise the
chance that an answer is not the true outcome (0 ≤ noise < 0.5; a wrong answer is taken to be one of the other
predicted values at that probe, uniformly) and weights the hypotheses' current probabilities (default: equal;
normalised). Probe.info is then that gain in bits: H(answer) − H(answer | hypothesis). With noise 0 and equal
weights it is the split's entropy (Probe.info of the default). Under noise the same probe may be the best again
after its answer (a repeat measurement); probes are not removed once asked. After read-only first, the larger gain,
then the lower cost, then the order given.
residual_check ¶
residual_check(refit: Callable[[Any], Any], level: float = 0.995, dof_sigma: int | None = None) -> Callable[[Any], bool]
→ a confirm(candidate) closure for LawSet(confirm=) that keeps a fitted numeric law only while the χ² of its
residuals after a refit is below the level quantile of χ² at n − k degrees of freedom. Opt-in; nothing changes
unless you pass the closure.
check = residual_check(refit) # level=0.995
laws = LawSet(g, apply=..., tol=math.inf, confirm=check)
refit(candidate) → (residuals, k), or None when the candidate cannot be fitted (refused). The refit is yours: fit the candidate's constants on the design rows and the held-out rows together, and return the residuals of every one of those rows in units of the measurement error, (observed − predicted) / σ, with k the number of fitted constants. The check is Σ r² ≤ χ²_level(n − k), n = len(residuals); n − k ≤ 0, a residual that is not finite, or None is a refusal. level: in (0, 1); 0.995 is 1 − α/2 at α = 0.01, the level measured.
σ must be known for that rule: with an estimated σ̂ (residuals in units of σ̂, σ̂² carrying ν degrees of freedom — e.g. ν = 10 from five independent groups of three repeated measurements, 15 observations in total) the plain χ² quantile refuses the true law far above the nominal level: on the true law y = 2x at σ = 1 % with σ̂ from 15 reps the refusal rate was 0.2315 against a nominal 0.005 (2,000 trials; tests/test_laws_handoff_fixes.py re-measures the three rates). Pass dof_sigma=ν: the check is then Σ r² ≤ (n − k) · F_level(n − k, ν) — the F quantile, of which the known-σ rule is the ν → ∞ limit (dof_sigma=None, the default, changes nothing). The exact F reference requires RSS/σ² to have a χ²(n − k) distribution and ν·σ̂²/σ² to have an independent χ²(ν) distribution. A correctly specified Gaussian linear refit with the appropriate fixed variance model supplies the numerator assumption; arbitrary refits do not establish it. Common rows can be used only when the two statistics have a justified independent decomposition. Merely counting leftover degrees of freedom does not justify independence: if σ̂² = RSS/(n − k) from the identical residuals, their standardized sum of squares is identically n − k, not an F-distributed statistic. Nonlinear fitting, random relative-error normalization, non-Gaussian noise and selecting the model on the same data do not automatically retain the exact reference distribution. The caller must justify these assumptions; dof_sigma supplies a degree count, not a check of them or a correction for model selection.
Use it with tol=math.inf: the row band is then off (every held-out row is a hit), min_support and net > 0 still
apply, and the χ² decides. Under tol=math.inf the sized bound is vacuous — with every row a hit it no longer
binds — so the false-accept control is the χ² level alone, and the level has no term for the number of candidates
tested (four were tested where it was measured). With a finite tol the check is asked after the row test, on
the candidates that passed it.
Why. The row test counts held-out rows inside tol: a 4σ band has almost no power against a bias of 1–2σ, and when
the law's constants are fitted it refuses the true law whose predictions carry a fit error larger than σ. Measured
once, on synthetic physics worlds re-created from another group's description: on a pendulum whose formula list
lacks the bob radius (200 worlds, 70 where the best wrong formula sits 1–2σ from the truth) the share of wrong laws
accepted in the 1–2σ bin went 0.60 → 0.14 over those 70 worlds (rows 42 of 70, this check 10 of 70) — both numbers
with a known σ; on Kepler orbits at Tycho's precision (0.03°) Kepler was accepted 4 → 12 of 12 skies; where the
truth is in the list nothing was lost (179 of 200 right and 0 wrong, the row rule 179 and 1). A sign test on the
same residuals refused nothing in 415 cases at 12–14 rows and is not part of the check.
Cautions. The refit χ² checks goodness of fit, not prediction: a flexible wrong model can pass it — an epicycle system with 25 constants did on 1 of 12 extra skies, where the row test on the held-out rows refused it — and so does a rigid one at a high noise: at 10 % noise, with the truth y = x(1 + 0.05x), the wrong law y = c·x (the right structure, the constant refitted every trial) passed 0.961 of 1,000 trials (σ known) — a wrong formula that fits within the noise is not refused, and the χ² alone does not carry "never a guess". Keep n − k large, and prefer a held-out check when the model is flexible. The level carries no candidate-count term. One task, physics only: a default change waits for a second task outside physics.
atom_features ¶
atom_features(lawset: LawSet | None, buffer: Buffer, action, effect, *, exclude=None, top_pairs: int = 64) -> tuple[list[str], np.ndarray]
Per atom of the grammar, against the rows of action where effect predicts right (the effect mask) on the
logged rows (without the group exclude): precision, recall, Jaccard, anti-match (rows without the atom that are
without the effect, among those without the atom), the atom's rate, the effect's base rate, and the best precision /
recall / Jaccard of a pair containing it — pairs only among the atoms of highest Jaccard, at most top_pairs pairs,
one atom per feature. → (atom names in grammar order, an (atoms, 9) array). A fixed length for any grammar.
atom_kind ¶
An atom's comparison type, read off its name: the first of != >= <= == = > < it contains ("==" reads as "="),
else "bool". A negated literal ("not:
candidate_features ¶
candidate_features(cand: Candidate, buffer: Buffer, lawset: LawSet | None = None, *, exclude=None, top_pairs: int = 64) -> np.ndarray
One candidate's features, a fixed length whatever its size: its number of atoms, the effect's base rate, and the
sum, the minimum and the maximum over its atoms of atom_features (zeros for the empty condition).
checks_to ¶
Checks until condition in a candidate order: its position among the distinct conditions (1-based), or None.
synthetic_task ¶
synthetic_task(grammar: Grammar, sample: Callable[[Random], Any], rng: Random, *, rows: int = 300, noise: float = 0.0, size: int | None = None, action: Any = 'act', groups: int = 4, min_support: int = 3) -> SyntheticTask
One synthetic task (see SyntheticTask): a condition of size atoms (default: uniform in 1..k) with at least
min_support matching rows (redrawn otherwise, up to 50 times), rows from sample(rng) in groups groups.
solvi.experimental.laws.factored¶
A multi-law (factored) transition model: one LawSet and held-out Buffer per channel, δ split over the channels,
row provenance. Experimental: more accurate than one joint LawSet on MiniGrid, no saving in transitions.
A multi-law (factored) transition model (experimental; use it for accuracy, not to learn from fewer transitions): several law sets that predict one step together, each over its own channel, with one joint error budget.
from solvi.experimental.laws.factored import Channel, Row, FactoredModel
model = FactoredModel([Channel("self", g_self, base=lambda s, a: "stay"),
Channel("enter", g_enter, base=lambda s, a: "pass"),
Channel("gone", g_gone, base=lambda s, a: "stays")], extract=my_rows, delta=0.1)
model.new_group(world_id) # a world, a level, a session: what must not judge its own laws
st = model.step(pre, action, post) # my_rows(model, pre, action, post) -> [Row, ...]; each learned
model.predict("enter", occupant_features) # the composer (the caller's turn skeleton) reads the channels
model.report()["bound"] # the joint promise: the sum of the per-channel deltas
What is the caller's. The channels (names, grammars, base predictors, effect encodings apply / effects), the atoms,
the row extraction (which situations of one transition are clean enough to be a row of which channel, and what the
row is labelled with — the credit assignment), the provenance of each row (Row.depends: the laws of other channels
whose predictions the extraction or the label relied on), the composer that turns channel answers into a next state,
the probe pool and ask for admissions, and the group boundaries (new_group). Nothing here knows a grid or a turn
order.
What the model does. One LawSet per channel (the channel name is its action slot) with one held-out Buffer
(every group; a candidate is never tested on the group it was proposed from) and one local Buffer of the current
group (the logged rows admit judges with), dedup within a group, the joint δ, provenance voiding, a version for a
planner's caches, a report, knowledge items and compiled modules per channel.
The guarantee.
Per channel c: as LawSet with sized acceptance at delta = δ_c — P[the held-out test accepts, over every candidate
ever tested on c's buffer, a law no better than its null] ≤ δ_c, assuming the held-out rows are independent draws,
the candidates were not chosen by looking at them, and the rows' labels are right.
Jointly: P[any such false law in any channel] ≤ Σ_c δ_c (the union bound; no independence between channels is
needed, the channels may share transitions). share="equal" (the default) sets δ_c = δ / C, so the model promises
delta; a dict {channel: δ_c} shares it unevenly, and report()["bound"] prints the sum (warned above 0.5).
Not covered: laws admitted by probes (admit: identifiability, no δ — counted per channel in the report), the
min_precision filter (a law wrong on up to 1 − min_precision of its rows passes by design), and labels that depend
on other channels' laws. For the last the model gives provenance, not a bound: when a law named in a row's
depends leaves the active set (retired, suspended, withdrawn), that row is voided in every buffer of its channel
(Buffer.void): it leaves every test and its count leaves the running precision of the law it was counted against;
N never decreases. A law accepted on voided rows is flagged stale in the report (no automatic re-test; it falls by
its running precision as before). While the named law stands, a channel's bound is conditional on it.
The action slot (action=, opt-in). A channel's name is its LawSet's action slot, so the caller's own action is an
atom of the channel's grammar like any other, and a channel law may leave it out — generalise over actions the data
never varied (measured on MiniGrid: front: key fits -> door open, right for toggle, wrong for every other action).
FactoredModel(action="act") names the feature of the action atoms: in every channel every candidate law must name an
action value (an atom of that feature), unless the channel is declared Channel(..., action_free=True) (a channel whose
effect does not depend on the action). A channel without an atom of that feature is refused at construction. The
candidates dropped are counted (report()[c]["action_dropped"]); fewer candidates also make N smaller.
Dedup (dedup="group", the default): a row whose key — Row.key, default (channel, state, observed) — was already
logged in the current group is dropped (no new draw); the same row in another group counts again. dedup=False logs
every row. Rows of one transition are not independent draws even within a channel: the report prints rows per
transition per channel so a caller sees a channel that logs many rows per step.
Measured on the push-world toy (benchmarks/laws_factored_toy.py: 1.000 next-state accuracy, 0 false laws, 5 seeds), a null simulation, and MiniGrid (3 environments, 5 fresh seeds: 0.999 against 0.984 for one joint LawSet, 0.94× and 1.05× its transitions against a bar of 0.6×, false laws on states the environments never reach in both); see the tests (tests/test_laws_factored.py).
Row
dataclass
¶
One row of one channel, made by the caller's extraction. state: what the channel's atoms (and its apply)
read; observed: what was observed for it; depends: the laws of other channels whose predictions this row's
extraction or label relied on — Law objects or (channel, law id) pairs; key: the dedup key (default (channel,
state, observed)).
Extractor ¶
Bases: Protocol
The caller's credit assignment: rows(model, pre, action, post) → the rows of one transition (or a plain
function of the same arguments).
Channel ¶
Channel(name: str, grammar: Grammar, base: Callable | None = None, *, apply: Callable | None = None, effects: Callable | None = None, proposer=None, action_free: bool = False, **lawset_kwargs)
One channel: its name (the LawSet's action slot), grammar, base predictor base(state, name) and the LawSet
options (apply, effects, proposer, min_support, min_precision, null, rivals, ...). delta is set by the model.
FactoredStep
dataclass
¶
FactoredStep(steps: dict = dict(), logged: dict = dict(), dropped: dict = dict(), refused: dict = dict(), unsettled: list = list(), voided: int = 0)
What one transition (or one batch of rows) did, per channel: steps (the LawSet Step of each logged row),
logged ((row, transition id) pairs), dropped (dedup), refused (rows whose depends named a law no longer active:
not logged), unsettled (rows whose mismatch the test did not settle — what the caller may hand to admit), and
voided (rows voided by the falls this step caused).
FactoredModel ¶
FactoredModel(channels: Iterable[Channel], extract: Extractor | Callable | None = None, *, delta: float = 0.1, share: str | dict = 'equal', dedup: str | bool = 'group', action: str | None = None)
Several law sets predicting one step together (see the module docstring for the guarantee).
channels: the Channels (distinct names); extract: the caller's Extractor (optional: log(rows) takes rows
directly); delta: the joint budget; share: "equal" (δ_c = δ / C) or {channel: δ_c} (the sum is the promise);
dedup: "group" or False.
laws[c] → LawSet, buffers[c] → the held-out Buffer, local[c] → the current group's Buffer.
version
property
¶
Goes up whenever any channel's laws change (accept, retire, suspend, reactivate, probe admission): the sum
of the channels' LawSet.versions. Unchanged by predictions and dry runs; kept across pickling.
new_group ¶
Start a group (a world, a level, a session): fresh local buffers and dedup set.
applied ¶
The law of channel that speaks in state (None: the base) — what a caller names in Row.depends.
predict ¶
The channel's prediction for state (its most specific law, else its base); memoised per version
when the state is hashable.
step ¶
Extract the transition's rows with the caller's extractor (from the model as it is before this transition)
and log them.
log ¶
Learn from rows, in order: dedup → for each row its channel's LawSet.step (predicted before it is learned
from; a mismatch tested on the channel's buffer without this group) → remembered in the held-out and the local
buffer. Laws that leave the active set void the rows that depend on them.
void_dependents ¶
Void every row whose depends named this law (normally automatic when it leaves the active set) → the
rows newly voided.
retire ¶
Retire a law of channel by hand → the rows voided because they depended on it.
admit ¶
admit(channel: str, row: Row, ask: Callable[[Any], Any], probes: Iterable[Any], *, budget: int = 8, **kw) -> Admission | None
Admit a law for a mismatched row by asking the world (LawSet.admit_by_probes) with the library's guards:
the current group's logged rows refute candidates (buffer=local[channel]), siblings="undetermined" (the pool
cannot tell two conditions of one length apart → refuse), coverage="varied" over the channel's held-out buffer
(an atom that varied elsewhere but not on the pool → refuse). ask(state) → the observation, or Unknown;
probes: channel states that may be asked. One attempt per (channel, row key, group); None when already tried, when the row is no mismatch, or
when nothing is proposed. Probe-admitted laws are outside δ (counted in the report). No held-out check: rows of
other groups that contradict the admitted law do not refuse it here (that check is not shipped; ROADMAP, "Laws:
admit_by_probes(..., held_out=buffer)"); the held-out test can retire it later. kw overrides admit_by_probes
options.
stale ¶
Active laws of channel accepted on rows since voided → {law id: how many of its source rows}.
report ¶
{channel: {laws, retired, suspended, tested (N), delta, null and need (the least support a law right on every row needs at this N, against the channel's least base null), rows, transitions, rows_per_transition, voided, dropped (dedup), refused (rows naming a fallen law), probe_admitted (outside δ), stale}, "bound": Σ δ_c, "probe_admitted": the total outside the bound}.
all_laws ¶
(channel, law) for every law ever accepted, channel by channel.
to_knowledge ¶
Each channel's laws into a KnowledgeStore (LawSet.to_knowledge), the scope carrying "channel" →
{channel: {law id: item id}}.
to_module ¶
{channel: the module source of its active laws} (LawSet.to_module; check each with
check_compiled(source, model.buffers[channel], model.laws[channel])).