Skip to content

solvi.experimental.skills

Learning in a world described by code (experimental): solvi.experimental.skills.SkillLearner(env, grammar, goals) — System 2 plans on an exact model of the environment, verified plans become System 1 skills, each used alone only while its outcomes keep a certificate; as_s2(learner) for solvi.Agent; export into the knowledge store. Since 1.1.1: footprints with variables — SkillLearner(..., lift="lattice", vocabulary=voc, verify_n=16) widens each new skill's footprint (constants to variables, relations widened, atoms dropped) and keeps a wider candidate only when the plan, instantiated on verification states that match it but not the original, reaches its goal on the exact copy (Footprint, Var, Vocabulary, Lift, lattice, generalize, replay).

Learning in a world described by code (experimental): System 2 plans on an exact model of the world — a copy of the environment — and only plans the code verified are compiled into System 1 skills, each used alone only while its outcomes keep a certificate. No label from a person anywhere: the world's code is the judge.

from solvi.experimental.laws import Atom, Grammar
from solvi.experimental.skills import Certificate, SkillLearner, as_s2

g = Grammar([Atom(f"x={i}", lambda s, i=i: s["pos"][0] == i, feature="x") for i in range(9)]
            + [Atom("has_key", lambda s: s["has_key"], feature="key")])
learner = SkillLearner(env, g, {"key": lambda s: s["has_key"], "door": lambda s: s["door_open"]},
                       horizon=12, budget=20_000, certificate=Certificate(promise=0.05, delta=0.1))
for seed in range(200):                                  # a stream of fresh worlds
    ep = learner.run(seed=seed, steps=100)
    ep["s1_share"], ep["planner_calls"], ep["goals"]      # System 1 grows, System 2 is called less
learner.skills.match(state, "door")                      # the most specific certified skill, or None
learner.skills.to_knowledge(store, {"env": "grid"})      # "skill" items: plan, start, simulation hash, outcomes
solvi.Agent(env, knowledge=km, s2=as_s2(learner))        # or: planning on the exact model as the agent's System 2

The exact model. ExactModel(env, clone=None): fork() is a deep copy of the environment as it is now (clone(env, state) for environments with a snapshot / restore of their own); simulate(actions) runs a sequence on a fork. The real environment is never stepped by the planner — fingerprint() hashes it, and the learner checks it before and after every planner and verify call.

The planner. plan(model, state, goal, actions=, horizon=, budget=, cost=, beam=) → Plan(actions, predicted, cost, searched, start, end, simulation): a breadth-first search on forks (uniform cost with cost=, a beam with beam=) with a visited set on the state key; refused steps are not plan steps; deterministic; None when nothing within the horizon reaches the goal or the budget of simulated steps is used up. verify(model, state, plan) re-simulates the plan on a fresh fork: every step accepted, the goal at the end, and — from the plan's own start — every predicted state. The version space (opt-in): plan(..., collect=k, distinct=fn) → a list of up to k plans whose ends differ by fn(end state) (is there ANOTHER program that fits the demonstrations but answers differently?), model.last ["complete_depth"] the last depth searched in full; a goal may read the path (goal(state, path)); verify takes a plain action list too.

Skills. Skill(condition, actions, goal, source, support, outcomes, certificate): "when these atoms hold, for this goal, do this sequence". SkillSet(grammar, certificate): compile(goal, plan, state) (the condition rule: the atoms true at the plan's start; a second verified plan of the same goal and actions from another start keeps only the atoms both starts share, unless that would cover a state where the skill failed), match(state, goal) (the most specific skill whose condition and certificate hold), observe(skill, ok, alone=) (its outcome), retirement and re-certification, to_knowledge(store, scope) ("skill" items, source "outcome", the plan, its start key, the verifying simulation's hash and the outcomes as evidence; retired skills retired in the store; unverified skills never written).

The certificate. Certificate(promise=0.05, delta=0.1, bet=0.9, scope="skill"): an anytime-valid e-process per skill (or pooled per goal / over the set) on its outcomes — certified once its wealth reaches 1/delta_v, which takes need clean outcomes (61 at 5%, 30 at 10%, 14 at 20%) and then stays certified; an e-detector on the outcomes of alone decisions restarts it (the default detector is a CUSUM on a run of failures: at 5% the alarm first fires on the 7th consecutive alone-failure; alternatives=None is the old burst alarm, under which one failure alone does). Since 1.4 delta is spent over the restarts: the v-th certification bets against delta_v = 6·delta/(π²·v²), so every certificate a process ever issues holds together with probability ≥ 1 − delta (61 / 91 / 108 clean outcomes for v = 1 / 2 / 3 at 5%); delta_split=False is the old full delta at every restart (50, 25, 12), under which each certificate holds alone, not together. Every use of a skill has an outcome: before certification System 2 re-verifies the skill on the exact model from the current state (the outcome is the code's answer, and the step is System 2's); after it the real episode's (did the goal hold when the skill's actions ran out). So outcomes do not come only from where the system acted alone — a certificate fed only by those can certify what it never tested.

The loop. SkillLearner(env, grammar, goals, planner=plan, horizon, budget, promise, delta): per step a running skill takes its next action; else for the first open goal a certified skill runs alone (System 1); else the top matching skill is re-verified (System 2, supervised); else System 2 plans, verifies, runs and compiles; else a reproducible fallback. Records per decision (who decided: s1 / s2 / fallback, the skill, the outcome) and per episode (System 1's share, goals reached, planner calls, simulated steps, alone runs and failures). Controls by argument: learn=False (plan always), verify_plans=False with a planner of your own (compiled without the judge), certificate=False (no certificate), grammar=None (cases keyed by the start state). as_s2(learner): a SlowPath for solvi.Agent(env, knowledge=km, s2=...) that answers "action" with the next step of a verified plan (one search, then its steps while the state is the predicted one); the option must pass the agent's hard checks.

Measured: benchmarks/skills_toy.py, a deterministic gridworld (a key, a door, a gem behind it, an exit; fresh layouts from a seed, now and then a boulder): see the toy's own docstring and tests/test_skills_toy.py (pytest -m slow).

Footprints with variables (1.1.1). SkillLearner(..., lift="lattice", vocabulary=voc, verify_n=16): every newly compiled skill is generalized on the exact copy. A footprint is a conjunction of atoms (relation, *args) whose arguments may be variables (Var("D", "dir")); the world's Vocabulary says what a relation means (holds), what a variable ranges over (domain), how a term is grounded (ground: an offset turned with a direction, a resource's material), what a plan read from its start (footprint) and which lifts a plan admits (lifts: a constant replaced by a variable everywhere it appears — in the atoms, the plan and the goal — or a relation widened: a cell → «a resource within 2 steps in direction D», a station's cell → «a station nearby», an exact count → a lower bound, the resource's tool → «has a tool for R»). lattice lists the candidates above the plan's own footprint, most general first and bounded (every subset of the lifts × the old condition kept or dropped, and single atoms dropped). generalize accepts a candidate only when the plan, instantiated for every verification case that matches it but not the original — for each goal its variables can take, the footprint's first binding — reaches its goal on a fresh copy of that state: up to verify_n cases (at least verify_min) from the stream's recorded states (a reservoir of copies taken at choice points) alternating with the world's perturb edits. No model judges; a candidate fails at its first failed check, and a drop counts only when it was exercised: each dropped atom false in at least drop_witness (2) passing cases. The most general passing candidate becomes the skill's footprint (and its plan a template: «move D, do» instead of «move_left, do»); the skill keeps the original footprint, the verified count and the lattice record; replay(skill) re-runs its verification cases and compares the simulation hashes. A generalized skill matches when its goal template unifies with the goal and its footprint finds a binding; it runs the template grounded by that binding; the certificate is unchanged in its mechanics. lift=False (the default) keeps the condition rule alone, exactly as before. Tested on a small grid (tests/test_skills_footprint.py): a plan learned for one direction is accepted turned in every direction where the copy agrees, refused where the world disagrees (an interaction refused in one direction), and a dropped tool atom is refuted by a state without the tool; measured once in an open world in 1.1.1 (the experimental page).

The search cost. Each episode records searched (simulated steps of the planner's searches), verify_calls and cost = {candidates, tests, probes, simulated} — report it next to System 1's share.

What is not done: - skills are admitted by the certificate on their outcomes, not by discriminating probes (laws have LawSet.admit_by_probes); a skill set does not abstain — two certified skills of one goal are both verified plans, and abstaining would need conflicting predicted end states, which skills do not carry; - the certificate needs its clean outcomes per certified unit: a skill used once per episode needs about sixty episodes at 5% before it acts alone (more after each restart); pool by goal or over the set (then the promise is the pool's) or loosen the promise; - the condition rule only widens by shared atoms; the lattice (lift="lattice") searches a bounded set of wider footprints, most general first, judged on the copy — it is not the smallest sufficient condition, its verification cases are a sample (a candidate wrong on a rare state can pass; the certificate is what catches it), and neither specializes a skill after a failure (the failure blocks a widening and feeds the certificate); - footprints with variables need a vocabulary written for the world (relations, domains, lifts, what a plan read): the library brings the lattice and the judge, not the relations; - the planner is a blind search: no heuristic, no learned model; a world too large to search within the budget falls back to a reproducible random step; - the exact model is a copy of the environment: a world that is not code (or not copyable) needs a clone of its own, and nothing here checks that a clone is faithful beyond the plan's re-simulation and the real outcomes.

Certificate

Certificate(promise: float = 0.05, delta: float = 0.1, *, bet: float = 0.9, scope: str = 'skill', alarm: float | None = None, alternatives: tuple | None | str = 'default', delta_split: bool = True)

The promise a skill acting alone keeps, and how it is certified: at most promise of its uses fail (its goal not reached), with error probability at most delta at any time — a test-martingale bet against "the failure rate is at least the promise" (Ville's inequality; no fixed sample size, outcomes may be read after every use).

Per outcome (did the skill reach its goal: ok / failed) the e-value is e = 1 + bet · (promise − failed) / (1 − promise) (ok: 1 + bet·promise/(1−promise); failed: 1 − bet) whose mean is ≤ 1 whenever the failure probability is ≥ the promise, for any fixed bet in [0, 1). The wealth is the product of the e-values since the process (re)started; it is certified once the wealth reaches 1/delta, and stays certified ("once at the need, certified": the statement holds at any stopping time). need: the clean outcomes that take a fresh process there, ceil(log(1/delta_1) / log(1 + bet·promise/(1−promise))) — 61 at 5% and 0.1, 30 at 10%, 14 at 20% (50, 25 and 12 with delta_split=False).

delta_split (default True): delta is spent over the restarts of one process. Its v-th certification (v = 1 for a fresh process, v + 1 after each restart — an alarm, or a skill condition widened by the condition rule; a lattice generalization widens a skill without a restart, so its certificate keeps its wealth and v) bets against delta_v = 6·delta / (π²·v²), and Σ_v delta_v = delta, so every certificate the process ever issues holds together with probability ≥ 1 − delta (a union bound over the restarts; each delta_v by Ville's inequality). The price is a larger need after each restart: at 5% and 0.1, 61 / 91 / 108 clean outcomes for v = 1 / 2 / 3. delta_split=False: every certification bets against the full delta (the behaviour before 1.4) — each certificate holds alone with probability ≥ 1 − delta, not together: after k restarts the chance that some certificate issued was wrong is bounded only by k·delta. The process reports v, delta_v and the delta spent so far either way.

The alarm (default): a change detector that bets on breaks of a stated size — alternatives, a tuple of failure-rate multiples r of the promise, default (1.5, 2) (a multiple with r·promise ≥ 1 is left out; at a promise of 2/3 or more none is left and the old alarm below is used). The detector is the mean over r of the Bernoulli likelihood-ratio CUSUMs against the rate r·promise (a failure multiplies one by r, an ok by (1 − r·promise)/(1 − promise); each restarts from 1 when it falls below 1) — a standard CUSUM against stated alternatives run in parallel (Lorden 1971), averaged (a mixture e-detector). At D ≥ alarm (default 50) the process restarts — uncertified, wealth 1 — and must reach the need again. At a failure rate at or below the promise each product has mean at most 1, so P(an alarm within L outcomes) ≤ L/alarm: a bound over a horizon, empty once L passes alarm. The alarm is a change detector, not a level-delta test: a unit running at its promise is retired within a few thousand outcomes and must re-certify; a unit well inside its promise rarely alarms. The level 50 (= 5/delta at the default delta 0.1; it does not follow delta) and the alternatives were chosen on simulated streams at promises 0.05–0.3 (docs/best_practices.md).

alternatives=None: the old alarm — an e-detector with CUSUM wealth D = max(D, 1)·(1 + μ·(failed − promise)), μ = ½ / promise (an ok halves it, a failure multiplies it by 1 + (1 − promise) / (2·promise)), default level 1/delta. It bets on a failure rate near one half, so it is a burst detector whose level must grow as the promise shrinks: at 5% one failure alone is an alarm; at 30% it takes three in a row.

scope: whose outcomes one process pools — "skill" (each skill its own, the default), "goal" (the skills of one goal) or "set" (every skill): a pooled process certifies the pool's failure rate, not each skill's; each skill keeps its own process too, whose alarm retires that skill alone.

need property

need: int

Clean outcomes that certify a fresh process (v = 1).

delta_at

delta_at(v: int) -> float

The delta the v-th certification of a process bets against (v = 1, 2, …): 6·delta/(π²·v²) with delta_split, else delta.

need_at

need_at(v: int) -> int

Clean outcomes that take the v-th certification of a process from wealth 1 to certified.

process

process() -> EProcess

A fresh process under this certificate.

EProcess

EProcess(cert: Certificate)

One certificate's running state: log wealth, certified (stays so until an alarm), the alarm's detector, counts. update(ok, alone) after every outcome; remaining() the clean outcomes still needed. v: which certification of this process is running (1 for a fresh one, one more after every restart); delta_v: the delta it bets against (Certificate.delta_at(v)); delta_spent: Σ delta_u over u ≤ v — at most delta with delta_split, v·delta without it.

loss_refuted property

loss_refuted: bool

A numeric loss given to update(loss=) has mean above the promise: its e-process >= 1/delta (lifetime).

delta_v property

delta_v: float

The delta the running certification bets against.

delta_spent property

delta_spent: float

Σ delta_u over the certifications so far (u = 1 … v): the bound on P(some certificate this process issued is wrong).

update

update(ok: bool, alone: bool = True, *, loss: float | None = None) -> dict

One outcome → {"certified": certified by this outcome, "alarm": restarted by this outcome}. loss: an optional numeric outcome in [0, 1] fed to a bounded betting e-process against "the mean loss is at most the promise" (loss_refuted; a statement, it does not move certification or the alarm).

restart

restart()

Back to a fresh process: uncertified, wealth 1, the detector reset — the next certification (v + 1), which bets against delta_{v+1} (with delta_split).

remaining

remaining() -> int

Clean outcomes still needed to certify (0 when certified).

Candidate dataclass

Candidate(footprint: Footprint, actions: tuple, goal: Any, lifts: tuple = (), dropped: tuple = (), context: bool = True, generality: int = 0)

One point of the lattice: a footprint, the plan template and goal template under the same variables, the lifts applied, the atoms dropped, whether the context (the caller's other conditions) is kept, and its generality (lifts + drops; the context counts one).

Footprint dataclass

Footprint(atoms: tuple = ())

A conjunction of atoms (constants or variables in their arguments).

bindings

bindings(vocab: Vocabulary, state, binding: dict | None = None) -> Iterator[dict]

Every binding of the footprint's variables under which all its atoms hold in state (depth first: the atom with the fewest unbound variables next; values in the vocabulary's domain order).

match

match(vocab: Vocabulary, state, binding: dict | None = None) -> dict | None

The first binding under which the footprint holds in state, or None.

Generalization dataclass

Generalization(accepted: bool, footprint: Footprint, actions: tuple, goal: Any, original: Footprint, lifts: tuple = (), dropped: tuple = (), context: bool = True, verified: int = 0, candidates: list = list(), states: list = list(), checks: int = 0, short: bool = False, evidence: list = list())

What generalize found. accepted: a candidate more general than the original passed; footprint / actions / goal / lifts / dropped / context: the chosen candidate's (the original's when none passed); original: the verified plan's own footprint; verified: the verification states the chosen candidate passed (0 when none); candidates: one record per candidate tried (its footprint, the states it was checked on, where it failed and why); states: the accepted candidate's verification states (key, binding, the ground plan, the simulation's hash, pool or perturbed); checks: the plan instantiations simulated in all. evidence (not exported) keeps (state, handle, actions, goal, simulation) of the accepted candidate's states for replay.

Lift dataclass

Lift(name: str, apply: Callable[[tuple, tuple, Any], tuple])

One generalizing move: apply(atoms, actions, goal) → (atoms, actions, goal), e.g. a constant replaced by a variable everywhere it appears, or a relation replaced by a wider one.

Var dataclass

Var(name: str, kind: str = '')

A variable of a footprint, a plan template or a goal template. kind: what it ranges over (the vocabulary's domain reads it).

Vocabulary

The world's relations — subclass it per world.

holds(atom, state): does a ground atom hold in state (required). domain(var, state, binding): the values a variable may take in state given the binding so far (finite, in a fixed order: the first binding found is the one used, at verification and at use alike). ground(term, binding): a term with its variables replaced (default structural; override for terms whose grounding is computed, e.g. an offset turned with a direction). goal(name) / goal_name(term): a goal's name as a term and back (default: the name itself); action(a) / act(term): an action as a term and back. lifts(atoms, actions, goal): the generalizing moves a verified plan admits, in the order they are applied (default: none — only drops). footprint(state, actions, predicted): the ground atoms a plan read from its start (what SkillLearner(lift="lattice") starts the lattice from).

ExactPlanPath

ExactPlanPath(learner: SkillLearner, *, decode=None, max_options: int = 64)

Bases: SlowPath

System 2 for solvi.Agent that plans on the exact model of a SkillLearner (as_s2(learner)): it answers "action" with the index of the option the verified plan's next action is, from the step's facts — at (the state key, decoded by decode, default JSON) and goals_open (the agent's open goals, matched by name). The option must pass the agent's hard checks (no blocks, not avoided); otherwise it does not answer. The record holds the goal, the plan, the step, the state key, whether the plan was cached and the simulated steps it took.

SkillLearner

SkillLearner(env, grammar, goals, *, planner=plan, horizon: int = 12, budget: int = 20000, promise: float = 0.05, delta: float = 0.1, certificate=None, actions=None, cost=None, key=None, clone=None, verify_plans: bool = True, learn: bool = True, generalize: bool = True, guard: bool = True, seed: int = 0, lift: bool | str = False, vocabulary: Vocabulary | None = None, verify_n: int = 16, verify_min: int | None = None, perturb=None, pool_size: int = 64, max_candidates: int = 12, drop_witness: int = 2)

The loop of learning in a world described by code (see the package docstring).

env: an Environment whose copy is an exact model (deepcopy, or clone=). grammar: a laws Grammar for skill conditions (None: cases keyed by the start state). goals: {name: done(state)} or [(name, done)] in order — the first goal not yet reached in the episode is worked on (the order is the prerequisites). planner: plan (or a function with its signature); horizon, budget, actions, cost: passed to it. promise, delta: the default Certificate(promise, delta); certificate: a Certificate of your own, or False (no certificate: skills act alone at once — a control). verify_plans: verify each plan on a fresh fork before it is executed or compiled (False: a control without the judge). learn: compile skills (False: System 2 plans every goal — "plan always"). generalize: the condition rule's merge. guard: check after every planner and verify call that the real environment's hash did not change (a RuntimeError if it did). seed: the fallback's tie-breaks.

Footprints with variables. lift="lattice" (default False: the condition rule only, as before) with vocabulary= (a Vocabulary: the world's relations and what a plan read): every newly compiled skill is generalized on the exact copy — the lattice above its plan's footprint (the vocabulary's lifts: constants to variables, a relation widened; atoms dropped; its old grammar condition kept or dropped as a block), most general first, at most max_candidates; a candidate is accepted only when the plan, instantiated for every verification state that matches the candidate but not the original, reaches its goal on a fresh copy of that state — up to verify_n states (at least verify_min, default verify_n // 2), alternating the stream's recorded states (a reservoir of pool_size (state, copy of the environment) pairs taken at choice points) and perturb(copy, state, rng, candidate) → [(state, copy)] (the world's edits; None: recorded states only); a dropped atom (or the dropped old condition) must be false in at least drop_witness (default 2) of the passing cases. The most general passing candidate becomes the skill's footprint; the skill records the original footprint and the verification count; replay(skill) re-runs its verification states. A skill whose lattice found no candidate refuted but too few verification states or witnesses is tried again (from the new plan's start) when a later verified plan merges into it, at most three runs. The certificate is unchanged. Per episode: generalized, generalize_accepted, generalize_checks.

Failed plan checks are retained in plan_rejections (also in report). Three consecutive same-start prediction mismatches warn once per learner about possible clone/RNG/hidden-state or prediction issues.

Per step (act(state) → an action; the caller steps the environment and calls observe(outcome); run(seed, steps) does both): a skill run in progress takes its next action; else, for the first open goal, System 1 runs the most specific matching skill whose certificate holds, alone; else the most specific matching skill is re-verified by System 2 on the exact model from here — its outcome feeds its certificate, and if it verifies it runs as System 2's; else System 2 plans on the exact model, verifies the plan, runs it and compiles it into a skill; when no plan is found, a reproducible pseudo-random offered action ("fallback").

Records. decisions: one per step — episode, step, by ("s1" / "s2" / "fallback"), kind ("alone" / "supervised" / "planned" / "fallback"), skill id, goal, action, certified (System 1 acted under a holding certificate), accepted, run_ok (did the run this step belonged to reach its goal; None while it is open — a run dropped by new_episode() never gets one; close_run(ok) closes an interrupted run with an outcome). episodes: one per episode — steps, decisions by System 1 / System 2 / fallback, s1_share, goals (reached, in order), goals_reached, all_goals, planner_calls, plans_found / plans_rejected, simulated (steps on the exact model), compiled, supervised (failed), alone_runs, alone_failures, s1_failed (System 1 decisions in failed runs), mismatch (a verified plan the real world did not follow), guard_violations, skills / usable / retired at the end; searched (simulated steps the planner's searches took), verify_calls, and cost — the search-cost record next to the outcome: {"candidates": searched (the states the planner expanded), "tests": verify_calls (re-simulations of a plan or skill), "probes": 0 (the real environment is never asked to tell hypotheses apart here), "simulated": every step on the exact model}.

new_episode

new_episode(seed=None)

Start an episode by hand (run() does it): no goal reached yet. A run still in progress is dropped without an outcome — close it first with close_run(ok) if the world showed one.

run

run(seed=None, steps: int = 200) -> dict

One episode: reset the environment with seed, act and observe for at most steps steps (or until the environment says done, or every goal is reached) → the episode's record.

act

act(state)

Decide the next action in state → an action, or None (every goal reached, or nothing offered). The real environment is not stepped here: call observe(outcome) after stepping it.

replay

replay(skill: Skill) -> dict

Re-run a generalized skill's verification states on fresh copies of their recorded worlds → {"states", "same", "differ"} (a state differs when its check fails now or its simulation hash changed).

observe

observe(outcome)

The environment's Outcome of the last action (or True / False): the run goes on, ends (its goal reached, an action refused, its actions used up) and its outcome feeds the skill's certificate when it ran alone.

close_run

close_run(ok: bool)

Close the run in progress with an outcome. A run interrupted mid-way — paused by a reflex and never resumed, or dropped by new_episode() — otherwise carries none: its decisions keep run_ok None and an alone run's skill never hears the world's verdict. ok: did the run reach its goal as far as the world showed; a run paused and not resumed is the caller's call (the Crafter run of docs/best_practices.md counted its 57 such runs as failures). → the run's record, or None when no run is open.

propose

propose(state, prefer=())

One action for state from System 2 on the exact model, without an episode of its own (what as_s2 asks): the goal is the first of prefer (the agent's open goals) this learner knows and that does not hold, else its first goal that does not hold. A plan is searched, verified and compiled once, then followed step by step while the state is the one it predicted. → (goal, action or None, a record).

report

report() -> dict

Per episode and in total: the records above, System 1's share, and the skill set's report.

Check dataclass

Check(ok: bool, why: str, end: Any = None, simulation: str = '', steps: int = 0)

What verify found: ok, why, the end state of the re-simulation, its hash and the steps it ran. Truthy when ok.

ExactModel

ExactModel(env, clone: Callable[[Any, Any], Any] | None = None, key: Callable[[Any], str] | None = None)

A world model that is the environment's own code. fork(state) → an independent copy to step: by default copy.deepcopy of the environment as it is now (the state argument is then only a label); clone(env, state) → an independent environment at state, for environments with their own snapshot / restore (it is also used to copy a fork during a search). A clone must be a faithful copy of the world as it is: the same future from the same state, its randomness carried with it (an RNG, a clock, a tick — anything the state alone does not hold). A clone with fresh randomness of its own gives every fork a different future from the fork that planned, and verify then rejects every plan of two steps or more (each step is compared with the predicted state) — verify is right, the model is not exact; check a clone of yours with solvi.testing.conformance.check_environment(make, clone=). The real environment is only read (copied), never stepped: fingerprint() hashes it, so a caller can check that a plan call left it as it was.

Counters: forks, steps (simulated steps, every search and every verify); last: what the last search did.

Without clone, the first fork checks what the deep copy carries and warns once when it holds a torch module (a trained model copied at every fork: a planner that seems to hang) or pickles to more than FORK_WARN_BYTES — give clone= a snapshot / restore of the world's state alone (check it with solvi.testing.conformance.check_environment(make, clone=)).

fork

fork(state=None)

An independent copy of the environment (at state when a clone hook is given).

copy

copy(world, state=None)

An independent copy of world (the real environment or a fork).

step

step(world, action)

One simulated step on a fork (counted).

simulate

simulate(actions, state=None) -> list

Run actions on a fresh fork → their Outcomes (it stops after a refused step or the end of the episode).

fingerprint

fingerprint() -> str

A hash of the real environment's contents (pickle; its repr when it cannot be pickled).

Plan dataclass

Plan(actions: tuple, predicted: tuple = (), cost: float = 0.0, searched: int = 0, start: str = '', end: str = '', simulation: str = '', goal: Any = None)

A plan found on the exact model. actions: the sequence; predicted: the states the code gave after each action; cost: its cost (the number of actions without cost=); searched: the simulated steps the search took; start / end: the keys of the start and end states; simulation: the hash of the simulation that found it; goal: the goal check it was found for (not part of its equality).

Skill dataclass

Skill(condition: tuple, actions: tuple, goal: str, source: str = 'verified', support: int = 1, outcomes: dict = dict(), certificate: EProcess | None = None, id: str = '', status: str = 'active', why: str | None = None, start: str = '', simulation: str = '', starts: list = list(), counterexamples: list = list(), footprint: Footprint | None = None, template: tuple | None = None, goal_template: object = None, original: list = list(), verified: int = 0, generalization: dict | None = None)

System 1's rule "in this situation, for this goal, do this sequence". condition: a conjunction of atom names (all must hold; without a grammar, the start state's key: a case); actions: the sequence; goal: the goal's name; source: "verified" (a plan the code verified) or "unverified" (a control); support: the verified plans it was compiled from; outcomes: counts of its uses — ok / fail (every use), alone_ok / alone_fail (System 1 alone), supervised_ok / supervised_fail (re-verified on the exact model before a use), source_ok / source_fail (the run of the plan it was compiled from), mismatch (a supervised or verified planned run of it the real world did not follow — the certificate never saw it); certificate: its own EProcess (None without a certificate). status: "active" or "retired" (its alarm fired: not used alone until its own process certifies again). start / simulation: the first plan's start key and simulation hash; starts: the start keys of every plan merged into it; counterexamples: the atoms of states where it failed (a generalization never covers one).

A skill generalized on the exact copy (SkillLearner(lift="lattice")) also carries: footprint — the accepted footprint (a solvi.experimental.skills Footprint; its atoms may hold variables) that must hold, besides condition, for it to match; template / goal_template — the plan and goal under the same variables (the actions it runs are the template grounded by the first binding the footprint finds in the state); original — the verified plan's own ground footprint (and its old condition); verified — the verification states the accepted footprint passed; generalization — the lattice record (every candidate tried, the verification states, the checks). A skill whose lattice found nothing keeps footprint None and the record.

variables property

variables: tuple

The variables of its footprint (empty for a skill with constants only).

SkillSet

SkillSet(grammar=None, certificate: Certificate | None | bool = True, *, generalize: bool = True, key=None, vocabulary: Vocabulary | None = None)

The skills System 1 may run, their certificates and their export into the knowledge store.

grammar: a solvi.experimental.laws Grammar (its atoms are read with action None; the conjunction size k does not bound a skill's condition) or None (a skill is a case: its start state's key). certificate: a Certificate (default Certificate()), or None — every skill acts alone at once (a control, not a guarantee). generalize: merge a new verified plan into an existing skill of the same goal and actions (below); key: the state key (canonical JSON).

The condition rule (conservative): a new skill's condition is every atom of the grammar true at its plan's start. A later verified plan of the same goal with the same actions from another start widens it to the atoms both starts share (anti-unification over atoms: an atom that differed between two verified starts was shown not to be needed for this sequence) — unless the widened condition would hold on a state where the skill is recorded to have failed; then the plan becomes a skill of its own. A widening that leaves no atom at all (the starts shared nothing) makes the skill match every state of its goal and warns. Widening by this rule restarts the skill's own certificate (its evidence was gathered on a narrower condition). The learner's lattice generalization (a footprint and template; the condition becomes () when no context is needed) also widens a skill, without a restart: its certificate keeps its wealth and its v.

No abstention between skills: two certified skills of one goal are both verified plans, and a different action sequence is not a disagreement about the world (laws abstain where they predict different outcomes: LawSet.predict(..., disagree="abstain")). A skill set could abstain only on conflicting predicted end states for the same goal, which skills do not carry; match stays most specific first.

vocabulary: the world's relations (a Vocabulary) for skills with a footprint (generalized on the exact copy): such a skill matches when its condition holds and its footprint finds a binding in the state (its goal template unified with the goal first); it runs its template grounded by that binding (actions_for). It is never widened by the condition rule; a verified plan it already covers (same goal, same ground actions) adds support only.

atoms

atoms(state) -> frozenset

The atoms true in state (without a grammar: its key, as one atom).

binding

binding(skill: Skill, state, goal) -> dict | None

The binding under which a skill with a footprint applies to goal in state ({} for a skill with constants only when its goal is goal), or None. Its condition is not read here.

actions_for

actions_for(skill: Skill, state, goal) -> tuple | None

The actions a skill runs for goal in state: its sequence, or its template grounded by the first binding of its footprint (None when it does not apply).

candidates

candidates(state, goal, *, atoms=None) -> list[Skill]

The skills of goal whose condition holds in state (active and retired), most specific first (more atoms, a footprint's atoms counted; ties: the earlier skill). A skill with a footprint must also find a binding.

pool

pool(skill) -> EProcess | None

The process whose certification a skill acts on (its own under scope "skill").

usable

usable(skill) -> bool

May System 1 run it alone: active, and its certificate (its own, or its pool's) holds — always, without a certificate.

match

match(state, goal, *, atoms=None) -> Skill | None

The most specific skill of goal whose condition holds in state and whose certificate holds, or None.

add

add(skill: Skill) -> Skill

Add a skill built by hand (it gets an id and a certificate).

compile

compile(goal: str, plan: Plan, state, *, verified: bool = True) -> Skill

A plan found (and, with verified=True, verified) from state → the skill it becomes: an existing one widened by the condition rule, or a new one with the atoms true at state.

observe

observe(skill: Skill, ok: bool, *, alone: bool = True, state=None, kind: str | None = None) -> dict

One use of skill: did it reach its goal (from the real episode when it ran alone; from the exact model when System 2 re-verified it first: alone=False). Updates its outcomes and its certificate(s); a failure from state records that state's atoms as a counterexample. A skill whose own alarm fires is retired; a retired skill whose own process certifies again is back ("recertified"). → {"certified", "alarm", "retired", "recertified", "pool_alarm"}.

to_knowledge

to_knowledge(store, scope: dict | None = None) -> dict[str, str]

Write the verified skills into a KnowledgeStore: each a "skill" item with body {"kind": "plan", "goal", "if", "do"}, source "outcome" (the code world answered), by "skills", evidence = the plan, its start key, the verifying simulation's hash, the starts merged, the outcomes and the certificate's state. A retired skill is retire()d with its reason (and asserted again when it is re-certified); a widened skill is a new item and the narrower version is retired. Unverified skills are never written. Idempotent. → skill id → item id.

generalize

generalize(atoms, actions, goal, vocab: Vocabulary, *, pool: Sequence = (), check: Callable, original: Callable[[Any, Any], bool], context_holds: Callable | None = None, perturb: Callable | None = None, start: tuple | None = None, goals: Callable[[Any], bool] | None = None, verify_n: int = 16, verify_min: int | None = None, max_candidates: int = 12, drop_atoms: bool = True, drop_witness: int = 2, seed: int | str = 0, key: Callable[[Any], str] | None = None) -> Generalization

Generalize a verified plan's footprint on the exact copy (the code of the world is the judge; no model).

atoms / actions / goal: the plan's ground footprint (what it read at its start), the plan and its goal name. pool: the recorded (state, handle) pairs of the stream — a handle is whatever check needs to rebuild the world at that state (a copy of the environment, a snapshot). check(handle, state, actions, goal) → (ok, simulation hash): run the ground plan on a fresh copy of the world at that state; ok when every step is accepted and the goal is reached. original(state, goal): the original skill applies there (its goal, its own footprint and old condition) — such a case proves nothing new and is never a verification state. context_holds(state, actions): the caller's other condition (kept by candidates with context=True; None: no context). perturb(handle, state, rng, candidate) → [(state, handle)]: states near a pool state or the plan's start (start = (state, handle)), edited by the world's code (None: the pool only). goals(name): the ground goals that can be checked (None: any).

A verification state is a (state, goal) case: for each value of the candidate's goal variables (a use is asked for a goal), the footprint's first binding; the plan and goal instantiated under it. A candidate is checked on up to verify_n cases that match it and not the original, alternating pool and perturbed states; it fails at its first failed check, and passes when every check passed, at least verify_min (default verify_n // 2) cases were found, and every drop was exercised: each dropped atom was false (grounded under the case's binding) in at least drop_witness passing cases, and a dropped context did not hold in at least as many — a case that differs from the original elsewhere says nothing about an atom it never falsified. short records that a candidate was not refuted but lacked cases or witnesses: worth another try on a larger pool. The most general passing candidate (the lattice's order) is returned; none → the original (accepted False). Deterministic given the seed, the pool order and the world.

lattice

lattice(atoms, actions, goal, vocab: Vocabulary, *, context: bool = False, drop_atoms: bool = True, max_candidates: int = 12) -> list[Candidate]

The bounded lattice above a verified plan's footprint, most general first (the original itself is not in it).

atoms: the ground atoms the plan read; actions / goal: the plan and its goal as they ran. context: the caller has other conditions (the old condition of a skill) a candidate may keep or drop as a block. Points: every subset of the vocabulary's lifts (applied in its order) × the context kept or dropped, and — when drop_atoms — each single atom dropped from the candidate with every lift applied and the context dropped. Ordered by generality (lifts + atoms dropped + 1 for a dropped context), then more lifts first, then the vocabulary's order; duplicates removed; the first max_candidates kept.

replay

replay(gen: Generalization, check: Callable) -> dict

Re-run the accepted candidate's verification states on fresh copies → {"states", "same", "differ"}: a state differs when its check fails now or its simulation hash is not the recorded one.

unify

unify(template, value, binding: dict | None = None) -> dict | None

Extend binding so that template grounds to value (structural: a variable takes the value at its place, a bound one must equal it; tuples element by element) → the binding, or None.

as_s2

as_s2(learner: SkillLearner, *, decode=None, max_options: int = 64) -> ExactPlanPath

The learner as System 2 of solvi.Agent(env, knowledge=km, s2=as_s2(learner)): planning on the exact model when System 1 abstains (the learner's env must be the agent's env; the agent's state keys must decode to states — its default key is the state's canonical JSON).

plan

plan(model: ExactModel, state, goal: Callable[..., bool], *, actions=None, horizon: int = 10, budget: int = 10000, cost: Callable[[Any, Any], float] | None = None, beam: int | None = None, collect: int | None = None, distinct: Callable[[Any], Any] | None = None)

Search the exact model from state for a sequence of actions after which goal(state) holds → a Plan, or None.

Every candidate step is taken on a copy of a fork — the real environment is never stepped. A refused step is not a step of a plan; a state already reached (by its key) is not searched again; a state where the episode ends and the goal does not hold is a dead end. Search: breadth-first (the fewest actions; ties in the order the environment offers its actions) by default; with cost(state, action) uniform-cost (the cheapest within horizon); with beam=b breadth-first keeping the b cheapest nodes per depth (an approximation). actions: None (the environment's actions(state)), a list, or a function of the state. budget bounds the simulated steps; when it is used up, or nothing within horizon reaches the goal, the result is None and model.last["why"] says which ("budget", "horizon", "exhausted"). Deterministic given the model.

goal: a function of the state, or of (state, path) — two positional parameters: the actions that led to it too. collect=k (opt-in): the version space — the search goes on past the first plan and returns a list of up to k plans whose ends differ by distinct(end state) (default: the end state's key), the first found for each (in search order: fewest actions, or cheapest with cost=); fewer than k (possibly none) when the search ends first, with model.last["why"] "found" (k collected), "budget", "horizon" or "exhausted". model.last["complete_depth"] (breadth-first without a beam): the last depth searched in full — "no other plan up to depth d" rather than "the budget ran out".

simulation_hash

simulation_hash(start: str, trace) -> str

The hash of one simulation: the start key and, per step, (action, accepted, the key reached).

state_key

state_key(state) -> str

The default key of a state: its canonical JSON (as solvi.Agent's).

verify

verify(model: ExactModel, state, plan, goal: Callable[..., bool] | None = None) -> Check

Re-simulate plan on a fresh fork from state and check it → a Check. Every step must be accepted and the episode must not end before the last one; the goal (goal, else the plan's own) must hold at the end; and when the plan was found from this very state (its start key), every state must be the one it predicted (so its simulation hash is the same). A plan counts as verified only by this re-simulation on the code world — never by what a model says about it. plan: a Plan, or a plain sequence of actions (nothing predicted: the steps and the goal are checked). goal: of the state, or of (state, path) as for plan.