solvi.core.knowledge.store¶
The knowledge store: a hash-chained journal of facts, rules, skills, actions and episodes, with provenance, versions, the source check, disputes to a person, the retraction cascade, staleness flags and snapshots for decisions.
The knowledge store: what a system has learned, from whom, and what rests on it — a hash-chained journal of items.
from solvi.core.knowledge import KnowledgeStore
ks = KnowledgeStore("knowledge.jsonl") # any TraceStorage backend (a path, a TraceStorage) or None
a = ks.add("fact", {"s": "abt:123", "r": "same_as", "o": "buy:456"}, source="person", by="ann")
b = ks.add("fact", {"s": "abt:123", "r": "brand", "o": "acme"}, source="outcome", derived_from=[a])
ks.retract(a, by="bob", why="wrong label") # b goes with it; what a refuted or disputed comes back
ks.verify(); ks.fingerprint(); ks.rebuild(skip={a}).fingerprint() # equal: a retraction is exact
ks.snapshot(about="abt:123") # what a decision is given: facts, hints, the fingerprint
An item is one of five kinds — fact (subject, relation, object), rule, skill, action (an action model's preconditions
and effects), episode (one finished case) — with a source: "person", "outcome" (an observation: what really happened),
"spec" (a written specification) or "verified" (a System 2 answer given alone under its guarantee, named by its stored
decision of=). The system's own unverified answers are never admitted: the source check is part of the store, not a
setting (SourceGate; solvi.core.sources.check_source).
State is a function of the live journal entries (JTMS-style; valid= is valid time and the journal position is
transaction time, as in a bitemporal store): an item is present while it has a live assertion whose
premises (derived_from) are present and not refuted. Per key (scope, subject, relation) the live assertions are walked
in time order: the same value confirms; an outcome refutes the current value (the world changed) — unless the outcome
assertion is inferred (a consequence computed from premises, not an observation: it competes by rank, so two inferences
of different values from equal-rank premises are a dispute); a higher-ranked source
(outcome > person > spec > verified) refutes a lower one; supersedes (a producer's own new version) refutes the version
it names; any other contradiction makes every side "disputed" and asks a person — one question per dispute EVENT (the
journal position it opened at): the same two items disputed again after a resolution are asked again. resolve(key,
value, by=) is the person's answer. A disputed item is never a fact.
Validity. valid=(since, until) is when a value holds in the world (half-open, None unbounded — a missing since is
the beginning of time, and an assertion without valid on a key where others carry one is the beginning-of-time
version, open-ended; ints, ISO date strings or datetime.date, which is kept as its ISO text). On a key where some
assertion carries one, values compete only when their intervals overlap (a dose label for 2023 and another from
mid-2024 are both facts, for their times). Versions of different sinces are versions in time, not a contradiction:
the later since closes the earlier version at that since and the earlier stays active (its effective interval ends
there; a bounded later version gives the earlier value back after its until; a retracted later version re-opens what
it closed), whatever order they were written in. Validity is per assertion: a value told for two periods (12, then 24,
then 12 again) holds in both under one id; valid(id) is the latest assertion's interval, periods(id) the effective
intervals and item(id)["valid"] lists every claim. A version is one item's claims of one since: when they give
different untils, the latest claim's until is the version's (a correction, as one validity per item was), so (45, 55)
then (45, None) reads open from 45 and the reverse order reads bounded at 55. The same since from two writers (two missing sinces too) is not an
observation of a change — rank decides, equal ranks dispute; that holds while any other writer still asserts the old
value, even when the new value's writer had asserted it too. A writer's later value at one since replaces its own
earlier one there (it no longer asserts the old value) — blindly: after an erratum withdraws one of its claims, the
writer's next value at that since replaces every value it had there, including ones the erratum did not correct; two writers who both move from one value to the same other
value leave that value holding. The winner takes only the overlap: a loser whose interval
reaches beyond it is closed there and holds outside (refuted only when the winner covers it whole), and a dispute is
contested only on the overlap — outside it each side holds; a person's answer (resolve) closes the loser the same
way. An item pending re-test takes no side on its key (no dispute, no refutation, no closing). usable(id, valid_at=)
and snapshot(valid_at=) read the effective intervals (a snapshot row carries "periods" when they are not the item's
own valid; Knowledge.value(valid_at=) reads them). An item derived from a premise disputed only on an overlap is a
hypothesis as a whole, but at a time where every premise of one of its derivations holds it is usable and a fact. Keys whose assertions carry no valid resolve as described
above.
A circular derivation (a ← p, b ← a, then a ← b) is accepted (with a warning at the write), but a chain back
through the item itself is no support: retracting the loop's outside premise taints every item in it, so such
content cannot keep itself alive past a retraction.
Statuses: "active" (a fact System 1 may answer on), "hypothesis" (a hint: a behaviour-changing item not yet promoted by the write gates, a derived item on a premise that is not active, an item under a pending flag, or one pending a re-test), "disputed", "refuted", "expired" (its reconfirm_after ran out on the store's clock, or retired) and "retracted".
Re-test. Content written again after it was retracted — the same item id, since an id is its content — is not revived
by the new assertion: it is a hypothesis "pending re-test" until retest(id, ok=True) (a person or a test confirms
it; ok=False retracts it again), and so is every item that rested on it when it was retracted, each until its own
re-test. pending() lists the items waiting, item(id)["pending"] says why.
The guard on content told with valid. Every retraction (plain, only_from, assertion=, retract_source, a failed
re-test, a rollback) withdraws its assertions' content (the key's value) on their validity. A later assertion of the
same content on an overlapping validity, whatever its item id, is a copy: it waits for a re-test (the item is pending;
when the item has other claims they still hold, and a failed re-test takes back only the copies) — unless writers it
was not withdrawn from still hold that content wherever the copy could hold; such a copy is admitted and marked
"unchecked" in item(id)["valid"]["claims"], and it goes pending once those holders are cut. A passed re-test vouches
for the item's assertions before it and clears the withdrawals of its content. Derived and inferred assertions follow
their premises instead. Items derived after the re-test are fresh; an item a rule only re-infers (inferred=True) follows its
premises, and so does an inferred item that rested on one: once the premise passes its re-test it is a fact again
without a re-test of its own. pending_retest(id) reads it.
Derived validity. An item with derived_from and no valid of its own gets the intersection of its premises'
validity (each premise's span over its live claims); when they never hold together, or their times do not compare
(an int and an ISO date), it is refused with the reason in the journal. It holds at a time (usable(id, valid_at=),
snapshot(valid_at=), periods(id)) only where, for one of its derivations, every premise holds — a premise closed by
a later version no longer carries it; an item with its own valid over premises whose times do not compare reads its
own valid only.
Staleness. flag(scope=..., why=...) (a drift or open-set flag) or reconfirm(id) marks items for re-confirmation: an
item learned before the flag is a hint, not a fact, until a newer observation or person confirms it or the flag ends
(end_flag). A flag is read where staleness is asked (stale(id)), not inferred from the item's status. rollback(id)
(supervision undoing an update) restores the previous version as a hint, never as a fact, while a flag covering the
rolled-back version or its scope is pending (a rollback during a pending drift flag once restored a stale threshold as
a fact on a shifted stream, and a stale check that read the item status missed it). reconfirm_after per
(kind, source) (RECONFIRM_DEFAULTS, the store's clock advanced by tick) expires items that weaker evidence keeps. A tick
re-resolves only the items whose expiry can flip between the old and the new clock (an index of their deadlines).
Every write — add, promote, refuse, retract, flag, resolve, tick, use — is a journal entry: in a TraceStorage (records of
kind "knowledge", in the same hash chain as the decisions when it is the decisions' store) or in memory with its own hash
chain. verify() checks the chain and that the state is what the journal gives; rebuild(upto=n, skip=ids) gives the
store as it was at entry n, or as if ids had never been written; a store opened on a backend is replayed from its
journal (an entry edited by hand breaks the chain). snapshot(...) returns what a decision reads, with the store's
fingerprint and journal position, so every decision that is given it records the knowledge it used and replays;
redecide(storage, retracted, decide) re-runs the decisions that rested on retracted items on their re-built snapshots
and splits them into "answer changes" (to a reviewer) and "justification only" (recorded).
Measured by benchmarks/knowledge/retraction.py: 1,000 of 1,000 random retractions exact on a 10,000-item synthetic store (the store after a retraction has the fingerprint of the store rebuilt without it). Cost grows with the connected component a write touches (premise edges and shared keys); the store has been measured to 10,000 items, not 10⁵–10⁶.
Verdict
dataclass
¶
A write gate's answer about one proposed item: admitted or not, why, and what it measured (recorded).
SelfDefeating ¶
Bases: ValueError
A derived item whose key is a key of one of its own premises with another value: it would refute its support.
KnowledgeStore ¶
KnowledgeStore(storage=None, *, gates=None, defaults=None, writers=None, reassert_writers=(), now=None, revival_window=None, _replay=True)
See the module docstring.
storage: where the journal is kept — None (in memory), a path (solvi.core.store.open_storage: .jsonl, .db, .duckdb,
postgresql://) or a TraceStorage (the decisions' store itself, so knowledge and decisions share one hash chain); an
existing journal there is replayed. gates: the write gates (solvi.core.knowledge.gates; default SourceGate and
ConsistencyGate); the source check runs first whatever is given. defaults: reconfirm_after per (kind, source).
writers: {writer: source (or sources), ...} — opt-in: a source (a rank) is then taken only from a writer (by=)
registered with it; any other item is refused with the reason (gates.WriterGate, run right after the source
check). None (default): any writer may declare any source, as before. now: a world clock, opt-in — a callable
returning the current world time (no arguments), or a fixed time; with it usable(id) and usable on derived
items read the effective intervals at that time instead of status alone, so a version closed by a later one
(or an item derived from it) is not usable with no valid_at given. None (default): as before — usable(id)
with no valid_at reads status and staleness only. revival_window: optional inclusive protection
after withdrawal, in logical tick units. New direct dated claims after the window may vouch
anew; old held claims never become clean merely because time passes. None preserves the
original overlap-based guard. Reopen persistent stores with the same explicit policy.
revival_window
property
¶
Optional inclusive withdrawal window in owner-declared logical clock units.
verify ¶
Is the journal's hash chain whole (in memory: its own chain; on a backend: the backend's verify()), and is the state the one the journal gives (the store rebuilt from it has the same fingerprint)?
rebuild ¶
The store the journal gives, in memory: entries applied in order (upto: the first upto — the store as it
was then); skip: item ids whose own entries (assertions, promotions, retractions) are left out — the store as if
they had never been written. Premises naming a skipped item stay in the entries derived from it (dead).
The result is independently writable, with its own hash chain, copied journal data, write gates, defaults,
world clock and reassert capabilities. Logical entry positions are retained for assertion proofs. Uncopyable
policy objects fail before replay. Functions used as callbacks retain their normal Python closure semantics;
external state accessed by a callback is not snapshotted. Stored verified decisions are not copied into this
in-memory reconstruction: new verified writes still require a decision in the destination TraceStorage.
branch ¶
Independent writable what-if over a journal prefix; optional empty destination.
No merge or decision-proof export. Persistent branches refuse inherited verified/of assertions. Reopening uses the normal constructor with the same explicit policy. A lineage note binds the original parent head and selected prefix hash.
proposal ¶
proposal(kind, body, scope=None, *, source, by=None, evidence=None, derived_from=(), supersedes=None, reconfirm_after=None, valid=None, level=None, of=None, inferred=False, consulted=())
A proposed item as the write gates see it (a dict with the record's fields; nothing is written).
add ¶
add(kind, body, scope=None, *, source, by=None, evidence=None, derived_from=(), supersedes=None, reconfirm_after=None, valid=None, level=None, of=None, shadow=None, inferred=False, consulted=(), reassert=False)
Propose an item → its id, or None when a gate refused it (a "refused" entry in the journal says why).
kind: one of KINDS; body: plain JSON (a fact: {"s", "r", "o"}); scope: a dict (a question, an environment);
source: "person" | "outcome" | "spec" | "verified" (anything else is refused: never the system's own answers);
by: who; evidence: stored ids, quotes, steps; derived_from: premise item ids (the retraction cascade follows them);
supersedes: the id of the producer's previous version; reconfirm_after: clock units until it needs re-confirmation
(default RECONFIRM_DEFAULTS); valid: (since, until) in the world, as you count time; level: a guarantee level
(for "verified"); of: the stored decision a "verified" item is (checked in the store's TraceStorage); shadow:
passed to the write gates (a shadow measurement for a behaviour-changing item); inferred: the assertion is a
consequence computed from derived_from (a rule applied to facts), not an observation — an inferred outcome
does not refute the current value as an observation does: it competes by rank, and equal ranks are a dispute.
consulted: item ids the conclusion looked at besides its premises — the evidence for the alternatives it did not
choose (opt-in). Retracting one does not retract this item (its premises stand) but marks it for review: a
hypothesis pending re-test (retest(id, ok=True)), as an item resting on a retracted premise; item(id)
["consulted"] lists them. Such a store differs from its rebuild without the retracted item until the re-test.
A fact or an episode the gates admit is present at once; a rule, skill or action the gates admit is promoted (its verdicts recorded); one a later gate holds stays a hypothesis (a "held" entry says why).
promote ¶
A behaviour-changing item passed its gates (or a person promotes it): it becomes a fact (measurements kept).
retract ¶
Take an item back, with everything derived from it (the cascade over derived_from); what it refuted or
disputed comes back. Nothing is deleted: the journal keeps the entry and the chain stays whole.
only_from: a writer (the by of its assertions) — take back only that writer's assertions of the item: the same
content told by others stays (one item for both, since an id is its content); the item goes only when no other
assertion is left. only_from=(writer, evidence): only that writer's assertions given with this evidence (one of
two publications of the same content). assertion: the journal position of one standing assertion of the item
(item(id)["valid"]["claims"] lists them as "at"; a list takes several) — only it is taken back (a journal
entry of its own, "retract_assertions"; a duplicate taken back while its writer's other claim of the content
stands is not a withdrawal of that content). → {id: (status before, status after)} of every item whose status
changed.
retract_source ¶
Take back every standing assertion from writer (an exposed source): items only it told go with their
derivations; items others told too stay on their support. → {id: (status before, status after)}.
forget ¶
Forget a source in one call: what writer told is taken back, the stored decisions that rested on it are
re-decided on the journal as it is now, and one "forget" journal entry records what was taken back and which
decisions changed. The three steps are the public calls
changed = store.retract_source(writer, by=by, why=why) # its items, and what was derived from them
gone = [i for i, (_, now) in changed.items() if now != "active"]
answer_changes, same = store.redecide(storage, gone, decide, fact=fact, upto="now")
then the entry. storage: the decisions' TraceStorage (None: only the retraction and the entry); decide: what
re-decides an input (a System, or init_state → {question: answer}; needed with storage); fact: the given fact
the snapshots were passed as. storage may be a path (opened as System(storage=) opens one, before anything is
written). Nothing is erased: the journal keeps the source's entries (retracted) and the
decision records keep their inputs — erasing personal data in a record is TraceStorage.redact(id), kept
explicit (where_is finds the records holding a value). → {"source", "retracted" ({id: (status before, status
after)}), "decisions" (the ids of the stored decisions whose snapshot changed), "changed" / "same" (as
redecide: {"id", "old", "new"}; the first are for a reviewer), "entry" (the forget entry's journal position)}.
retest ¶
The re-test of an item pending one (re-asserted after a retraction, resting on one that was, or a copy of withdrawn content): ok=True — it is a fact again (a "promote" entry; its premises must be active too); ok=False — it is retracted again (only its pending copies when other claims of it hold).
pending ¶
The items waiting for a re-test (retest(id, ok=)), sorted; item(id)["pending"] says why.
supporters ¶
The claims standing behind the item as of journal position at (default: now — entries up to at
applied): one row {"by", "at", "source"} per claim written by then and not yet taken back there — by its
own assertion cut, its writer's cut, or the item's retraction. The answer to "who supported this when the
decision at that position was made": a retraction later than at does not rewrite it (S233). A derived
claim's row carries its premises as "just"; a claim with valid carries it.
rollback ¶
Supervision undoes an update: retract iid; the version it superseded or refuted comes back — as a fact, or
only as a hint while a flag covering iid (or its scope) is pending: a stale version is never restored as a fact.
→ the status changes (as retract).
retire ¶
The item's producer no longer stands behind it (no longer compiled, no longer offered): expired.
reconfirm ¶
Mark one item for re-confirmation (a flag on it alone) → the flag's id.
flag ¶
A drift / open-set flag: the items it covers — ids, every item whose scope falls under scope, or the
items on key — learned before it are hints, not facts, until a newer observation or a person confirms each
(or end_flag). → the flag's id (its journal position).
end_flag ¶
The flag is resolved (re-confirmation done, a person cleared it): its items are facts again if nothing else holds them back.
resolve ¶
A person's answer to a dispute on key (an item's key, as disputes() lists it): value wins, the other
sides are refuted.
tick ¶
The store's clock moved by k (decisions on a stream, moves in an environment): reconfirm_after counts it.
used ¶
Decisions used these items (wrong=True: and were found wrong): counts kept in each item's confidence.
note ¶
A journal entry that changes nothing (a gate's rejected measurement, a reason): recorded and chained.
upstream ¶
Items whose state can change the state of ids: premises (transitively) and same-key competitors.
component ¶
The connected component(s) of ids: premise edges both ways and shared keys.
downstream ¶
Items whose state a change to ids can change: ids, what is derived from them (transitively) and same-key
competitors — what a write re-resolves (the rest keeps its state).
status ¶
The item's status (STATUSES); "retracted" for an item with no live support (retracted, or derived from one). Inside a batch the writes so far are resolved first: an item added in the batch reads as what it is, not as absent (a premise added earlier in the same batch passes the consistency gate).
stale ¶
Must this item be re-confirmed before System 1 answers on it alone? Reads the pending flags (and the clock's expiry) — not the status, which a later write could have set back.
usable ¶
Is the item a fact System 1 may answer on alone: active and not stale? valid_at: a world time — and does the
item hold then: inside its effective intervals (periods(id); an item without valid always is), and, for a
derived item, where its premises hold. A disputed item is usable at a time outside the contested overlap; so
is an item derived from it, where for one of its derivations every premise is usable then (a premise met again
on its own derivation chain is not). No valid_at and no world clock (KnowledgeStore(now=)): status and
staleness only — a version closed by a later since still reads usable.
periods ¶
The world times the item holds as its key's value → [[since, until], ...] (half-open, None: unbounded), or
None for an item that holds at any time (no valid on it or under it). On a key with valid a version holds
from its since until a later version's since (or its own until; a bounded later version gives it back after
the bounded one ends), outside any contested overlap. A version pending a re-test takes no
side, but its writer's standing later different value leaves a gap in that writer's older
version until review/withdrawal; other writers' pending data do not create this gap. A derived item
holds where its own validity and, for one of its derivations, every premise holds.
valid ¶
The item's validity in the world: the valid of its latest live assertion, or None (no valid given).
find ¶
Records of the items that match every filter given (status: one or a tuple of STATUSES).
disputes ¶
The open disputes (person questions not resolved yet) → [{"question", "key", "items", "opened_at"}].
snapshot ¶
What a decision is given, as one plain fact: the items that match (about: a subject or a list of them;
relation; scope; kinds, default every kind) — "facts" (active and not stale) and "hints" (hypotheses,
expired or stale items, with why) — the ids they rest on, the store's fingerprint and journal position, and the
query (so redecide can rebuild it). Refuted and retracted items are not given, nor disputed ones except below.
valid_at: a world time — an item whose valid does not cover it is a hint ("not valid at ..."), not a fact. A
row of an item with a valid carries it ("valid": [since, until]). On keys with valid, two kinds of rows are
facts though their item-level status is not "active": a disputed item outside its contested overlap, and a
hypothesis derived from such an item where every premise of one of its derivations holds. Without valid_at
each is a fact row carrying "periods" (where it holds) when that is not empty, and is left out (disputed) or
a hint (derived) when it holds at no time.
fingerprint ¶
Versioned SHA-256 Merkle identity of the existing belief-row payload.
Version 1.5 uses km2:sha256:<64 hex>; old persisted identities must be
compared with legacy_fingerprint(). Journal hashes and item IDs are
unchanged. Updates cost expected O(changed log N), reads O(1). Direct
mutation of internal state/items is unsupported and bypasses this index.
legacy_fingerprint ¶
Exact pre-1.5 16-hex identity; rebuilds the full store after writes.
report ¶
What was learned and from whom, what is refuted, retracted, expired, disputed, flagged → a dict.
redecide ¶
redecide(storage, retracted, decide, *, fact='knowledge', upto='decision', skip_retracted=True, valid_at_key=None)
The stored decisions that rested on retracted items, re-run on their re-built snapshots.
storage: the decisions' TraceStorage; retracted: the retracted item ids; decide: init_state → {question: answer} (a function, or a System: its ask, not stored); fact: the given fact the snapshot was passed as. A decision is listed when its recorded snapshot changes once the retracted items' entries are left out of the journal up to its position; each is re-run on the old and the new snapshot. upto: "decision" (default) — the journal up to the decision's position, so the new snapshot is what the decision would have read without the retracted items; "now" — the whole journal without them, so a correction told after the decision (a wrong dose retracted and the right one told) is in the new snapshot and the new answer is the corrected one, not "unknown". → (answer_changes, justification_only): lists of {"id", "old", "new"} — the first goes to a reviewer, the second is recorded.
key_of ¶
Items whose body has a subject "s" and a relation "r" have a key: one value per (scope, s, r) at a time.
item_id ¶
The content id of an item: the same knowledge written twice is one item.
scope_matches ¶
Does an item's scope fall under a flag's scope (every key of pattern has the same value in scope)?
covers ¶
Is the world time at inside an item's validity [since, until)? (None: unbounded; no valid: always.) A
datetime.date is read as its ISO text.