Skip to content

solvi.core.knowledge.store

The knowledge store: a hash-chained journal of facts, rules, skills, actions and episodes, with provenance, versions, the source check, disputes to a person, the retraction cascade, staleness flags and snapshots for decisions.

The knowledge store: what a system has learned, from whom, and what rests on it — a hash-chained journal of items.

from solvi.core.knowledge import KnowledgeStore
ks = KnowledgeStore("knowledge.jsonl")              # any TraceStorage backend (a path, a TraceStorage) or None
a = ks.add("fact", {"s": "abt:123", "r": "same_as", "o": "buy:456"}, source="person", by="ann")
b = ks.add("fact", {"s": "abt:123", "r": "brand", "o": "acme"}, source="outcome", derived_from=[a])
ks.retract(a, by="bob", why="wrong label")          # b goes with it; what a refuted or disputed comes back
ks.verify(); ks.fingerprint(); ks.rebuild(skip={a}).fingerprint()   # equal: a retraction is exact
ks.snapshot(about="abt:123")                        # what a decision is given: facts, hints, the fingerprint

An item is one of five kinds — fact (subject, relation, object), rule, skill, action (an action model's preconditions and effects), episode (one finished case) — with a source: "person", "outcome" (an observation: what really happened), "spec" (a written specification) or "verified" (a System 2 answer given alone under its guarantee, named by its stored decision of=). The system's own unverified answers are never admitted: the source check is part of the store, not a setting (SourceGate; solvi.core.sources.check_source).

State is a function of the live journal entries (JTMS-style; valid= is valid time and the journal position is transaction time, as in a bitemporal store): an item is present while it has a live assertion whose premises (derived_from) are present and not refuted. Per key (scope, subject, relation) the live assertions are walked in time order: the same value confirms; an outcome refutes the current value (the world changed) — unless the outcome assertion is inferred (a consequence computed from premises, not an observation: it competes by rank, so two inferences of different values from equal-rank premises are a dispute); a higher-ranked source (outcome > person > spec > verified) refutes a lower one; supersedes (a producer's own new version) refutes the version it names; any other contradiction makes every side "disputed" and asks a person — one question per dispute EVENT (the journal position it opened at): the same two items disputed again after a resolution are asked again. resolve(key, value, by=) is the person's answer. A disputed item is never a fact.

Validity. valid=(since, until) is when a value holds in the world (half-open, None unbounded — a missing since is the beginning of time, and an assertion without valid on a key where others carry one is the beginning-of-time version, open-ended; ints, ISO date strings or datetime.date, which is kept as its ISO text). On a key where some assertion carries one, values compete only when their intervals overlap (a dose label for 2023 and another from mid-2024 are both facts, for their times). Versions of different sinces are versions in time, not a contradiction: the later since closes the earlier version at that since and the earlier stays active (its effective interval ends there; a bounded later version gives the earlier value back after its until; a retracted later version re-opens what it closed), whatever order they were written in. Validity is per assertion: a value told for two periods (12, then 24, then 12 again) holds in both under one id; valid(id) is the latest assertion's interval, periods(id) the effective intervals and item(id)["valid"] lists every claim. A version is one item's claims of one since: when they give different untils, the latest claim's until is the version's (a correction, as one validity per item was), so (45, 55) then (45, None) reads open from 45 and the reverse order reads bounded at 55. The same since from two writers (two missing sinces too) is not an observation of a change — rank decides, equal ranks dispute; that holds while any other writer still asserts the old value, even when the new value's writer had asserted it too. A writer's later value at one since replaces its own earlier one there (it no longer asserts the old value) — blindly: after an erratum withdraws one of its claims, the writer's next value at that since replaces every value it had there, including ones the erratum did not correct; two writers who both move from one value to the same other value leave that value holding. The winner takes only the overlap: a loser whose interval reaches beyond it is closed there and holds outside (refuted only when the winner covers it whole), and a dispute is contested only on the overlap — outside it each side holds; a person's answer (resolve) closes the loser the same way. An item pending re-test takes no side on its key (no dispute, no refutation, no closing). usable(id, valid_at=) and snapshot(valid_at=) read the effective intervals (a snapshot row carries "periods" when they are not the item's own valid; Knowledge.value(valid_at=) reads them). An item derived from a premise disputed only on an overlap is a hypothesis as a whole, but at a time where every premise of one of its derivations holds it is usable and a fact. Keys whose assertions carry no valid resolve as described above. A circular derivation (a ← p, b ← a, then a ← b) is accepted (with a warning at the write), but a chain back through the item itself is no support: retracting the loop's outside premise taints every item in it, so such content cannot keep itself alive past a retraction.

Statuses: "active" (a fact System 1 may answer on), "hypothesis" (a hint: a behaviour-changing item not yet promoted by the write gates, a derived item on a premise that is not active, an item under a pending flag, or one pending a re-test), "disputed", "refuted", "expired" (its reconfirm_after ran out on the store's clock, or retired) and "retracted".

Re-test. Content written again after it was retracted — the same item id, since an id is its content — is not revived by the new assertion: it is a hypothesis "pending re-test" until retest(id, ok=True) (a person or a test confirms it; ok=False retracts it again), and so is every item that rested on it when it was retracted, each until its own re-test. pending() lists the items waiting, item(id)["pending"] says why.

The guard on content told with valid. Every retraction (plain, only_from, assertion=, retract_source, a failed re-test, a rollback) withdraws its assertions' content (the key's value) on their validity. A later assertion of the same content on an overlapping validity, whatever its item id, is a copy: it waits for a re-test (the item is pending; when the item has other claims they still hold, and a failed re-test takes back only the copies) — unless writers it was not withdrawn from still hold that content wherever the copy could hold; such a copy is admitted and marked "unchecked" in item(id)["valid"]["claims"], and it goes pending once those holders are cut. A passed re-test vouches for the item's assertions before it and clears the withdrawals of its content. Derived and inferred assertions follow their premises instead. Items derived after the re-test are fresh; an item a rule only re-infers (inferred=True) follows its premises, and so does an inferred item that rested on one: once the premise passes its re-test it is a fact again without a re-test of its own. pending_retest(id) reads it.

Derived validity. An item with derived_from and no valid of its own gets the intersection of its premises' validity (each premise's span over its live claims); when they never hold together, or their times do not compare (an int and an ISO date), it is refused with the reason in the journal. It holds at a time (usable(id, valid_at=), snapshot(valid_at=), periods(id)) only where, for one of its derivations, every premise holds — a premise closed by a later version no longer carries it; an item with its own valid over premises whose times do not compare reads its own valid only.

Staleness. flag(scope=..., why=...) (a drift or open-set flag) or reconfirm(id) marks items for re-confirmation: an item learned before the flag is a hint, not a fact, until a newer observation or person confirms it or the flag ends (end_flag). A flag is read where staleness is asked (stale(id)), not inferred from the item's status. rollback(id) (supervision undoing an update) restores the previous version as a hint, never as a fact, while a flag covering the rolled-back version or its scope is pending (a rollback during a pending drift flag once restored a stale threshold as a fact on a shifted stream, and a stale check that read the item status missed it). reconfirm_after per (kind, source) (RECONFIRM_DEFAULTS, the store's clock advanced by tick) expires items that weaker evidence keeps. A tick re-resolves only the items whose expiry can flip between the old and the new clock (an index of their deadlines).

Every write — add, promote, refuse, retract, flag, resolve, tick, use — is a journal entry: in a TraceStorage (records of kind "knowledge", in the same hash chain as the decisions when it is the decisions' store) or in memory with its own hash chain. verify() checks the chain and that the state is what the journal gives; rebuild(upto=n, skip=ids) gives the store as it was at entry n, or as if ids had never been written; a store opened on a backend is replayed from its journal (an entry edited by hand breaks the chain). snapshot(...) returns what a decision reads, with the store's fingerprint and journal position, so every decision that is given it records the knowledge it used and replays; redecide(storage, retracted, decide) re-runs the decisions that rested on retracted items on their re-built snapshots and splits them into "answer changes" (to a reviewer) and "justification only" (recorded).

Measured by benchmarks/knowledge/retraction.py: 1,000 of 1,000 random retractions exact on a 10,000-item synthetic store (the store after a retraction has the fingerprint of the store rebuilt without it). Cost grows with the connected component a write touches (premise edges and shared keys); the store has been measured to 10,000 items, not 10⁵–10⁶.

Verdict dataclass

Verdict(admit: bool, reason: str = '', measured: dict | None = None, gate: str = '')

A write gate's answer about one proposed item: admitted or not, why, and what it measured (recorded).

SelfDefeating

Bases: ValueError

A derived item whose key is a key of one of its own premises with another value: it would refute its support.

KnowledgeStore

KnowledgeStore(storage=None, *, gates=None, defaults=None, writers=None, reassert_writers=(), now=None, revival_window=None, _replay=True)

See the module docstring.

storage: where the journal is kept — None (in memory), a path (solvi.core.store.open_storage: .jsonl, .db, .duckdb, postgresql://) or a TraceStorage (the decisions' store itself, so knowledge and decisions share one hash chain); an existing journal there is replayed. gates: the write gates (solvi.core.knowledge.gates; default SourceGate and ConsistencyGate); the source check runs first whatever is given. defaults: reconfirm_after per (kind, source). writers: {writer: source (or sources), ...} — opt-in: a source (a rank) is then taken only from a writer (by=) registered with it; any other item is refused with the reason (gates.WriterGate, run right after the source check). None (default): any writer may declare any source, as before. now: a world clock, opt-in — a callable returning the current world time (no arguments), or a fixed time; with it usable(id) and usable on derived items read the effective intervals at that time instead of status alone, so a version closed by a later one (or an item derived from it) is not usable with no valid_at given. None (default): as before — usable(id) with no valid_at reads status and staleness only. revival_window: optional inclusive protection after withdrawal, in logical tick units. New direct dated claims after the window may vouch anew; old held claims never become clean merely because time passes. None preserves the original overlap-based guard. Reopen persistent stores with the same explicit policy.

revival_window property

revival_window

Optional inclusive withdrawal window in owner-declared logical clock units.

__len__

__len__()

The number of journal entries.

head

head()

{"count", "hash"}: the number of journal entries and the last entry's hash.

verify

verify()

Is the journal's hash chain whole (in memory: its own chain; on a backend: the backend's verify()), and is the state the one the journal gives (the store rebuilt from it has the same fingerprint)?

rebuild

rebuild(upto=None, skip=())

The store the journal gives, in memory: entries applied in order (upto: the first upto — the store as it was then); skip: item ids whose own entries (assertions, promotions, retractions) are left out — the store as if they had never been written. Premises naming a skipped item stay in the entries derived from it (dead). The result is independently writable, with its own hash chain, copied journal data, write gates, defaults, world clock and reassert capabilities. Logical entry positions are retained for assertion proofs. Uncopyable policy objects fail before replay. Functions used as callbacks retain their normal Python closure semantics; external state accessed by a callback is not snapshotted. Stored verified decisions are not copied into this in-memory reconstruction: new verified writes still require a decision in the destination TraceStorage.

branch

branch(storage=None, *, upto=None)

Independent writable what-if over a journal prefix; optional empty destination.

No merge or decision-proof export. Persistent branches refuse inherited verified/of assertions. Reopening uses the normal constructor with the same explicit policy. A lineage note binds the original parent head and selected prefix hash.

batch

batch()

Several writes, one resolution at the end (a consolidation writes many items).

proposal

proposal(kind, body, scope=None, *, source, by=None, evidence=None, derived_from=(), supersedes=None, reconfirm_after=None, valid=None, level=None, of=None, inferred=False, consulted=())

A proposed item as the write gates see it (a dict with the record's fields; nothing is written).

add

add(kind, body, scope=None, *, source, by=None, evidence=None, derived_from=(), supersedes=None, reconfirm_after=None, valid=None, level=None, of=None, shadow=None, inferred=False, consulted=(), reassert=False)

Propose an item → its id, or None when a gate refused it (a "refused" entry in the journal says why).

kind: one of KINDS; body: plain JSON (a fact: {"s", "r", "o"}); scope: a dict (a question, an environment); source: "person" | "outcome" | "spec" | "verified" (anything else is refused: never the system's own answers); by: who; evidence: stored ids, quotes, steps; derived_from: premise item ids (the retraction cascade follows them); supersedes: the id of the producer's previous version; reconfirm_after: clock units until it needs re-confirmation (default RECONFIRM_DEFAULTS); valid: (since, until) in the world, as you count time; level: a guarantee level (for "verified"); of: the stored decision a "verified" item is (checked in the store's TraceStorage); shadow: passed to the write gates (a shadow measurement for a behaviour-changing item); inferred: the assertion is a consequence computed from derived_from (a rule applied to facts), not an observation — an inferred outcome does not refute the current value as an observation does: it competes by rank, and equal ranks are a dispute. consulted: item ids the conclusion looked at besides its premises — the evidence for the alternatives it did not choose (opt-in). Retracting one does not retract this item (its premises stand) but marks it for review: a hypothesis pending re-test (retest(id, ok=True)), as an item resting on a retracted premise; item(id) ["consulted"] lists them. Such a store differs from its rebuild without the retracted item until the re-test.

A fact or an episode the gates admit is present at once; a rule, skill or action the gates admit is promoted (its verdicts recorded); one a later gate holds stays a hypothesis (a "held" entry says why).

promote

promote(iid, measured=None)

A behaviour-changing item passed its gates (or a person promotes it): it becomes a fact (measurements kept).

retract

retract(iid, *, by=None, why=None, only_from=None, assertion=None)

Take an item back, with everything derived from it (the cascade over derived_from); what it refuted or disputed comes back. Nothing is deleted: the journal keeps the entry and the chain stays whole. only_from: a writer (the by of its assertions) — take back only that writer's assertions of the item: the same content told by others stays (one item for both, since an id is its content); the item goes only when no other assertion is left. only_from=(writer, evidence): only that writer's assertions given with this evidence (one of two publications of the same content). assertion: the journal position of one standing assertion of the item (item(id)["valid"]["claims"] lists them as "at"; a list takes several) — only it is taken back (a journal entry of its own, "retract_assertions"; a duplicate taken back while its writer's other claim of the content stands is not a withdrawal of that content). → {id: (status before, status after)} of every item whose status changed.

retract_source

retract_source(writer, *, by=None, why=None)

Take back every standing assertion from writer (an exposed source): items only it told go with their derivations; items others told too stay on their support. → {id: (status before, status after)}.

forget

forget(writer, storage=None, decide=None, *, by=None, why=None, fact='knowledge')

Forget a source in one call: what writer told is taken back, the stored decisions that rested on it are re-decided on the journal as it is now, and one "forget" journal entry records what was taken back and which decisions changed. The three steps are the public calls

changed = store.retract_source(writer, by=by, why=why)          # its items, and what was derived from them
gone = [i for i, (_, now) in changed.items() if now != "active"]
answer_changes, same = store.redecide(storage, gone, decide, fact=fact, upto="now")

then the entry. storage: the decisions' TraceStorage (None: only the retraction and the entry); decide: what re-decides an input (a System, or init_state → {question: answer}; needed with storage); fact: the given fact the snapshots were passed as. storage may be a path (opened as System(storage=) opens one, before anything is written). Nothing is erased: the journal keeps the source's entries (retracted) and the decision records keep their inputs — erasing personal data in a record is TraceStorage.redact(id), kept explicit (where_is finds the records holding a value). → {"source", "retracted" ({id: (status before, status after)}), "decisions" (the ids of the stored decisions whose snapshot changed), "changed" / "same" (as redecide: {"id", "old", "new"}; the first are for a reviewer), "entry" (the forget entry's journal position)}.

retest

retest(iid, *, ok, by=None, why=None)

The re-test of an item pending one (re-asserted after a retraction, resting on one that was, or a copy of withdrawn content): ok=True — it is a fact again (a "promote" entry; its premises must be active too); ok=False — it is retracted again (only its pending copies when other claims of it hold).

pending_retest

pending_retest(iid)

Is the item waiting for a re-test (see the module docstring)?

pending

pending()

The items waiting for a re-test (retest(id, ok=)), sorted; item(id)["pending"] says why.

supporters

supporters(iid, *, at=None)

The claims standing behind the item as of journal position at (default: now — entries up to at applied): one row {"by", "at", "source"} per claim written by then and not yet taken back there — by its own assertion cut, its writer's cut, or the item's retraction. The answer to "who supported this when the decision at that position was made": a retraction later than at does not rewrite it (S233). A derived claim's row carries its premises as "just"; a claim with valid carries it.

rollback

rollback(iid, *, by=None, why=None)

Supervision undoes an update: retract iid; the version it superseded or refuted comes back — as a fact, or only as a hint while a flag covering iid (or its scope) is pending: a stale version is never restored as a fact. → the status changes (as retract).

retire

retire(iid, *, why=None)

The item's producer no longer stands behind it (no longer compiled, no longer offered): expired.

reconfirm

reconfirm(iid, *, why=None, by=None)

Mark one item for re-confirmation (a flag on it alone) → the flag's id.

flag

flag(*, scope=None, ids=None, key=None, why=None, by=None)

A drift / open-set flag: the items it covers — ids, every item whose scope falls under scope, or the items on key — learned before it are hints, not facts, until a newer observation or a person confirms each (or end_flag). → the flag's id (its journal position).

end_flag

end_flag(flag, *, by=None, why=None)

The flag is resolved (re-confirmation done, a person cleared it): its items are facts again if nothing else holds them back.

resolve

resolve(key, value, *, by)

A person's answer to a dispute on key (an item's key, as disputes() lists it): value wins, the other sides are refuted.

tick

tick(k=1)

The store's clock moved by k (decisions on a stream, moves in an environment): reconfirm_after counts it.

used

used(ids, *, wrong=False)

Decisions used these items (wrong=True: and were found wrong): counts kept in each item's confidence.

note

note(**data)

A journal entry that changes nothing (a gate's rejected measurement, a reason): recorded and chained.

upstream

upstream(ids)

Items whose state can change the state of ids: premises (transitively) and same-key competitors.

component

component(ids)

The connected component(s) of ids: premise edges both ways and shared keys.

downstream

downstream(ids)

Items whose state a change to ids can change: ids, what is derived from them (transitively) and same-key competitors — what a write re-resolves (the rest keeps its state).

status

status(iid)

The item's status (STATUSES); "retracted" for an item with no live support (retracted, or derived from one). Inside a batch the writes so far are resolved first: an item added in the batch reads as what it is, not as absent (a premise added earlier in the same batch passes the consistency gate).

stale

stale(iid)

Must this item be re-confirmed before System 1 answers on it alone? Reads the pending flags (and the clock's expiry) — not the status, which a later write could have set back.

usable

usable(iid, *, valid_at=None, _seen=frozenset())

Is the item a fact System 1 may answer on alone: active and not stale? valid_at: a world time — and does the item hold then: inside its effective intervals (periods(id); an item without valid always is), and, for a derived item, where its premises hold. A disputed item is usable at a time outside the contested overlap; so is an item derived from it, where for one of its derivations every premise is usable then (a premise met again on its own derivation chain is not). No valid_at and no world clock (KnowledgeStore(now=)): status and staleness only — a version closed by a later since still reads usable.

periods

periods(iid)

The world times the item holds as its key's value → [[since, until], ...] (half-open, None: unbounded), or None for an item that holds at any time (no valid on it or under it). On a key with valid a version holds from its since until a later version's since (or its own until; a bounded later version gives it back after the bounded one ends), outside any contested overlap. A version pending a re-test takes no side, but its writer's standing later different value leaves a gap in that writer's older version until review/withdrawal; other writers' pending data do not create this gap. A derived item holds where its own validity and, for one of its derivations, every premise holds.

valid

valid(iid)

The item's validity in the world: the valid of its latest live assertion, or None (no valid given).

item

item(iid)

The record of one item (the schema of DESIGN_KM §4) → dict, or None.

find

find(*, kind=None, s=None, r=None, scope=None, status=None)

Records of the items that match every filter given (status: one or a tuple of STATUSES).

active

active(key)

The active item on a key, or None.

disputes

disputes()

The open disputes (person questions not resolved yet) → [{"question", "key", "items", "opened_at"}].

snapshot

snapshot(*, about=None, relation=None, scope=None, kinds=None, valid_at=None)

What a decision is given, as one plain fact: the items that match (about: a subject or a list of them; relation; scope; kinds, default every kind) — "facts" (active and not stale) and "hints" (hypotheses, expired or stale items, with why) — the ids they rest on, the store's fingerprint and journal position, and the query (so redecide can rebuild it). Refuted and retracted items are not given, nor disputed ones except below. valid_at: a world time — an item whose valid does not cover it is a hint ("not valid at ..."), not a fact. A row of an item with a valid carries it ("valid": [since, until]). On keys with valid, two kinds of rows are facts though their item-level status is not "active": a disputed item outside its contested overlap, and a hypothesis derived from such an item where every premise of one of its derivations holds. Without valid_at each is a fact row carrying "periods" (where it holds) when that is not empty, and is left out (disputed) or a hint (derived) when it holds at no time.

fingerprint

fingerprint()

Versioned SHA-256 Merkle identity of the existing belief-row payload.

Version 1.5 uses km2:sha256:<64 hex>; old persisted identities must be compared with legacy_fingerprint(). Journal hashes and item IDs are unchanged. Updates cost expected O(changed log N), reads O(1). Direct mutation of internal state/items is unsupported and bypasses this index.

legacy_fingerprint

legacy_fingerprint()

Exact pre-1.5 16-hex identity; rebuilds the full store after writes.

counts

counts()

{status: number of items} (absent items counted as "retracted").

report

report()

What was learned and from whom, what is refuted, retracted, expired, disputed, flagged → a dict.

redecide

redecide(storage, retracted, decide, *, fact='knowledge', upto='decision', skip_retracted=True, valid_at_key=None)

The stored decisions that rested on retracted items, re-run on their re-built snapshots.

storage: the decisions' TraceStorage; retracted: the retracted item ids; decide: init_state → {question: answer} (a function, or a System: its ask, not stored); fact: the given fact the snapshot was passed as. A decision is listed when its recorded snapshot changes once the retracted items' entries are left out of the journal up to its position; each is re-run on the old and the new snapshot. upto: "decision" (default) — the journal up to the decision's position, so the new snapshot is what the decision would have read without the retracted items; "now" — the whole journal without them, so a correction told after the decision (a wrong dose retracted and the right one told) is in the new snapshot and the new answer is the corrected one, not "unknown". → (answer_changes, justification_only): lists of {"id", "old", "new"} — the first goes to a reviewer, the second is recorded.

key_of

key_of(body, scope)

Items whose body has a subject "s" and a relation "r" have a key: one value per (scope, s, r) at a time.

item_id

item_id(kind, body, scope)

The content id of an item: the same knowledge written twice is one item.

scope_matches

scope_matches(scope, pattern)

Does an item's scope fall under a flag's scope (every key of pattern has the same value in scope)?

covers

covers(valid, at)

Is the world time at inside an item's validity [since, until)? (None: unbounded; no valid: always.) A datetime.date is read as its ISO text.