solvi.experimental.numguard¶
Exact values through a side channel (experimental): the model writes [k] for an input numeral and <calc> /
<date> calls with @k for their results; fill(text, source) writes the exact values in, check(text, source)
names the source of every number left in a text, NumberGuard(on_fail=...) runs both and refuses, raises or reports,
NumberGuard.run_with_retry(generate, source) asks once more with what failed. Typed calls share the @k
numbering: <table> (cells, column and row aggregates of the source's pipe tables, table_calc), <cmp>
(comparisons, a yes/no, cmp_calc), <scale> (scale words, percent, time and speed units, scale_calc) and
<count> (counts and membership, count_calc). MODES is the registry of call kinds (slot grammar, result type,
validator, executor); register_mode adds one that fill runs. PI is an opt-in constant. It guarantees that every
number has a source, not that it is the right one (status).
Exact values through a side channel: the model writes pointers, code writes the numbers, a guard checks that every number in the final text has a source (experimental).
A model that types numbers into free text makes some of them up. Here it writes pointers instead:
[k]— the k-th numeral of the input text (1-based, in reading order;annotate(source)shows the numbering);<calc>expr</calc>/<date>expr</date>— a call computed exactly by code (Decimalarithmetic, calendar arithmetic); the calls are numbered in order of appearance, all kinds together, and@k(or{@k}) is the value of the k-th call;<table>(a cell, or sum / avg / count / max / min / argmax / argmin over a column or a row of a table in the source),<cmp>(comparisons and and / or / not, a yes/no),<scale>(thousand ↔ million, percent ↔ fraction, seconds ↔ hours ↔ days ↔ weeks, months ↔ years, km/h ↔ m/s) and<count>(how many items match, membership) — typed calls computed by code the same way (table_calc,cmp_calc,scale_calc,count_calcon their own).MODESlists every call kind with its slot grammar, result type and validator;register_modeadds one.
fill replaces the pointers with the exact values (an input numeral as the input writes it: "$1,234", "12.5%",
"4.2 million") and each call with its result; check reads every number left in a text and names its source — an
input numeral, a call's result, or an allowed constant — or marks it unsourced; NumberGuard runs both with one
configuration and a fail-closed mode (raise, or return a refusal) or a report mode; NumberGuard.run_with_retry asks
your model once more with the list of what failed (any callable: no LLM client here).
from solvi.experimental.numguard import NumberGuard, annotate
source = "Revenue was $1,250 million in 2023 and $1,100 million in 2022. Invoice dated 2024-03-01, net 45 days."
prompt_text, _ = annotate(source) # "Revenue was [1]$1,250 million in [2]2023 ..." for the prompt
reply = "Revenue grew by <calc>[1] - [3]</calc> million (<calc>(@1) / [3] * 100</calc>%). Due <date>[5] + [6]D</date>."
out = NumberGuard().run(reply, source)
out.text # 'Revenue grew by 150 million (13.63636364%). Due 2024-04-15.'
out.verdict.ok # True: every number has a source
What it gives is provenance, not correctness: a pointer at the wrong input numeral is filled and passes. Numbers
written as words ("three") and numbers inside identifiers ("A-10457", "COVID-19", "v2") are not checked (identifiers
can be checked against the input verbatim with ids="check"). Only integers 0–12 and 100 pass as constants by
default (SMALL), as written counts without decimals, currency, percent or scale; MEASURED_CONSTANTS is the wider set the measurements used (integers 0–100 and a few common
factors), opt-in, and years() or your own set or predicate extend either. pi (PI, 28 digits, an
approximation) is opt-in the same way: constants=SMALL | {PI}.
SMALL
module-attribute
¶
Integers 0–12 and 100: counts, months, a percent base. The default constants, for literals in calls and free numbers alike.
MEASURED_CONSTANTS
module-attribute
¶
MEASURED_CONSTANTS = frozenset(Decimal(i) for i in range(101)) | frozenset(Decimal(x) for x in (365, 360, 52, 1000, 10 ** 6, 10 ** 9))
Integers 0–100, days in a year (365, 360), weeks (52), 1,000, a million, a billion: the set the measurements behind
the docs used, for calc literals and free constants alike. Opt-in (constants=MEASURED_CONSTANTS): an invented
number from 13 to 99 passes it.
PI
module-attribute
¶
π to 28 significant digits — an approximation (the error is below 3e-28), not an exact value. pi in a <calc>
expression (and as a value in the typed calls) stands for it. Not in SMALL or MEASURED_CONSTANTS: inside fill it
passes only when it is an allowed constant (constants=SMALL | {PI}) or with literals="any".
MODES
module-attribute
¶
The call kinds fill runs, by name (read-only; register_mode adds one): the single list of typed calls.
Numeral
dataclass
¶
Numeral(start: int, end: int, surface: str, kind: Literal['number', 'date', 'id'], number: Decimal | None = None, scale: int = 1, unit: str = '', date: object = None)
A number as a text writes it.
number is the written number with its sign ("(12.3)" and "-12.3" → -12.3; "4.2 million" → 4.2); scale the
multiplier of a scale word or suffix (1,000 for "thousand" / "k"); value their product. unit is "%", a currency
sign, an ordinal suffix or "". A parenthesised number right after another number on its line ("16.6% (18.8%)": a
comparative, or an accounting negative — the text does not say) keeps its parentheses and reads unsigned, the
sign not guessed (what solvi.solutions.document abstains on, as a reader without a decision). A date has
kind="date" and date (a datetime.date, or (month, day) when the
text gives no year). An identifier ("A-10457", "COVID-19") has kind="id" and is not a number.
plain ¶
The surface as prose would write it: a negative in parentheses as "-12.3". A comparative "(18.8%)" — parentheses after another number on its line, whose sign was never read — keeps them.
ProgramError ¶
Bases: ValueError
A call that cannot be computed, or a pointer that does not resolve: the message says why, for a retry.
Cell
dataclass
¶
A table cell a result came from: table table<table>, its 1-based data row and column (the header row and the
row-label column are not counted), their labels and the cell as written.
Table
dataclass
¶
A table for <table> and <count>: rows row-major, the header row first (the column labels); with
row_labels=True the first column holds the row labels. scale multiplies the money cells under norm (1,000
for a caption "in thousands" or a header "SEK million"; percent cells are not scaled) and is applied to the
cells by check; currency is what a header "SEK million" / "£ million" states about them ("" when it does
not). Cells are text ("$1,250", "(3.2)", "12.5%", "—"),
read like the input's numerals: a parenthesised number is negative, a dash or "n/a" is blank. Short rows are
padded with blanks.
CallContext
dataclass
¶
What an extra mode's executor sees inside fill: the input's numerals ([k] is inputs[k - 1]), the earlier
calls (@j is calls[j - 1], a CallResult; check its error), the tables, this call's number k, and
literal_ok(Decimal), the literal policy of this fill.
Mode
dataclass
¶
Mode(name: str, slots: str, result: str, validator: Callable[[object], bool], executor: Callable | None, description: str = '', builtin: bool = False)
A call kind <name>…</name>: its slot grammar in short, the type of its result, validator(value) -> bool
on a result, the executor and a one-line description. A built-in mode's executor is its standalone function
(calc, date_calc, table_calc, cmp_calc, scale_calc, count_calc); an extra mode's is
executor(expr, ctx: CallContext) -> value, raising ProgramError for a bad call, or None (the mode is
described but has no executor: its calls fail).
CallResult
dataclass
¶
CallResult(k: int, kind: str, expr: str, value: object = None, shown: str = '', error: str = '', cells: tuple = ())
The k-th call: its kind, the expression as written, the value (a Decimal, a date, an int of days or a count,
a weekday or a table label, a bool) and the text it was shown as; error instead of a value when it failed;
cells, the table cells a <table> / <count> value (or a <cmp> / <scale> on one) came from.
Inserted
dataclass
¶
A span of the filled text written by code: from the input numeral [index] (source="input") or from call
@index (source="tool").
Filled
dataclass
¶
Filled(text: str, calls: list[CallResult], inserted: list[Inserted], errors: list[str], source_numerals: list[Numeral], tables: list = list())
fill's result: the final text, the calls, the spans code wrote, and every error (an empty list = clean).
NumberCheck
dataclass
¶
NumberCheck(surface: str, start: int, end: int, kind: str, source: Source, index: int = 0, how: str = '', cells: tuple = ())
One number of the checked text and where it comes from.
source: "input", "tool", "constant" or "unsourced"; index: the input numeral [k] or call @k (0 otherwise);
how: "pointer" (written by code), "exact", "sign" (equal but for the sign), "scale_word" (equal once a scale word
is applied: "4.2 million" against "4,200,000"), "rescaled x1000" / "rescaled /1000" (and 10^6, 10^9: digits off by
a whole scale, only with rescale=True), "rounded" (only with rounding=True), "date_year" (the year of a source
date), "verbatim" (an identifier found in the input), "table cell" (a cell of a table passed as tables), or ""
when unsourced. A number from a <table>, <cmp>, <scale> or <count> call has the mode as how ("table",
...) where a calc or date call has "pointer" or "exact", and cells, the table cells it came from.
Verdict
dataclass
¶
Every number of a text with its source; ok when none is unsourced and the program had no error.
feedback ¶
What to tell the model in a retry: each error and each unsourced number.
UnsourcedNumbers ¶
Bases: ValueError
Raised by a fail-closed guard (on_fail="raise"): outcome holds the verdict and the filled text.
Outcome
dataclass
¶
Outcome(text: str, ok: bool, refused: bool, verdict: Verdict, filled: Filled, attempts: list['Outcome'] = list())
A guarded answer: text (the filled text, or the refusal when refused), ok, refused, the verdict, the
calls and, after run_with_retry, every attempt (attempts, this one last).
NumberGuard
dataclass
¶
NumberGuard(on_fail: Literal['report', 'refuse', 'raise'] = 'refuse', refusal: str = REFUSAL, constants: object = SMALL, literals: Literal['constants', 'input', 'any'] = 'constants', signed: bool = False, rescale: bool = False, rounding: bool = False, ids: Literal['skip', 'check'] = 'skip', answer_marker: str | None = None, digits: int = 10, places: int | None = None, grouping: bool | None = None, date_format: str | None = None, show_calls: bool = True, booleans: tuple = ('yes', 'no'), decimal: str = '.', locale: str | None = None)
fill and check with one configuration and a mode for a failure.
on_fail: "report" returns the outcome as it is (the verdict says what failed), "refuse" returns refusal as the
text, "raise" raises UnsourcedNumbers. The other fields are fill's and check's options.
check ¶
Check a text that was not filled (a model typing its numbers, a native tool caller's answer).
run ¶
Fill the text, check every number left in it, apply on_fail. results: values from tools run outside
(a native tool call's output) that also count as sources. tables: tables given apart from source (None:
the pipe tables of source, their units header applied to the cells in the check).
run_with_retry ¶
run_with_retry(generate: Callable[[str | None], str], source: str, retries: int = 1, results: Iterable = (), tables=None) -> Outcome
Ask, guard, and on a failure ask again with what failed. generate(feedback) returns the model's text:
it is called with None first, then with Verdict.feedback() (the errors and the unsourced numbers) up to
retries more times. The last attempt goes through on_fail; Outcome.attempts holds them all.
years ¶
The years lo..hi as constants: constants=SMALL | years(2000, 2030).
numerals ¶
Every number, date and identifier with digits in text, in order. Dates: ISO "2024-03-15", "15.03.2024" /
"15/03/2024" (day first), "March 15, 2024", "15 March 2024", "March 15" (no year). Numbers: a sign or a negative
in parentheses, a currency sign, comma thousands, a decimal point, then "%" / "percent", an ordinal suffix
("3rd"), a scale suffix ("12k", "3bn") or a scale word ("4.2 million", bound to the number on its own line —
"£1,000\nmillion" is the number £1,000 and the word on the next line). A number in parentheses right after
another number on its line ("16.6% (18.8%)" — a comparative or a negative, the text does not say) keeps its
parentheses and reads unsigned. A token where digits follow letters
("A10", "v2.1", "COVID-19", "1Z999AA1") is an identifier; a number followed by a unit ("5km") is a number.
Unsupported scale words (English -illion words and Russian scale abbreviations) and comma groups that are
not comma thousands raise ValueError; no decimal-comma interpretation is guessed. Explicit decimal="," instead reads decimal commas
and space-grouped thousands; dot decimals and mixed punctuation then raise ValueError. Surfaces and spans
remain those of the original text. Call expressions retain their Python-style decimal point syntax.
annotate ¶
The text with [k] before each numeral (numbers and dates, not identifiers), for the model's prompt, and the
numerals in that order: [1] is numerals[0].
calc ¶
Exact arithmetic on number literals: + - * / ^ (or **), parentheses, unary minus, a postfix % (divides by 100);
Decimal with 50 digits. An integer power up to 1,000 is exact; a fractional one needs a positive base. pi is
PI (28 digits, an approximation). Raises ProgramError with a readable reason.
date_calc ¶
Calendar arithmetic: DATE ((+|-) N(D|W|M|Y))* → a date (a month added to Jan 31 gives the month's last day);
DATE - DATE → days (an int; DAYS in front is allowed); WEEKDAY DATE ... → the weekday's name. Dates are ISO
(YYYY-MM-DD) or DD.MM.YYYY. Raises ProgramError.
parse_tables ¶
The pipe (markdown) tables of a text, in order: table1, table2, ... A run of lines that start with "|" or
hold two "|" or more is a table; a |---|---| line is skipped; the first row is the header. A caption "in thousands"
/ "in millions" / "in billions" / "in trillions" (or "'000", "000s"), or a header that pairs a currency with the
scale — "SEK million", "£ million", "In $ millions" — in the last two non-blank lines of the three above the table
or in its header row sets Table.scale (and Table.currency for the currency + scale form).
table_calc ¶
A <table> call on its own. tables: a text with pipe tables, a Table, or a list of tables (each a Table
or rows, the header first). [tableN:] cell(row, column[, norm]);
[tableN:] sum|avg|max|min|count|argmax|argmin([column] col[, a..b][, where <condition>][, norm]) over a column
(a..b: rows) and ...(row r[, a..b][, where <condition>][, norm]) over a row (a..b: columns). A row or column is a
1-based number or a "label"; a condition compares a column (for a row aggregate: another row, in each column)
with a value (< <= > >= == != between in), joined by and / or / not. Values are the written numbers; norm
multiplies by each cell's scale word and the table's caption scale. Returns the value (a Decimal, an int for
count, a label for argmax / argmin — the row's over a column, the column's over a row — a date or a string for a
cell), or (value, cells) with cells=True. Raises ProgramError.
cmp_calc ¶
A <cmp> call on its own: numbers, ISO dates or "strings" compared with < <= > >= == != (strings only == and
!=, case-insensitive), x between a and b (inclusive), x [not] in (a, b, ...), joined by and / or / not.
Raises ProgramError for a number against a date, two different units ("%" against "$") or scale words.
scale_calc ¶
A <scale> call on its own: VALUE [UNIT] to UNIT [rule 360|365|366]. Units: units, thousand, million,
billion, trillion (k, mn, bn, tn); percent (%), fraction, bp; seconds (s, sec), minutes (min), hours (h, hr),
days, weeks; months, quarters, years; m/s, km/h (kmh, kph), km/s, mph. Seconds to weeks against months, quarters
and years need a rule (360: a month is 30 days; 365 or 366: a year). Exact Decimal, one division at the end
(1 m/s is 3.6 km/h exactly; 1 km/h to m/s is 1000/3600 to 50 digits); raises ProgramError.
count_calc ¶
A <count> call on its own: (a, b, ...) [if <condition>] or [tableN:] column(col[, a..b][, where ...]) [if
<condition>] (or row(r[, a..b][, where ...])) → how many items (non-blank cells) match (an int); x [not] in
<list> → a bool. Raises ProgramError.
register_mode ¶
register_mode(name: str, executor: Callable | None = None, *, slots: str = '', result: str = '', validator: Callable[[object], bool] | None = None, description: str = '', replace: bool = False) -> Mode
Add a call kind <name>…</name> that fill runs with executor(expr, ctx) (see Mode, CallContext); its
calls share the @k numbering, a result failing validator (default: any) is an error, and check reports its
numbers with the name as how. executor=None registers a described mode whose calls fail. A built-in mode
cannot be replaced; an extra one only with replace=True.
fill ¶
fill(text: str, source: str | list[Numeral], *, answer_marker: str | None = None, literals: Literal['constants', 'input', 'any'] = 'constants', constants=SMALL, digits: int = 10, places: int | None = None, grouping: bool | None = None, date_format: str | None = None, show_calls: bool = True, tables=None, booleans: tuple[str, str] = ('yes', 'no'), decimal: str = '.', locale: str | None = None) -> Filled
Run the calls of a model's text and write in the exact values.
locale="ru" renders Russian decimal commas, scale/currency abbreviations and month names.
decimal always selects the SOURCE reader (default English decimal point); output locale never
reinterprets source punctuation. Use matching check(..., locale="ru", decimal=".") to read the
Russian output against English input without trusting inserted spans. None/"en" retains prior output;
unsupported locale names raise ValueError. No process-wide locale is changed.
source is the input text (or its numerals). Each <calc> / <date> call is computed in order ([k] is the
k-th input numeral's written number — "4.2 million" gives 4.2, "12.5%" gives 12.5 — or its date; @k an earlier
call's value) and replaced by its value where it stands (show_calls=False removes it: refer to it as @k). Then
[k] and @k / {@k} in the prose are replaced: an input numeral as the input writes it (a currency sign or "%"
the text repeats around the pointer is not doubled), a result with digits significant digits or places
decimals, comma-grouped when grouping (None: when the input groups its thousands), a date as ISO or
date_format (strftime). With answer_marker ("ANSWER:") only the text after its last occurrence is returned;
the calls before it still run.
A literal number inside a call must be a constant (literals="constants"), or also an input numeral written
verbatim ("input"), or anything ("any"). A pointer out of range, an unknown or failed @k, a malformed call or
a missing marker is an error in Filled.errors; its pointer is left as written.
<table>, <cmp>, <scale>, <count> and modes added with register_mode share the numbering (see
table_calc, cmp_calc, scale_calc, count_calc, MODES); [k] and @k work inside them, a literal number in a value position follows
literals (a literal date needs literals="any", or "input" and an input date). tables: the tables they
read (a Table, a list of them or of rows; None: the pipe tables of source, table1 first). A bool is shown as
booleans ("yes" / "no").
check ¶
check(text: str, source: str | list[Numeral] = '', results: Iterable = (), *, inserted: Iterable[Inserted] = (), constants=SMALL, signed: bool = False, rescale: bool = False, rounding: bool = False, ids: Literal['skip', 'check'] = 'skip', errors: Iterable[str] = (), tables=None, decimal: str = '.', locale: str | None = None) -> Verdict
Name the source of every number in text.
locale="ru" selects a Russian OUTPUT reader (decimal comma, space/NBSP grouping,
Russian scales, currency and months). decimal independently selects source/results/table
punctuation. Localized inserted spans are re-read and matched against values, never trusted.
A number is sourced when code wrote it (inserted, from fill), when it equals an input numeral (sign ignored
unless signed; a scale word applied: "4.2 million" equals "4,200,000"), a result in results (CallResults, or
plain values: numbers, dates, or strings such as a native tool's output), or a constant (constants: a set of
numbers or a predicate on a Decimal). A date is sourced when it equals an input date or a result date; a year
when it is the year of one. With rescale=True a number off by a whole scale (8,220 against 8,220,000) is
sourced as "rescaled"; with rounding=True a number equal to a source rounded to its own decimals (two significant
digits at least) as "rounded". Identifiers are skipped, or with ids="check" must appear verbatim in the input.
tables: tables given apart from source (a Table, a list, or a text with pipe tables); a number equal to
one of their cells is an input with index 0 and how="table cell". A table's units header — "in thousands" or
"SEK million" / "£ million" — applies to its cells: a cell 18,742 under a "SEK million" header sources
"18,742 million" (a percent cell is not scaled). The tables inside a source text are already its numerals;
their header scale is applied to them the same way, as an addition (a number quoting a cell as written still
matches the numeral).