Skip to content

solvi.experimental.numguard

Exact values through a side channel (experimental): the model writes [k] for an input numeral and <calc> / <date> calls with @k for their results; fill(text, source) writes the exact values in, check(text, source) names the source of every number left in a text, NumberGuard(on_fail=...) runs both and refuses, raises or reports, NumberGuard.run_with_retry(generate, source) asks once more with what failed. Typed calls share the @k numbering: <table> (cells, column and row aggregates of the source's pipe tables, table_calc), <cmp> (comparisons, a yes/no, cmp_calc), <scale> (scale words, percent, time and speed units, scale_calc) and <count> (counts and membership, count_calc). MODES is the registry of call kinds (slot grammar, result type, validator, executor); register_mode adds one that fill runs. PI is an opt-in constant. It guarantees that every number has a source, not that it is the right one (status).

Exact values through a side channel: the model writes pointers, code writes the numbers, a guard checks that every number in the final text has a source (experimental).

A model that types numbers into free text makes some of them up. Here it writes pointers instead:

  • [k] — the k-th numeral of the input text (1-based, in reading order; annotate(source) shows the numbering);
  • <calc>expr</calc> / <date>expr</date> — a call computed exactly by code (Decimal arithmetic, calendar arithmetic); the calls are numbered in order of appearance, all kinds together, and @k (or {@k}) is the value of the k-th call;
  • <table> (a cell, or sum / avg / count / max / min / argmax / argmin over a column or a row of a table in the source), <cmp> (comparisons and and / or / not, a yes/no), <scale> (thousand ↔ million, percent ↔ fraction, seconds ↔ hours ↔ days ↔ weeks, months ↔ years, km/h ↔ m/s) and <count> (how many items match, membership) — typed calls computed by code the same way (table_calc, cmp_calc, scale_calc, count_calc on their own). MODES lists every call kind with its slot grammar, result type and validator; register_mode adds one.

fill replaces the pointers with the exact values (an input numeral as the input writes it: "$1,234", "12.5%", "4.2 million") and each call with its result; check reads every number left in a text and names its source — an input numeral, a call's result, or an allowed constant — or marks it unsourced; NumberGuard runs both with one configuration and a fail-closed mode (raise, or return a refusal) or a report mode; NumberGuard.run_with_retry asks your model once more with the list of what failed (any callable: no LLM client here).

from solvi.experimental.numguard import NumberGuard, annotate

source = "Revenue was $1,250 million in 2023 and $1,100 million in 2022. Invoice dated 2024-03-01, net 45 days."
prompt_text, _ = annotate(source)       # "Revenue was [1]$1,250 million in [2]2023 ..." for the prompt
reply = "Revenue grew by <calc>[1] - [3]</calc> million (<calc>(@1) / [3] * 100</calc>%). Due <date>[5] + [6]D</date>."
out = NumberGuard().run(reply, source)
out.text            # 'Revenue grew by 150 million (13.63636364%). Due 2024-04-15.'
out.verdict.ok      # True: every number has a source

What it gives is provenance, not correctness: a pointer at the wrong input numeral is filled and passes. Numbers written as words ("three") and numbers inside identifiers ("A-10457", "COVID-19", "v2") are not checked (identifiers can be checked against the input verbatim with ids="check"). Only integers 0–12 and 100 pass as constants by default (SMALL), as written counts without decimals, currency, percent or scale; MEASURED_CONSTANTS is the wider set the measurements used (integers 0–100 and a few common factors), opt-in, and years() or your own set or predicate extend either. pi (PI, 28 digits, an approximation) is opt-in the same way: constants=SMALL | {PI}.

SMALL module-attribute

SMALL = frozenset(Decimal(i) for i in range(13)) | {Decimal(100)}

Integers 0–12 and 100: counts, months, a percent base. The default constants, for literals in calls and free numbers alike.

MEASURED_CONSTANTS module-attribute

MEASURED_CONSTANTS = frozenset(Decimal(i) for i in range(101)) | frozenset(Decimal(x) for x in (365, 360, 52, 1000, 10 ** 6, 10 ** 9))

Integers 0–100, days in a year (365, 360), weeks (52), 1,000, a million, a billion: the set the measurements behind the docs used, for calc literals and free constants alike. Opt-in (constants=MEASURED_CONSTANTS): an invented number from 13 to 99 passes it.

PI module-attribute

PI = Decimal('3.141592653589793238462643383')

π to 28 significant digits — an approximation (the error is below 3e-28), not an exact value. pi in a <calc> expression (and as a value in the typed calls) stands for it. Not in SMALL or MEASURED_CONSTANTS: inside fill it passes only when it is an allowed constant (constants=SMALL | {PI}) or with literals="any".

MODES module-attribute

MODES = MappingProxyType(_MODES)

The call kinds fill runs, by name (read-only; register_mode adds one): the single list of typed calls.

Numeral dataclass

Numeral(start: int, end: int, surface: str, kind: Literal['number', 'date', 'id'], number: Decimal | None = None, scale: int = 1, unit: str = '', date: object = None)

A number as a text writes it.

number is the written number with its sign ("(12.3)" and "-12.3" → -12.3; "4.2 million" → 4.2); scale the multiplier of a scale word or suffix (1,000 for "thousand" / "k"); value their product. unit is "%", a currency sign, an ordinal suffix or "". A parenthesised number right after another number on its line ("16.6% (18.8%)": a comparative, or an accounting negative — the text does not say) keeps its parentheses and reads unsigned, the sign not guessed (what solvi.solutions.document abstains on, as a reader without a decision). A date has kind="date" and date (a datetime.date, or (month, day) when the text gives no year). An identifier ("A-10457", "COVID-19") has kind="id" and is not a number.

plain

plain() -> str

The surface as prose would write it: a negative in parentheses as "-12.3". A comparative "(18.8%)" — parentheses after another number on its line, whose sign was never read — keeps them.

ProgramError

Bases: ValueError

A call that cannot be computed, or a pointer that does not resolve: the message says why, for a retry.

Cell dataclass

Cell(table: int, row: int, col: int, row_label: str, col_label: str, surface: str)

A table cell a result came from: table table<table>, its 1-based data row and column (the header row and the row-label column are not counted), their labels and the cell as written.

Table dataclass

Table(rows: list, row_labels: bool = True, scale: int = 1, caption: str = '', currency: str = '')

A table for <table> and <count>: rows row-major, the header row first (the column labels); with row_labels=True the first column holds the row labels. scale multiplies the money cells under norm (1,000 for a caption "in thousands" or a header "SEK million"; percent cells are not scaled) and is applied to the cells by check; currency is what a header "SEK million" / "£ million" states about them ("" when it does not). Cells are text ("$1,250", "(3.2)", "12.5%", "—"), read like the input's numerals: a parenthesised number is negative, a dash or "n/a" is blank. Short rows are padded with blanks.

columns property

columns: list[str]

The column labels (without the row-label column's header).

labels property

labels: list[str]

The row labels ([] without a row-label column).

shape property

shape: tuple[int, int]

(data rows, data columns).

text

text(row: int, col: int) -> str

The cell at 1-based data row and column, as written (column 0: the row label).

describe

describe(name: str = 'table1') -> str

The table's pointers for a prompt: its rows and columns with their numbers and labels.

CallContext dataclass

CallContext(inputs: list, calls: list, tables: list, k: int, literal_ok: Callable[[Decimal], bool])

What an extra mode's executor sees inside fill: the input's numerals ([k] is inputs[k - 1]), the earlier calls (@j is calls[j - 1], a CallResult; check its error), the tables, this call's number k, and literal_ok(Decimal), the literal policy of this fill.

Mode dataclass

Mode(name: str, slots: str, result: str, validator: Callable[[object], bool], executor: Callable | None, description: str = '', builtin: bool = False)

A call kind <name>…</name>: its slot grammar in short, the type of its result, validator(value) -> bool on a result, the executor and a one-line description. A built-in mode's executor is its standalone function (calc, date_calc, table_calc, cmp_calc, scale_calc, count_calc); an extra mode's is executor(expr, ctx: CallContext) -> value, raising ProgramError for a bad call, or None (the mode is described but has no executor: its calls fail).

CallResult dataclass

CallResult(k: int, kind: str, expr: str, value: object = None, shown: str = '', error: str = '', cells: tuple = ())

The k-th call: its kind, the expression as written, the value (a Decimal, a date, an int of days or a count, a weekday or a table label, a bool) and the text it was shown as; error instead of a value when it failed; cells, the table cells a <table> / <count> value (or a <cmp> / <scale> on one) came from.

Inserted dataclass

Inserted(start: int, end: int, source: str, index: int)

A span of the filled text written by code: from the input numeral [index] (source="input") or from call @index (source="tool").

Filled dataclass

Filled(text: str, calls: list[CallResult], inserted: list[Inserted], errors: list[str], source_numerals: list[Numeral], tables: list = list())

fill's result: the final text, the calls, the spans code wrote, and every error (an empty list = clean).

NumberCheck dataclass

NumberCheck(surface: str, start: int, end: int, kind: str, source: Source, index: int = 0, how: str = '', cells: tuple = ())

One number of the checked text and where it comes from.

source: "input", "tool", "constant" or "unsourced"; index: the input numeral [k] or call @k (0 otherwise); how: "pointer" (written by code), "exact", "sign" (equal but for the sign), "scale_word" (equal once a scale word is applied: "4.2 million" against "4,200,000"), "rescaled x1000" / "rescaled /1000" (and 10^6, 10^9: digits off by a whole scale, only with rescale=True), "rounded" (only with rounding=True), "date_year" (the year of a source date), "verbatim" (an identifier found in the input), "table cell" (a cell of a table passed as tables), or "" when unsourced. A number from a <table>, <cmp>, <scale> or <count> call has the mode as how ("table", ...) where a calc or date call has "pointer" or "exact", and cells, the table cells it came from.

Verdict dataclass

Verdict(numbers: list[NumberCheck], errors: list[str] = list())

Every number of a text with its source; ok when none is unsourced and the program had no error.

feedback

feedback() -> str

What to tell the model in a retry: each error and each unsourced number.

UnsourcedNumbers

UnsourcedNumbers(outcome: 'Outcome')

Bases: ValueError

Raised by a fail-closed guard (on_fail="raise"): outcome holds the verdict and the filled text.

Outcome dataclass

Outcome(text: str, ok: bool, refused: bool, verdict: Verdict, filled: Filled, attempts: list['Outcome'] = list())

A guarded answer: text (the filled text, or the refusal when refused), ok, refused, the verdict, the calls and, after run_with_retry, every attempt (attempts, this one last).

NumberGuard dataclass

NumberGuard(on_fail: Literal['report', 'refuse', 'raise'] = 'refuse', refusal: str = REFUSAL, constants: object = SMALL, literals: Literal['constants', 'input', 'any'] = 'constants', signed: bool = False, rescale: bool = False, rounding: bool = False, ids: Literal['skip', 'check'] = 'skip', answer_marker: str | None = None, digits: int = 10, places: int | None = None, grouping: bool | None = None, date_format: str | None = None, show_calls: bool = True, booleans: tuple = ('yes', 'no'), decimal: str = '.', locale: str | None = None)

fill and check with one configuration and a mode for a failure.

on_fail: "report" returns the outcome as it is (the verdict says what failed), "refuse" returns refusal as the text, "raise" raises UnsourcedNumbers. The other fields are fill's and check's options.

check

check(text: str, source: str, results: Iterable = (), **kw) -> Verdict

Check a text that was not filled (a model typing its numbers, a native tool caller's answer).

run

run(text: str, source: str, results: Iterable = (), tables=None) -> Outcome

Fill the text, check every number left in it, apply on_fail. results: values from tools run outside (a native tool call's output) that also count as sources. tables: tables given apart from source (None: the pipe tables of source, their units header applied to the cells in the check).

run_with_retry

run_with_retry(generate: Callable[[str | None], str], source: str, retries: int = 1, results: Iterable = (), tables=None) -> Outcome

Ask, guard, and on a failure ask again with what failed. generate(feedback) returns the model's text: it is called with None first, then with Verdict.feedback() (the errors and the unsourced numbers) up to retries more times. The last attempt goes through on_fail; Outcome.attempts holds them all.

years

years(lo: int = 1900, hi: int = 2100) -> frozenset

The years lo..hi as constants: constants=SMALL | years(2000, 2030).

numerals

numerals(text: str, *, decimal: str = '.', locale: str | None = None) -> list[Numeral]

Every number, date and identifier with digits in text, in order. Dates: ISO "2024-03-15", "15.03.2024" / "15/03/2024" (day first), "March 15, 2024", "15 March 2024", "March 15" (no year). Numbers: a sign or a negative in parentheses, a currency sign, comma thousands, a decimal point, then "%" / "percent", an ordinal suffix ("3rd"), a scale suffix ("12k", "3bn") or a scale word ("4.2 million", bound to the number on its own line — "£1,000\nmillion" is the number £1,000 and the word on the next line). A number in parentheses right after another number on its line ("16.6% (18.8%)" — a comparative or a negative, the text does not say) keeps its parentheses and reads unsigned. A token where digits follow letters ("A10", "v2.1", "COVID-19", "1Z999AA1") is an identifier; a number followed by a unit ("5km") is a number. Unsupported scale words (English -illion words and Russian scale abbreviations) and comma groups that are not comma thousands raise ValueError; no decimal-comma interpretation is guessed. Explicit decimal="," instead reads decimal commas and space-grouped thousands; dot decimals and mixed punctuation then raise ValueError. Surfaces and spans remain those of the original text. Call expressions retain their Python-style decimal point syntax.

annotate

annotate(text: str, *, decimal: str = '.') -> tuple[str, list[Numeral]]

The text with [k] before each numeral (numbers and dates, not identifiers), for the model's prompt, and the numerals in that order: [1] is numerals[0].

calc

calc(expr: str) -> Decimal

Exact arithmetic on number literals: + - * / ^ (or **), parentheses, unary minus, a postfix % (divides by 100); Decimal with 50 digits. An integer power up to 1,000 is exact; a fractional one needs a positive base. pi is PI (28 digits, an approximation). Raises ProgramError with a readable reason.

date_calc

date_calc(expr: str) -> dt.date | int | str

Calendar arithmetic: DATE ((+|-) N(D|W|M|Y))* → a date (a month added to Jan 31 gives the month's last day); DATE - DATE → days (an int; DAYS in front is allowed); WEEKDAY DATE ... → the weekday's name. Dates are ISO (YYYY-MM-DD) or DD.MM.YYYY. Raises ProgramError.

parse_tables

parse_tables(text: str, row_labels: bool = True) -> list[Table]

The pipe (markdown) tables of a text, in order: table1, table2, ... A run of lines that start with "|" or hold two "|" or more is a table; a |---|---| line is skipped; the first row is the header. A caption "in thousands" / "in millions" / "in billions" / "in trillions" (or "'000", "000s"), or a header that pairs a currency with the scale — "SEK million", "£ million", "In $ millions" — in the last two non-blank lines of the three above the table or in its header row sets Table.scale (and Table.currency for the currency + scale form).

table_calc

table_calc(expr: str, tables, *, cells: bool = False)

A <table> call on its own. tables: a text with pipe tables, a Table, or a list of tables (each a Table or rows, the header first). [tableN:] cell(row, column[, norm]); [tableN:] sum|avg|max|min|count|argmax|argmin([column] col[, a..b][, where <condition>][, norm]) over a column (a..b: rows) and ...(row r[, a..b][, where <condition>][, norm]) over a row (a..b: columns). A row or column is a 1-based number or a "label"; a condition compares a column (for a row aggregate: another row, in each column) with a value (< <= > >= == != between in), joined by and / or / not. Values are the written numbers; norm multiplies by each cell's scale word and the table's caption scale. Returns the value (a Decimal, an int for count, a label for argmax / argmin — the row's over a column, the column's over a row — a date or a string for a cell), or (value, cells) with cells=True. Raises ProgramError.

cmp_calc

cmp_calc(expr: str) -> bool

A <cmp> call on its own: numbers, ISO dates or "strings" compared with < <= > >= == != (strings only == and !=, case-insensitive), x between a and b (inclusive), x [not] in (a, b, ...), joined by and / or / not. Raises ProgramError for a number against a date, two different units ("%" against "$") or scale words.

scale_calc

scale_calc(expr: str) -> Decimal

A <scale> call on its own: VALUE [UNIT] to UNIT [rule 360|365|366]. Units: units, thousand, million, billion, trillion (k, mn, bn, tn); percent (%), fraction, bp; seconds (s, sec), minutes (min), hours (h, hr), days, weeks; months, quarters, years; m/s, km/h (kmh, kph), km/s, mph. Seconds to weeks against months, quarters and years need a rule (360: a month is 30 days; 365 or 366: a year). Exact Decimal, one division at the end (1 m/s is 3.6 km/h exactly; 1 km/h to m/s is 1000/3600 to 50 digits); raises ProgramError.

count_calc

count_calc(expr: str, tables=None)

A <count> call on its own: (a, b, ...) [if <condition>] or [tableN:] column(col[, a..b][, where ...]) [if <condition>] (or row(r[, a..b][, where ...])) → how many items (non-blank cells) match (an int); x [not] in <list> → a bool. Raises ProgramError.

register_mode

register_mode(name: str, executor: Callable | None = None, *, slots: str = '', result: str = '', validator: Callable[[object], bool] | None = None, description: str = '', replace: bool = False) -> Mode

Add a call kind <name>…</name> that fill runs with executor(expr, ctx) (see Mode, CallContext); its calls share the @k numbering, a result failing validator (default: any) is an error, and check reports its numbers with the name as how. executor=None registers a described mode whose calls fail. A built-in mode cannot be replaced; an extra one only with replace=True.

unregister_mode

unregister_mode(name: str) -> None

Remove an extra mode (a built-in one stays).

fill

fill(text: str, source: str | list[Numeral], *, answer_marker: str | None = None, literals: Literal['constants', 'input', 'any'] = 'constants', constants=SMALL, digits: int = 10, places: int | None = None, grouping: bool | None = None, date_format: str | None = None, show_calls: bool = True, tables=None, booleans: tuple[str, str] = ('yes', 'no'), decimal: str = '.', locale: str | None = None) -> Filled

Run the calls of a model's text and write in the exact values.

locale="ru" renders Russian decimal commas, scale/currency abbreviations and month names. decimal always selects the SOURCE reader (default English decimal point); output locale never reinterprets source punctuation. Use matching check(..., locale="ru", decimal=".") to read the Russian output against English input without trusting inserted spans. None/"en" retains prior output; unsupported locale names raise ValueError. No process-wide locale is changed.

source is the input text (or its numerals). Each <calc> / <date> call is computed in order ([k] is the k-th input numeral's written number — "4.2 million" gives 4.2, "12.5%" gives 12.5 — or its date; @k an earlier call's value) and replaced by its value where it stands (show_calls=False removes it: refer to it as @k). Then [k] and @k / {@k} in the prose are replaced: an input numeral as the input writes it (a currency sign or "%" the text repeats around the pointer is not doubled), a result with digits significant digits or places decimals, comma-grouped when grouping (None: when the input groups its thousands), a date as ISO or date_format (strftime). With answer_marker ("ANSWER:") only the text after its last occurrence is returned; the calls before it still run.

A literal number inside a call must be a constant (literals="constants"), or also an input numeral written verbatim ("input"), or anything ("any"). A pointer out of range, an unknown or failed @k, a malformed call or a missing marker is an error in Filled.errors; its pointer is left as written.

<table>, <cmp>, <scale>, <count> and modes added with register_mode share the numbering (see table_calc, cmp_calc, scale_calc, count_calc, MODES); [k] and @k work inside them, a literal number in a value position follows literals (a literal date needs literals="any", or "input" and an input date). tables: the tables they read (a Table, a list of them or of rows; None: the pipe tables of source, table1 first). A bool is shown as booleans ("yes" / "no").

check

check(text: str, source: str | list[Numeral] = '', results: Iterable = (), *, inserted: Iterable[Inserted] = (), constants=SMALL, signed: bool = False, rescale: bool = False, rounding: bool = False, ids: Literal['skip', 'check'] = 'skip', errors: Iterable[str] = (), tables=None, decimal: str = '.', locale: str | None = None) -> Verdict

Name the source of every number in text.

locale="ru" selects a Russian OUTPUT reader (decimal comma, space/NBSP grouping, Russian scales, currency and months). decimal independently selects source/results/table punctuation. Localized inserted spans are re-read and matched against values, never trusted.

A number is sourced when code wrote it (inserted, from fill), when it equals an input numeral (sign ignored unless signed; a scale word applied: "4.2 million" equals "4,200,000"), a result in results (CallResults, or plain values: numbers, dates, or strings such as a native tool's output), or a constant (constants: a set of numbers or a predicate on a Decimal). A date is sourced when it equals an input date or a result date; a year when it is the year of one. With rescale=True a number off by a whole scale (8,220 against 8,220,000) is sourced as "rescaled"; with rounding=True a number equal to a source rounded to its own decimals (two significant digits at least) as "rounded". Identifiers are skipped, or with ids="check" must appear verbatim in the input. tables: tables given apart from source (a Table, a list, or a text with pipe tables); a number equal to one of their cells is an input with index 0 and how="table cell". A table's units header — "in thousands" or "SEK million" / "£ million" — applies to its cells: a cell 18,742 under a "SEK million" header sources "18,742 million" (a percent cell is not scaled). The tables inside a source text are already its numerals; their header scale is applied to them the same way, as an addition (a number quoting a cell as written still matches the numeral).