Skip to content

solvi.core.costs

Run-time costs: CostBook (system.cost_book), the dollars of recorded model calls (price_of, cost_of), and Budget / Cost / BudgetStop — the limits and costs solvi.core.dispatch, solvi.core.slow.generate and solvi.core.slow.refine share (defined in solvi.core.dispatch until 1.0, which re-exports them); MeasuredCosts (System(cost_policy="measured")) was removed in 1.0. Part of solvi.learned up to 0.7; the learned order and producer policy are in solvi.core.plan.strategist.

Run-time costs: CostBook, the moving average of each part's run time (ms) that every System keeps (system.cost_book) and updates after every ask; the model calls a trace record holds and their dollars (_usages, price_of, recorded_calls, cost_of); Cost (what a decision or a part of one cost: dollars, model calls, milliseconds, tokens) and Budget (limits on them) with BudgetStop — shared by solvi.core.dispatch, solvi.core.slow.generate and solvi.core.slow.refine (defined in solvi.core.dispatch until 1.0, which re-exports them). Moved out of solvi.learned in 0.8 (nothing here is learned). MeasuredCosts (System(cost_policy="measured")) was removed in 1.0: it showed no measured benefit.

CostBook

CostBook(alpha=0.3)

Moving average of each part's run time in ms (exponential, alpha).

observe

observe(name, ms)

One run of the part named name took ms.

get

get(part, default=1.0)

The moving average for a part (a catalog part, or its name — what observe takes); else the part's declared cost, else default.

Budget dataclass

Budget(usd: float | None = None, calls: int | None = None, ms: float | None = None, tokens: int | None = None)

Limits on model calls: dollars, calls, milliseconds and tokens (input + output) — None: no limit on that one. The same Budget is per decision or in total, wherever it is given (solvi.core.dispatch, solvi.core.slow.generate, solvi.core.slow.refine).

over

over(cost, plus=None)

The first limit that cost (+ plus, an expected further cost) goes over → its reason, or None. A cost without dollars (no price known) is never over a limit in dollars: give a price with a budget in dollars.

used_up

used_up(cost)

The first limit cost has reached → its reason, or None.

Cost dataclass

Cost(usd: float | None = 0.0, calls: int = 0, ms: float = 0.0, input_tokens: int = 0, output_tokens: int = 0)

What a decision (or a part of one) cost: dollars (None when no price is known), model calls, milliseconds and tokens.

BudgetStop

Bases: RuntimeError

Stopped between two steps: what is left of the budget would not cover the next one (a slow path's next round, a refinement's next round, a generator's next request).

price_of

price_of(price, calls, recorded=False)

Dollars of recorded calls: price (dollars per million input and output tokens), a function (model, usage) → dollars, or None (unknown → None). recorded=True: with no price, the dollars the calls recorded themselves (a generator built with price= writes them in each call's usage as "usd") when every call has them.

recorded_calls

recorded_calls(responses, generated=())

The model calls recorded in responses' traces (and in generator records outside a trace, as refine keeps the proposer's) → [(model, usage)].

cost_of

cost_of(responses, price, ms=0.0, generated=(), recorded=False)

The Cost of what responses (and generator records) recorded, with ms as the time. recorded=True: without a price, the dollars the calls recorded (see price_of).