solvi.core.costs¶
Run-time costs: CostBook (system.cost_book), the dollars of recorded model calls (price_of, cost_of), and
Budget / Cost / BudgetStop — the limits and costs solvi.core.dispatch, solvi.core.slow.generate and solvi.core.slow.refine share
(defined in solvi.core.dispatch until 1.0, which re-exports them); MeasuredCosts
(System(cost_policy="measured")) was removed in 1.0. Part of
solvi.learned up to 0.7; the learned order and producer policy are in solvi.core.plan.strategist.
Run-time costs: CostBook, the moving average of each part's run time (ms) that every System keeps (system.cost_book)
and updates after every ask; the model calls a trace record holds and their dollars (_usages, price_of, recorded_calls,
cost_of); Cost (what a decision or a part of one cost: dollars, model calls, milliseconds, tokens) and Budget (limits
on them) with BudgetStop — shared by solvi.core.dispatch, solvi.core.slow.generate and solvi.core.slow.refine (defined in solvi.core.dispatch until
1.0, which re-exports them). Moved out of solvi.learned in 0.8 (nothing here is learned). MeasuredCosts
(System(cost_policy="measured")) was removed in 1.0: it showed no measured benefit.
CostBook ¶
Budget
dataclass
¶
Budget(usd: float | None = None, calls: int | None = None, ms: float | None = None, tokens: int | None = None)
Limits on model calls: dollars, calls, milliseconds and tokens (input + output) — None: no limit on that one. The same Budget is per decision or in total, wherever it is given (solvi.core.dispatch, solvi.core.slow.generate, solvi.core.slow.refine).
over ¶
The first limit that cost (+ plus, an expected further cost) goes over → its reason, or None. A cost
without dollars (no price known) is never over a limit in dollars: give a price with a budget in dollars.
Cost
dataclass
¶
Cost(usd: float | None = 0.0, calls: int = 0, ms: float = 0.0, input_tokens: int = 0, output_tokens: int = 0)
What a decision (or a part of one) cost: dollars (None when no price is known), model calls, milliseconds and tokens.
BudgetStop ¶
Bases: RuntimeError
Stopped between two steps: what is left of the budget would not cover the next one (a slow path's next round, a refinement's next round, a generator's next request).
price_of ¶
Dollars of recorded calls: price (dollars per million input and output tokens), a function (model, usage) → dollars, or None (unknown → None). recorded=True: with no price, the dollars the calls recorded themselves (a generator built with price= writes them in each call's usage as "usd") when every call has them.
recorded_calls ¶
The model calls recorded in responses' traces (and in generator records outside a trace, as refine keeps the proposer's) → [(model, usage)].
cost_of ¶
The Cost of what responses (and generator records) recorded, with ms as the time. recorded=True: without a
price, the dollars the calls recorded (see price_of).