solvi.many¶
A choice among many options: a shortlist, then the decider reads the few that remain.
A choice among many options: dozens or hundreds of candidates, more than a decider reads in one pass.
A decider reads the question — its task, options and descriptions — and the input in one sequence of max_len tokens. Thirty catalog rows as options no longer fit ("task and options do not fit"), and twenty that do fit leave the input a few dozen tokens. An agent's step — the actions in a room, the elements of a page, the rows of a catalog — has that many candidates, and they change with every step.
from solvi.many import Many, decide_many
d = decide_many(model, text, "Which action achieves the goal?", actions, many=Many(query=goal))
d.value # one of `actions`
d.extra["many"] # {"mode": "shortlist", "considered": 8, "of": 45, "unconsidered": 37, "calls": [...], ...}
Modes (Many(mode=...)):
direct one ordinary decision over all the options — when they fit
shortlist a selector ranks the options against a query (BM25 over the options' labels and descriptions, or your
function), the decider chooses among the best k. One call. The options left out were not considered: the
record says how many, and when the first one left out scores nearly as well as the last one kept
(gap), the decision escalates
tournament the options in blocks of block: the decider chooses in each block, the winners meet in the next round,
the last round decides. Every option is considered; about N / (block − 1) calls. When a block's
call escalates, its winner still goes on, and the final decision escalates too
auto direct when the question fits and leaves the input at least min_input tokens (default: half of
max_len); else shortlist when there is a selector, else tournament
Whatever the mode, extra["many"] records the mode applied, how many options were considered of how many, and every
model call (its options, its answer, its probabilities) — a decision that replays from its inputs, since the selector
and the model are deterministic. The decision's probs cover the options of the last call only.
What it does not do. A guarantee calibrated on one set of options (act_guard, conformal) does not carry over to
options that change: a threshold holds for the question it was calibrated on. In shortlist mode the right option may
not be among the k (the selector's recall bounds the accuracy), which unconsidered and the gap escalation make
visible, not impossible. Expect direct, where the options fit, to be the most accurate, then shortlist, then the
tournament (more calls, more chances to drop the right option); scoring each option alone is not offered. Where facts
decide (price within budget, the category), code should narrow the candidates before the model is asked: a model
choosing among many similar catalog rows by their text alone does poorly.
Many
dataclass
¶
Many(mode: str = 'auto', k: int = 8, block: int = 10, query: str | None = None, selector: str | Callable | None = 'bm25', min_input: int | None = None, gap: float = 0.1, max_options: int = 2000)
How to choose among many options (see the module docstring). k: the shortlist's size. block: the options per tournament call. query: what the selector searches by (default: the input's text — a short query, the goal or the request, finds better). selector: "bm25", a function (query, labels, texts) → one score per option, or None (no shortlist: auto goes to the tournament). min_input: the tokens the input must be left with for a direct decision (None: half of max_len). gap: shortlist escalates when (score of the last kept − score of the first left out) / the best score is below it and the one left out matched the query at all (BM25: a score above 0; your selector: scores are measured from the lowest one given, so negative scores work, and a tie at the cut is a close cut). max_options: more is an error at once.
bm25_scores ¶
BM25 of each option (label and description) for the query → one score per option, in the options' order.
fits ¶
Does the question with all its options fit one pass, and what is left for the input → {"question_tokens", "input_budget", "max_len", "min_input", "fits"} (token counts are the model's own, or approximate when the model has no tokenizer: an LLM, a hosted decision model).
decide_many ¶
Choose one of many options → Decision (value: one of the options; probs: over the options of the last call; extra["many"]: what was done). model: a DecideModel (a local checkpoint, solvi.llm, solvi.systemone). options: a list, or {option: description}. many: a Many (default: Many()). spec: passed to model.decide (escalate_below, ...).