Probabilities
Built and tested, not yet deployed. Early access.
On this page
What the number is
- A predicate answered
true: the probability of true. Answeredfalse: one minus it. - A choice: the chosen option’s probability, or
nullif the model gave no distribution (thenabstainis true). - A score: the most likely level’s probability, which need not be the level returned.
- Settled by a rule: 1.
Calibration isn’t fitted yet
The code can recalibrate a decision’s probabilities by temperature scaling, fitted on held-out labelled cases. It would change the probability, and so whether it clears your line, but never which answer wins.
No decision has a fitted calibration. One will be set only from a published reliability table.
Planned. Per-decision calibration.
What we don’t claim
That an answer at 0.9 is right nine times in ten. The eval doesn’t measure calibration yet.
# Probabilities Built and tested, not yet deployed. Early access. ## What the number is - A predicate answered `true`: the probability of true. Answered `false`: one minus it. - A choice: the chosen option’s probability, or `null` if the model gave no distribution (then `abstain` is true). - A score: the most likely level’s probability, which need not be the level returned. - Settled by a rule: 1. ## Calibration isn’t fitted yet The code can recalibrate a decision’s probabilities by temperature scaling, fitted on held-out labelled cases. It would change the probability, and so whether it clears your line, but never which answer wins. No decision has a fitted calibration. One will be set only from a published reliability table. > Planned. Per-decision calibration. ## What we don’t claim That an answer at 0.9 is right nine times in ten. The eval doesn’t measure calibration yet. Source: /api/docs/probabilities