Evaluation
The Understanding API hasn’t been run on our eval yet, and it gets no number until it is.
Built and tested, not yet deployed. Early access.
On this page
What the eval tests
Our eval asks AI models five narrow money decisions: is this merchant a gambling business, which card charge paid for this receipt, what total does this receipt email show, which transaction refunds this purchase, and is this kind of spending essential or discretionary.
Our engine’s baseline
On our own four labelled tests, TUC's rule-based engine, which uses no AI model, averaged 74.9% (95% interval 71.7% to 78.0%). On spend type, the engine agreed with our own written rule on 72.9% of 284 merchant codes (67.6% to 78.2%).
Measured on 2026-10-28, on tests we built. The spend-type task measures agreement with a rule we wrote, not accuracy against an outside standard, so it is shown on its own and left out of the overall score.
Conflict of interest
TUC built these tests and sells the Understanding engine and API that are scored here. The data, prompts, scoring code and every model's raw answers on the public tests are published so anyone can rerun or challenge them.
Read the eval
# Evaluation The Understanding API hasn’t been run on our eval yet, and it gets no number until it is. Built and tested, not yet deployed. Early access. ## What the eval tests Our eval asks AI models five narrow money decisions: is this merchant a gambling business, which card charge paid for this receipt, what total does this receipt email show, which transaction refunds this purchase, and is this kind of spending essential or discretionary. ## Our engine’s baseline On our own four labelled tests, TUC's rule-based engine, which uses no AI model, averaged 74.9% (95% interval 71.7% to 78.0%). On spend type, the engine agreed with our own written rule on 72.9% of 284 merchant codes (67.6% to 78.2%). Measured on 2026-10-28, on tests we built. The spend-type task measures agreement with a rule we wrote, not accuracy against an outside standard, so it is shown on its own and left out of the overall score. ## Conflict of interest TUC built these tests and sells the Understanding engine and API that are scored here. The data, prompts, scoring code and every model's raw answers on the public tests are published so anyone can rerun or challenge them. ## Read the eval [See the eval](/eval)[How it’s run](/eval#method) Source: /api/docs/evaluation