Public benchmarks

Protocol

Every example sees the full candidate label set for that dataset. Label wording is frozen to the publishers’ category strings. Calibration / threshold selection never touches the official test set. Two columns: current checkpoint (OneSystem v0.3.0 as published) and domain-adapted (fine-tuned on that dataset’s train split). How to train your own: see Domain-specific model in the cookbook.

BANKING77

Fine-grained banking intents. CC BY 4.0 · 3,080 test · 77 labels. source

CLINC150

150 in-scope intents + out-of-scope. 4,500 + 1,000 test. source

HWU64

Home-assistant style intents · 64 labels. source

GoEmotions

Multi-label emotion tagging · 27 emotions (neutral-only rows skipped). source

What these numbers do not claim

• They are not “exact performance” of OneSystem on every task — only on these datasets.
• They do not measure LLM-calls-avoided end-to-end quality; that needs a workflow eval.
• They are short-utterance benchmarks; the advertised 8,192-token window is a separate stress test.
• Domain-adapted scores require fine-tuning on that dataset’s train split — see the cookbook.