Glossary
Calibration
A model is calibrated when its predicted probabilities match observed frequencies: of actions scored 0.7 unsafe, about 70% really are.
Checks run on our GPUs in Switzerland. Request payloads are not stored.Security
What Calibration means
A classifier is well calibrated when its scores can be read as probabilities. If a calibrated guard assigns P(unsafe) = 0.2 to a thousand actions, about two hundred of them should in fact be unsafe. Calibration is separate from ranking quality: a model can rank well (AUROC high) and still be badly calibrated.
Calibration matters because it is what makes a threshold meaningful. “Ask a human when P(unsafe) ≥ 0.3” only has a predictable effect if 0.3 means roughly what it says. Guo et al. (2017) showed that modern neural networks are often overconfident, and that simple post-hoc fixes such as temperature scaling help.
Calibration is also local to a data distribution. Scores calibrated on one set of agents and tools can drift on another (see distribution shift), which is why it is worth checking and, if needed, re-fitting thresholds on a sample of your own labelled actions.
Related terms
How Calibration connects to the rest of the vocabulary.