Glossary
Cognitive Bias
Systematic patterns in human thinking that lead to errors in judgment, often unconsciously — including biases that can affect annotation, evaluation, and behavioral design.
Cognitive biases are the predictable ways human reasoning goes astray — anchoring on the first piece of information received, preferring the status quo, overweighting vivid examples, or being influenced by how a question is framed rather than its substance. These biases matter in AI development because they affect every human-in-the-loop process: annotators’ labels, evaluators’ ratings, users’ feedback, and designers’ intuitions are all susceptible to systematic cognitive errors. For example, the “halo effect” can cause annotators to rate a well-written response as more accurate than it actually is. For behavior architects, understanding cognitive biases helps in designing annotation tasks, interpreting evaluation results, and avoiding the trap of trusting your own behavioral intuitions more than the data warrants.