Ulric B. and Evelyn L. Bray Social Sciences Seminar

Abstract: We study whether and how people respond to a fundamental tension in model selection: simple models generate high bias and low variance, whereas complex models generate low bias and high variance. We report results from an experiment in which participants rely on a limited sample of observations to construct mental models describing the relationship between two variables and are rewarded based on the out-of-sample predictive accuracy of these models. In aggregate, behavior is broadly consistent with people acting as intuitive modelers who respond to the simplicity–complexity tradeoff. Model complexity varies systematically with the size and nonlinearity of the sample, but willingness to improve in-sample fit declines when doing so requires more complex models. The main deviation from optimal benchmarks is in the direction of underinference, driven by both a preference for overly simple models and mistakes in model construction conditional on the chosen complexity level.

Joint with Emanuel Vespa and Sevgi Yuksel.