Correcting Measurement Error Bias in Conjoint Survey Experiments (Harvard Experiments Working Group)

Conjoint survey designs are spreading across the social sciences due to their unusual capacity to estimate many causal effects from a single randomized experiment. Unfortunately, by their ability to mirror complicated real-world choices, these designs often generate substantial measurement error and thus bias. We replicate both the data collection and analysis from eight prominent conjoint studies, all of which closely reproduce published results, and show that a large proportion of observed variation in answers to conjoint questions is effectively random noise. We then discover a common empirical pattern in how measurement error appears in conjoint studies and, with it, introduce an easy-to-use statistical method to correct the bias.

Based on a paper with Katherine Clayton, Yusaku Horiuchi, Aaron Kaufman, and Mayya Komisarchik; see GaryKing.org/conjointE.