- We recommend asking the self-assessment early in the survey and the vignettes some time later.
- The vignettes should not be ordered in the survey according to your understanding of their actual value. One problem with this approach is that respondents will tend to try to make their answers consistent over the set of responses, or may use simple heuristics such as placing one vignette in each response category. Both of these outcomes would compromise the requirement of response consistency between vignette ratings and self-assessments. Further, respondents may have different abilities to remember all the previous vignettes, and those who pick the wrong or an unusual value for the first vignette may feel locked in for the rest. The result will be essentially constant responses for the rest that do not discriminate well and so provide little information.
- We find that vignettes are best presented to the respondent in randomized or mixed order. In addition, if you have more than one set of vignettes, it is helpful to shuffle the two sets together. Since a separate question follows each vignette, this does not cause respondents to have any additional trouble in understanding the survey instrument.
- Why not ask the self-assessment question after the vignettes? This would lead to an undesirable priming effect, and a different one depending on the order in which the vignettes were presented. Although we might like respondents to read, internalize, and remember the set of vignettes prior to being asked the self-assessment question, this is infeasible for most respondents.
Anchoring Vignettes
- Frequently Asked Questions
- About this document
- Introduction
- Statistical Analyses with Anchoring Vignettes
- Survey Instrumentation
- Are universally applicable, culture-independent survey questions possible?
- Can I use anchoring vignettes if I don't have variables to predict the thresholds?
- Can I use anchoring vignettes to understand why respondents understand survey questions in such different ways?
- Can we avoid DIF by using a panel design, without vignettes?
- Do I need one vignette for each response category?
- Doesn't Anchoring Vignettes merely move the problem of coming up with DIF-free survey questions back one level (from self-assessments to vignettes), and so in the end you have the same problem?
- How is this strategy affected by the finding in social-psychology that assessments of oneself and others differ?
- How many anchoring vignettes should I ask for each concept I want to measure?
- How much expense will anchoring vignettes add?
- If I have a direct physical measurement, such as a medical test, do I need anchoring vignettes?
- In what order should vignettes and self-assessment questions be asked?
- Is there a simpler way of asking questions so we can avoid any statistical analysis?
- Should I ask vignettes questions of all people or a random subset?
- Should the vignette describe the age, sex, etc., of the hypothetical person? Should it be self-referential?
- What are the requirements
- What empirical evidence do you have that it works?
- What has to go wrong for anchoring vignette corrections to bias my results?
- What issues should I consider when writing vignettes?
- When do anchoring vignettes make the most difference?
- Why Use Anchoring Vignettes?
- Why will anchoring vignettes work when we know that putting educational achievement tests on a common scale has not been possible?
Copyright © 2024 The President and Fellows of Harvard College | Accessibility | Digital Accessibility | Report Copyright Infringement