The one research area where our approach clearly does not work is educational testing. The difficulty with educational testing is that no matter how carefully you write the common test questions as anchors, test takers will differ in their responses to them according to both DIF and their knowledge or achievement. Anchoring vignettes solve the problem in other areas because a respondent's answer is only a function of DIF (and estimation variability), and so can be used to adjust the self-assessments. An appropriate anchoring vignette in educational testing would be a test question where all test takers have identical knowledge of the subject being examined, but this is obviously infeasible.
Anchoring Vignettes
- Frequently Asked Questions
- About this document
- Introduction
- Statistical Analyses with Anchoring Vignettes
- Survey Instrumentation
- Are universally applicable, culture-independent survey questions possible?
- Can I use anchoring vignettes if I don't have variables to predict the thresholds?
- Can I use anchoring vignettes to understand why respondents understand survey questions in such different ways?
- Can we avoid DIF by using a panel design, without vignettes?
- Do I need one vignette for each response category?
- Doesn't Anchoring Vignettes merely move the problem of coming up with DIF-free survey questions back one level (from self-assessments to vignettes), and so in the end you have the same problem?
- How is this strategy affected by the finding in social-psychology that assessments of oneself and others differ?
- How many anchoring vignettes should I ask for each concept I want to measure?
- How much expense will anchoring vignettes add?
- If I have a direct physical measurement, such as a medical test, do I need anchoring vignettes?
- In what order should vignettes and self-assessment questions be asked?
- Is there a simpler way of asking questions so we can avoid any statistical analysis?
- Should I ask vignettes questions of all people or a random subset?
- Should the vignette describe the age, sex, etc., of the hypothetical person? Should it be self-referential?
- What are the requirements
- What empirical evidence do you have that it works?
- What has to go wrong for anchoring vignette corrections to bias my results?
- What issues should I consider when writing vignettes?
- When do anchoring vignettes make the most difference?
- Why Use Anchoring Vignettes?
- Why will anchoring vignettes work when we know that putting educational achievement tests on a common scale has not been possible?
Copyright © 2024 The President and Fellows of Harvard College | Accessibility | Digital Accessibility | Report Copyright Infringement