Rare Events

Case Control and Rare Events Bias Corrections

Develops corrections for the biases in logistic regression that occur when predicting or explaining rare outcomes (such as when you have many more zeros than ones). Corrections developed for standard prospective studies, as well as case-control designs. How to use "case-control designs" to save 99% of your data collection costs. These articles overlap:

For general mathematical proofs and other technical material: Gary King and Langche Zeng. Logistic Regression in Rare Events Data, Political Analysis, Vol. 9, No. 2 (Spring, 2001): Pp. 137-163. (Article: PDF | Abstract: HTML)
An applied companion paper to the previous article that has more examples and pedagogical material but none of the mathematical proofs. Gary King and Langche Zeng. Explaining Rare Events in International Relations, International Organization, Vol. 55, No. 3 (Summer, 2001): Pp. 693-715. (Article: PDF | Abstract: HTML)

Example of an analysis of case-control data. Also an independent evaluation of the U.S. State Failure Task Force, including improved methods of forecasting state failure and assessing its causes. King, Gary and Langche Zeng. Improving Forecasts of State Failure, World Politics, Vol. 53, No. 4 (July, 2001): 623-58. (Article: PDF | Abstract: HTML)
A method to estimate base probabilities or any quantity of interest from case-control data, even with no (or partial) auxilliary information. Discusses problems with odds-ratios.

The original article: King, Gary and Langche Zeng. Estimating Risk and Rate Levels, Ratios, and Differences in Case-Control Studies, Statistics in Medicine, Vol. 21 (2002): Pp. 1409-1427. (Article: PDF | Abstract: HTML)
A revised and extended version of the previous article. Gary King and Langche Zeng. Inference in Case-Control Studies, in Shein-Chung Chow, ed., Encyclopedia of Biopharmaceutical Statistics, 2nd edition. New York: Marcel Dekker, 2004. (Article: PDF | Abstract: HTML)

James E. Alt; Gary King; and Curtis Signorino. Aggregation Among Binary, Count, and Duration Models: Estimating the Same Quantities from Different Levels of Data, Political Analysis, Vol. 9, No. 1 (Winter, 2001): Pp. 21-44. (Article: PDF | Abstract: HTML)
The first extensive empirical study of the probability of your vote changing the outcome of a U.S. presidential election? Most previous studies of the probability of a tied vote have involved theoretical calculation without data. Andrew Gelman; Gary King; and John Boscardin. Estimating the Probability of Events that Have Never Occurred: When Is Your Vote Decisive? Journal of the American Statistical Association, Vol. 93, No. 441 (March, 1998): Pp. 1-9. (Article: PDF | Abstract: HTML)

Methods to evaluate automated information extraction systems when coding rare events, the success of one such system, along with considerable data. Gary King and Will Lowe. An Automated Information Extraction Tool For International Conflict Data with Performance as Good as Human Coders: A Rare Events Evaluation Design, International Organization, Vol. 57, No. 03 (July, 2003): pp. 617-642. (Article: PDF | Abstract: HTML)
Will Lowe and Gary King. Some Statistical Methods for Evaluating Information Extraction Systems, in K. Pastra, ed. Proceedings of the Workshop on Evaluation Initiatives in Natural Language Processing. 10th Conference of the European Association for Computational Linguistics, Budapest, Hungary (2003): Pp. 19-26.

Zelig: Everyone's Statistical Software, includes ReLogit. (Website: Zelig)
Michael Tomz; Gary King; and Langche Zeng. ReLogit: Rare Events Logistic Regression, Journal of Statistical Software, Vol. 8, Issue 2 (2003). Abstract published in Journal of Computational and Graphical Statistics, Vol. 12, No. 1 (March, 2003): 246-247. (Website: ReLogit)
ReLogit: Rare Events Logistic Regression, versions for Stata and Gauss. (Software: ReLogit)

10 Million International Dyadic Events, coding conflict and cooperation in international relations, 1990-2004, as evaluated by King and Lowe (2003). (Website: Events | Abstract: HTML)