Sunday, July 21, 2019

Confidence Intervals -- If Only It Were That Simple

Confidence Interval: Statistics such as means (or averages) and medians are often calculated from data from a portion—or sample—of a population rather than from data for an entire population. Statistics based on sample data are called “sample statistics,” whereas those based on an entire population are called “population parameters.” A confidence interval is the range of values of a sample statistic that is likely to contain a population parameter, and that likeliness is expressed with a specific probability. For example, if a study of a sample of 1,500 Americans finds their average weight to be 150 pounds with a 95 percent confidence interval of plus/minus 25 pounds, this means that there is a 95 percent probability that the average weight of the entire American population is between 125 and 175 pounds. --Wm. Nöel & Judy Wang, Is Cannabis a Gateway Drug? Key Findings and Literature Review: A Report Prepared by the Federal Research Division, Library of Congress, Under an Interagency Agreement with the Office of the Director, National Institute of Justice, Office of Justice Programs, U.S. Department of Justice, Nov. 2018, at 3.

{T]here is a 5 percent chance the true value [of a 95% one-sided confidence interval] exceeds the bound. --President’s Council of Advisors on Science and Technology, Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods, Sept. 2016, at 153.
[T]he confidence level does not give the probability that the unknown parameter lies within the confidence interval. ... According to the frequentist theory of statistics, probability statements cannot be made about population characteristics: Probability statements apply to the behavior of samples. That is why the different term ‘confidence’ is used. --David H. Kaye & David A. Freedman, Reference Guide on Statistics, in Reference Manual on Scientific Evidence 211, 247 (Federal Judicial Center & National Research Council Committee on the Development of the Third Edition of the Reference Manual on Scientific Evidence eds., 3d ed. 2011).

Warning! ... [T]he fact that a confidence interval is not a probability statement about [an unknown value] is confusing. --Larry Wasserman, All of Statistics: A Concise Course in Statistical Inference 92-93 (2004) (emphasis in original).

No comments:

Post a Comment