Study guides · Intermediate

Understanding confidence-interval coverage

Interpret interval estimation as a repeated-sampling procedure and distinguish interval width, coverage, and selection effects.

Ask Logeia about this articleTry the three questionsReview your answers

Updated

On this page
  1. Separate the parameter and the interval
  2. Construct a normal-mean interval
  3. Compare width and coverage
  4. Estimate coverage with uncertainty
  5. Include selection and stopping rules
  6. Check representativeness before precision
  7. Ask Logeia about this articleCheck your understanding

Separate the parameter and the interval

In a frequentist confidence procedure, the parameter is fixed while the interval varies with the sample. A nominal ninety-five percent level describes how often intervals constructed by the same procedure contain the parameter over repeated sampling. It is distinct from assigning a posterior probability to a parameter lying inside the particular realized interval. Begin by specifying both the sampling population and the construction rule. An arbitrary symmetric band around an estimate does not acquire a confidence interpretation just because its endpoints look plausible or its label says ninety-five percent.

Construct a normal-mean interval

For independent normal observations with known population standard deviation sigma, the sample mean plus or minus 1.96 sigma divided by the square root of n has approximately ninety-five percent coverage. With n=100, sigma=10, and observed mean 52, the standard error is one and the endpoints are about 50.04 and 53.96. The standard deviation describes individual observations; the standard error describes sample means. Substituting one for the other changes the width substantially. When sigma is estimated rather than known, a different procedure, such as a t interval under normal sampling, must be justified.

Compare width and coverage

Increasing sample size from one hundred to four hundred halves the standard error under the same population and construction rule. The expected width halves as well. Reducing the multiplier from 1.96 to one also narrows the interval, but changes coverage. Narrower alone does not mean a more informative or better-calibrated procedure. Distinguish more observations, a different confidence level, and a genuinely less variable population. Report sample size, nominal confidence level, whether sigma was assumed known, and the construction formula when comparing intervals from different analyses or experiments.

Estimate coverage with uncertainty

A coverage simulation fixes a known population mean, draws a fresh sample for each repetition, constructs an interval, and counts whether it contains that mean. Nine hundred fifty successes in one thousand repetitions estimate coverage at ninety-five percent. The simulation proportion itself has uncertainty: using 0.95 as the success probability gives a Monte Carlo standard error near 0.0069. A run need not land exactly on its nominal level. Fix the random seed for reproduction, inspect assertions about the interval calculation, and increase repetitions when investigating whether an apparent discrepancy exceeds ordinary simulation noise.

Include selection and stopping rules

Trying many outcomes and reporting only a convenient interval changes the procedure being evaluated. So does repeatedly increasing the sample until a desired result appears and then applying an ordinary fixed-sample rule without accounting for stopping. Multiple comparisons and sequential monitoring require appropriate methods or a predeclared design. This does not make interval estimation unusable; it expands the procedure to include sampling, selection, and stopping. Record which analyses were planned and which were chosen after inspecting results, rather than attaching the original nominal label to a materially different process.

Check representativeness before precision

Measuring only long-time users can misrepresent new users even with a very large sample. A narrow interval quantifies variability under the assumed sampling model; it does not diagnose representation bias. For practice, calculate half-widths under a known-sigma normal model with sigma 10 and sample sizes 25 and 100. The results are about 3.92 and 1.96. Explain the factor of two using sample size, then identify what neither calculation establishes about the recruitment process. Report the target population, collection mechanism, exclusions, and missing observations alongside endpoints so readers can assess both precision and relevance.

Reading and practice are recorded separately in this browser, for each language.

Learn this in order

Prepare for AI systems · STEP 11

Choose a learning path