An interval’s interpretation comes from the procedure and probability model that produced it.
By the end: Explain repeated-sample coverage, distinguish credible and prediction intervals, and inspect assumptions.
A confidence procedure has coverage
A point estimate hides sampling uncertainty. A frequentist confidence interval comes from a rule that maps a dataset to bounds. A 95% procedure covers the fixed true parameter in 95% of repeated samples, or at least 95% for a conservative procedure, under its assumptions. Once the dataset is fixed, the computed interval either contains that parameter or does not.
The experiment uses equal-tailed exact binomial intervals, often called Clopper–Pearson intervals. A candidate p is excluded below or above the bounds when the observed count would lie in a tail smaller than 2.5%. Discrete counts make the coverage at least 95%, often more. “Exact” refers to using binomial probabilities without a normal approximation; it does not make the model assumptions exact.
Construct an interval by testing candidate probabilities
Suppose a process has one fixed but unknown success probability, and ten independent trials give a count of eight. The data are fixed now, but consider different candidate probabilities for the process. A candidate of 0.1 makes such a high count unusual. A candidate of 0.8 makes it much less surprising. Interval construction asks which candidate values survive a specified compatibility rule. It does not require assigning probabilities to those candidate values themselves, which would be a different kind of model.
For a two-sided equal-tailed 95% exact interval, each direction receives a tail allowance of 2.5%. To find the lower bound for eight successes, increase a candidate probability until its chance of eight or more successes reaches 0.025. Smaller candidates make the observed high count too rare under this rule. To find the upper bound, use the other direction, the chance of eight or fewer successes. Large candidates are excluded when that lower-tail probability falls below 0.025. Endpoints remain included.
This is inversion because the usual binomial calculation fixes a probability and finds probabilities of counts. Here we know the count and search over probabilities. For eight successes in ten trials, the resulting bounds are approximately 0.4439 and 0.9748. The observed proportion is 0.8, but the interval is not symmetric around it. The lower distance is about 0.3561 and the upper distance about 0.1748. A fixed symmetric error band would not reproduce this construction, especially near the boundaries of possible probabilities.
An asymmetric interval on the parameter scale
Two-sided equal-tailed 95% exact binomial interval. Endpoint values are rounded. The segment length shows a range of candidate probabilities, not an amount of posterior probability.
Why does the rule provide coverage? Fix any true probability before imagining repeated datasets. Counts that exclude it must fall in one of two extreme sets, each with probability at most 0.025 under that true model. Adding those two error probabilities gives at most 0.05. The complementary probability of including the truth is therefore at least 0.95. This argument concerns the behavior of the rule over possible data. It does not distribute 95% probability across candidate parameters after one count has been observed.
Counts are discrete, so a tail cannot usually be adjusted to have exactly the desired probability at each fixed true parameter. Adding or removing one count can change a tail's mass by a noticeable amount. This explains why exact binomial coverage often exceeds 95%. Exact means the tail calculation uses the binomial distribution rather than a smooth approximation. It does not mean the interval is shortest, that the estimate is close on this run, or that independence and constant success probability have been verified.
A higher confidence target changes the rule rather than the underlying process. At 99%, each tail receives only 0.5%, so fewer candidate values can be excluded and the interval generally widens. Reducing sample size also tends to leave more uncertainty, though the observed count matters too. These are design tradeoffs between width and coverage. They should be chosen for the intended reporting procedure, not switched after seeing the data merely to obtain a more appealing interval or a preferred conclusion.
The interval rule allocates two tail allowances
Coverage target
Lower-tail allowance
Upper-tail allowance
Total miss bound
95%
2.5%
2.5%
5%
99%
0.5%
0.5%
1%
Equal-tailed exact binomial construction. Smaller tail allowances retain more candidate probabilities. Actual miss probabilities can be below these bounds because counts are discrete.
Do not count possible datasets equally
Ten independent trials can give counts zero through ten. These eleven counts are not equally likely. Coverage adds the probabilities of the counts whose intervals cover the chosen true p. At p = 0.5, counts zero, one, nine, and ten miss. Their combined probability is 22/1,024. Coverage is therefore 1,002/1,024, about 97.85%, not seven elevenths.
A confidence level is not automatically a posterior probability about this particular interval. A Bayesian credible interval instead contains a specified amount of posterior probability for the parameter, using an explicit prior and likelihood. Numerical bounds can sometimes resemble confidence bounds while their interpretations differ.
Weight coverage by how often each interval occurs
An interval procedure assigns a pair of bounds to each possible dataset. In the ten-trial experiment, all datasets with the same count receive the same bounds. Grouping by count leaves eleven rows, but it does not make those rows equally probable. At a true probability of one half, a count of five can occur in 252 different sequences. A count of zero has only one sequence. Each sequence has probability 1/1,024, so the central row occurs 252 times as often as the zero row.
Coverage weights each row's inclusion indicator by that row's probability. At true probability one half, the exact intervals for counts two through eight include the truth. The excluded rows have weights 1, 10, 10, and 1 out of 1,024. Adding them gives missed coverage of 22/1,024, about 2.15%. Subtracting from one gives approximately 97.85%. Counting seven included rows out of eleven instead would describe a fictitious experiment in which a count label was selected uniformly, not ten fair trials.
For another view, imagine repeating the complete ten-trial experiment many times and recording each resulting interval. Most rows in this list will correspond to central counts when the true probability is one half. A displayed catalogue with one interval for every possible count deliberately removes that repetition so the possibilities fit on the screen. The probability column restores the information about how often each interval would occur. Coverage must use those weights, whether the catalogue has eleven rows or eleven thousand.
Coverage combines probability weights
Ten independent fair trials. Gold ends represent counts 0–1 and 9–10; the central blue length represents counts 2–8. The whole length is probability one. These lengths encode repeated-sample probability, not a posterior over p.
The truth is fixed in this repeated-sampling calculation. It is the interval that changes with each possible dataset. After a particular count arrives, there is only one computed interval. It either covers the fixed truth or misses it, although we generally do not know which. The confidence label still tells us something useful about the method used to produce it. It is not a guarantee for the particular case, nor does the observed interval reveal whether this run belongs to the method's failures.
A further selection step can change that method. Suppose an analyst computes intervals for several outcomes but reports only the narrowest one. The reported object now comes from a selection procedure, not simply the original interval rule for one prespecified outcome. Likewise, repeating measurements until an interval excludes a target value changes how the dataset arises. The original fixed-sample coverage calculation does not automatically apply to the selected or stopped result. The whole reporting procedure needs its own evaluation under repeated sampling.
Confidence coverage is also distinct from a test's size and power. All describe repeated-data behavior, but their events differ. Coverage asks whether a random interval includes a fixed parameter. Test size asks how often a rule rejects under its null conditions; power asks how often it rejects under a specified alternative. None directly supplies the probability that a parameter claim is true given the observed report. That reverse conditional requires further assumptions about parameter uncertainty and how the evidence was selected or generated.
Work through the mechanism
Weight the intervals by their sampling probabilities
Coverage: 97.85%. Sum the probability weights of the ✓ rows, not their fraction of the eleven rows.
Each row is one possible success count from ten independent trials. Its horizontal interval comes from exact binomial-tail inversion. The dashed line is the chosen true p. ✓ means the interval covers it; × means it misses. Bounds stay fixed when p changes; row probabilities and coverage change immediately. These eleven counts are not eleven equally likely samples. Displayed percentages are rounded; coverage uses unrounded weights.
True p = 0.5, n = 10
Probability mass
Counts 0 and 1, miss
11/1,024
Counts 2 through 8, cover
1,002/1,024
Counts 9 and 10, miss
11/1,024
Exact 95% binomial intervals have conservative coverage here. Count outcomes must be probability-weighted.
Change the generating truth without changing the interval rule
Start with the middle preset in the coverage experiment. Each horizontal segment is the interval the rule would produce for its row's count. The vertical reference marks the stipulated true success probability. A segment that crosses that reference covers the truth; a segment entirely to either side misses it. The right column gives the probability of obtaining that count under the selected truth. The display therefore combines an interval catalogue with a model for how often each catalogue entry would be used.
Now choose the low probability preset. Predict which features should change before reading the result. The interval for a count of eight must stay the same because its calculation uses the count, sample size, and confidence rule, not knowledge of the generating truth. The vertical reference moves, and count probabilities shift toward zero. Some segments change from covering to missing or the reverse. This is an evaluation of one fixed procedure at another possible truth, not a new interval fitted to the slider value.
At a true probability of 0.1, the exact intervals for counts zero through three cover. For count four, the lower endpoint is about 0.1216, above 0.1, so that row misses. The probability of zero through three successes is approximately 0.987205, or 98.72%. Thus the plot can show only four covering rows while coverage exceeds the seven-row coverage at probability 0.5. The difference comes from probability weights. At probability 0.1, counts near zero account for nearly all possible experiments.
Fewer covering rows can have greater probability
True p
Counts whose intervals cover
Coverage probability
0.1
0 through 3
≈ 98.7205%
0.5
2 through 8
≈ 97.8516%
0.9
7 through 10
≈ 98.7205%
Ten independent trials and the same two-sided 95% exact interval rule in every row. Coverage sums count probabilities at the stated truth; it does not divide covering rows by eleven.
The high probability preset provides a contrasting model. Reflecting every success into a failure turns probability 0.1 into 0.9 and count k into count ten minus k. The interval bounds reflect in the same way. Counts seven through ten then cover, with the same total coverage of about 98.72%. This symmetry belongs to the binomial model and equal-tailed construction. It should not be assumed for every interval method or every data collection process.
Notice that two different candidate truths can both lie inside the interval obtained from one observed count. This does not make their sampling probabilities equal, and it does not assign each half of some posterior weight. An interval reports membership under its construction rule, not a complete ranking or distribution of uncertainty over parameters. To compare how strongly the observed data favor two candidate values, calculate their likelihoods. To turn that comparison into posterior probabilities, additionally specify a prior and the full set of allowed alternatives.
The displayed percentage is a model calculation, not an empirical estimate from a thousand simulated experiments. Small changes in the truth can alter which interval endpoints are crossed, so coverage need not vary smoothly like a simple straight line. Counts retain discrete probability masses while the truth varies continuously. If two rounded percentages look equal, their underlying weights may still differ. The useful exploration is to connect changing weights and membership to total coverage, rather than search for a slider position that certifies the real process assumptions.
Three entries from the same interval catalogue
Ten trials and two-sided equal-tailed 95% exact binomial intervals. Segment endpoints use the common parameter scale and are rounded to six decimals. Changing the generating probability changes row weights and coverage, not these bounds.
Ask what the interval covers
A confidence interval for a population mean or proportion concerns that parameter. A prediction interval concerns a future observation, which also has individual variation. A tightly estimated average waiting time does not imply that the next person’s wait is nearly fixed.
Neither interval protects against all model misspecification, bad measurements, biased recruitment, or an unmodeled shift in conditions. Report the estimand, population, sampling procedure, interval construction, and important assumptions. Bayesian work also needs prior-sensitivity and model checks; frequentist work needs the stated coverage conditions.
Match the probability statement to its target
For a separate Bayesian construction, suppose a machine has one fixed success setting, either 0.2 or 0.6, with equal prior weights. Conditional on its setting, observations are independent. Observe success followed by failure. The sequence likelihood is 0.2 times 0.8, or 0.16, at the lower setting and 0.6 times 0.4, or 0.24, at the higher setting. These likelihoods describe data under settings, not posterior probabilities of settings.
Multiplying by the prior weights gives 0.08 and 0.12. Their sum is 0.20, the model's total probability of the sequence. Dividing by that sum gives posterior weights 0.4 and 0.6. A credible set containing only the higher setting holds 60% of posterior mass, not 95%. A set containing both holds 100%. This discrete model cannot supply a set with exactly every requested probability. The gap comes from the allowed parameter states, whereas conservative confidence coverage comes from probabilities across possible datasets.
Change the prior to assign weight 0.8 to the lower setting and 0.2 to the higher setting. The same likelihoods now give weighted values 0.128 and 0.048, totaling 0.176. The lower setting's posterior weight becomes 0.128/0.176, or about 72.73%. Evidence still favors the higher setting relative to its prior, but it does not erase the prior. A credible statement needs its prior assumptions. Confidence coverage instead needs the sampling model and interval rule.
The same likelihoods with two different priors
Prior for p = 0.2 / 0.6
Weighted likelihoods
Posterior for p = 0.2 / 0.6
0.5 / 0.5
0.08 / 0.12
0.4 / 0.6
0.8 / 0.2
0.128 / 0.048
8/11 / 3/11
Observed sequence is success then failure. Its likelihoods are 0.16 and 0.24. Each weighted likelihood is prior times likelihood; normalization divides by their sum. The setting is fixed across trials.
Prediction asks another question. Under the original equal prior example, the posterior chance of a success on the next trial is 0.4 times 0.2 plus 0.6 times 0.6, or 0.44. The machine does not redraw its setting each time. We average because we do not know its fixed setting. Future trials are independent conditional on that setting, but are generally dependent after averaging over uncertainty about it. A new success changes what we believe about the setting and therefore changes predictions for later trials.
An interval for a parameter and a prediction interval for an observation can consequently have very different widths. Estimating the average waiting time precisely does not remove person-to-person variation in future waits. Similarly, an interval for a difference between groups is not automatically an interval for a causal treatment effect. Random sampling concerns population representation; randomized assignment can support the causal comparison. If exposure was self-selected, narrower statistical bounds around the association do not eliminate confounding or identify what would happen under an intervention.
Finally, uncertainty about a probability does not choose an action without consequences. Two investigators could agree on a posterior and credible set but prefer different actions because missed failures and unnecessary interventions carry different losses for them. A confidence interval alone also cannot settle that loss comparison. When reading a report, identify whether its target is a population parameter, a future observation, a causal effect, or an action's expected loss. Then inspect the assumptions connecting the available data to that target, rather than treating every pair of bounds as interchangeable.
Different targets require different interpretations
Reported object
Target
Needed connection
Confidence interval
Fixed population parameter
Sampling model and coverage rule
Credible set
Uncertain parameter
Prior and likelihood
Prediction interval
Future observation
Future variation and parameter uncertainty
Causal interval
Intervention effect
Causal identification and uncertainty model
Expected loss
Consequences of an action
State probabilities and conditional losses
These rows are not competing names for the same uncertainty. A numerical interval does not establish representativeness, causal identification, or the value of an action.
Exercises
Read zero observed failures
You observe zero failures in ten independent trials. Does a confidence interval have upper bound zero?
Hint
Even a positive failure probability can produce ten successes.
Worked answer
Candidate failure rate
Chance of zero failures in ten
0
1
0.1
0.9¹⁰ ≈ 0.3487
≈ 0.3085
0.025
The upper endpoint depends on the interval procedure and confidence level.
No. For the two-sided equal-tailed 95% exact interval, the upper bound solves the probability of zero failures equal to 0.025. It is 1 − 0.025^(1/10), about 0.3085. No observed failures does not prove a zero failure rate.
A posterior over two settings
A device has one fixed success-probability setting, either 0.25 or 0.75. Give each setting prior probability one half. Conditional on its setting, trials are independent. You observe success, success, failure. Find the posterior weights. Is the single setting 0.75 a 95% credible set? Predict success on the next trial.
Hint
Multiply each setting’s prior by the likelihood of the entire sequence, then normalize. Average the next-trial probabilities over the posterior.
Worked answer
Device setting
Prior weight
Sequence likelihood
Posterior weight
p = 0.25
0.5
0.046875
0.25
p = 0.75
0.5
0.140625
0.75
These are the only two allowed settings in this model. A credible set is a set of parameter values, weighted by their posterior probabilities.
The sequence likelihoods are 0.25² × 0.75 = 0.046875 and 0.75² × 0.25 = 0.140625. With equal priors, their ratio is 1 to 3, giving posterior weights 0.25 and 0.75. The single higher setting contains 75%, not 95%, of posterior mass. A set containing at least 95% must include both settings in this discrete model. Next-trial success probability is 0.25 × 0.25 + 0.75 × 0.75 = 0.625. This prediction averages uncertainty about a fixed setting; it does not assume the device changes settings.
Sources and further reading
The explanations and exercises are complete without these optional links. The references distinguish probability identities from inference procedures and reporting guidance. The examples are constructed teaching models, not empirical estimates or professional decision advice.
The subsection on exact intervals specifies binomial-tail inversion. Other sections discuss different interval methods; the lab uses equal-tailed exact intervals.