Confidence intervals explained for CS1

CS1 · Actuarial Statistics · 9 min read

What the 95% is a statement about

Start with the sentence almost everyone says and the syllabus will not accept: "there is a 95% probability that the true mean lies in this interval". It is wrong, and knowing precisely why is most of this topic.

The true mean is a fixed number. It is unknown, but it is not random — it is not wobbling about, and it does not have a probability of being anywhere. What is random is the interval. Collect a different sample and you get a different one, because it is computed from data.

So the 95% describes the procedure, not this particular interval. It says: if you were to repeat the sampling many times and build an interval the same way each time, about 95% of those intervals would contain the true value. The one in front of you either does or does not; you cannot know which.

Right: "95% of intervals built this way contain the true mean."
Wrong: "There is a 95% chance the true mean is in this interval."

This is not pedantry for its own sake. The whole frequentist framework treats parameters as fixed and data as random, and the Bayesian framework — which is also in CS1 — does the opposite, which is exactly why a Bayesian credible interval can be described the second way. Being loose here makes the later comparison between the two impossible to state.

How an interval is actually built

Every confidence interval in the paper comes from the same manoeuvre. Find a quantity built from the data and the unknown parameter whose distribution does not depend on the unknown parameter. Such a quantity is called a pivot. Then write down a probability statement about the pivot, and rearrange it so the parameter sits in the middle.

For a mean, the pivot is the standardised sample mean. With σ known,

(X̄ − μ) / (σ/√n) ≈ N(0, 1)

which does not involve μ anywhere in its distribution. So P(−1.96 < pivot < 1.96) = 0.95, and multiplying through and adding X̄ to both sides moves μ to the centre:

X̄ ± 1.96 σ/√n

Notice what each piece is doing. X̄ says where; σ/√n is the standard error from the central limit theorem, saying how much a sample mean typically wobbles; 1.96 says how many wobbles to allow for 95% coverage. Change the confidence level and only the 1.96 changes — 1.645 for 90%, 2.576 for 99%.

When to use t instead of z

The interval above needs σ, the population standard deviation, which in practice is never known. Replacing it with the sample standard deviation s introduces a second source of uncertainty: the centre is estimated and so is the width. The pivot then no longer follows a normal distribution but a Student's t with n − 1 degrees of freedom:

(X̄ − μ) / (s/√n) ~ t_n−1 → X̄ ± t_{n−1, 0.975} × s/√n

The t distribution is the normal with heavier tails, and the extra weight in the tails is the price of not knowing σ. Its multiplier is therefore always larger than 1.96, so a t interval is always wider than the corresponding z interval. How much wider depends on n: at n = 25 the multiplier is 2.0639, at n = 100 it is 1.9842, and by a few hundred observations the difference stops mattering, which is why large samples are handled with z without apology.

The decision rule in one line: if σ came from outside the data, use z; if it was computed from the data, use t with n − 1 degrees of freedom.

An interval for a proportion

When each observation is a yes or no, the estimate is the sample proportion p̂, whose standard error has the parameter inside it. Substituting the estimate for the unknown gives the standard interval:

p̂ ± z √(p̂(1 − p̂)/n)

This one has a condition attached, and it is routinely ignored. The normal approximation behind it needs both np̂ and n(1 − p̂) to be reasonably large — ten is the usual working threshold. With 3 claims out of 10 policies the interval would be nonsense, and can even run below zero, which is an impossible value for a proportion and a clear signal that the approximation has broken.

The interval is the set of hypotheses you cannot reject

This is the most useful fact on the page and it is also the arithmetic check. A two-sided test at the 5% level rejects a hypothesised value μ₀ exactly when

|(X̄ − μ₀) / (s/√n)| > t_{n−1, 0.975}

Rearranging that inequality gives precisely the complement of the 95% interval. So the interval is the collection of all values of μ₀ that a 5% test would not reject, and the endpoints are the two values at which the test statistic equals the critical value exactly.

Two things follow. If an interval has been computed correctly, plugging either endpoint back in as μ₀ must return the critical value to every decimal place — which is the check performed in the worked example. And whether a stated value is "significant" can be read straight off an interval, with no test to compute: if the value is outside, it is rejected.

Where marks are lost

The probability statement about the parameter. Worth real marks in a written answer, and the error costs them every time.

Using z when s was estimated and n is small. With n = 25 the z interval is about 5% too narrow, which overstates how much the data have settled.

The wrong degrees of freedom. For a single mean it is n − 1, not n. One degree of freedom has been spent estimating the centre before the spread is measured.

Forgetting the √n. Using s rather than s/√n produces an interval far too wide — it describes where one observation might fall, not where the mean might be. If the interval is as wide as the data themselves, this is the error.

Expecting the width to halve when n doubles. The width goes as 1/√n, so halving it needs four times the data. This is the single most important practical fact about precision, and the reason data collection gets expensive quickly.

Questions students ask

Why 95% rather than some other figure?

Convention and nothing more. 95% corresponds to about two standard errors, which is a convenient round idea, and 1.96 is close enough to 2 that the arithmetic can be checked in your head. Nothing in the mathematics prefers it; 90% and 99% intervals are built the same way with a different multiplier. In solvency work much higher confidence levels are normal, because the consequence of being wrong is different.

Does a wide interval mean the study was bad?

It means the data were not sufficient to pin the quantity down, which is useful information honestly reported. A wide interval that is correct is worth more than a narrow one obtained by using z where t was required, or by quietly dropping inconvenient observations. The width is a finding, not a failure.

How is this different from a prediction interval?

A confidence interval is for a parameter, such as the mean. A prediction interval is for the next observation, and it must carry both the uncertainty in the estimated mean and the variability of a single observation about that mean. That makes it much wider — it contains s√(1 + 1/n) where the confidence interval contains s/√n. Confusing the two is how a reserve gets set to the average claim rather than to something a claim might plausibly be.

A worked example

Two parts: an interval for a mean, which is then checked by inverting the test it corresponds to, and an interval for a proportion. All figures were invented for this example.

Part one: a mean, with s estimated from the data

Twenty-five students sit a mock paper. Their marks average 58.4 with a sample standard deviation of 12.5. Build a 95% confidence interval for the mean mark of the population these students represent.

The standard deviation was computed from the data, so this is a t interval with n − 1 = 24 degrees of freedom.

se = s/√n = 12.5/√25 = 12.5/5 = 2.5
t_{24, 0.975} = 2.0638986
margin = 2.0638986 × 2.5 = 5.1597464

58.4 ± 5.1597464 → (53.2403, 63.5597)

Rounded for reporting: the mean mark is estimated at 58.4, with a 95% confidence interval of 53.2 to 63.6.

The check: invert the test

If the interval is right, then testing the hypothesis μ = 53.2402536 — the lower endpoint — must produce a t statistic equal to the critical value exactly:

t = (58.4 − 53.2402536) / 2.5 = 5.1597464 / 2.5 = 2.0638986

Identical to t_{24, 0.975} to seven decimal places. The upper endpoint gives −2.0638986 by the same arithmetic. The interval is exactly the set of hypothesised means a 5% two-sided test would not reject, which is what it is supposed to be.

What using z instead would have done

Had σ been known to be 12.5, the multiplier would be 1.96 and the margin 1.96 × 2.5 = 4.9, giving (53.50, 63.30). That interval is 0.52 marks narrower, about 5% too tight. The difference is small here but it is in the wrong direction: it claims more precision than twenty-five observations support.

What four times the data would do

Suppose 100 students were tested instead, with the same mean and the same sample standard deviation.

Sample size	Standard error	Multiplier	Margin	Full width
25	2.5000	2.0639	5.1597	10.3195
100	1.2500	1.9842	2.4803	4.9605

Four times the data has cut the width by a factor of 2.08. Slightly better than the factor of 2 that 1/√n predicts, because the t multiplier also shrinks from 2.0639 to 1.9842 as the degrees of freedom rise. The lesson stands: to halve the width you need four times the data, not twice.

Turning that round, to get the margin down to 2.5 marks you would need about (1.96 × 12.5 / 2.5)² = 96.04, so 97 students — and with the t multiplier at that sample size the margin lands at 2.52 rather than 2.50, close enough for planning.

Part two: a proportion

Of 320 enquiries received in a month, 92 led to an enrolment. Estimate the conversion rate with a 95% interval.

p̂ = 92/320 = 0.2875
p̂(1 − p̂) = 0.2875 × 0.7125 = 0.2048438
se = √(0.2048438/320) = √0.0006401 = 0.0253009
margin = 1.96 × 0.0253009 = 0.0495898

0.2875 ± 0.0495898 → (0.2379, 0.3371)

So the conversion rate is estimated at 28.8%, with a 95% interval of roughly 23.8% to 33.7%.

Check the condition before trusting it: np̂ = 92 and n(1 − p̂) = 228. Both comfortably above ten, so the normal approximation is appropriate here and the interval sits well inside (0, 1) as it must.

One honest caveat on interpretation, which has nothing to do with the arithmetic: this interval describes sampling variability only. If the 320 enquiries were not representative of the enquiries you care about, no confidence level repairs that. A confidence interval quantifies randomness, never bias.

Where this fits in CS1

This page explains one idea from CS1 Actuarial Statistics. The full CS1 syllabus, with what each part is examined on, sits on the course page, and past-paper style mock exams with full solutions are where you find out whether the idea has stuck under time pressure.

Confidence intervals: what the 95% actually means