CS2 · explained

Extreme value theory: block maxima, thresholds and the generalised Pareto distribution

A model fitted to all your claims is fitted mostly to the small ones, and it will be wrong about the large ones that matter. Extreme value theory models only the tail. This page explains the two ways in and works a full tail probability and return level from threshold exceedances.

CS2 · Risk Modelling & Survival Analysis · 12 min read

After this page you can

Fit a generalised Pareto distribution to threshold exceedances and use it to estimate a tail probability and a return level

Why the tail needs its own model

You have 2,000 claims and you want to know the chance of one above ₹1.5 crore. You have never seen a claim that large.

The instinct is to fit a distribution to all 2,000 claims and read off the tail. This fails, and it fails for a reason worth understanding rather than memorising. A fitted distribution is fitted where the data is, and the data is almost all small claims. A lognormal that matches the body of the distribution beautifully can be out by an order of magnitude at the 99.9th percentile, and nothing in the fit statistics will tell you so, because the handful of large observations that would have objected carry almost no weight in the likelihood.

Worse, the answer depends entirely on which family you picked. Lognormal, gamma and Pareto can all fit the same body of claims acceptably and give wildly different probabilities for a claim above anything you have not seen. Choosing between them on goodness of fit in the body is choosing a tail by accident.

Extreme value theory takes the opposite approach. Throw the body away and model only the extremes. What makes this more than a counsel of despair is that there is a theory about what the extremes must look like — one that does not depend on knowing the distribution the data came from.

The two ways in

Block maxima, and the GEV

Divide the data into blocks — years, say — and keep only the largest observation from each. Those maxima have a limiting distribution, and the remarkable fact is that there are only three possible shapes, which combine into one family: the generalised extreme value distribution.

G(x) = exp( −[1 + ξ(x − μ)/σ]−1/ξ )

This is the analogue of the central limit theorem for maxima rather than averages: whatever the underlying distribution, the maxima tend to this family. The parameter ξ is the shape and it carries all the information about the tail; μ and σ are location and scale. The three classical cases are ξ > 0 (Fréchet, heavy tailed, power-law decay), ξ = 0 (Gumbel, light tailed, exponential decay, the formula read as a limit), and ξ < 0 (Weibull form, a tail with a finite upper limit).

Block maxima is wasteful, and that is its drawback. A year with three enormous claims contributes one observation; a quiet year contributes one as well, and that one may be small enough to carry no information about the tail at all.

Peaks over threshold, and the GPD

The better approach for most insurance data. Choose a threshold u, keep every observation above it, and model the excesses X − u. Those excesses have a limiting distribution too, and it is the generalised Pareto distribution:

P(X − u ≤ y | X > u) = 1 − [1 + ξy/β]−1/ξ, for y > 0

In words: given that a claim has exceeded the threshold, the amount by which it exceeds it follows this two-parameter family, whatever the claims distribution was.

The GPD and the GEV share the shape parameter ξ — the same number, carrying the same meaning, which is why the two approaches are two views of one theory rather than two theories. The β is a scale that depends on the threshold chosen.

Three cases again. ξ > 0 is heavy tailed and is what large insurance claims look like; the distribution has infinite moments beyond order 1/ξ, so ξ ≥ 0.5 means infinite variance and ξ ≥ 1 means an infinite mean. ξ = 0 reduces to the exponential distribution, which is the dividing line. ξ < 0 has a hard upper limit at u − β/ξ, which is the right model for something that physically cannot exceed a maximum.

The formula that does the work

Having fitted a GPD above u, the tail probability for any x > u follows from the chain rule for probabilities:

P(X > x) = P(X > u) × [1 + ξ(x − u)/β]−1/ξ

In words: the chance of getting past the threshold at all, times the chance of going on from there to x.

P(X > u) is estimated by the obvious proportion — the number of exceedances divided by the total number of observations. That is the one place in the calculation where the body of the data is used, and it is used only as a count.

Inverting this gives the return level: the claim size exceeded with a given probability p.

xp = u + (β/ξ)[ (P(X > u)/p)ξ − 1 ]

This is the output a reinsurance programme or a capital calculation actually needs: not "what is the chance of a claim above 1.5 crore" but "what claim size should I plan for once in 200 claims".

Choosing the threshold, which is the real decision

The whole method rests on one judgement, and a question will usually ask about it.

Too low a threshold and the limiting result has not taken hold — you are modelling part of the body with a tail model, and the fit is biased. Too high and you have twenty observations left, and the parameter estimates are so uncertain that the answer is worthless. It is a bias-variance trade-off in its purest form.

Two diagnostics are standard. The mean excess plot uses the fact that for a GPD the mean excess over a threshold is linear in the threshold:

E[X − u | X > u] = (β + ξu′) / (1 − ξ) for a threshold raised by u′, valid when ξ < 1

So plot the sample mean excess against the threshold and look for the point beyond which it becomes a straight line; that is where the GPD starts to hold. An upward slope indicates ξ > 0 and a heavy tail, a flat line indicates the exponential case, and a downward slope a short tail. The second diagnostic is a stability plot: fit at a range of thresholds and look for the region where ξ̂ stops drifting.

Other ways to describe a tail

Three more measures appear in this part of the syllabus and they are all ways of asking the same question — how fast does the density die away?

Moments. A distribution with infinite higher moments is heavy tailed. The existence of moments is a crude but decisive classifier: the lognormal has every moment finite, the Pareto does not.

The hazard rate. A hazard that decreases towards zero in the tail means a heavy tail — the larger a claim already is, the less likely it is to stop growing. A constant hazard is the exponential borderline.

The mean residual life. For a heavy tail, the expected further amount of a claim given it has already exceeded x increases with x. This is the most counter-intuitive and the most useful property: on heavy-tailed claims, the news that a claim is already large is bad news about how large it will end up.

All three agree with the GPD shape parameter, which is the point. They are different instruments reading the same quantity.

Questions students ask

Why is there a theorem at all? Surely the tail depends on the distribution.

It does, but only through one parameter. The reason is that maxima and exceedances are governed by a limiting argument much like the central limit theorem: rescale properly and the specifics of the underlying distribution wash out, leaving a one-parameter family of possible shapes. That is why the result is useful — you do not have to know, or guess, what generated the data.

How many exceedances do I need?

There is no clean answer, which is itself the answer. Fewer than about fifty and the shape parameter is very poorly determined; the honest treatment is to fit at several thresholds and report how much the conclusion moves. A single point estimate from a short tail hides the uncertainty that matters most.

Is the GPD the same as the Pareto distribution?

Closely related, not identical. The ordinary Pareto is the ξ > 0 case with a particular parametrisation; the GPD covers the light and short-tailed cases as well, and it arises as a limit for exceedances rather than being assumed. The difference matters when you are asked where the model came from.

Does this apply to anything other than claim sizes?

Anywhere the extremes are what hurts: operational losses, flood levels, the worst daily fall in a portfolio, peak electricity demand. Insurance is the natural home because reinsurance programmes and capital requirements are defined by tail quantiles, which is exactly what the return level formula produces.

A worked example

A general insurer has 2,000 claims from one class of business, in ₹ lakh. Every figure below is invented for this example and is not drawn from any real portfolio.

A threshold of u = 50 is chosen from a mean excess plot, and 80 claims exceed it. Among those 80, the excesses over 50 have a sample mean of 25 and a sample variance of 1250, which is a standard deviation of about 35.4.

  1. Fit the GPD by matching moments

    For a GPD with shape ξ and scale β, and ξ < 0.5 so both exist:

    mean = β/(1 − ξ)      variance = β2 / [(1 − ξ)2(1 − 2ξ)]

    Divide the variance by the square of the mean and the β cancels, leaving one equation in ξ:

    variance / mean2 = 1/(1 − 2ξ), so ξ = ½(1 − mean2/variance)
    mean2/variance = 625/1250 = 0.5000000
    ξ̂ = ½(1 − 0.5) = 0.2500000
    β̂ = mean × (1 − ξ̂) = 25 × 0.75 = 18.7500000

  2. Check the fit against its own moments

    β/(1 − ξ) = 18.75/0.75 = 25.0000000   against a sample mean of 25
    β2/[(1 − ξ)2(1 − 2ξ)] = 351.5625 / (0.5625 × 0.5) = 351.5625/0.28125 = 1250.0000000   against a sample variance of 1250

    Both reproduced exactly, which is what method of moments guarantees and therefore confirms the algebra rather than the model. Note what ξ = 0.25 already tells you: moments above order 1/0.25 = 4 are infinite, so the tail is genuinely heavy, and the variance exists only because 0.25 is below 0.5.

  3. The probability of exceeding the threshold

    P(X > 50) = 80/2000 = 0.0400000

  4. Tail probabilities

    P(X > x) = 0.04 × [1 + 0.25(x − 50)/18.75]−4

    At x = 150:

    1 + 0.25 × 100/18.75 = 1 + 25/18.75 = 1 + 4/3 = 7/3 = 2.3333333
    (7/3)4 = 2401/81 = 29.6419753
    P(X > 150) = 0.04 × 81/2401 = 3.24/2401 = 0.0013494

    Keeping the bracket as the fraction 7/3 rather than the decimal is worth the small effort: the fourth power of a rounded 2.3333333 drifts in the seventh digit, while 2401/81 is exact.

    So about one claim in 741 exceeds ₹150 lakh. Two more, for the shape of the tail:

    Claim size x1 + ξ(x−u)/βP(X > x)One claim in
    501.00000000.040000025
    1001.6666667 = 5/30.0051840193
    1502.3333333 = 7/30.0013494741
    2003.00000000.00049382,025

    The x = 50 row is the check worth making: putting the threshold itself into the formula gives a bracket of exactly 1, so the power is 1 and the answer is the unconditional exceedance probability 0.04 you started with. A tail formula that does not reproduce its own threshold has the P(X > u) factor in the wrong place.

    Notice how fast the probability falls and how slowly compared with an exponential tail: doubling the claim from 100 to 200 divides the probability by about ten, not by the hundreds an exponential would give. That slow decay is what ξ > 0 means in money.

  5. The return level

    What claim size is exceeded by one claim in a thousand?

    xp = 50 + (18.75/0.25)[ (0.04/0.001)0.25 − 1 ]
    (0.04/0.001) = 40, and 400.25 = √√40 = √6.3245553 = 2.5148669
    xp = 50 + 75 × 1.5148669 = 50 + 113.6150175 = 163.6150175

    So about ₹163.6 lakh. Substitute it straight back into the tail formula as an independent check:

    1 + 0.25 × 113.6150175/18.75 = 1 + 1.5148669 = 2.5148669
    2.51486694 = 40.0000026
    P = 0.04 / 40.0000026 = 0.0010000

    Exactly the one in a thousand asked for. Two routes, inverse and forward, agreeing to seven decimal places; the 26 in the eighth digit of the fourth power is the fourth root having been rounded to seven places on the way out.

  6. And the largest claim in the year

    If those 2,000 claims are one year's worth, what is the chance the year's largest claim is below ₹150 lakh? Treating the claims as independent:

    P(all 2,000 below 150) = (1 − 0.0013494)2000 = 0.0671637

    and the Poisson approximation, which is the bridge to the block-maxima view:

    exp(−2000 × 0.0013494) = exp(−2.6988) = 0.0672862

    0.0671637 against 0.0672862. These two are not the same quantity — the second is an approximation to the first, good because 0.0013494 is small — and they agree to three decimal places, which is the right amount of agreement to expect and a useful way to see that a block maximum and a threshold exceedance are two descriptions of one tail. There is about a 93% chance the year's largest claim exceeds ₹150 lakh, which is the number a reinsurance discussion would actually start from.

  7. A mean excess, to confirm the heavy tail

    Raise the threshold from 50 to 150 and the fitted model says the mean excess becomes

    (β + ξ × 100)/(1 − ξ) = (18.75 + 25)/0.75 = 43.75/0.75 = 58.3333333

    against 25.0000000 at a threshold of 50. The expected further amount of a claim has more than doubled now that the claim is known to be large. That is the mean residual life increasing, it is the signature of a heavy tail, and it is the sentence to remember from this page: with ξ > 0, finding out that a claim is already big is bad news about how big it will get.

  8. What this calculation does not justify

    Everything above rests on 80 exceedances and a threshold chosen by eye. The 1-in-1,000 figure of ₹163.6 lakh is an extrapolation a little way past the data and is reasonable. A 1-in-100,000 figure from the same 80 points would not be, however confidently the formula produces one. Refitting at thresholds of 40 and 60 and reporting how far the answer moves is part of the work, not an optional refinement.

Where this fits in CS2

This page explains one idea from CS2 Risk Modelling & Survival Analysis. The full CS2 syllabus, with what each part is examined on, sits on the course page, and past-paper style mock exams with full solutions are where you find out whether the idea has stuck under time pressure.

Other CS2 topics explained

Stuck on a term rather than the idea? The actuarial glossary defines the notation used above. If something here is wrong or unclear, tell us — these pages get corrected.