CM2 · explained

Measures of investment risk: variance, semi-variance, VaR and Tail VaR

Five ways of putting one number on how risky an investment is, each answering a different question. This page explains what each measure captures, what it throws away, and why a fund can look safer on one and riskier on another.

CM2 · Financial Engineering & Loss Reserving · 11 min read

After this page you can

Calculate variance, semi-variance, shortfall probability, Value at Risk and Tail Value at Risk, and say which measure suits which decision

Why there is more than one measure

"How risky is it?" is not one question. A pension fund trustee asking whether the fund will be able to pay the pensions is not asking the same thing as a bank asking how much capital it needs to survive a bad week, and neither is asking what a statistician means by spread. Each measure in this topic answers one of those questions well and the others badly, and the examinable skill is knowing which is which.

What they all have in common is that they compress a whole distribution of outcomes into a single number. Something must be lost in that compression. Knowing what each one discards is the fastest way to remember what it is for.

Variance and standard deviation

Var(R) = E[(R − μ)2] = E[R2] − μ2

In words: the average squared distance of the return from its mean. The standard deviation is its square root, which puts the answer back into the units of the return.

Variance is the measure that the rest of investment theory is built on, and for one reason: it adds up. The variance of a portfolio can be computed from the variances of its holdings and the covariances between them, which is what makes mean-variance portfolio theory and the Capital Asset Pricing Model possible at all. No other measure on this page has that property.

What it discards is direction. A return 10 percentage points above the mean and one 10 points below contribute identically, because the deviation is squared. For a symmetric distribution that is harmless. For an insurance liability, or a portfolio holding options, or anything else with a long tail on one side, it is a real distortion: variance treats a windfall as a risk.

Downside semi-variance

semi-variance = E[((R − μ)−)2] = ∑ pi(ri − μ)2 over the outcomes with ri < μ

In words: the variance calculation, but only the outcomes below the mean are counted. The ones above it contribute nothing.

This is the obvious repair to variance, and it is a genuine improvement in interpretation: an investor is worried about losses, not about gains. It comes with two costs. It no longer adds up across a portfolio, so it cannot support the theory that variance supports. And it uses only part of the data, so it is estimated less reliably from the same sample.

One check is worth building in. For a symmetric distribution the semi-variance is exactly half the variance, because the two halves of the calculation are mirror images. So the ratio of semi-variance to variance is a reading on skewness: above a half means the downside dominates, below a half means the upside does. On the example below the ratio comes out at 0.5056, which says the distribution is very slightly weighted to the downside — a fact that the variance alone could never have told you.

Shortfall probability

shortfall probability = P(R < L) for a chosen benchmark L

In words: the chance of ending up below some level you care about — zero, the risk-free rate, the return the liabilities need.

Its strength is that a trustee understands it immediately: "there is a one-in-five chance of losing money this year" needs no explanation. Its weakness is severe and is almost always the point of an exam question: it says nothing whatever about how bad the shortfall would be. An investment with a 20% chance of losing one rupee and one with a 20% chance of losing everything have the same shortfall probability. Used alone it will prefer a small chance of ruin to a larger chance of a trivial loss.

Value at Risk

VaRα is the loss that is exceeded with probability 1 − α
for a normal return, the α percentile of loss is zασ − μ

In words: pick a confidence level, say 95%. Value at Risk is the loss you are 95% confident of not exceeding over the stated period. It is a percentile of the loss distribution and nothing more.

Two things have to be stated with a Value at Risk figure or it means nothing: the confidence level and the time horizon. A 95% one-day figure and a 99% one-year figure for the same portfolio differ by an order of magnitude.

Value at Risk is in every regulatory framework and has one well-known defect, which is the reason the next measure exists. It is a single point on the distribution, so it is blind to everything beyond it. Two portfolios can have identical Value at Risk while one loses a little more past that point and the other loses everything. It also depends heavily on the distributional assumption: assume normality for something with fat tails and the figure will be comfortably, systematically too small.

Tail Value at Risk

TVaRα = E[loss | loss > VaRα]
for a normal return, E[R | R < μ − zασ] = μ − σ · φ(zα) / (1 − α)

In words: given that things have gone worse than the Value at Risk level, how bad are they on average? It is the average of the tail rather than its boundary.

That one change fixes the defect. Tail Value at Risk uses the whole tail, so it distinguishes a portfolio that loses a little past the threshold from one that is wiped out. It is always at least as large as the Value at Risk at the same confidence level, and the gap between them is itself informative: a wide gap means a nasty tail.

It has one further property that matters in practice and is worth stating in an exam. Tail Value at Risk is sub-additive — combining two portfolios can never make it larger than the sum of its parts. Value at Risk can fail that, which means it can report that diversification has increased risk. For a measure used to set capital, that is a serious flaw.

Which measure for which job

  • Building a portfolio. Variance, because it is the only one the theory can carry through a portfolio calculation.
  • Explaining risk to a trustee. Shortfall probability against the return the liabilities actually need, supported by something that measures size.
  • Setting capital. Tail Value at Risk, with Value at Risk reported alongside it because regulators and comparisons require it.
  • Comparing skewed investments. Semi-variance, or the ratio of semi-variance to variance, which reads the asymmetry directly.

Questions students ask

Is standard deviation really a bad measure of risk?

It is an excellent measure of spread and an incomplete measure of risk. For a roughly symmetric distribution the two coincide closely enough that the distinction does not bite. For anything skewed — and an insurance liability is always skewed — it understates what the decision-maker cares about, because half of what it counts is good news.

Why is Tail Value at Risk not simply used everywhere?

Partly history, partly estimation. Value at Risk arrived first and is embedded in rules, reporting and comparisons. More substantively, Tail Value at Risk depends on the shape of the extreme tail, which is where there is least data, so it is harder to estimate reliably. The usual practical answer is to report both and treat a large gap between them as the finding.

Does a fat-tailed distribution change the answer much?

Yes, and increasingly so as you move out into the tail. Take a normal distribution and a fatter-tailed one with the same mean and the same variance. At a 95% level the two give similar figures, because 95% is not far out. At 99% or 99.5% the fat-tailed figure can be dramatically larger. That is why a question will specify the distribution, and why matching only the mean and the variance of a real return series is not enough for capital work.

Can a risk measure be gamed?

A portfolio can be built to look good on any single measure. Selling far-out-of-the-money options collects a steady premium and leaves a small probability of a very large loss — which flatters the standard deviation, flatters the shortfall probability and, if the strike is far enough out, flatters the Value at Risk too. Tail Value at Risk is the measure in this list that notices. That is the strongest practical argument for it.

A worked example

Two investments are compared over one year.

Investment A has the discrete distribution of annual returns below. Investment B has returns that are normally distributed with mean 4% and standard deviation 7% a year.

Both sets of figures are invented for this example. They are not real funds and not a forecast of anything.

Return (% a year)Probability
−100.05
−30.15
20.40
70.30
140.10

The probabilities sum to 1.00, which is always the first thing to check.

  1. Investment A: the mean

    μ = 0.05(−10) + 0.15(−3) + 0.40(2) + 0.30(7) + 0.10(14)
       = −0.50 − 0.45 + 0.80 + 2.10 + 1.40 = 3.35%

  2. The variance, both ways

    Route one, through the second moment:

    E[R2] = 0.05(100) + 0.15(9) + 0.40(4) + 0.30(49) + 0.10(196)
        = 5.00 + 1.35 + 1.60 + 14.70 + 19.60 = 42.25
    Var = 42.25 − 3.352 = 42.25 − 11.2225 = 31.0275

    Route two, straight from the definition:

    rr − μ(r − μ)2p(r − μ)2
    −10−13.35178.22258.911125
    −3−6.3540.32256.048375
    2−1.351.82250.729000
    73.6513.32253.996750
    1410.65113.422511.342250
    Total——31.027500

    31.0275 both ways, with no rounding in either. The standard deviation is √31.0275 = 5.5702%.

  3. The downside semi-variance about the mean

    Keep only the rows with a return below 3.35% — the first three — and add the same last column:

    8.911125 + 6.048375 + 0.729000 = 15.6885

    The downside semi-standard-deviation is √15.6885 = 3.9609%.

    Now the check that makes the number mean something. The ratio of semi-variance to variance is

    15.6885 / 31.0275 = 0.5056

    For a symmetric distribution this would be exactly 0.5000. At 0.5056 the distribution is very slightly tilted so that the below-average outcomes are, in squared terms, marginally more spread than the above-average ones. It is a small effect, and the point is that variance on its own could not have detected it at all.

  4. The shortfall probability below zero

    Only the first two outcomes are negative:

    P(R < 0) = 0.05 + 0.15 = 0.20

    A one-in-five chance of losing money. Note immediately what this figure cannot tell you: that one of those two outcomes is a 10% loss and the other a 3% loss. It counts them the same.

  5. Investment B: 95% Value at Risk

    For a normal distribution, the 5th percentile sits 1.6449 standard deviations below the mean.

    5th percentile of return = 4 − 1.6448536 × 7 = 4 − 11.5140 = −7.5140%

    So the 95% Value at Risk is a loss of 7.514% measured from today's value, or 11.514% below the expected return. Both are correct answers to differently worded questions, and they differ by exactly the mean of 4%. Say which one you have given.

  6. Investment B: 95% Tail Value at Risk

    The average return within that worst 5%:

    φ(1.6448536) = 0.1031356
    φ(z) / (1 − α) = 0.1031356 / 0.05 = 2.0627128
    TVaR return = 4 − 7 × 2.0627128 = 4 − 14.4390 = −10.4390%

    So the Tail Value at Risk is a loss of 10.439%, against the Value at Risk's 7.514%. Conditional on a bad year, the average bad year is about three percentage points worse than the threshold that defined it.

  7. The second route, as a check

    The formula above can be verified without using it. Average the return numerically over the tail — weight each return by the normal density, integrate from far below up to the 5th percentile of −7.5140%, and divide by the mass found. Doing that over 200,000 intervals returns a tail mass of 0.0500000, as it must, and a conditional mean of −10.438990%. The closed form gave −10.438990%. They agree to six decimal places, so the formula has been used correctly and the two figures are the same calculation seen twice.

  8. Putting the two investments side by side

    MeasureInvestment AInvestment B
    Expected return3.35%4.00%
    Standard deviation5.5702%7.0000%
    Downside semi-standard-deviation3.9609%4.9497%
    Shortfall probability below 00.20000.2839
    95% Value at Risk (loss from zero)—7.514%
    95% Tail Value at Risk (loss from zero)—10.439%

    Investment B's semi-standard-deviation is 7 / √2 = 4.9497% because a normal distribution is symmetric, so exactly half its variance is downside — the check from the body of this page, used in reverse. Its shortfall probability below zero is Φ(−4/7) = Φ(−0.5714) = 0.2839.

    B offers 0.65 percentage points more expected return for 1.43 points more standard deviation and a materially higher chance of a losing year. Which is preferable is not a question any of these numbers answers; it depends on the investor, which is where the utility theory of the previous topic comes back in. What the numbers do is make the trade-off explicit, and that is all a risk measure is ever asked to do.

Where this fits in CM2

This page explains one idea from CM2 Financial Engineering & Loss Reserving. The full CM2 syllabus, with what each part is examined on, sits on the course page, and past-paper style mock exams with full solutions are where you find out whether the idea has stuck under time pressure.

Other CM2 topics explained

Stuck on a term rather than the idea? The actuarial glossary defines the notation used above. If something here is wrong or unclear, tell us — these pages get corrected.