When Hurricane Fran swept towards America鈥檚 eastern seaboard last month, it
was not just the residents of North Carolina who were quaking. Insurance
companies also kept a fearful eye on the weather reports. Would Fran run out of
puff after striking land, or would she turn into a monster like Hurricane Andrew
of 1992, and engulf them in a $16 billion tidal wave of claims?
In the event, Fran was bad, but not as bad as Andrew. She killed about 20
people and hit the insurers with a bill for $650 million. The insurance
market had weathered yet another storm. But for how much longer? Just what are
the chances of a truly 鈥渂ig one鈥 striking in the next 10 years? This is no idle
speculation. Without a way to estimate the risk of extreme events, insurance
actuaries cannot set sensible premiums. Put them too high, and no one is
interested, but set them too low, and the company goes belly-up come that
ultimate rainy day.
Now a small band of statisticians believe they have the mathematical tools
for doing the seemingly impossible: predicting the chances of events that have
never happened before. Known as extreme value theory, it is an outgrowth of
pioneering research carried out 70 years ago, but only now is its astonishing
power being recognised. It can help actuaries predict the likelihood of events
that are incredibly rare, but so devastating that they would threaten the
survival of their companies. Engineers can use it to work out what rigours a
bridge, ship or oil rig should be built to withstand. And in academic research,
EVT is already being used to probe the mystery of how long humans can live.
Advertisement
At the heart of EVT is the standard statistical idea that the frequency of
random events follows a mathematical rule, known as a distribution. Measure the
heights of a stadium-full of soccer fans, for example, and you鈥檒l find a few
very short adults, a few long-shanks, and the bulk somewhere in between. The
frequency with which different heights turn up follows a bell-shaped curve
called the normal distribution. First derived from probability theory more than
two hundred years ago, the normal distribution is now a mainstay of modern
statistics.
In the 1920s, a number of statisticians including the founder of modern
statistical theory, the English mathematician R. A. Fisher, discovered that it
is not just run-of-the-mill phenomena like heights that follow a predictable
distribution. They found that extreme values, such as the very tallest or
shortest of a set of heights, also follow their own special family of curves.
And, once again, it is possible to deduce the shape of these curves from basic
probability theory.
The idea is that when a new extreme value occurs鈥攁 record downpour,
say鈥攊t will usually change the average value for all the extremes seen to
date, but leave the shape of the extreme value curve the same. Using probability
theory, it is possible to show mathematically that only three curves behave this
way. Known technically as the Gumbel, Frechet and Weibull distributions, they
give a precise quantitative statement of the otherwise vague notion that the
more extreme the event, the less likely it is to happen.
Bigger floods
These distributions do for EVT what the normal distribution does for everyday
events: they provide a mathematically rigorous way to analyse past records of,
say, extreme floods, and to put figures on the risk of an even bigger flood
being seen over the next century.
It sounds wonderfully simple and powerful, so where has EVT been for all
these years? According to Richard Smith at the University of North Carolina, one
of the chief advocates of EVT, early attempts at applying the theory suggested
that vast amounts of data were needed before its results could be reliable.
鈥淭hese results seemed to imply that the theory only worked if you had around a
million million data points, which appeared to rule out any real applications,鈥
says Smith. 鈥淏ut more recent research shows that it can work pretty well with
just a few hundred.鈥 Statisticians also worried because the early EVT work
assumed that extreme events were independent of each other. Yet many phenomena,
from quakes to financial markets, have some 鈥渕emory鈥 of past events. Past
tremors may presage a massive quake, while market crashes can make traders
jittery for years. 鈥淚t took a long time for the theory to go beyond that
assumption鈥, says Smith. 鈥淏ut now it has, it鈥檚 possible to apply it to
time-series data, such as stock market fluctuations.鈥
Now that the basic theory has been sorted out, EVT is beginning to flex its
muscles. In the Netherlands, Laurens de Haan and his colleagues at Erasmus
University, Rotterdam, have an understandable interest in EVT. With almost half
of their country below sea level, predicting extreme floods is a life or death
matter and, says de Haan, 鈥淓rasmus University is in the lowest-lying area of
all鈥. For millions of Dutch people, disaster is only the height of a sea wall
away. In February 1953, for example, a severe storm surge broke through sea
defences to kill 1800 people. Ever since, the Dutch authorities have taken a
keen interest in using the latest statistical methods to guide their efforts to
keep the sea at bay.
Following the 1953 disaster, a panel of experts was formed to analyse flood
records and come up with a sea wall design that would withstand a surge so
extreme it would be expected just once every 10 000 years. It quickly became
clear that the 1953 flood鈥攁t 3.85 metres above sea level鈥攚as not the
worst that had hit the country. A flood on All Saints Day in 1570 topped 4
metres. Using a simple curve to fit its relatively small amount of data, the
panel estimated that a wall of 5 metres or so above sea level should meet the
once-in-10 000 year criterion. But just how much faith can the Dutch put in this
figure?
Since the late 1980s, de Haan and his colleagues have tried to answer this
question with help from EVT. They used a computer to fit the EVT distributions
to the historical data, and then used its shape to estimate the most extreme
height of flood likely to happen once in the next 10 000 years. By coincidence,
the original panel had used a simple version of the EVT distribution to model
the original data, so applying the full theory led to a height standard only a
shade above the panel鈥檚 recommendation. The real benefit, says de Haan, is that
it is possible to have far more confidence in the EVT result, as it is based on
solid mathematical principles about the behaviour of extreme events. 鈥淚t鈥檚 now
been officially adopted, and is being applied to new and existing structures,鈥
he says.
The British government is also taking EVT seriously. It recently commissioned
statisticians at Lancaster University to use it to assess the ability of
Britain鈥檚 eastern sea defences to withstand battering from the North Sea.
EVT鈥檚 ability to put guesstimates on a firm footing is likely to lead to
major applications in the financial sector. With extreme events presenting
insurance companies with an annual bill of over 拢10 billion, mere
empiricism seems a recipe for disaster. Paul Embrechts and colleagues at the
Swiss Federal Institute of Technology in Z眉rich are using EVT to help
insurance actuaries to sleep easier at night. Actuaries are always worried that
among the risk sectors in their portfolios lurk a few very risky areas where a
single 鈥渉it鈥 could be enough to empty the entire portfolio鈥檚 coffers.
鈥淰arious rules of thumb are being used by actuaries to measure the
dangerousness of their portfolios,鈥 says Embrechts. 鈥淥ne of these is the
so-called 20-80 rule: 20 per cent of the claims in a particular portfolio are
responsible for more than 80 per cent of the total portfolio claim amount.
Extreme value theory allows us to make this rule more precise by specifying
where the 20-80 rule applies.鈥
Colossal claims
To do this, an actuary can take historical data about all the areas covered
in a portfolio, and fit EVT distributions to them. The resulting curves will
then show how dangerous the portfolio is, by revealing the presence of areas
where a single event could spell financial ruin. The good news is that the 20-80
rule works pretty well for many insurance sectors, such as shipping, capable of
throwing up colossal claims. The bad news is that for others it is hopelessly
wide of the mark. EVT analysis of hurricane data, for example, reveals that a
0.1-95 rule applies. In other words, insurers can be chugging along nicely for
years, and then find themselves hit by a 1-in-1000 hurricane which swallows up
95 per cent of the total cover in one go.
Embrechts believes EVT will give insurers more confidence to cover high-risk
areas such as earthquake and wind storm insurance by reducing the danger that
they will not be covered when the 鈥渂ig one鈥 hits. 鈥淚t offers the industry a
sound, consistent theory within which premiums for cover against catastrophic
events can be worked out,鈥 he says.
One of the most intriguing applications of EVT centres on a subject of
interest to biologists and actuaries alike: human longevity. Attempts to find a
mathematical explanation of the relationship between age and death rates have
long been dominated by a rule of thumb first pointed out in 1985 by the English
actuary Benjamin Gompertz. He showed that, in essence, the older you are, the
shorter your life expectancy. The precise way in which life expectancy falls
with age is captured by the 鈥淕ompertz curve鈥 (see
Diagram). The curve appears to
fit the empirical data on human life spans very well, and actuaries have based
their life assurance premiums on the Gompertz curve for years.
Can humans live forever?
But for biologists, that success has an intriguing implication鈥攖he
Gompertz curve simply carries on forever, suggesting there is no limit to the
human life span. Can this apparently sensational finding be relied on? Current
biological theories of ageing are too primitive to give a definitive answer. But
pushing out from the known into the unknown is meat and drink to EVT, and at
Erasmus University de Haan and his student Karin Aarssen have used it to probe
the question of ultimate human longevity. They investigated, in the language of
EVT, the most extreme age to which humans can live.
As so often, they found that the empirical curve does have some mathematical
justification. The Gompertz curve turns out to be a special case of an extreme
value distribution. But does the curve give the best fit to the known data?
Taking mortality data for the Netherlands, Aarssen and de Haan found that
although the Gompertz curve may indeed work for the bulk of the population, it
starts to fail when dealing with extremes. Computer analysis showed that the
data for the 鈥渙ldest old鈥 fits a quite different extreme value distribution. And
crucially, it has a mathematical form which predicts that there is, after all, a
maximum life span for humans. Pinning down its precise value is made difficult
by the relatively small amount of data, says de Haan. 鈥淭he picture is not very
clear but we can say that a reasonable choice for the confidence bounds is 113
to 124.鈥
Despite being based on Dutch longevity data alone, this result certainly fits
well with one observational fact about human longevity: to date, no one has
lived beyond the upper age limit set by the EVT analysis. According to Smith,
such results highlight the value of EVT for those groping for solid results in
otherwise murky scientific areas. 鈥淭here is always going to be an element of
doubt, as one is extrapolating into areas one doesn鈥檛 know about鈥, he says. 鈥淏ut
what EVT is doing is making the best use of whatever data you do have about
extreme phenomena.鈥
With such extremes鈥攆rom natural disasters to stock market
crashes鈥攂eing the source of so much fascination, it can only be a matter
of time before EVT finds itself at the focus of controversy, and the first signs
are already emerging.
Last year, Michael Robinson and Jonathan Tawn from Lancaster University
turned the power of EVT on one of the most controversial athletic achievements
of recent years: the astonishing world records of China鈥檚 track star Wang
Junxia. At the national championships in Beijing on 12 September 1993, she ran
3000 metres in 8 minutes 12.19 seconds, smashing the previous record set nine
years earlier by an astonishing 10.43 seconds. The following day she did it
again, beating her own record by more than 6 seconds. Junxia鈥檚 amazing
performances triggered huge controversy in the West, including inevitable
accusations of drug-taking. The Chinese authorities hit back by pointing out
that drug tests had been negative.
Robinson and Tawn decided to see what light EVT could cast on the
controversy. Taking past records for the women鈥檚 3000 metres, they calculated
the best-fitting EVT distribution, and used it to calculate the most
extreme鈥攖hat is, quickest鈥攖ime likely to be possible on current
trends. The analysis revealed that while Junxia鈥檚 time was certainly unusual, it
was not incredible. According to EVT, the ultimate 3000 metres time is almost
certainly somewhere between 8 minutes 3 seconds and 8 minutes 17 seconds.
Junxia鈥檚 record lies within these bounds.
Legal disputes
Such research findings raise the possibility of EVT-based evidence being used
to settle legal disputes. 鈥淚 can see cases where it could be used in court鈥,
says Tawn. 鈥淔or example, in cases of alleged drug taking by athletes, it could
be used to quantify the evidence and help show that `obvious鈥 evidence of
cheating on the basis of past records does not in fact exist.鈥
Yet despite its solid mathematical foundations and huge range of potential
applications, EVT itself has a credibility gap to leap, even among
statisticians. It stems from EVT鈥檚 apparent ability to do the
impossible鈥攇ive reliable estimates of the chances of events that have
never been seen. 鈥淭here is doubt within the statistical community that you can
ever extrapolate data,鈥 admits Tawn. But such critics, he says, ignore the fact
that EVT is much more than extrapolation: 鈥淲e exploit everything we know about
the basic nature of the problem, together with all the available data鈥攁nd
then bring in probabilistic arguments to guide the extrapolation as well.鈥
The key message, says Tawn, is that EVT cannot do magic鈥攂ut it can do a
whole lot better than empirical curve-fitting and guesswork. 鈥淢y answer to the
sceptics is that if people aren鈥檛 given well-founded methods like EVT, they鈥檒l
just use dubious ones instead.鈥
- Further reading: 鈥淓xtreme Value Theory鈥 by Richard Smith in Handbook
of Applicable Mathematics: Supplement, Wiley, 1990, p437