¿ìè¶ÌÊÓƵ

Roses, their sharp spines being gone, Not royal in their smells alone,
But in their hue, Maiden pinks, of odour faint, Daisies smell-less, yet
most quaint, And sweet thyme true

Thus begins Act 1 of The Two Noble Kinsmen, a play first performed in
about 1613 by a troupe of London actors known as The King’s Men. It tells
the story of two knights, devoted cousins who find themselves rivals for
the favours of a woman. Despite its enduring theme, the play quickly fell
out of the troupe’s repertoire, and was not performed again for more than
three centuries. But as long ago as 1634 a purported script of the play
was being circulated, bearing an intriguing inscription on its title page:
‘Written by the memorably Worthies of their time Mr John Fletcher, and Mr
William Shakespeare, Gent’.

Could this obscure play be an unrecognised work of the greatest dramatist
of all? Scholars have pored over The Two Noble Kinsmen, and some have hailed
it as a genuine collaboration between Shakespeare and Fletcher, who is known
to have succeeded the Bard after his death in 1616 as chief dramatist to
the King’s Men. Others, however, remain unconvinced.

The debate over the origin of The Two Noble Kinsmen highlights an issue
that has raged in scholarly circles for centuries. Can questions of authorship
be settled objectively – perhaps even quantitatively? Many would recoil
at the suggestion that something as ineffable as ‘literary style’ could
be captured by numbers. And yet statisticians and computer scientists have
now developed techniques that do seem to provide a quantitative assessment
of style.

Computer scholar

One such technique, which we have developed over the past year, mimics
the way human scholars acquire expertise. We set up a computer to behave
as if it were a collection of very simple brain cells – a so-called neural
network – and then trained it to recognise literary style by showing it
numerous examples of Shakespeare’s work. Applying our neural network to
disputed works such as The Two Noble Kinsmen has produced some interesting
results, and may help to settle some bitter arguments over the authorship
of controversial literary texts.

But the neural network is only the latest technique in a field of research
known as stylometry, which dates back to the middle of the 19th century.
In 1851 the English logician Augustus De Morgan suggested that mathematics
might resolve a debate over the authorship of certain biblical texts. In
particular, he wondered if the work of different authors might be distinguished
by differences in the length of the words they used.

De Morgan’s hypothesis came to the attention of Thomas Corwin Mendenhall,
an American physicist, in the 1880s and he built on the idea to create ‘word
spectra’. These were designed to detect different literary styles by changes
in the frequency of words of different lengths.

Bacon and the Bard

Mendenhall applied his technique to the notorious argument over whether
Francis Bacon was the true author of Shakespeare’s works. In 1901, Mendenhall
published the results of an impressive display of academic labour in which
he measured the lengths of 200 000 words from works known to have been written
by Bacon and 400 000 from writing generally attributed to Shakespeare, and
built up the word spectra for each.

At first, Mendenhall believed that he had found a unique mathematical
‘signature’ for Shakespeare: a preponderance of four-letter words. However,
he later found that Christopher Marlowe – another contemporary and alleged
author of the Bard’s work – shared the same characteristics. Mendenhall’s
colossal undertaking had merely proved that word spectra could not uniquely
identify authors. Intriguingly, however, the possibility that the young
Shakespeare was strongly influenced by Marlowe is now being taken seriously
by a number of scholars.

Mendenhall did leave one major contribution to modern stylometry: he
showed that large amounts of text are needed to make any progress. Humans
do not stop after every paragraph to ensure they have achieved a particular
frequency of words. Only over lengthy passages – many hundreds, even thousands
of words long – do stylistic characteristics stand any chance of rising
above statistical noise.

Not surprisingly, the daunting amount of work this implies deterred
many researchers from following Mendenhall. More than 30 years elapsed before
the statistician G. Udny Yule in England and the linguist George Zipf in
the US moved stylometry on. By then statistical methods for testing hypotheses,
unknown in Mendenhall’s time, had been developed. These have come to play
a key role in boosting the objectivity of stylometry.

Yule and Zipf discovered that the richness of vocabulary in a text,
as measured by the frequency of use of different words, seemed to follow
patterns that might distinguish one author from another. Specifically,
the number of words occurring with a particular frequency declines as the
frequency grows. This means that there are more unique words, used just
once by an author, than words used twice, and so on. Only a handful of words,
such as ‘the’, are used very frequently. Yule devised an objective measure
of this feature of word frequencies, based on the Poisson probability distribution,
which he called Characteristic K. Early results suggested that this might
be the reliable and objective test of authorship scholars sought, with different
styles producing different values of K. But it later emerged that a wide
range of K values could be produced by the same author – a fatal flaw in
any test intended to pin down authorship.

Not until the early 1960s was the academic world given a convincing
demonstration of the power of stylometry. Two distinguished American statisticians,
Frederick Mosteller and David Wallace, decided to use stylometry to attack
the mystery of the authorship of the so-called Federalist Papers. Published
in New York newspapers in 1787 and 1788, the papers consist of political
essays written to persuade voters to ratify the constitution for the new
United States.

All 85 essays were signed ‘Publius’, but three politicians were held
to be behind this pseudonym: Alexander Hamilton, John Jay and James Madison,
who later became the fourth US president. Although scholars were broadly
agreed on the authorship of most of the essays, a dozen remained the subject
of dispute. Some essays were thought to be by Madison, others by Hamilton.

As stylometrists, Mosteller and Wallace had two major advantages over
their predecessors: powerful statistical techniques and that most vital
and uncomplaining of research assistants, an electronic computer. They used
the computer to trawl through undisputed works of Madison and Hamilton looking
for words used in significantly different ways by the two authors – and
which might thus serve as potential signatures of each author’s style.

Mosteller and Wallace conducted a meticulous search, covering tens of
thousands of words, in which they weeded out context-dependent words and
words that both authors used with the same frequency. The end result was
a collection of words capable of separating Madison’s writings from those
of Hamilton.

For example, ‘upon’ averaged 33.5 appearances per 10 000 words in the
writings of Hamilton, but only 1.4 in those of Madison. Hamilton never used
the word ‘whilst’ in the test samples, but in Madison’s writings it appeared
4.8 times.

Federal investigations

To improve their chances of separating the two authors, Mosteller and
Wallace derived mathematical distributions of many discriminatory words
to measure how Madison-like or Hamilton-like a disputed text was. These
showed that all the disputed papers appeared to have been written by Madison
– a view shared by the majority of historians albeit on mainly subjective
grounds.

This confirmation of a prevailing scholarly belief gave stylometry a
much-needed boost in academic circles. Until then, its methods had seemed
ad hoc, and its application academically dubious. Indeed, the findings of
Mosteller and Wallace were an ideal way to open the modern, computerised
age of stylometry: they were interesting without being overly controversial.
Mosteller and Wallace’s painstaking and scholarly analysis, later turned
into a book (Applied Bayesian and Classical Inference: The Case of the Federalist
Papers, Springer, 1984), is still widely seen as the classic work of literary
detection.

Since then, growing computer power and machine-readable versions of
many literary works have helped overcome the appalling drudgery otherwise
needed to carry out stylometric analysis. They have also led to a plethora
of techniques being brought to bear on authorship issues.

Most researchers believe that common words are of most value in characterising
an author’s stylometric ‘signature’. They study the so-called function words,
which include conjunctions (‘and’, ‘so’), prepositions (‘on’, ‘upon’), articles
(‘an’, ‘the’) and certain verbs and adverbs. Such words are needed to construct
virtually any statement, and are not particularly context-dependent. They
also tend to be used unconsciously. So, the reasoning goes, their frequency
will not be altered when a writer is attempting to change – or mimic – a
literary style. Also, such words are very common, giving plenty of data
and thus a better chance of detecting a stylistic signature above statistical
noise.

But some stylometric research has been based on the opposite view: that
it is the rare words that most clearly characterise a literary style. This
is the idea underpinning a controversial stylometric technique whose findings
brought stylometry out of academia and into the newspapers.

Developed by statisticians Bradley Efron at Stanford University in California
and Ronald Thisted at the University of Chicago, the technique’s origins
lie in a curious question: how many unknown animal species are there still
to be found? According to legend, this question was first raised in the
1940s by a naturalist back from an expedition in search of new butterflies.
Fortunately, he was talking to the great British statistician R. A. Fisher,
who showed that this apparently silly question did not have a silly answer.
By making probabilistic assumptions about the way the capture process works,
Fisher used the number of species already caught to estimate the number
that would be caught if naturalists carried on searching forever.

In the mid-1970s, Efron and Thisted pointed out that species could just
as easily be different words. In this case, the number of words already
‘caught’ are those in the existing works of an author, and the words still
to be discovered are those that the author knew but never used in extant
writings.

Efron and Thisted applied Fisher’s technique to the works of Shakespeare.
The aim was to estimate how large a vocabulary the Bard actually had. They
took as their starting point the 884 647 words contained in all writing
that is officially recognised as being genuine Shakespeare. Of these words,
14 376 appear once, 4343 twice and so on. It turns out that Shakespeare
used 31 534 different words in his works. Using Fisher’s idea, Efron and
Thisted were then able to show that a published vocabulary of this size
suggests that Shakespeare knew at least a further 35 000 words which never
appear in his plays and poems.

It is an intriguing claim, but could it ever be tested? Unknown to Efron
and Thisted, the answer was lying neglected on the shelves of the Bodleian
Library of the University of Oxford. There, in November 1985, the Shakespeare
scholar Gary Taylor found an ageing volume containing an untitled and anonymous
poem that begins ‘Shall I die’. Taylor’s instincts led him to hail the poem
as a previously undiscovered work of Shakespeare, a controversial claim
that was quickly picked up by the media.

A few weeks later Thisted read about it in his Sunday newspaper, and
realised that Fisher’s method could be used to put Taylor’s claim to the
test. If ‘Shall I die’ were an addition to Shakespeare’s works, it should
contain some previously unseen words taken from the playwright’s ‘hidden’
vocabulary. Working with Efron, Thisted calculated that if ‘Shall I die’
were Shakespearean, it should contain about seven previously unseen words.
In fact, nine new words appeared – striking backing for Taylor’s claim.

Measure for measure

Thisted and Efron went much further, however: they turned Fisher’s basic
theory into a series of authorship tests. They developed three measures
which, starting from an analysis of known works by a particular author,
predict the total number of different words expected to appear in a work
of given length by the author, the number of previously unseen words and
the distribution of word frequencies. By comparing the predicted results
for different authors with the corresponding quantities found in a mystery
work, they could test the likelihood of the work being by a particular author.

Thisted and Efron applied their three tests to ‘Shall I die’. As experimental
controls, they also tested some undisputed Shakespearean poems and poems
by contemporaries such as Marlowe and Ben Jonson. Although the tests for
different words and for unseen words were only moderately successful, they
found that the test characterising word frequency did seem able to ascribe
poems to their correct authors. The test also backed Taylor’s claim for
‘Shall I die’.

With their solid theoretical foundation and apparent discriminatory
power, Thisted and Efron’s authorship tests have been eagerly examined by
stylometrists since their publication in the journal Biometrika in 1987.
Using both genuine and computer-generated texts, Robert Valenza at Claremont
McKenna College in California has discovered that Shakespeare’s use of words
in his plays is remarkably consistent with Thisted and Efron’s theory.
He also found that some of the tests discriminated between works of Shakespeare
and Marlowe.

However, Valenza discovered that – as so often in stylometry – there
are traps for the unwary. The Thisted-Efron tests failed to convincingly
ascribe widely accepted Marlovian works to Marlowe. The tests were also
poor at correctly ascribing some of Shakespeare’s poems. More worrying
still, Valenza found that Shakespeare’s use of language differed significantly
between his poetry and his plays. In retrospect, this is not surprising:
language in poetry is clearly more tightly constrained than it is in plays.
This finding does, however, raise grave doubts over Thisted and Efron’s
attribution of ‘Shall I die’ to Shakespeare. Their conclusion was based
on comparison with all Shakespeare’s known works – plays and poetry – lumped
together.

What should have been done was a comparison of word use in ‘Shall I
die’ and Shakespeare’s extant poems. Frus-tratingly, however, there is too
little Shakespearean poetry to guarantee the reliability of any such comparison.
Although Thisted and Efron’s techniques are of some stylometric value, it
seems they will never solve the mystery of ‘Shall I die’.

A more clear-cut stylometric result on a controversial subject was secured
recently by David Holmes of the University of the West of England, Bristol.
To members of the Church of Jesus Christ of Latter-day Saints, the Book
of Mormon is a sacred text. It recounts the experiences of a family of Jews
who fled from Jerusalem shortly before its destruction in 597 BC, and arrived
in America. Their experiences are said to have been engraved on golden
plates, found more than 2400 years later by Joseph Smith, the son of a poor
New England farmer. Using magical stones, Smith is said to have translated
the plates to provide the basis of Mormon religion.

Smith the prophet

Sceptics take the more prosaic view that the Book of Mormon was concocted
by Smith himself. To investigate the controversy, Holmes turned to a range
of statistical measures of vocabulary richness. These he applied to the
Book of Mormon, to Smith’s personal writings and, crucially, to the so-called
Doctrine and Covenants of the Mormon church, divine guidance given to the
early Mormons via Smith himself.

Holmes discovered that the style of the Book of Mormon was indeed distinct
from that of Smith’s personal writings – apparently backing Mormon claims.
However, the stylometric tests also showed that the Book of Mormon appeared
to be the work of just one author; an author whose style, moreover, was
identical to that of Smith when he was allegedly given the Doctrine and
Covenants. Holmes concluded that the Book of Mormon is just another example
of Smith writing in a prophetic style.

Like Mosteller and Wallace, Holmes used a series of tests to capture
as much of the stylistic ‘signal’ in the text as possible. Even so, statistical
noise was still a serious problem for him. And it was this problem in stylometry
that, in October 1992, prompted us to investigate the use of neural networks
to attack questions of authorship.

Loosely based on ideas drawn from neurophysiology, a neural network
is usually an ordinary computer programmed to behave as if it were a network
of very simple neurons. One layer of these neurons acts as an input, while
another produces an output. Solving problems then becomes a matter of training
the network so that a particular input produces the appropriate output.
For example, the input could be a pattern of squares looking like a dog,
while the output could be the word ‘dog’.

Neural networks have proved particularly useful in classifying complex
data in the presence of statistical noise. This is akin to the human brain’s
ability to pick out a face in a crowd. Also, they reach their decisions
without having to make a lot of potentially false assumptions about the
statistical properties of the data – unlike most stylometric techniques.
All this suggests that neural networks may be particularly useful for investigating
questions of authorship. To find out, we decided to train a neural network
to recognise the literary styles of Shakespeare and his contemporaries.
Our stylometric neural network consists of five inputs, each of which takes
a stylometric measure extracted from text, and outputs that together give
an estimate of how ‘Shakespearean’ the network deems a text to be.

The first task was to train the network. This we did by exposing it
to data extracted from a large number of samples of Shakespeare’s undisputed
work, together with that of his successor with The King’s Men, John Fletcher.
The samples ran to around 1000 words, and we used 50 samples of text from
each author; 100 000 words in all. Five stylometric measures, based on the
function words ‘are’, ‘in’, ‘no’, ‘of’ and ‘the’, were then extracted from
each sample text. The measures took the form of ratios of the number of
times each function word appeared to the total number of words in the text,
and were used as the inputs to the network. The neural network was then
repeatedly shown the sample texts, up to 100 times per text, until it could
confidently recognise the works of both Shakespeare and Fletcher.

Once trained, the network turned out to be extremely good at recognising
works of both Shakespeare and Fletcher it had never seen before. The system
produced an output of near to 1 when presented with work it deemed to be
by Shakespeare, and an output of near to 0 when it deemed the author to
be Fletcher. The network came out with values of around 0.5 when it was
unsure. As well as identifying whole plays correctly, we found that the
neural network could even correctly identify the author of individual acts
of undisputed provenance.

We then set the neural network loose on the mystery of The Two Noble
Kinsmen. Drawing on a wide variety of essentially subjective evidence,
scholars have claimed that Shakespeare’s hand dominates Acts I and V, with
much of the rest appearing to be by Fletcher. In March last year, our neural
network agreed with these attributions – and proffered the extra opinion
that Fletcher may have received considerable help from Shakespeare in Act
IV. In short, our neural network quantitatively supports the subjective
view of its much more sophisticated human counterparts that The Two Noble
Kinsmen is a genuine collaboration between Shakespeare and one of his contemporaries.

Neural discrimination

Two months later, we also trained the neural network to discriminate
between the works of Shakespeare and Marlowe. Although we used the same
training principles, we adopted different stylometric measures that conventional
statistical techniques suggested were likely to produce accurate results.
Essentially, we altered the function words and the ratios we used.

Again, the trained network turned out to be adept at classifying plays
it had never seen before, and has shed light on a number of scholarly questions.
In particular, the neural network suggests that the anonymous play Edward
III was written by Shakespeare under considerable influence from Marlowe,
and that The Third Part of King Henry VI is a Shake-spearean revision of
a lost Marlovian play. Two research papers detailing the work we have done
are due to be published over the next few months in the journal Literary
and Linguistic Computing.

Like Mosteller and Wallace, we have focused on relatively well-defined
authorship questions, involving only two likely contenders, but there is
scope for further research with neural networks trained on the corpus of
other potential writers for works whose authorship is in doubt. Such techniques
could, for example, cast light on the origin of the anonymous play Edmund
Ironside, which some scholars claim represents Shakespeare’s earliest work.
They may also prove capable of sensing Shakespeare’s hand in works as short
as ‘Shall I die’.

Although neural networks have some advantages over more traditional
techniques, they have no claims to infallibility. As with any new stylometric
method, their importance lies in their ability to add further weight of
evidence in support of a particular theory.

Scholars trained in the humanities have tended to be suspicious of attempts
to quantify the essence of literary style, but we believe this attitude
is changing. Neural networks, together with more conventional stylometric
techniques, could yet form a permanent bridge between the Two Cultures.

Robert Matthews is visiting fellow in computer science at Aston University
and science correspondent of The Sunday Telegraph. Tom Merriam is a retired
history lecturer, who in 1992, received a PhD from King’s College London,
for his investigations of the links between authorship and word frequency
distributions.