Last Year in Marienbad
Who Wrote The Two Noble Kinsmen?
Abstract
Stylometry—Analysing Writing Styles
Stylometry
Robert Matthews, science correspondent of The Sunday Telegraph and a visiting fellow in computer science at Aston University at the time, and Tom Merriam, a stylometrist and retired history lecturer, discussed in the New Scientist (1994) the establishing of authorship by analysing writing. The Two Noble Kinsmen is a play first performed in about 1613 by The King’s Men, a company of London actors, the very company that William Shakespeare was associated with. It was not popular, it seems, and fell out of the company’s repertoire, not to be performed again for over three centuries. In 1634, a script appeared purporting to have been written by “Mr John Fletcher, and Mr William Shakespeare, Gent”. Modern scholars have given the script a mixed reception. Some say it is a genuine collaboration between Shakespeare, who died in 1616, and his successor as chief dramatist to the King’s Men, Fletcher. Others, however, remain unconvinced.
Can questions of authorship be settled objectively? Literary types do not like to think so, but statisticians and computer scientists have found ways of doing it. In 1851, the English logician Augustus De Morgan suggested that mathematics might resolve the debate over the authorship of certain biblical texts. He wondered if the work of different authors might be distinguished by differences in the length of the words they used. An American physicist, Thomas Corwin Mendenhall, heard of De Morgan’s idea in the 1880s and tried to find “word spectra” based on the frequency of words of different lengths. In 1901, Mendenhall published a laborious analysis of the lengths of 200,000 words from works known to have been written by Bacon and 400,000 from the works of Shakespeare. He had built up the word spectrum of each. Shakespeare seemed to be fond of four letter words, but later the same characteristic was found of Marlowe. So, Marlowe, if anyone, had written Shakespeare. That the young Shakespeare was strongly influenced by Marlowe is seriously considered by some scholars.
Mendenhall at least demonstrated that a lot of writing was needed to do any textual analysis. People are quite capable of writing differently according to the occasion. Only over lengthy passages—many hundreds, even thousands of words long—do stylistic characteristics rise above statistical noise. In the days before computers, it was a deterrent to progress. And it took years before statistical methods for testing hypotheses, unknown in Mendenhall’s time, had been developed, then a statistician and a linguist moved stylometry on.
G Udny Yule in England and George Zipf in the US discovered that the frequency of use of different words seemed to follow patterns from one author from another. The number of different words with a given frequency declines as the frequency grows. This means that there are more unique words, used just once by an author, than words used twice, and so on. Only a handful of words, such as “the”, are used very frequently. Yule devised an objective measure of this feature of word frequencies, based on the Poisson probability distribution, which he called “Characteristic K”. Different styles had different values of K. But, in similar fashion, people can write in different styles with different K values, so authorship could not be accurately determined.
Who Wrote the Federalist Papers?
In the early 1960s, two distinguished American statisticians, Frederick Mosteller and David Wallace, used stylometry on the mystery of the authorship of the Federalist Papers, published in New York newspapers in 1787 and 1788. They were political essays written to persuade voters to ratify the constitution of the new United States. All 85 essays were signed “Publius”, and three politicians were held to be behind the pseudonym—John Jay, Alexander Hamilton, and James Madison, the latter to be the fourth US president. Although scholars agreed on the authorship of most essays, a few were still disputed between Madison and Hamilton. Mosteller and Wallace had the advantage over predecessors of the use of a computer. They used it to trawl through undisputed works of Madison and Hamilton looking for words used in significantly different ways by the two authors which might thus serve as signatures of each author’s style.
From tens of thousands of words, they weeded out context-dependent ones and those that both authors used with the same frequency. For example, “upon” averaged 33.5 appearances per 10,000 words in the writings of Hamilton, but only 1.4 in those of Madison. Hamilton never used the word “whilst” in the test samples, but in Madison’s writings it appeared 4.8 times per 10,000. They derived mathematical distributions of many discriminatory words to measure how Madison-like or Hamilton-like a disputed text was. In the end, they showed that all the disputed papers seemed to have been written by Madison—a view most historians endorsed. Mosteller and Wallace’s analysis published as Applied Bayesian and Classical Inference: The Case of the Federalist Papers (Springer, 1984) is a seminal work of literary analysis.
Now, some researchers think common words best characterize an author’s stylometric “signature”. They study the so-called function words, which include conjunctions (“and”, “so”), prepositions (“on”, “upon”), articles (“an”, “the”) and certain verbs and adverbs, words that are needed in most expressions, not particularly context-dependent, and used unconsciously. So, a writer trying to change their style will find it hard to change how they use these common words, and their commonness means they are present even in quite short pieces. Others think it is the rare words that most clearly characterize a literary style.
Efron and Thisted—Words as Species
In the mid-1970s, statisticians, Bradley Efron at Stanford University in California and Ronald Thisted at the University of Chicago, pointed out that a method found by the great British statistician R A Fisher for estimating how many species remained undiscovered could be applied to words. “Discovered” words were those in the existing works of an author, and the words still to be discovered were those that the author knew but never used in extant writings. Used on Shakespeare, the full vocabulary of the Bard could be estimated just as the unknown species of animals in the world could. Efron and Thisted started from the 884,647 words officially recognised as being Shakespeare’s. Shakespeare used 31,534 different words in his works. Of these words, 14,376 appear once, 4343 twice and so on. A published vocabulary this big implies Shakespeare had a further 35,000 words which never appeared in his plays and poems.
Could the idea and the claim be tested? In November 1985, the Shakespeare scholar, Gary Taylor, found in an old volume an untitled and anonymous poem that begins “Shall I die…”. Taylor felt the poem was an undiscovered work of Shakespeare, a controversial claim that was quickly picked up by the media. Thisted read about it and realised that Fisher’s method could be used to test Taylor’s claim. If Shall I die were an addition to Shakespeare’s works, it should contain about seven previously unseen words in the playwright’s unknown vocabulary. Nine new words appeared—strikingly close to expectations.
Thisted and Efron went much further, turning Fisher’s calculation into a series of authorship tests. They developed three measures which, starting from an analysis of known works by a particular author, predict the total number of different words expected to appear in a work of given length by the author, the number of previously unseen words, and the distribution of word frequencies. By comparing the predicted results for different authors with the corresponding quantities found in a mystery work, they could test the likelihood of the work being by a particular author.
Thisted and Efron applied their three tests to Shall I die. As experimental controls, they also tested some undisputed Shakespearean poems, and poems by contemporaries such as Marlowe and Ben Jonson. Although the tests for different words and for unseen words were only moderately successful, they found that the test characterizing word frequency did seem able to ascribe poems to their correct authors. The test also backed Taylor’s claim for Shall I die.
Thisted and Efron’s authorship tests have been examined by stylometrists since their publication in the journal Biometrika in 1987. Using both genuine and computer-generated texts, Robert Valenza at Claremont McKenna College in California has discovered that Shakespeare’s use of words in his plays is remarkably consistent with Thisted and Efron’s theory. Some of the tests even discriminated between Shakespeare and Marlowe.
However, the Thisted-Efron tests failed to convincingly ascribe widely accepted Marlovian works to Marlowe. The tests were also poor at correctly ascribing some of Shakespeare’s poems. More worrying still, Valenza found that Shakespeare’s use of language differed significantly between his poetry and his plays—language in poetry is clearly more tightly constrained than it is in plays—raising doubt over the attribution of Shall I die to Shakespeare. Their conclusion was based on comparison with all Shakespeare’s known works—plays and poetry—lumped together when the poems only should have been used for reference. But there is too little Shakespearean poetry for reliability.
Joseph Smith, Prophet
To members of the Church of Jesus Christ of Latter-day Saints, the Book of Mormon is a sacred text, which recounts the experiences of a family of Jews who fled to America from Jerusalem just before 597 BC when Babylon attacked. The book was engraved on golden plates, allegedly found more than 2400 years later by Joseph Smith, the son of a poor New England farmer. Using magical stones, Smith is said to have translated the plates to provide the basis of Mormon religion. Skeptics see the Book of Mormon as concocted by Smith himself.
To investigate the controversy, David Holmes of the University of the West of England, Bristol, applied a range of statistical textual measures to the Book of Mormon, to Smith’s personal writings and, crucially, to the so-called Doctrine and Covenants of the Mormon Church, divine guidance given to the early Mormons by Smith himself. He showed that the style of the Book of Mormon was distinct from that of Smith’s personal writings—apparently backing Mormon claims—but the tests showed that the Book of Mormon was the work of a single author, an author whose style was identical to that of the Doctrine and Covenants. Holmes concluded that The Book of Mormon is in the style of Smith writing archaically or biblically.
Neural Networks
A modern computer based approach to style mimics the way humans do it—neural networks—which can be trained to recognize, say, Shakespeare’s work by showing them many examples of his work. In October 1992, R Matthews and T Merriam decided to investigate the use of neural networks in questions of authorship. A neural network is usually a computer programmed to behave as if it were a simple network of neurons. Some neurons act as input, while others provide output. Solving problems is training the network so that a particular input produces the appropriate output. If the input was a pattern of squares looking like a dog, then the output would be the word “dog”.
Neural networks are particularly useful for classifying complex and noisy data. It is akin to the human brain picking out a face in a crowd. No assumptions are needed. It is all left to whatever the neural network can discern in the data. They trained a neural network to recognise the literary styles of Shakespeare and his contemporaries, but not directly. Their neural network consisted of five inputs, each of which was a stylometric measure extracted from text. The outputs were categories of how “Shakespearean” the text was deemed to be. So, the training was exposing the network to stylometric measures of a large number of samples of Shakespeare’s undisputed work, together with that of his successor with The King’s Men, John Fletcher. The samples ran to around 1000 words, with 50 samples of the text of each author—100,000 words in all.
The five stylometric measures extracted from each sample text were based on the function words “are”, “in”, “no”, “of” and “the”. The measures were the ratios of the number of times each function word appeared to the total number of words in the text, and were used as the inputs to the network. The neural network was then repeatedly shown the samples, up to 100 times each, until it could confidently recognise the works of both Shakespeare and Fletcher, an output near 1 suggesting Shakespeare and near 0 Fletcher, values of around 0.5 indicating it was unsure. Once trained, the network was good at recognising works of both Shakespeare and Fletcher it had never seen before. As well as identifying whole plays correctly, the neural network could correctly identify the author of individual acts by known playwrights.
Turning to The Two Noble Kinsmen, the neural network agreed with the subjective evidence of scholars, which is that Shakespeare’s hand dominates Acts I and V, with much of the rest being by Fletcher, and even that Fletcher seemed to have had help from Shakespeare in Act IV. The neural network supports the view that The Two Noble Kinsmen is a collaboration between Shakespeare and his apprentice, Fletcher.
Later, the neural network was trained to to discriminate between works of Shakespeare and Marlowe, albeit using different function words and ratios. The network was adept at classifying plays it had never seen before. It suggests the anonymous play Edward III was written by Shakespeare under Marlowe’s influence, and that King Henry VI Part III is a Shakespearean revision of a lost Marlovian play.
As with any new stylometric method, the importance of neural networks is in their ability to add evidence in support of a particular theory that scholars trained in the humanities have formulated subjectively. Rather than being suspicious of attempts to measure literary style objectively, literary scholars might find themselves offering reasons why the objective analysis by neural networks and other stylometric techniques is wrong.
