Stylometry

Stylometry grew out of earlier techniques of analyzing texts for evidence of authenticity, author identity, and other questions.

The modern practice of the discipline received major impetus from the study of authorship problems in English Renaissance drama. Researchers and readers observed that some playwrights of the era had distinctive patterns of language preferences, and attempted to use those patterns to identify authors in uncertain or collaborative works. Early efforts were not always successful: in 1901, one researcher attempted to use John Fletcher's preference for "⁠ ⁠’em", the contractional form of "them", as a marker to distinguish between Fletcher and Philip Massinger in their collaborations—but he mistakenly employed an edition of Massinger's works in which the editor had expanded all instances of "⁠ ⁠’em" to "them".[6]

The basics of stylometry were set out by Polish philosopher Wincenty Lutosławski in Principes de stylométrie (1890). Lutosławski used this method to build a chronology of Plato's Dialogues.[7]

The development of computers and their capacities for analyzing large quantities of data enhanced this type of effort by orders of magnitude. The great capacity of computers for data analysis, however, did not guarantee quality output. In the early 1960s, Rev. A. Q. Morton produced a computer analysis of the fourteen Epistles of the New Testament attributed to St. Paul, which showed that six different authors had written that body of work. A check of his method, applied to the works of James Joyce, gave the result that Ulysses, Joyce's multi-perspective, multi-style masterpiece, was written by five separate individuals, none of whom apparently had any part in the crafting of Joyce's first novel, A Portrait of the Artist as a Young Man.[8]

In time, however, and with practice, researchers and scholars have refined their approaches and methods, to yield better results. One notable early success was the resolution of disputed authorship in twelve of The Federalist Papers by Frederick Mosteller and David Wallace.[9] While questions of initial assumptions and methodology still arise (and, perhaps, always will), few now dispute the basic premise that linguistic analysis of written texts can produce valuable information and insight. (Indeed, this was apparent even before the advent of computers: the successful application of a textual/linguistic approach to the Fletcher canon by Cyrus Hoy and others yielded clear results in the late 1950s and early '60s.)

Applications of stylometry include literary studies, historical studies, social studies, gender studies, and many forensic cases and studies.[10][11] It can also be applied to computer code.[12] Stylometry can also be used to predict whether someone is a native or non native English speaker through their typing speed.[13]

Stylometry as a method is vulnerable to the distortion of text during revision.[14] There is also the case of the author adopting different styles in the course of his career as was demonstrated in the case of Plato, who chose different stylistic policies such as the those adopted for the early and middle dialogues addressing the Socratic problem.[15]

Modern stylometry draws heavily on the aid of computers for statistical analysis, artificial intelligence and access to the growing corpus of texts available via the Internet.[16] Software systems such as Signature[17] (freeware produced by Dr Peter Millican of Oxford University), JGAAP[18] (the Java Graphical Authorship Attribution Program—freeware produced by Dr Patrick Juola of Duquesne University), stylo[19][20] (an open-source R package for a variety of stylometric analyses, including authorship attribution, developed by Maciej Eder, Jan Rybicki and Mike Kestemont) and Stylene[21] for Dutch (online freeware by Prof Walter Daelemans of University of Antwerp and Dr Véronique Hoste of University of Ghent) make its use increasingly practicable, even for the non-expert.

Stylometric methods are discussed in several academic fields, mostly as a tangential field of application as with machine learning, natural language processing, and lexicography.

• Around 1370 to 1070 BC, as recorded in the Book of Judges, one tribe identified members of another tribe in order to kill them by asking them to say the word Shibboleth which in the dialect of the intended victims sounded like "sibboleth".[30]
• In 1439, Lorenzo Valla showed that the Donation of Constantine was a forgery, an argument based partly on a comparison of the Latin with that used in authentic 4th-century documents.
• In 1952, the Swedish priest Dick Helander was elected bishop of Strängnäs. The campaign was competitive and Helander was accused of writing a series of a hundred-some anonymous libelous letters about other candidates to the electorate of the bishopric of Strängnäs. Helander was first convicted of writing the letters and lost his position as bishop but later partially exonerated. The letters were studied using a number of stylometric measures (and also typewriter characteristics) and the various court cases and further examinations, many contracted by Helander himself during the years up to his death in 1978 discussed stylometric methodology and its value as evidence in some detail.[31][32]
• In 1975, after Ronald Reagan had served as governor of California, he began giving weekly radio commentaries syndicated to hundreds of stations. After his personal notes were made public on his 90th birthday in 2001, a study to determine which of those talks were written by him and which were written by various aides used stylostatistical methods.[33]
• In 1996, the stylometric analysis of the controversial, pseudonymously authored book Primary Colors, performed by Vassar College professor Donald Foster[34] brought the field to the attention of a wider audience after correctly identifying the author as Joe Klein. (This case was only resolved after a handwriting analysis confirmed the authorship).
• In 1996, stylometric methods were used to compare the Unabomber manifesto with letters written by one of the suspects, Theodor Kaczynski to his brother, which led to his apprehension and later conviction.[35]
• In April 2015, researchers using stylometry techniques identified a play, Double Falsehood, as being the work of William Shakespeare.[36] Researchers analyzed 54 plays by Shakespeare and John Fletcher and compared average sentence length, studied the use of unusual words and quantified the complexity and psychological valence of its language.

• In 2016, MacDonald P. Jackson, Emeritus Professor of English at the University of Auckland, New Zealand and a Fellow of the Royal Society of New Zealand, who had spent his entire academic career analyzing authorship attribution, wrote a book titled Who Wrote "the Night Before Christmas"?: Analyzing the Clement Clarke Moore Vs. Henry Livingston Question,[20], in which he evaluates the opposing arguments and, for the first time, uses the author-attribution techniques of modern computational stylistics to examine the long-standing controversy. Jackson employs a range of tests and introduces a new one, statistical analysis of phonemes; he concludes that Livingston is the true author of the classic work.
• In 2017, Simon Fuller and James O'Sullivan published a study claiming that bestselling author James Patterson does not do any writing in his co-authored novels.[37][38][39] According to O'Sullivan, his collaboration with former U.S. president Bill Clinton, The President is Missing, is an exception to this rule.[40]

• In 2017, a group of linguists, computer scientists, and scholars analysed the authorship of Elena Ferrante. Based on a corpus created at University of Padua containing 150 novels written by 40 authors, they analyzed Ferrante's style based on seven of her novels. They were able to compare her writing style with 39 other novelists using, for example, stylo.[19] The conclusion was the same for all of them: Domenico Starnone is the secret hand behind Elena Ferrante.[41]
• In 2018, Mark Glickman, senior lecturer in statistics at Harvard University worked with Ryan Song, a former statistics student at Harvard, and Jason Brown, a professor at Dalhousie University in Nova Scotia, applied stylometry to find that, most likely, The Beatles' song "In My Life" was composed by John Lennon, but with a 50% chance that Paul McCartney wrote the middle eight.[42]

• In 2019, the ETSO project: Stylometry applied to the Spanish Golden Age Theater, led by Álvaro Cuéllar González and Germán Vega García-Luengos (University of Valladolid) managed to gather more than 1200 plays of the Spanish Golden Age. After applying stylometrical analysis, the attribution of Mujeres y criados to Lope de Vega[43][44] was ratified, and an authorship problem was detected in La monja alférez, a play attributed to Pérez de Montalbán which, thanks to these analyzes and through historical and philology research, was eventually attributed to Juan Ruiz de Alarcón.[45][46][47][48]

Since stylometry has both descriptive use cases, used to characterise the content of a collection, and identificatory use cases, e.g. identifying authors or categories of texts, the methods used to analyse the data and features above range from those built to classify items into sets or to distribute items in a space of feature variation. Most methods are statistical in nature, such as cluster analysis and discriminant analysis, are typically based on philological data and features, and are fruitful application domains for modern machine learning approaches.

Whereas in the past, stylometry emphasized the rarest or most striking elements of a text, contemporary techniques can isolate identifying patterns even in common parts of speech. Most systems are based on lexical statistics, i.e. using the frequencies of words and terms in the text to characterise the text (or its author). In this context, unlike in information retrieval, the observed occurrence patterns of the most common words are more interesting than the topical terms which are less frequent.[49][50]

The primary stylometric method is the writer invariant: a property held in common by all texts, or at least all texts long enough to admit of analysis yielding statistically significant results, written by a given author. An example of a writer invariant is frequency of function words used by the writer.

In one such method, the text is analyzed to find the 50 most common words. The text is then broken into 5,000 word chunks and each of the chunks is analyzed to find the frequency of those 50 words in that chunk. This generates a unique 50-number identifier for each chunk. These numbers place each chunk of text into a point in a 50-dimensional space. This 50-dimensional space is flattened into a plane using principal components analysis (PCA). This results in a display of points that correspond to an author's style. If two literary works are placed on the same plane, the resulting pattern may show if both works were by the same author or different authors.

The diffusion of Internet has shifted the authorship attribution attention towards online texts (web pages, blogs, etc.) electronic messages (e-mails, tweets, posts, etc.), and other types of written information that are far shorter than an average book, much less formal and more diverse in terms of expressive elements such as colors, layout, fonts, graphics, emoticons, etc. Efforts to take into account such aspects at the level of both structure and syntax were reported in.[56] In addition, content-specific and idiosyncratic cues (e.g., topic models and grammar checking tools) were introduced to unveil deliberate stylistic choices.[57]

Standard stylometric features have been employed to categorize the content of a chat over instant messaging,[58] or the behavior of the participants,[59] but attempts of identifying chat participants are still few and early. Furthermore, the similarity between spoken conversations and chat interactions has been neglected while being a key difference between chat data and any other type of written information.

• Linguistics and the Book of Mormon, Stylometry (Wordprint Studies)
• Moshe Koppel
• Writeprint

• Brocardo, Marcelo Luiz; Issa Traore; Sherif Saad; Isaac Woungang (2013). Authorship Verification for Short Messages Using Stylometry. IEEE Intl. Conference on Computer, Information and Telecommunication Systems (CITS). doi:10.1109/CITS.2013.6705711.
• Can F, Patton JM (2004). "Change of writing style with time". Computers and the Humanities. 38 (1): 61–82. CiteSeerX 10.1.1.1.8850. doi:10.1023/b:chum.0000009225.28847.77.
• Brennan, Michael Robert; Greenstadt, Rachel. "Practical Attacks Against Authorship Recognition Techniques". Innovative Applications of Artificial Intelligence.
• Hope, Jonathan (1994). The Authorship of Shakespeare's Plays. Cambridge: Cambridge University Press.
• Hoy C (1956–62). "The Shares of Fletcher and His Collaborators in the Beaumont and Fletcher Canon". Studies in Bibliography. 7–15.
• Juola, Patrick (2006). "Authorship Attribution" (PDF). Foundations and Trends in Information Retrieval. 1 (3): 3. CiteSeerX 10.1.1.219.1605. doi:10.1561/1500000005.
• Kenny, Anthony (1982). The Computation of Style: An Introduction to Statistics for Students of Literature and Humanities. Oxford: Pergamon Press.
• Romaine, Suzanne (1982). Socio-Historical Linguistics. Cambridge: Cambridge University Press.
• Samuels, M. L. (1972). Linguistic Evolution: With Special Reference to English. Cambridge: Cambridge University Press.
• Schoenbaum, Samuel (1966). Internal Evidence and Elizabethan Dramatic Authorship: An Essay in Literary History and Method. Evanston, IL, USA: Northwestern University Press.
• Van Droogenbroeck, Frans J. (2016) "Handling the Zipf distribution in computerized authorship attribution"
• Van Droogenbroeck, Frans J. (2019) "An essential rephrasing of the Zipf-Mandelbrot law to solve authorship attribution applications by Gaussian statistics"
• Zenkov A.V. (2017) "Method of Text Attribution Based on the Statistics of Numerals", Journal of Quantitative Linguistics

• Association for Computers and the Humanities
• Literary and Linguistic Computing
• Computational Stylistics Group
• Signature Stylometric System
• JGAAP Authorship Attribution Program
• Uncovering the Mystery of J.K. Rowling's Latest Novel