The devil strip

In the 1970s, a girl disappeared in Illinois, and her parents found a ransom note on their doorstep, written in pencil. It asked for $10,000 in a diaper bag, and went on (Shuy, 2001):

Put it in the green trash kan on the devil strip at corner 18th and Carlson. Don’t bring anybody along. No kops!!

What Shuy saw

• “Kan” and “kops” look like the spelling of a poorly educated writer. But the rest of the note, punctuation included, is fully standard. A likely explanation is an educated writer trying to look uneducated
• A devil strip is the strip of grass between the pavement and the road. The term belongs to the area around Akron, Ohio; people from nearby Cleveland would hardly use it
• Shuy asked whether any suspect was an educated man from Akron. One was, and he confessed when confronted, although one regional word on its own is thin evidence

Today

1. Profiling and comparison
2. Idiolect, and two British cases
3. Counting words
4. What can go wrong

The same two questions as in speech

What a text can give away

• Region shows in words like devil strip, in spellings like colour or color, and in local grammar
• A first language can show through errors and structures carried over from it
• Education and training show in spelling, punctuation, vocabulary and the conventions of a profession
• Age and gender appear only as tendencies in large samples

Profiling by machine

Schler et al. (2006) analysed blogs by 19,320 authors, balanced for gender within each age group.

Simulated values, in the direction of the published results: the features that separate male from female bloggers also separate older from younger ones.

Limits of profiling

• Classifiers trained on thousands of texts guess gender or age group well above chance, and still get many individuals wrong
• A forensic text is usually short and written for one purpose, and writers can disguise their style, as the “kan” and “kops” of the ransom note show

A profile can shorten a list of suspects, as it did with the devil strip. Identifying the writer takes a comparison with texts of known authorship.

Everyone has their own version

Linguists use idiolect for the version of a language that one person knows and uses (Coulthard, 2004).

• Whether every idiolect is unique is hard to prove. Casework needs two weaker properties: the writer is fairly consistent across their texts, and distinctive among the other likely writers (Grant, 2010)
• Nini (2023) proposes a theory grounded in cognitive linguistics: each person entrenches their own set of linguistic units, and the combination is individual. He tests it in computational experiments that report likelihood ratios

Jenny Nicholl

Jenny Nicholl, 19, disappeared in North Yorkshire on 30 June 2005. Text messages kept coming from her phone (Grant, 2010).

• Tim Grant compared the disputed messages with messages known to be hers and with those of her former lover, David Hodgson
• Jenny wrote my and myself. Hodgson often used the regional me and meself, like the disputed messages, which were closer to his style overall

Discussion

Suppose Hodgson had noticed that Jenny wrote myself, and copied it in every message.

• Which other habits would he have had to imitate to pass as her?
• Which of them would be hardest to control?

Derek Bentley

In 1952, 19-year-old Derek Bentley was arrested with Christopher Craig after a break-in in Croydon, during which Craig shot a police officer dead. Bentley could not read or write.

• His signed statement was presented in court as a verbatim record of what he had dictated. He was convicted of murder and hanged in January 1953
• Coulthard argued that the statement had been produced by question and answer and then written up as a monologue, and that some of its wording belonged to the police officers

“I then”

In a general corpus, then usually comes before the subject (Fox, 1993). Bentley’s statement has Chris Craig and I then caught a bus and Chris then jumped over: an order typical of police reports. In 1998, after a referral partly based on this analysis, the Court of Appeal quashed his conviction.

The Federalist papers

In 1787–88, Alexander Hamilton, James Madison and John Jay published essays in favour of the new US constitution under one pen name, “Publius”. Of the 77 papers that appeared in newspapers, 12 were claimed by both Hamilton and Madison.

• Sentence length did not help: on average 34.55 words for Hamilton and 34.59 for Madison
• The historian Douglass Adair noticed that Hamilton wrote while and Madison whilst. Mosteller and Wallace (1963) added more such marker words, including upon and enough, both typical of Hamilton

Upon

Simulated papers, following the pattern Mosteller and Wallace reported. Using Bayesian statistics on dozens of such words, they concluded that Madison wrote all 12 disputed papers.

Why small words?

• Function words such as the, of, upon and whilst are frequent, so even a short text gives usable counts
• They change little with the topic. Mosteller and Wallace found stable rates for an, of and upon, but not for words like war or executive
• Writers choose them with little awareness, so they are harder to control or imitate than vocabulary

Burrows’ Delta

Burrows (2002) proposed a simple distance between a text and each candidate author:

1. Take the most frequent words in all the texts: Burrows used up to 150, later work up to 800 (Hoover, 2004)
2. Turn each word’s frequency into a z-score: how far it is from the mean of all texts, in standard deviations
3. For each candidate, average the absolute differences between their z-scores and those of the disputed text. That average is Delta
4. The candidate with the smallest Delta is the most similar, and so the likeliest author

Delta in practice

Simulated: four authors with five known texts each, 30 frequent words, and one disputed text written by Author C. The code computes Delta as described.

Robert Galbraith

In 2013, The Cuckoo’s Calling came out as the first novel of one Robert Galbraith. After an anonymous tip, the Sunday Times asked two researchers whether J. K. Rowling had written it.

• Patrick Juola compared it with Rowling’s The Casual Vacancy and with novels by three British crime writers, Ruth Rendell, P. D. James and Val McDermid (Juola, 2013)
• He ran four tests: word lengths, the 100 most common words, pairs of adjacent words, and sequences of four characters. Peter Millican ran separate tests

Rowling came out as the most likely author, and she then confirmed that the book was hers.

The Unabomber

Between 1978 and 1995, bombs, many sent by mail, killed three people and injured many more. In 1995, newspapers published the bomber’s manifesto, Industrial Society and Its Future.

• Linda Patrik, and then her husband David Kaczynski, recognised the style of David’s brother Ted, including the phrase “cool-headed logicians”
• Ted’s letters and the manifesto both reversed a common idiom: you can’t eat your cake and have it too
• The FBI analyst James Fitzgerald compared the manifesto with Ted’s known writings. His comparison supported the 1996 search of Ted’s cabin in Montana

Hard cases

• Short texts, such as text messages or tweets, give few counts for any one feature
• Genre and register change style: a writer’s emails and reports may differ more than two writers’ emails do. Delta is known to be sensitive to genre
• The likely author may not be among the candidates at all
• Texts can be edited or co-written, as Bentley’s statement was by the police
• People can imitate another person’s style, or disguise their own

How strong is the evidence?

As in speaker comparison, the question is how much more likely the observed features are if the suspect wrote the text than if someone else from the relevant population did.

• Grant (2007) proposed ways to quantify consistency and distinctiveness in casework, and Nini (2023) reports his results as likelihood ratios, the approach of session 4
• Reference data are still the weak point: how common is meself among young men in North Yorkshire texting in 2005?

Discussion

• The devil strip pointed to one suspect. What else would you want to know before acting on a single regional word?
• Should an authorship analysis that cannot give a likelihood ratio be allowed in court?

Activity

For the next session, in pairs:

1. Each of you collects 10 of your own text messages or chat messages, with any personal details removed
2. List five habits in your own messages: spellings, abbreviations, punctuation, greetings, emoji
3. Swap the lists, then write one new message each in your partner’s style
4. Give both the real and the imitated messages to another pair. Can they tell which are imitations, and which habits gave them away?

Summary

Slides and code

These slides

mcanzi.de/ws27_fss/09_authorship_analysis.html

Code and all sessions

github.com/mcanzi/ws27_fss