Read next
NICS 2020Counting the words that set Vietnamese stories apart from other texts12 min read
FAIR 2020Picking 100 million words that stand for Vietnamese14 min readAll publications · Get in touch about this research
PACLIC 34 · 2020 · Interactive explainer
Everyone repeats small habits when they write: favourite short words, a typical word length, a certain range of vocabulary. Few studies had checked whether those habits identify writers in Vietnamese. This paper collected 1,304 opinion texts by ten writers and tested three classic counting methods on them.
Which of ten writers produced this text?
Pick a held-out text. The method measures how far its word counts are from each writer’s known texts; the shortest bar is its answer.
The method is Kilgarriff’s chi-squared on the 500 most frequent words. For every one of the ten held-out texts, the shortest bar belongs to the true author. Source: Table 3. Author names are anonymised IDs in the paper.
measures writing style with numbers. Its oldest job is : given a text and a list of possible writers, decide who wrote it. The same tools are used to detect plagiarism, check a disputed signature on a document, or guess a writer’s age or region.
The methods were developed and tested mostly on English and other languages with large resources. The authors note that very few studies apply them to Vietnamese, partly because there was no corpus with enough text per known writer.
Where do the three methods come from?
Tap a date. The history is the one the paper gives in its introduction.
Word length (Mendenhall) and word frequency (Zipf, Mosteller and Wallace) are the two families of clues this paper tests.
The holds 1,304 texts in which ten writers give their opinion or share experiences on social issues. Most come from VnExpress, a Vietnamese online newspaper; the rest from Facebook and blogs. Links, images and captions were removed.
Vietnamese writes a space between every syllable, so a single word such as xã hội (“society”) spans two space-separated pieces. The text was split into words () semi-automatically with the CLC toolkit, then the whole corpus was checked by hand.
Each writer’s text was split in two. A known sample of exactly 58,088 words (punctuation removed) acts as the writer’s profile. The rest is a held-out test text: the “mystery” document the methods must attribute.
How much text does each writer contribute?
Each bar is one writer. Tap a writer for their numbers.
The known samples are equal by design, so no writer has an advantage from writing more. The test texts are not: Author1262’s is 89,997 words, Author83’s only 14,525. Source: Table 1 (known = corpus words without punctuation − test-set words).
In 1887 Thomas Mendenhall proposed that a writer’s “stylistic signature” shows in how often they use words of each length. Count the characters of every word, tally the lengths, and draw the tally as a curve: the characteristic curve.
Before looking at the writers, try it on the most common Vietnamese words in this corpus.
How long is a common Vietnamese word?
These are the 30 most frequent words in the writers’ known samples, listed in the paper. Tap one to count its letters. computed here
Among these 30 words, 3-letter words such as của, một, tôi are the largest group. The letter counts are computed on this page from the paper’s word list; diacritics do not add letters.
The authors drew one curve per writer from the full 58,088-word known samples. They report that every writer uses words of 2 to 4 characters most, with 3 characters at the top.
What do the ten characteristic curves look like?
One line per writer: how many of their 58,088 words have each length. Pick a writer to highlight. traced, approximate
All ten curves rise to a sharp peak at 3 characters and fall away almost together. Author1262 has the tallest peak and, with Author1035, a higher point at 4 than at 2. The paper shows these curves only as plots (Figure 5, where each panel’s x-axis is sorted by frequency); we traced them and redrew them on one shared length axis. Values are approximate: each traced curve adds up to within about 1% of 58,088.
The curves overlap so closely that the differences are easier to see as a ranking: which length comes first, second and third for each writer.
Which word lengths does each writer use most?
Each row lists word lengths from most to least used, as ordered on the x-axis of the paper’s curves. Tap a writer.
Eight writers go 3 → 2 → 4. Author1035 and Author1262 go 3 → 4 → 2. Otherwise the orders are very similar. The paper uses these curves to describe style; it does not use them to attribute the test texts. Source: Figure 5 (order of the x-axis labels; the counts themselves are not given as numbers).
The paper’s Figure 5 is ten NLTK frequency plots whose x-axis is sorted by count, not by length. This figure reproduces only that order. One panel is printed as “Author108”, a name not in Table 1; the author confirmed (2026-11-10) that it is Author203, and it is labelled Author203 here. One detail still differs from the text: the Author203 and Author1050 panels list a length of 15, while the text says the longest words of these two writers have 14 characters. The text also reports that Author59’s longest word has 17 characters. In the Author97 panel the 2- and 4-character counts are nearly equal.
Alongside the curves, the authors measured with Carroll’s index R: the number of different words () divided by the number of words. Because every known sample has the same 58,088 words, R simply tracks vocabulary size.
Whose vocabulary is richest?
R = different words ÷ 58,088. Higher means more varied. Tap a writer to see the division.
Author1035 has the richest vocabulary (R = 0.146, 8,456 different words); Author83 the narrowest (R = 0.092, 5,327). Our running example, Author1262, sits near the top with 0.129. Source: Table 2.
The second method, Kilgarriff’s , looks at the 500 most frequent words. It pools a writer’s known sample with the mystery text and asks: if both were random samples of the same writer, how often should each word appear in each? Then it adds up how far the real counts are from those expectations.
How does chi-squared turn word counts into one distance? illustrative numbers
A toy example with two words instead of 500. Step through it.
The counts here are made up to show the arithmetic. The real test used the 500 most frequent words, a 58,088-word known sample and the full held-out text; everything was lowercased first.
For candidate file Ki and unknown file U, with joint corpus J and AuShare = len(Ki) / len(J):
x runs over the 500 most common words of J. Expected counts split each word’s joint count by AuShare and 1 − AuShare. The smallest χ² over the ten candidates gives the attribution.
Run this for every pair of known writer and held-out text and you get a 10 × 10 table of distances. If the method works, each column’s smallest value should sit on the diagonal, where the writer meets their own text.
Does each text land closest to its own writer?
Columns are held-out texts, rows are known writers. Darker = closer. Tap a column header to focus it.
Yes, in all ten columns: chi-squared attributes 100% of the test texts correctly. Source: Table 3.
Table 3 labels both axes “Authors”. We read columns as tested texts, as the paper states for Tables 4 and 5. For Table 3 the orientation does not change the result: each diagonal value is the minimum of its row and of its column.
John Burrows’ takes a narrower view. It keeps the n most frequent words of all known samples combined (the paper tried n = 20 and n = 30), and for each writer turns the frequency of every word into a : how far above or below the group average that writer is. Delta is the average gap between the mystery text’s z-scores and a writer’s z-scores. Smaller is closer.
With 20 or 30 signature words, who gets matched to whom?
Switch the number of words. Columns are held-out texts; the outlined cell is the writer Delta picks.
Many picks fall off the diagonal. With 20 words Delta is right for 5 of 10 texts; adding 10 more words drops it to 4 of 10. Source: Tables 4 and 5; outlines follow the paper’s highlighted cells.
Ci is the observed frequency of feature word i; μi and σi are its mean and standard deviation across the ten known sub-corpora; Zt is the test text’s z-score. In Table 5, three columns tie at three decimals (texts of Author88, Author97 and Author1262); we follow the paper’s highlighted cell, which counts Author1262 as a hit and Author97 as a miss.
Picking a writer at random would be right 1 time in 10. All three results clear that bar, but only chi-squared gets every text right; the authors note that Delta did not do as well as they expected.
How many of the ten test texts does each method attribute correctly?
Each square is one writer’s held-out text: red = right writer, orange = wrong. Higher is better.
100% for chi-squared, 50% for Delta with 20 words, 40% with 30. Only Author1262 and Author1289 are right under all three. Source: Section 3.3 and 4.1.
A wrong pick can be a near miss or a long way off. Ranking all ten candidates for each text shows where the true writer ended up.
Where does the true writer rank among the ten candidates?
1 = the method’s first choice. Tap a square for the distances behind it.
Delta’s misses are not all near misses: with 30 words, Author1035’s own profile is only 7th closest to Author1035’s text. Ranks computed on this page from Tables 3–5.
The authors point to the texts themselves. The VnExpress pieces come from the “Perspective” section, which covers finance, society, lifestyle, health and more, and not every writer covers the same topics. With only 20 or 30 very common words, a few topic-driven differences can push a text toward the wrong writer. Equal-sized known samples do not remove this topic imbalance.
Chi-squared has its own weak spots. It is a coarse measure: the most frequent words carry most of the weight, so subtle habits with rarer words go unnoticed. Lowercasing everything usually helps, by counting a sentence-initial word with its lowercase form, but can merge a proper noun with a common noun spelled the same way.
Next, the authors plan a corpus with a wider variety of topics and a richer annotated corpus for machine-learning approaches to authorship.
Read next
NICS 2020Counting the words that set Vietnamese stories apart from other texts12 min readFAIR 2020Picking 100 million words that stand for Vietnamese14 min readAll publications · Get in touch about this research