Multilingual Stylometry

CLS INFRA · ELTeC

Multilingual Stylometry

Can machine learning recognize an author's style across languages and translations?

This research showcase explores how language, translation, feature selection, sample size, and corpus composition affect machine learning-based authorship attribution.

4 languages: English, French, Hungarian, Ukrainian
4 language families: Germanic, Romance, Finno-Ugric, Slavic
16 original and translated sub-corpora
12 machine-translated corpus versions
18,610 classification runs

Background

From Style To Signal

Stylometry turns writing style into measurable data. The key idea is simple: texts can be represented as feature vectors, and authorship attribution predicts which known author a new text most resembles.

Stylometry

The quantitative study of writing style: frequent words, character sequences, grammatical patterns, lemmas, and other recurring traces of authorial habit.

Authorship Attribution

A classification task: given texts by known authors, assign an unknown, disputed, or pseudonymous text to the most likely author.

Where It Is Used

Literary studies, forensic linguistics, historical research, plagiarism and ghostwriting detection, digital humanities, and large-scale text provenance work.

What Affects Performance

Text length, corpus composition, genre, chronology, feature type, language, translation, classifier choice, and evaluation metrics such as accuracy and Cohen's Kappa.

Data Foundation

Four Languages, Sixteen Sub-Corpora

The study uses four corpora derived from ELTeC, the European Literary Text Collection: English, French, Hungarian, and Ukrainian. Each language represents a different language family, and each source corpus was translated into the other three languages.

Select A Corpus To Compare

Original corpus Machine translation

eng-fra means: the English source corpus translated into French. Diagonal codes such as fra-fra are originals. Click any corpus to open it in the comparison heatmap.

Methods

Experimental Design

The attribution tests compare original and translated corpora across feature types, n-gram sizes, sample sizes, and MFF settings. Performance is reported as accuracy and Cohen's Kappa.

Data
Four nineteenth-century novel corpora derived from the European Literary Text Collection.
Translation
DeepL Pro translations into the other three languages, producing 16 sub-corpora.
Features
Plain tokens, lemmas, POS tags, word features, character features, and n-grams from 1 to 5.
Evaluation
Authorship attribution performance is reported as accuracy and Cohen's Kappa.

Interactive Visualization

Explore The Authorship Signal

Change language pairs and feature settings to see how attribution accuracy shifts. Try originals against translations, words against characters, and shorter samples against full novels.

eng-fra = English corpus translated into French Full novel often gives the strongest signal

Primary View

Corpus

Loading...

Comparison View

Corpus

Loading...
Lower accuracy Higher accuracy

Results

Key Findings

01

Longer Samples Help

Full novels usually improve attribution performance compared with shorter samples because frequency estimates become more stable.

02

Originals Beat Translations

Original-language corpora generally perform better than translated ones, suggesting that translation can reshape part of the authorial signal.

03

More Features, Up To A Point

Adding frequent features can help, but too many features may introduce noise and reduce classification accuracy.

04

Corpus Composition Matters

Genre, chronology, narrative perspective, and diversity affect how difficult authorship attribution becomes.

Research Background

Standing On Delta

Rybicki and Eder's “Deeper Delta across Genres and Languages” is a central reference for this project: it showed that feature-vector choices and language or genre conditions strongly affect attribution performance. This study extends that line by separating language effects from corpus-composition effects through machine-translated versions of the same ELTeC-derived corpora.

Project Team

Contributors

This showcase is based on joint research within the CLS INFRA project.

Further Information

References And Links

Full Paper

Schöch, C., Dudar, J., Fileva, E. & Šeļa, A. (2024). Multilingual Stylometry: The Influence of Language on the Performance of Authorship Attribution using Corpora from the European Literary Text Collection (ELTeC). Proceedings of the Computational Humanities Research Conference 2024, 386-408.

Access paper

Explanatory Paper

Schöch, C. (2024). CLS INFRA D3.3 Showcases for the application of CLS methods and tools. Zenodo.

DOI: 10.5281/zenodo.10912517

Background On Stylometric Methods

Schöch, C., Dudar, J., & Fileva, E. (2023). CLS INFRA D3.2: Series of Five Short Survey Papers on Methodological Issues (= Survey of Methods in Computational Literary Studies) (v1.1.0). Zenodo.

DOI: 10.5281/zenodo.7892112

Core Research Background

  • Rybicki, J., & Eder, M. (2011). Deeper Delta across Genres and Languages: Do We Really Need the Most Frequent Words?
  • Burrows, J. (2002). Delta: A Measure of Stylistic Difference and a Guide to Likely Authorship.
  • Stamatatos, E. (2009). A Survey of Modern Authorship Attribution Methods.
  • Koppel, M., Schler, J., & Argamon, S. (2009). Computational Methods in Authorship Attribution.
  • Byszuk, J. (2023). What is Authorship Attribution?