UChicago Researchers Demonstrate the Quantifiable Uniqueness of Former President Donald Trump’s Language Use

As November approaches, so too does the presidential election for the United States. Along with any major election, the electorate can expect a bevy of debates and political speeches from candidates. The 2024 election candidates include former President Donald Trump and current Vice President Kamala Harris. Previous reporting on Trump from prior campaigns has pointed to his distinctive language and claimed that it’s unlike any previous candidates’ language, but is that really true?

In a new paper, authors from the University of Chicago quantify the uniqueness of Trump using several metrics and tools, including large language models (LLMs). Doctoral student Karen Zhou, who is advised by Assistant Professor of Computer Science and Data Science Chenhao Tan, led the research, to be published at PNAS Nexus.

“This work was inspired by the desire to quantify the differences between Donald Trump and prior American presidents,” Zhou stated. “In particular, we hypothesized that portrayal of political opponents might be a distinguishing feature of his speech.”

Zhou and her co-authors developed a new language measurement technique and found that Donald Trump’s use of language is indeed unique among all presidential candidates since 1960. They categorized Trump’s language as divisive, particularly when mentioning political opponents.

How do we define unique, divisive speech?

To support such claims, the authors had to create tools up to the challenge. They implemented three complementary approaches to allow for a multifaceted comparison. One of the approaches employed an innovative use of a large language model (LLM). Large language models are essentially text prediction algorithms that calculate the likelihood of the next words or phases, given a certain context. Based on the ability of LLMs to make these predictions, the authors created a metric called “uniqueness” that measured an individual’s probability of word phrases or sequences compared to other political candidates. For this group of political candidates, this measure compares how likely one speaker is to use the same language and phrasing as another candidate.

“This is quite a creative way of using large language models,” Tan said. “The common practice is to evaluate large language models by how well it predicts the next word, but this approach uses large language models to inspect the political discourse.”

The second measure employed in this paper was a standardized definition of “divisive speech.” Defined as language meant to delegitimize or challenge the speaker’s target, classic divisive speech examples include calling a political opponent “stupid,” or “racist,” or “ridiculous.” The researchers created a lexicon of 178 words that are readily recognized as divisive and could be used by political figures regardless of party affiliation.

The third measurement employed by the authors was specifically interested in speech directed at a debate opponent. The authors selected mentions of opponents from presidential debates and calculated how a speaker’s rhetoric changes when referring to opponents versus in other political speech.

Donald Trump doesn’t speak like any other political candidate

To compare Donald Trump’s speaking style with past political candidates’ language, the researchers pulled from three sources: State of the Union addresses since 1961, presidential debates for general elections since 1960, and a selection of campaign speeches collected by the American Presidency Project.

“Whether from the campaign trail or in office, these are among the most significant opportunities that presidents and presidential candidates have to speak to the American people,” Sydney Stein Professor in American Politics and a collaborator on this project, William Howell, stated. “The language they choose to employ, as such, comes to define their communication styles as presidents.”

Using these sources, they found Trump’s use of language was by far the most unique across all three data sources, with the most noticeable difference found during debates. Even when just comparing Trump to the average performance of both Democrat and Republican speakers, Trump remained an anomaly.

Similarly, using the lexicon of divisive words, Trump was found to employ divisive language most frequently. In campaign speeches, he was particularly likely to use “crazy” (135 times), “corrupt” (111 times), and “stupid” (69 times). His most divisive language appeared during debates, but he ranked as the most divisive across all forms of political speech measured here.

Finally, Trump was more likely than others to mention his political opponents specifically. And when talking about his opponents, he referred to them with more distinctive language than other candidates did.

“Trump routinely cast himself as apart from, and in a fight against, the dominant political order,” Howell remarked. “This oppositional stance, in vital respects, shaped the particular ways that Trump talked about his many political opponents.”

It’s easy to discard the opinion of a pundit or politico when they make sweeping statements about the uniqueness of Donald Trump. But at least when it comes to language use, Trump is an outlier case. Since the early 1960s, there hasn’t been a political candidate quite like him. And we now have data across three separate measures that demonstrates that.

“Presidential speech has far-reaching social effects,” Zhou said. “ It is important to identify the features of such rhetoric so that we can measure their impacts, especially for Donald Trump, who many intuitively consider an outlier politician.”

A preprint version is available here.