ABSTRACT

Numbers don’t speak for themselves – yet taking numbers for granted (numerism) is widespread. In fact, journalists often rely heavily on numbers precisely because they are widely considered objective. As a team of journalists and social scientists, we undertook a qualitative exploration of clauses and entire news reports that are particularly quantitatively dense. The dense clauses were often grammatically complex and assumed familiarity with sophisticated concepts. They were rarely associated with explanations of data collection methods. Meanwhile, the dense news reports were all about economy or health topics, chiefly brief updates on an ongoing event (e.g., stock market fluctuations; COVID-19 cases). We suggest that journalists can support public understanding by:

  • Providing more detail about research methods;

  • Writing shorter, clearer sentences;

  • Providing context behind statistics;

  • Being transparent about uncertainty; and

  • Indicating where consensus lies.

We also encourage news organizations to consider structural changes like rethinking their relationship with newswires and working closely with statisticians.

Acknowledgments

Discussions with a great number of colleagues at both PBS NewsHour and Knology have informed this work, particularly Rupu Gupta, Erica Hendry, Nicole LaMarca, and Megan McGrew. We are also grateful to our project evaluators and advisors, who have participated in those conversations — and several of whom have read earlier drafts of this piece: Jim Corter, Jim Hammerman, Eric Hochberg, Danny Bernard Martin, Caitlin Petre, Jonathan Stray, Nikki Usher, and Darryl Yong. We are also grateful to two anonymous reviewers for comments which have clarified our thinking.

Disclosure Statement

No potential conflict of interest was reported by the author(s).

Funding Statement

This material is based upon work supported by the National Science Foundation under grant number 1906802. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Data Availability

The full data set, including all coded clauses, is available at https://bit.ly/3jIF3K0.

Notes

1 The many ways in which this is so are beyond the scope of the present paper, but see D’Ignazio and Klein (Citation2020) for a synthesis of many recent critiques.

2 PP (journalist): “This ideology calls to mind the debate over whether algorithms can be biased. Of course they can, but for a time people assumed they were as pure as numbers.”

3 Within the context of journalism, the notion of mechanical objectivity has also been used to critique the unreflexive symmetry we know informally as “both-sidesism.” For example, Galison (Citation2015) quotes an editorial published in TIME in 1950 that uses this term to take The New York Times to task.

4 Their categories are somewhat more granular than ours, and they found a large number of statistics in topic areas that would likely have been subsumed under one of ours: 58.5% of energy stories contained statistics, as did 54.1% of social policy stories, 49.7% of business stories, and 47.9% of taxation stories.

5 IIT (journalist): “My sense is that [the desire for credibility] is a fairly prominent motivator, underscored by the idea that numbers are objective and legitimate.”

6 Upon beginning to examine the clauses with 4 codes or more, we found some parsing errors. We thus excluded those that should have been further divided into multiple clauses (by our operational definition), leaving us with 166 clauses for close reading. In at least some cases, at least one of the component clauses may have received 4 codes on its own. However, we excluded these to be conservative.

7 It is important to note that we removed duplicate stories from the data set: some stories appeared on multiple dates, and some stories appeared in multiple content areas. That is, these stories appeared only once in the data set for analysis.

8 Fox News, the Associated Press, and – to a lesser extent – CNBC are more heavily represented among high-code clauses than in the data corpus as a whole: 11.7% of clauses in the database were produced by the Associated Press, 6.1% by CNBC, 2.5% by Fox News, 9% by CNN (including CNN’s wire service), and 8.2% by the New York Times.

9 National general outlets are under-represented compared to their representation in the database as a whole, while all other outlet types are over-represented: 53% of all clauses were produced by national outlets, 12.5% by economy specialists, 13.3% by newswires, and 3.2% by partisan outlets.

10 We are indebted to a colleague for observing in a conference session that “journalists say everything they know up front.”

11 As an example, we've rewritten the first example to make it more accessible: In 2019, the Virginia Joint Commission on Healthcare did a study. They found that Virginia has 122 licensed hospitals. Only 16 of them provide sexual assault forensic exams. The state also has 94,000 registered nurses. But only about 150 of them are credentialed forensic nurses.

12 By contrast, the densest Politics story has 1.87 codes per clause. It is the 23rd most dense story in the dataset. The densest Science story has 1.72 codes per clause. It is the 31st most dense story in the dataset - behind 13 Health stories, 16 Economy stories, and 1 Politics story.

13 For example, Irwin (Citation2020) walks through common economic indicators – the unemployment rate, the 'establishment survey,' and GDP – to outline how COVID-19 shutdowns interact with data collection and reporting conventions to produce irregular results.

14 IIT (journalist): “I'd argue that this problem still [in May 2021] largely has gone unaddressed in a lot of reporting, especially mortality.” She recommends https://ourworldindata.org/covid-mortality-risk

15 We expected to see both left-wing and right-wing outlets, but all three outlets in this category were right-wing: Fox News, The Daily Wire, and The Blaze.

16 Rather than attempting a theoretical definition of news, which remains contested (see, e.g., Armstrong et al. Citation2015; Cunningham et al. Citation2016; Edgerly and Vraga Citation2019; Ekström and Westlund Citation2019), we defined it functionally and accepted Google News’ inclusion criteria at face value. The single story from NASA was a YouTube video from a series called “NASA Explorers” intended for general audiences, which we judged to be comparable to other science stories in the sample. The story was relatively light on quantification and thus not one of the stories we focus on for analysis in this paper.

Additional information

Funding

This work was supported by US National Science Foundation [Grant Number 1906802].

Log in via your institution

Log in to Taylor & Francis Online

PDF download + Online access

  • 48 hours access to article PDF & online version
  • Article PDF can be downloaded
  • Article PDF can be printed
USD 53.00 Add to cart

Issue Purchase

  • 30 days online access to complete issue
  • Article PDFs can be downloaded
  • Article PDFs can be printed
USD 315.00 Add to cart

* Local tax will be added as applicable

Related Research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations.
Articles with the Crossref icon will open in a new tab.