The Quality of COVID-19 Coverage: Investigating Relevance and Viewpoint Diversity in German Mainstream and Alternative Media

ABSTRACT The COVID-19 pandemic was a major challenge for journalism. In the case of Germany, critics very early on voiced concerns over the quality of mainstream media coverage. Mainstream media were attacked for the intensity of their coverage, for reporting too much and unnecessarily frightening audiences, for not presenting a diversity of viewpoints on the pandemic and the measures taken to mitigate it, and for being one-sided and too close to the government. Against this background, this paper examines the intensity and viewpoint diversity of the coverage of 13 mainstream and two alternative media in Germany in 2020 and 2021. Based on a comprehensive manual content analysis of almost 9000 news stories, we conclude that the mainstream media did not cover the pandemic in a fundamentally exaggerated or one-sided way. However, they did not always report intensively when the pandemic was particularly severe, and they tended to warn against the pandemic rather than criticize the measures taken. In contrast, alternative media coverage tended to be more one-sided and generally biased against government measures.

Who are reliable experts?Which assessments and opinions are well-founded and deserving of publicity-and which are not?In crisis situations in particular, journalistic uncertainty is often countered by an orientation toward official or established sources, as well as intensive co-orientation within newsrooms and across different media, at least until new news routines and frames are established (e.g., Vasterman 2005).This makes crisis situations prone to a reduction of media diversity and the development of intra-and cross-media consonance.On the one hand, this can lead to media coverage that is increasingly disconnected from the real development of events, as editorial decisions are overwhelmingly driven by official sources pushing their own agenda and/or processes of mutual reinforcement within the media system (e.g., Vasterman 2005).On the other hand, these processes may contribute to a lack of diversity, which may be reflected in a reduced variety of opinions and frames present in the coverage (e.g., Maurer et al. 2022).Clearly, such developments would be a cause for concern, as they could jeopardize the media's ability to fulfill its basic democratic function of reporting on the most important issues of the day (relevance) and presenting a variety of perspectives on these issues (viewpoint diversity) (e.g., Joris et al. 2020;Masini et al. 2018;McQuail 1992;Napoli 1999).
Against this background, and building on recent research on the dimensions of quality of journalistic reporting (e.g., Bachmann, Eisenegger, and Ingenhoff 2022), on journalistic reporting in conflicts and crises (e.g., Maurer et al. 2019;2022), and on the differences between mainstream and alternative media (e.g., Heft et al. 2020), we examine two dimensions of quality-relevance and viewpoint diversity-in the context of the COVID-19 pandemic.One reason to concentrate on these two dimensions was that they were a focus of media criticism during the pandemic (see details below).Another reason to look at diversity is that it is often regarded as one, if not the key dimension of media performance (e.g., Loecherbach et al. 2020).An additional reason to investigate relevance was that the availability of official data provided us with the opportunity to compare media coverage and real-world developments that serve as an external benchmark for news decisions (e.g., Jacobs et al. 2018;Maurer et al. 2019;2022).
The empirical basis of our study is a large-scale quantitative manual content analysis of 13 mainstream and two alternative German media in the 2 years between the beginning of 2020 and the end of 2021.The longitudinal design allows us to track the development of coverage over several waves of the pandemic and to contrast it with real-world developments.The relevance of our research is twofold: On the one hand, the content characteristics examined provide insights into the journalistic performance of the German media during one of the most significant social crises in recent times.On the other hand, the characteristics studied are also important because we can assume that journalistic performance along our dimensions was decisive for citizens' attention to the pandemic and their opinion formation regarding political efforts to mitigate the crisis. 1

Relevance of Media Coverage
Relevance as a criterion of journalistic quality refers to the question of whether the media devote attention to issues, events, and actors according to their actual importance.It is regarded as an important criterion across different models of the democratic public sphere (e.g., Bachmann, Eisenegger, and Ingenhoff 2022;Jandura and Friedrich 2014;McQuail 1992).As importance cannot be judged without a standard, relevance is regarded as a relational concept, which means that the (external) relevance of an event, issue, or actor can only be judged in comparison to other events, issues, or actors.Scholars have come up with various quantitative and qualitative indicators for relevance that sometimes are inspired by news value theory.Among them are the number of people affected, individual or social importance, the likelihood of an event or a consequence, the gravity of effects or consequences, the geographical proximity of an event, etc. (e.g., McQuail 1992, 198).
Another way to arrive at conclusions about the relevance criterion is to compare coverage with external benchmarks like data on real-world developments, judgments by experts, or the coverage of other media.However, most studies focus on comparisons with external data (intra-extra-media comparison, e.g., Jacobs et al. 2018;Maurer et al. 2019;Vasterman 2005).In the context of the pandemic, several studies have investigated the intensity of coverage (e.g., Krawczyk et al. 2021) and some have also used mediaexternal epidemiological data to assess the adequacy of the intensity of Corona coverage over time (e.g., Mach et al. 2021; Ort et al. 2023; Pearman et al. 2021). 2 For example, in an automated analysis of coverage in 50 countries between January and November 2020, Pearman et al. (2021) found that coverage dramatically increased in February and March, but then declined despite a continued average increase of new infections across the 50 countries (see also Mach et al. 2021;Ort et al. 2023).
In general, journalistic reporting would score higher on the relevance dimension the more it is in line with the official indicators of the general evolution of the pandemic.However, we should not expect a perfect match between real-world data and coverage.First, it can be assumed that coverage of the pandemic is not only driven by the epidemiological situation itself, but also by other events such as political discussions about Corona measures that might, and this is crucial, precede or follow the rise and fall of infections and deaths.Second, it is likely that after an initial period of uncertainty, newsrooms established new news routines and generally became better able to assess what was really going on and how information about the pandemic was to be assessed and presented.This could suggest that the relationship between the actual evolution of the pandemic and media coverage might become stronger over time.Third, the pandemic itself changed in very dynamic ways during the period we studied.For example, there were shifts in the dominant variants of the virus that changed the level of threat posed by it; COVID treatment improved; and vaccines became available in early 2021.From a normative point of view however, we would still expect a certain degree of congruence between infections, deaths, and media coverage-also because especially in uncertain crisis situations citizens' problem perceptions strongly depend on the amount of media coverage on a given issue (e.g., Maurer and Holbach 2016).We therefore ask: RQ1.How intense was the coverage of the COVID-19 pandemic in German news media and how did the intensity of coverage relate to the real-world development of new COVID-19 infections and COVID-19-related deaths?

Viewpoint Diversity in Media Coverage
Diversity as a criterion of journalistic quality refers to the variety of topics, actors, or opinions that are present in journalistic reporting.It is therefore concerned with the heterogeneity of structural or content features of media coverage (e.g., Van Cuilenburg 1999, 188).Diversity is regarded as one, if not the key dimension of journalistic quality across different normative models of democratic public spheres, because it is seen as a fundamental prerequisite for citizens' free opinion formation and political decision-making (e.g., Jandura and Friedrich 2014;McQuail 1992;Strömbäck 2005). 3However, different models of the public sphere differ in whether they expect diversity at the level of individual news items, individual news outlets (internal diversity), or the media system (external diversity).For example, while liberal and discursive models suggest a preference for internal diversity to provide all audience members with diverse perspectives irrespective of their media repertoire, participatory models argue that diversity can also be provided by "alternative media outlets, which give a voice to citizens who normally would find it difficult to gain access to the mainstream media" (Jandura and Friedrich 2014, 364).
However, diversity in media reporting is particularly at-risk during crises because a consensus often develops among political elites that is then adopted by the news media.Accordingly, in crises, the news media may largely rely on official government sources and convey their messages in a relatively uniform manner (indexing theory ;Bennett 1990;Maurer et al. 2022).In the context of the pandemic, Matthews et al. (2024) investigated the diversity of sources of pandemic reporting in UK news media and found that official government sources dominated this topic.Krawczyk et al. (2021, 8) used automated sentiment analysis to investigate the emotional homogeneity of COVID coverage across eleven countries and compared to other topics (e.g., cancer).They concluded that coverage was not exclusively negatively polarized.To the best of our knowledge, no studies have been published yet that focus on more specific aspects of viewpoint diversity in COVID coverage.
In this paper, we concentrate on viewpoint diversity for two reasons: First, viewpoint diversity is most closely related to the reasons why diversity is considered important in democratic discourse in the first place (e.g., Baden and Springer 2017).Second, media criticism in the context of the pandemic very often suggested that the media were onesided and did not voice criticism specially against the COVID-measures thereby silencing dissenting voices and manipulating public opinion (e.g., Holtz-Bacha 2022; Newman et al. 2021).Viewpoint diversity refers to the degree of variety or the dispersion of perspectives or opinions about a given issue, actor, etc. that are represented in media coverage (e.g., Baden and Springer 2017;McQuail and Van Cuilenburg 1983).In our analysis, we include two indicators of viewpoint diversity.We first investigate how diverse the evaluation of the Corona measures was that were imposed by the executive on the federal, state, or local level (e.g., lockdowns, closing of schools and businesses, mask mandates, etc.).We therefore put forward our RQ2: RQ2.How diverse was the coverage of the COVID-19 pandemic in German news media both internally and externally with respect to the evaluation of Corona measures?Second, we will look at the overall framing of the debate on how to best deal with the pandemic.Framing has also been suggested as a way to measure viewpoint diversity (e.g., Loecherbach et al. 2020).More specifically, we focus on value frames that typically "depict debates about public policy issues as clashes of deep-rooted values" (Schemer, Wirth, and Matthes 2012, 335), and investigate whether news stories framed debates and decisions around COVID measures as questions of safety or freedom.We concentrate on the value conflict between freedom and safety because, in many countries around the world, Corona measures included unprecedented restrictions of constitutionally guaranteed individual freedoms.Especially in democracies, the balance between the state's duty to secure individual civic rights and to safeguard public health was therefore heatedly debated (e.g., Saam, Friedrich, and Engelhardt 2022).The importance of the freedomsecurity dilemma in the context of the pandemic is also apparent in the interest it has garnered in various academic fields like philosophy (e.g., Daemen 2022), psychology (e.g., Cheek, Reutskaja, and Schwartz 2022), and communication (e.g., Papadopoulou and Maniou 2021).We therefore put forward RQ3: RQ3.How diverse was the coverage of the COVID-19 pandemic in German news media both internally and externally with respect to the framing of decisions about measures as questions of freedom or safety?

Mainstream and Alternative Media Coverage
The development of a high-choice media environment has also contributed to the establishment of an ecosystem of so-called alternative media that attract considerable audience across many established democracies (e.g., Heft et al. 2020).Alternative media portray themselves as a "censorship-free" alternative to established traditional or "mainstream media" considered to be mouthpieces of the dominating political and social elites.In contrast to that, alternative media consider themselves to be purveyors of "true, genuine, authentic information" (e.g., Heft et al. 2020;Holt, Figenschou, and Frischlich 2019).However, the landscape of alternative media is far from being homogenous.In Germany, for example, the spectrum of what is regarded as an alternative outlet ranges from YouTube channels and websites run by conspiracy theorists to quite professional outlets by right-wing conservative or left-wing journalists and publicists to highly professional and well-funded outlets like "RT Deutsch" (Russia Today) that is run by the Russian state (Freudenthaler and Wessler 2022a;2022b).Recent studies investigating whether alternative media really foster content diversity have concentrated on speaker and topic diversity (Buyens and Van Aelst 2022;Freudenthaler and Wessler 2022a;2022b).However, these studies did not investigate viewpoint diversity, which is important, because regarding speakers or topics as proxies for viewpoint diversity is problematic (Baden and Springer 2017;Loecherbach et al. 2020).
In the context of the pandemic, prior studies mostly focused on media differences across the political spectrum (Mach et al. 2021) or between national and regional media outlets (e.g., Santos-Gonçalves and Napp 2023).In Germany, a study covering the very first months of the COVID-19 pandemic found that alternative media spread disinformation and conspiracy theories (e.g., Boberg et al. 2020).In addition, a study of Telegram channels shows that alternative media like the German outlet Russia Today (RT Deutsch) became important points of reference for the so-called "Querdenker"-movement ("lateral thinkers") of Corona deniers and extreme critics of Corona measures (e.g., Zehring and Domahidi 2023).And, finally, another study showed that citizens with a low trust in mainstream and a high trust in alternative media were especially skeptical about the dangers of COVID and the necessity of Corona measures right from the start of the pandemic (Reinemann, Haas, and Rieger 2022).
What does this mean for the two quality criteria that we want to investigate?With respect to relevance, we first assume that alternative media did not take the pandemic as seriously and thus paid less attention to it than mainstream media.Second, as they regard themselves as the alternative to "the system", they might as well have addressed the pandemic just as intensively as the mainstream media to fulfill their "duty" to "counter the dominating false narrative".As the above-mentioned arguments do not provide clear grounds for a directional hypothesis either way, we put forward RQ4: RQ4.Did mainstream and alternative media differ in their intensity of coverage of the COVID-19 pandemic?
With respect to diversity, we first can assume that the mainstream media themselves might differ.More specifically, we can assume that public service media would provide a more diverse picture of the pandemic than commercial media due to rather strict impartiality rules for public service providers (e.g., Hopmann et al. 2017).In addition, we can assume that the alternative media we investigate provide a less balanced picture of events.Because they regard the mainstream media as a monolithic block that provides a one-sided version of reality, they probably regard it as their responsibility to primarily present the "other side" of the story (e.g., Fuchs 2010;Schwaiger 2022).This means that, in terms of their contribution to diversity, they can be expected to add to the overall external diversity of the media system as a whole despite being not so diverse internally.The normative assessment of such a result would of course differ depending upon the normative perspective taken.We therefore ask: RQ5.Did mainstream and alternative media differ in the viewpoint diversity of their coverage of the COVID-19 pandemic?

Background: The COVID-19 Pandemic and the Media in Germany
During our study period in 2020 and 2021, Germany saw four completed COVID-waves with a fifth wave starting in the last week of 2021 (Schilling, Buda, and Tolksdorf 2022).The Corona waves were rather different with respect to the numbers of new infections, hospitalizations, and deaths.Vaccines became widely available in the spring of 2021, dramatically reducing the risk of a lethal outcome.Because of that, the second wave became the most lethal with the highest number of deaths recorded in late December 2020 and early January 2021.In contrast, the fourth wave had the highest number of new infections in November 2021.
Since the imposition of the first drastic measures by Federal and State Governments in March 2020, the coverage of the German media has became the subject of public criticism and debate.Critics voiced concerns over an alleged lack of diversity of opinions and experts, a lack of analytical depth and context, reporting in favor of the government, dramatization, and fearmongering, as well as a general and far-reaching uniformity in reporting (Betsch et al. 2020;Rieg 2020).Overall, according to some critics, German Corona coverage was not a "beacon of orientation" (e.g., Rieg 2020).Another central and re-occurring accusation regarding the performance of German media during the pandemic was that there was hardly any criticism of the measures (Betsch et al. 2020;Newman et al. 2021).We investigate these claims by asking how appropriate the intensity of coverage was, how critical or supportive the media were with respect to the measures discussed and taken, and which values the media referred to in framing the debates around Corona measures.

Method
This study analyses the diversity and consonance of COVID-19 coverage of 13 German news and two alternative media outlets between January 1, 2020, and December 31, 2021, using quantitative content analysis.The sample includes outlets of different genres and with different editorial lines (left-liberal to conservative and "alternative").The selection of the media outlets was informed by surveys conducted during the pandemic that included measures of individual media outlet use (see Reinemann, Haas, and Rieger 2022 for details).Among the mainstream media selected were the online editions of the three most-read broadsheet daily national newspapers Frankfurter Allgemeine Zeitung ( faz.net), Süddeutsche Zeitung (sueddeutsche.de),Welt (welt.de), the most-read tabloid newspaper Bild (bild.de), of two most-read national news magazines Spiegel-Online (spiegel.de)and Focus ( focus.de), of two regional newspapers from the western (Rheinische Post, rp-online) and the eastern part of the country (Sächsische Zeitung, sächsische.de), the news page of the most important Web provider (t-online.de),the main editions of the three most important national TV newscasts Tagesschau (ARD; public service), heute (ZDF; public service) and RTL aktuell (commercial), and the most-watched TV special that frequently aired at prime time during the pandemic (ARD Extra Corona).In addition, we analyzed two so-called alternative media, the Russian state-sponsored RT deutsch (RT German) and the formerly left-wing nachdenkseiten.de.Although differing in ownership and financial power, both of these outlets belong to the type of alternative media that position themselves as a counterpart to "manipulation by the mainstream media" and are often discussed in the context of disinformation and right-wing populism (e.g., Schwaiger 2022, 145-154).We chose these specific outlets because the results of a quota-based survey of the German population in April 2020 showed that they had the biggest audience among a list of ten alternative media at the beginning of the pandemic (see Reinemann, Haas, and Rieger 2022 for details; also Schwaiger 2022). 4 We included news items on the pandemic in which it was clear from the first paragraph (online media) or the beginning of the report (television news) that they dealt with the causes, development, or consequences of the COVID-19 pandemic and were related to Germany.Accordingly, we did not include reports that dealt exclusively with the pandemic in other countries.In addition, we did not include news tickers or regularly updated compilations of pandemic statistics (infection incidence, vaccination rates, etc.) in online media in our analysis.From TV news and special broadcasts, all news stories on the pandemic were included in the analysis.From news websites, articles that appeared on the starting pages in the main News section and in the Politics, Corona(virus), or Science(science) sections below the main news section were included.This way, we concentrate the analysis on particularly relevant and high-reach reports.The online articles were accessed from the start pages of the online editions, which were stored and retrievable via the internet archive (https://web.archive.org/).Due to the high volume of reporting, we then drew another systematic random sample from these stories for the analyses.For online news sites and television news, we selected only every other day.In the case of the news sites, we also included only every second relevant news item.In contrast to that, all special broadcasts on the pandemic were analyzed (full survey).As this makes for disproportionate samples for different types of media, we compensated for that by weighting our data before the analysis.All in all, nine trained coders analyzed 8956 news stories.However, due to weighting all analyses are based on 29,535 weighted cases (see Table 1).
For reliability testing, the nine coders coded a random sample of 47 news articles selected from the analyzed media outlets.The reliability tests showed a common understanding for all categories ranging from very good to good values of Holsti's and unstandardized and standardized Lotus coefficients between 1.0/1.0/1.0 (mention of various medical parameters) and 0.78/0.86/0.82(marking of uncertainty in forecasts). 5 Of the more than 30 categories of the codebook, two are relevant for this analysis: The first category captured the overall tone of a news story regarding Corona measures, i.e., measures decided upon by national, state, or local governments and public health officials to contain the spread of the virus or to mitigate its consequences.In this category on the article level, coders were supposed to decide whether measures were evaluated at all and-if that was the case-whether the overall impression conveyed was that measures were going too far, were about right, or should be tougher (5-point-scale).The inter-coder reliability reached a Holsti's coefficient of 0.82, an unstandardized Lotus of 0.89, and a standardized Lotus of 0.86 showing a good agreement.All in all, 36 percent of news items (n = 10,584) conveyed an overall evaluation of Corona measures.
Similarly, coders were instructed to decide whether an article framed the question of Corona measures mainly as an issue of freedom or safety, the key values that had to be balanced out by decision-makers when deciding upon measures.Coders were instructed to decide whether these issues were addressed at all and-if this was the case-articles conveyed the impression that the problem should be seen as a question of safety was more important in the context of fighting the pandemic, that freedom was more important, or that both were equally important (5-point-scale).The inter-coder reliability showed a good agreement with Holsti's coefficient of 0.86, an unstandardized Lotus of 0.91, and a standardized Lotus of 0.89.All in all, 32 percent of news items (n = 9,493) conveyed an overall evaluation of Corona measures.
Finally, we include two sets of real-world data in our analysis that come from the Robert-Koch-Institute (RKI), the key scientific institution responsible for safeguarding public health in Germany.The institute was responsible for monitoring the development of the pandemic, for federal health reporting, and for providing a scientific basis for policy decisions.In this paper, we use the number of new infections and the number of COVIDrelated deaths as real-world indicators.These appear to us to be the best possible indicators, although both also come with problems.The number of (detected) new infections depends heavily on the number of tests carried out.This fluctuated considerably over time and was presumably dependent on the risk situation portrayed in the media.The number of deaths naturally lags behind the number of new infections with a delay of a few weeks.In addition, the statistics do not distinguish between people who died with Covid and people who died of Covid.Finally, the number of deaths became less meaningful in the second half of our study period because significantly fewer infected people died because of the newly developed vaccines.Nevertheless, we can assume that the journalists based their reporting primarily on these two indicators because they were mentioned particularly often in the media (Maurer, Reinemann, and Kruschinski 2021).

Relevance of Media Coverage
The first important aspect of journalistic quality we turn to is relevance.This regards the question of whether the media cover the most important issues or accurately reflect the importance of a topic in their coverage.However, it often is far from easy to come up with an objective standard for "importance".Therefore, scholars have in the past turned to realworld data that can serve as benchmarks for the editorial decisions made in newsrooms.
In the case of COVID-19, there are plenty of data that cover various aspects of the pandemic.We chose to concentrate on two rather straightforward indicators that were available very early on in the pandemic: new Corona infections and new Corona-related deaths.As these numbers were published very early on in the pandemic by RKI and as they featured very prominently in media coverage for most parts of the period under study, we can assume that journalists were generally aware of them (Maurer, Reinemann, and Kruschinski 2021).
From the data we gathered, it is apparent that the intensity of coverage varied a lot over the course of the pandemic with peaks at the very beginning from March to May 2020 (weeks 11-19), in the fall of 2020 (weeks 42-45), winter 2020/2021 (weeks 49-51), spring 2021 (weeks 8-11), and again in November of 2021 (weeks 42-47).A visual examination of the time series shows, on the one hand, fundamental similarities over time.Increases in media coverage are usually accompanied by increases in the number of infections.The clearest exception to this rule can be found in early 2021 (weeks 2-8) when media reports increased significantly despite falling infection numbers.Without additional data, we can only speculate about the potential reasons for that mismatch, but we come back to this in our discussion.
At the same time, however, a long-term view also shows that media reporting had already peaked at the beginning of the pandemic, although the number of (detected) new infections was still comparatively low here.In the later waves of the pandemic, some of which had significantly higher numbers of infections, the media did not lose interest in the pandemic.However, the significantly higher number of infections was not reflected in the amount of reporting.Statistically, the number of infections and media reports correlate only moderately over the entire period (0.347**).However, if the investigation period is divided into four semi-annual quarters, significantly higher correlations (0.678**-0.713***)can be seen for the last year and a half of the investigation period.This suggests that the extreme uncertainty at the beginning of the pandemic has caused reporting to increase disproportionately, while reports later in time were a much better indicator of the true magnitude of the problem (Figure 1).
Turning to a comparison with the number of deaths, the overall picture seems similar at first glance with the overall correlation being slightly higher (0.44**).Here, too, there is a notable exception: the by far highest number of deaths at the turn of the year 2020/ 2021 was not accompanied by an unusually high amount of reporting.Media reporting here was obviously more geared toward the number of infections, which was already falling again.Overall, however, the correlations between media coverage and deaths are greater than between media coverage and new infections, mainly because the decreasing number of deaths due to vaccinations over the course of 2021 coincided with a slowly waning media interest in the pandemic (Figure 2).
Regarding differences between mainstream and alternative news media, we take a look at the intensity of coverage over time.Because the overall output of news items and stories differs considerably between individual outlets and because our group of alternative media only consists of two rather different media, we do not compare the average number of items in mainstream vs. alternative media.Instead, we focus on the overtime variance of attention the two types of media paid to the pandemic.As it turns out, the intensity of coverage in both groups of media is highly correlated across the study period (0.77***).This means at least that the alternative media did not choose to refrain from covering the pandemic because they thought it was a hoax and often increased or decreased their coverage at the same time the mainstream media did (Figure 3).

Evaluation of Corona Measures
We first look at external diversity by investigating the evaluation of Corona measures at the aggregate level of all media.In the 2 years under study, most of the stories dealing with the issue either considered Corona measures at the time as adequate (44%) or conveyed the impression that they did not go too far (28%).Another 28 percent suggested  that measures went too far.Looking at the development over time, a visual inspection of Figure 4 suggests first that there was a great deal of variability in the share of critical and supportive news stories.We can assume that this variability is a result of the fact that our category is relational in that it refers to specific measures discussed at a specific point in time against the backdrop of the specific epidemiological situation at the time.It therefore, second, might not come as a surprise that no clear trends are apparent in the overall evaluation of the measures that were discussed.Third, except for the first 8 weeks of 2020, in which the number of relevant articles was extremely low because the pandemic had not even really started, there always was a considerable number of articles that voiced criticism toward the measures as being either too strict or too weak (Figure 4).
For the analysis of internal diversity, we now look at the evaluation of Corona measures in the different media outlets.Among the mainstream media analyzed, there seems to have been a great deal of agreement on the general evaluation of Corona measures.In the majority of news stories, Corona measures were either regarded as being about right or not going too far (if an evaluation was apparent at all).Over the whole study period, the number of stories criticizing measures as going too far did not exceed 35 percent.Taking a closer look, some mainstream media did in fact strike at least slightly differing chords.In TV newscasts and special broadcasts, on regional newspapers' websites, and in the journalistic section of the Web provider t-online, the number of critical stories did not exceed 20 percent.The websites of national magazines and newspapers were slightly more critical with a share of more than 20 percent critical articles.Among them, Welt and Bild, both from conservative publisher Springer, stood out as being the most critical of all mainstream media.Here, more than one-third of stories saw the measures as going too far.Turning to the two alternative media RT Deutsch and nachdenkseiten.deit becomes clear that they were by far the most critical with half (RT Deutsch) or even almost 90 percent (nachdenkseiten.de) of stories having a negative tone (Figure 5).

Value Framing (Safety vs. Freedom)
Regarding external diversity of safety and freedom framing, the picture was even more pronounced with 58 percent of relevant stories stressing safety, 22 percent arguing for an equal importance of safety and freedom, and just 20 percent preferring freedom.Whereas the overall evaluation of measures remained rather stable over time, there is greater variation in value framing clearly depending on the current threat of the pandemic.Safety concerns were most dominant at the very beginning of the pandemic in early 2020, but they became less important during the spring when the first Corona wave faded out, and concerns about freedom became much more important.That changed again in June 2020 when safety became the dominant value in the discussion of COVID-19, probably due to first warnings of a second wave.This dominance of safety basically lasted throughout the deadly second wave until late March 2021.After that, the picture again changed, and freedom became the dominant value for most of the time until the next wave started in October 2021 (Figure 6).
Regarding the comparison of internal diversity between different media, only a small minority of news items in most mainstream media used a clear freedom frame to cover the pandemic.This was slightly different in the rather conservative Bild.de and Welt.de, which used a freedom framing in about one-quarter of their reports.In contrast, the safety frame was most prominent in commercial and public service TV newscasts, but also in some other media with more than two-thirds and up to 70 percent of stories taking the safety perspective.However, at least five of the mainstream outlets (spiegel.de,sächsische.de,bild.de,faz.net, welt.de)used frames that either focused on freedom or freedom and safety in at least 40 percent of their stories.And finally, the picture is slightly to extremely different for the two alternative outlets.RT Deutsch used freedom and security frames to the same extent and nachdenkseiten.dewas almost completely devoted to the freedom frame (Figure 7).

Discussion
From very early on in the Covid-19 pandemic, news media in Germany and other countries were massively criticized for their reporting.Two main points of criticism were an allegedly excessive intensity of reporting and a one-sidedness of coverage uncritically supporting government policies (e.g., Holtz-Bacha 2022; Newman et al. 2021).Against the backdrop of the important role of news media in informing the public in times of uncertainty and crisis (e.g., Spence and Lachlan 2016), this paper investigated two dimensions of media quality: relevance (e.g., Bachmann, Eisenegger, and Ingenhoff 2022) and external and internal viewpoint diversity (e.g., Baden and Springer 2017).Although the normative expectations regarding these quality criteria vary between different models of democracy, all regard them as important (e.g., Bachmann, Eisenegger, and Ingenhoff 2021;Jandura and Friedrich 2014).To assess whether the media covered the pandemic adequately (relevance) we compared the intensity of coverage to official data on new infections and COVID-related deaths.To assess whether the media provided audiences with a multifaceted or one-sided picture (viewpoint diversity) we investigated the evaluation of Corona-measures and framing of the crisis as a problem of safety or freedom (value framing).In addition, we wanted to know how the reporting of established news media differed from the reporting of socalled alternative media (e.g., Heft et al. 2020).To answer these questions, we carried out a manual content analysis of 13 German mainstream and two alternative media in the first 2 years of the pandemic (January 2020-December 2021).
The analyses of relevance show that there was only a medium overall correlation between the amount of coverage and external statistics representing new infections and deaths.However, this was largely because mainstream and alternative media reported disproportionately heavily at the beginning of the pandemic.Significantly higher numbers of infected and dead in later pandemic waves led to comparatively less reporting, i.e., the number of news stories did not increase in the same way that new infections and deaths did.This pattern might be explained by the fact that a great deal of uncertainty in newsrooms was more and more replaced by newly established news routines regarding the pandemic.We assume that a lack of experience with a pandemic caused extreme uncertainty and led to intense journalistic co-orientation and reliance on official sources in early 2020, contributing to a somewhat exaggerated amount of coverage at the time.In addition, journalistic competition and the extreme novelty of the topic might have played a role here.Over time, however, with journalists learning about the virus, the illness, the interpretation of epidemiological statistics, etc., new news routines were established, obviously changing the way real-world developments were translated into news stories.It would have been difficult to increase reporting in a proportionate way in later waves because column inches and broadcasting time in media reporting are limited.The coverage-infection-ratio (of enormous coverage to comparatively low numbers of cases) at the advent of the pandemic would have been technically impossible to sustain as the pandemic progressed with even higher infection rates.At the same time, however, it can be argued that the German population did not always receive a completely adequate picture of the danger posed by the pandemic.If we assume that people use the intensity of reporting as an indicator of the relevance of a problem (agenda-setting), we can assume that the awareness of the increasing threat posed by COVID-19 might have been negatively affected during later phases of the pandemic.But there are other potential explanations for the relative decline of coverage in the second wave: Maybe journalists reacted to an increasing fatigue among audiences regarding the pandemic (Wiedicke, Stehr, and Rossmann 2023) or to the potential negative mental health consequences that researchers began to worry about (Su et al. 2021) and adapted the intensity of their coverage.Whether these explanations hold true should be investigated in future studies.
The analyses of internal and external diversity of media coverage show that it was not quite as one-sided as critics claimed.This was particularly true for the assessment of the Corona measures, which always varied greatly and in all mainstream news media, indicating a considerable degree of both external and internal diversity.Although support for the measures taken by the government outweighed critical assessments in almost all media most of the time, a significant proportion of the news items also considered at least some of them as going too far or not going far enough.This means that critical voices were indeed present.On the other hand, the value framing (safety versus freedom frame) was less diverse, and most news media overwhelmingly sided with security most of the time.
In contrast to the mainstream new media, the alternative nachdenkseiten.dereported in an extremely one-sided manner, almost consistently criticizing Corona measures and prioritizing individual freedom.From the perspective of a liberal model of democracy that highly values internal diversity, this lack of diversity can be regarded as a problem.From the perspective of the participatory model and in the eyes of an alternative medium that is explicitly positioning itself as a representative of a counter-public, this result could be interpreted as evidence that they acted in line with their self-proclaimed editorial concept.The German branch of Russia Today, RT deutsch, on the other hand, also set a somewhat different tone than the established German news media but remained significantly more diverse than nachdenkseiten.de.This can be seen as the result of an attempt to create the appearance of a medium that guarantees at least a minimum of journalistic professionalism in order not to frighten off its users and to have the chance to convey its most important messages.
The interpretation of our findings significantly hinges on the role attributed to the news media during times of crisis and on the normative democratic model used.If one's expectation is that the media should primarily function as critics and watchdogs and provide the public with a diversity of viewpoints regardless of potential negative outcomes, then any reporting perceived as favoring a particular perspective will be deemed biased and inadmissible.Conversely, those who argue that the media's primary responsibility is to support and promote measures that are essential for combating a pandemic and that are in accordance with a scientific consensus may accept or even call for a certain level of one-sidedness in favor of these measures.However, this standpoint raises a fundamental question about the acceptability and necessity of limited diversity vis-à-vis the danger of false balance in media reporting in general (Humprecht 2022).For instance, Brossard and Nisbet (2006) emphasize the need for a careful selection of sources to maintain credibility.Similarly, Boykoff and Boykoff (2004) discuss the pitfalls of "balance" in media reporting where presenting non-expert opinions alongside expert ones can lead to false equivalency and misinformation.Therefore, while our study advocates for viewpoint diversity, we also acknowledge the responsibility of media outlets to vet information and sources rigorously.This is particularly crucial in an era where the proliferation of digital media has blurred the lines between expert and non-expert contributions (Chadwick 2017).
The normative question concerning the conditions under which restricted diversity in media reporting can be deemed acceptable or necessary will probably remain a contested issue and topic of discussion in journalism, among the public and in science.Thus, it warrants in-depth exploration and robust discussion in future theoretical papers.Examining this question can help to illuminate the complex interplay between media responsibilities, public interest, and crisis management strategies.As such, this study calls for further scholarly inquiry into the ethical and societal implications of media coverage in crisis situations.

Contribution to the Literature
This study contributes to the literature on media performance in several ways.Theoretically, this study falls in line with recent studies that connect the debate on media performance more closely to the debate on normative standards.In this respect, our study highlights the extent to which the interpretation of empirical results hinges on the democratic model applied as well as the factual basis that the respective coverage is based on.Empirically, this study investigates media quality in a situation of an acute and longlasting crisis which is relatively rare in the literature.Second, it uses real-world data as a benchmark to evaluate the relevance of reporting which also is not done very often.In this context, it points to the challenges and opportunities that come along with the comparison of intra-and extra-media data.Third, our study includes a diverse media sample that also includes alternative media.As such media have only rarely been investigated and compared to mainstream media with respect to their quality we regard this as another especially important contribution of this study.

Limitations
This study has several limitations.First, although we examined the coverage of a relatively large number of media outlets over a rather long period of time, our data are limited to Germany.Future studies should therefore be internationally comparative and also connect the patterns of coverage to the real-world development of infections and measures as well as to other macro-level contextual factors (e.g., commercialization of media and political polarization).
Second, in our analyses of intensity (relevance), we connected the number of news stories to official data in a rather simple way.This enabled us to show basic matches and mismatches between coverage and certain real-world developments.However, our results suggest that the way the pandemic situation was translated into a specific intensity of coverage is far from trivial.In fact, this does not really come as a surprise because some potential drivers of Corona coverage may have had a greater disconnect to the epidemic situation than others (e.g., political debates before or after a wave of infections).To incorporate other types of real-world events into more complex models to explain the intensity of coverage would therefore be an important next step in research.It would also be possible to investigate this question with retrospective in-depth interviews with journalists.
Third, we only examined two indicators of viewpoint diversity and these indicators are rather abstract in nature.In additional analyses, we plan to look at the diversity of actors, the specific Corona measures addressed as well as the specific epidemiological data presented (e.g., infections, hospitalizations, ICU admissions, and deaths).However, analyses we have performed for a shorter period of time and a smaller set of media outlets (Maurer, Reinemann, and Kruschinski 2021) suggest that other indicators of diversity point in a similar direction.

Conclusion
Overall, this study of COVID-19 coverage shows that the reporting of German media was neither fundamentally exaggerated in its intensity nor fundamentally one-sided in its evaluation of Corona measures.However, the media did not always report with particular intensity on the pandemic when it was in fact particularly severe, and they acted more as an early warning system about the pandemic than as critics of the measures taken.The extent to which this is typical for crises of this type should also be examined in further studies.Overall, the established news media largely took a warning stance toward the pandemic, indicating a limited degree of external and internal diversity in value framing.However, they did not uncritically support all the measures the government proposed or implemented and repeatedly presented deviating positions.We therefore conclude that the German mainstream media indeed performed better than their harshest critics claimed.Notes 1.Preliminary results for a subset of media and the time between January 1, 2020, and April, 30, 2021 have been published in German (Maurer, Reinemann, and Kruschinski 2021).2. Krawczyk et al. (2021) also compared the intensity of Corona coverage for the first 10 month in 2020 across different countries and in relation to the intensify of coverage about various heads of government.These benchmarks are not ideal, however, because the pandemic situation varied across countries and a topic is compared with actors.3. Diversity is closely related to balance, but balance refers to an adequate representation of topics, actors, or opinions.Therefore, from the perspective of balance, variety is not a value per se (e.g., Hopmann et al. 2017;McQuail 1992).In addition, balance raises the question of benchmarks (e.g., Hopman et al. 2017).Moreover, balanced proportionality might even impede the attempt to provide a diverse picture of viewpoints.For example, an emphasis on majority opinions might make minority positions less visible, favor the status-quo and impede social change (e.g., Jandura and Friedrich 2014).And finally, scholars have pointed to the problem of "false balance", which refers to the equal representation of valid, evidence-based views and views that are demonstrably false or stand against a scientific consensus (e.g., Boykoff and Boykoff 2004).4. Details of the quota-based survey of the German population (18-65 years) can be found in Reinemann, Haas, and Rieger (2022).The alternative media included in the survey were Compact online, Epoch Times, Junge Freiheit, Ken FM, Kopp online, nachdenkseiten.de,PI News, RT Deutsch, Sputnik and Tichys Einblick.The only alternative outlets with a weekly reach of more than 10 percent turned out to be RT Deutsch (17.9%) and nachdenkseiten.de(11.3%).The success of the two outlets is also documented by Schwaiger (2022, 128).
Taking website visits as an indicator, she also put Russia Today (later renamed RT Deutsch) and nachdenkseiten.deamong the 10 most successful alternative media outlets in Germany.In addition, Russia Today also was very successful on Facebook and YouTube (Schwaiger 2022, 129)-at least before it got banned from both platforms by the EU. 5.The Lotus coefficient represents the raw percentage agreement of coders with the category most frequently used by all coders and the standardized Lotus coefficient is a chance-adjusted Lotus.According to this method, the correct code serves as the reference category, or "gold standard", based upon which the chance-corrected reliability is computed (Fretwurst 2015).

Figure 3 .
Figure3.Intensity of coverage in mainstream and alternative media (N; weekly basis).Note: 29,535 articles (weighted) in 13 mainstream and two alternative news outlets.

Figure 4 .
Figure 4. External diversity of evaluation of Corona measures (% of articles presenting measures as going not far enough, being about right or going too far) (weekly basis).Note: 29,535 articles (weighted) in 15 news outlets.Only articles that included an overall evaluation of Corona measures (n = 10,584).

Figure 5 .
Figure 5. Internal diversity of evaluation of Corona measures in mainstream and alternative media (% of articles presenting measures as going not far enough, being about right or going too far) (weekly basis).Note: 29,535 articles in 13 mainstream and two alternative news outlets.Only articles that included an overall evaluation of Corona measures (n = 10,584).

Figure 6 .
Figure 6.External diversity of value framing (% of articles using safety, freedom or mixed framing) (weekly basis).Note: 29,535 articles (weighted) in 15 news outlets.Only articles that included an overall value framing of Corona measures (n = 9,493).

Figure 7 .
Figure 7. Internal diversity of value framing of COVID measures in mainstream and alternative media (% of articles using safety, freedom or mixed framing) (weekly basis).Note: 29,535 articles (weighted) in 15 news outlets.Only articles that included an overall value framing of Corona measures (n = 9,493).

Table 1 .
News stories per medium included in the analysis (weighted).