Illusion of knowledge: is the Dunning-Kruger effect in political sophistication more widespread than before?

ABSTRACT The dramatic expansion of social and digital media during the past fifteen years has fundamentally transformed the media landscape in Western democracies. Nevertheless, we still know little about the impact it has had on aggregate levels of political sophistication. Previous research has suggested that increased information availability may result in inflated self-perceptions of (political) sophistication. Based on the Dunning-Kruger effect, it is plausible to think that low-sophistication individuals are particularly vulnerable for overconfidence in political sophistication. Using a repeated-measures cross-sectional survey data with samples representative of the Finnish voting-age population, this study analyzes the prevalence of the Dunning-Kruger effect in 2008 and 2020, before and after social media revolutionized the political information landscape. Although the Dunning-Kruger effect is more widespread in 2020 than in 2008, the increase is not statistically significant at the individual-level. However, the findings suggest that at the individual-level, overconfidence is linked to relying on internet and social media for political news, but not to relying on traditional media. This aligns with recent research showing that social media contributes at best minimally to political learning. Instead, it seems to amplify overconfidence, especially among the least knowledgeable segment of the population.


Introduction
The expansion of social and digital media during the past approximately fifteen years has fundamentally transformed the media landscape in Western democracies.Possibility for exposure to political information has increased dramatically, but we still know very little about whether aggregate levels of political sophistication have remained stable.
Recent scholarship has demonstrated that using social media for political news increases self-assessed knowledgeability, but not necessarily objectively measurable knowledge (Müller, Schneiders, and Schäfer 2016;Ran, Yamamoto, and Xu 2016;Schäfer 2020).The mere availability of political information online induces a sense of self-perceived knowledgeability (Schäfer 2020), which risks resulting in an "illusion of knowledge", where access to more information reduces correctness while simultaneously increasing self-confidence (Hall, Ariss, and Todorov 2007).An inflated self-confidence is more likely to occur among low-sophistication individuals.As posited by the widely documented Dunning-Kruger effect, low-performers have a tendency to overestimate their own abilities; not only is their level of performance low, they are also unable to recognize it (Dunning 2011;Kruger and Dunning 1999).
While analyses of self-assessed political sophistication are not new (see e.g.Druckman 2004), political scientists have only seldom explored the Dunning-Kruger effect.The pioneering work by Anson (2018) demonstrated its significance for political cognition, and linked it to partisan identity and motivated reasoning.This short study examines whether the Dunning-Kruger effect has become more widespread over time.The analysis utilizes a unique repeated cross-sectional survey design from 2008 and 2020 with population-based samples from Finland.The two surveys have similar samples of the voting age population, include identical political knowledge items and identical items for measuring self-assessed sophistication and the relevant control variables.The combined data allows a robust comparison of the Dunning-Kruger effect in political sophistication between two relevant points in time: one in the beginning of the new (social) media era, the other twelve years later.Through this comparison, the study addresses the wider debate about the impact of digitalized (social) media on democratic citizenship.

Overconfidence in political sophistication
According to an optimistic account, the ever-presence of online political content could bridge the gap between low and highly informed citizens, thus contributing to aggregate sophistication levels (e.g.Bode 2016).The assumed impact comes from a kind of "trickle-down" effect, as people constantly become exposed to political content even when they do not seek it (Shehata and Strömbäck 2021).In a more pessimistic view, the expansion of online content makes it possible to choose whatever one wants.In this scenario, political learning is driven by personal motivation, rather than by (involuntary) exposure (Prior 2007;Wolfsfeld, Yarchi, and Samuel-Azran 2016).
One possible consequence of the explosive growth of online political content is self-perceived sophistication.Even here, an optimistic interpretation suggests that the increased availability of political information could be associated with more accurate self-perceptions of sophistication.However, plenty of research suggests instead that access to more information is in fact associated with overconfidence (Hall, Ariss, and Todorov 2007; see also Schwartz 2004).Calling it the "illusion of knowledge", the authors argue that exposure to additional information often leads people to believe that it enhances their ability to arrive at better and more accurate decisions.According to Dunning and Kruger, however, such an illusion is likely to vary between individuals, depending on their level of aptness (in any domain).As the Dunning-Kruger effect posits, low-performers not only do poorly in whatever skill is being measured, but also lack the ability to understand their own low level of performance.
Taken together, it seems plausible to assume that compared with the presocial media environment, the current media environment, where political information is constantly accessible through various online platforms, is more conducive to a Dunning-Kruger effect in political sophistication.Consequently, the number of low-performers in political sophistication who overestimate their own level of sophistication, may have grown between the two points of comparison in the forthcoming analysis.
The matter has significance for our understanding of the possible consequences of the online (social) media revolution for the dynamics of representative democracy.A knowledgeable citizenry is widely considered to be fundamental to democracy, because democracy is ultimately based on public opinion that links political choices to real-life circumstances (Delli Carpini, Michael, and Keeter 1996).The media plays a decisive role in creating politically informed citizensor failing to do so.However, for the individual citizen, a lack of political knowledge can either be a matter of ignorance, i.e. unfamiliarity with politics, or of misinformation, i.e. factually incorrect beliefs that are held as being correct (Kuklinski et al. 2000).From the two, misinformation poses potentially a bigger threat from a normative viewpoint, because confidently held inaccurate beliefs are likely to drive opinion formation.This is particularly troublesome if and when misinformation systematically distorts the opinions of a certain section of the electorate, as opposed to randomly distributed misinformation effects that cancel out each other in the aggregate (see e.g.Jerit and Zhao 2020).Indeed, as Ortoleva and Snowberg (2015) show, overconfidence in political sophistication fuels ideological extremeness by strengthening partisan identification (see also Anson 2018).Increases in the Dunning-Kruger effect in political sophistication could therefore be a significant democratic dilemma.
It should be duly noted that many scholars have questioned whether the effect itself is only a statistical artefact.Burson, Larrick, and Klayman (2006), for example, could not replicate the findings by Dunning-Kruger, suggesting that there could be an effect of regression to the mean at play, rather than the purported poor performance of the most incompetent.Gignac and Zajenkowski (2020) and Nuhfer et al. (2017) have demonstrated similar results, which have, at least partly, challenged the original claim by Dunning and Kruger.Some scholars have chosen to refer to the more general term (epistemic) overconfidence when referring to the same phenomena, as a way to circumvent the problem.Emphasizing overconfidence, rather than the Dunning-Kruger effect, removes the focus from poorest performers.It is not possible to settle the contest between the vast literature that supports the existence of the Dunning-Kruger effect and the (growing) literature that questions it, within the scope of this study.However, the methodological criticism serves as a reminder to approach the findings with some caution.
Despite these important criticisms, the analysis follows the original logic of Dunning and Kruger, which focuses on low-performers.Although overconfidence can plausibly affect anyone, a lack of "meta-skills" or "self-monitoring skills" are not distributed equally across all individuals, but are instead more prevalent among the incompetent, as Dunning and Kruger argue.In practice, this means that low-performers lack precisely those skills that are needed to evaluate one's performance in that particular domain, where they display deficiencies.In other words, it is hard to realize one's incompetence in an area where one is incompetent (Kruger and Dunning 1999).The Dunning-Kruger effect has since become "one of the most highly replicable findings in social psychology" (Mazor and Fleming 2021); see also (Jansen, Rafferty, and Griffiths 2021), which suggests that focusing on the overconfidence lowperformers aligns well with the basic idea behind the Dunning-Kruger effect.
Moreover, as demonstrated by Hall, Ariss, and Todorov (2007), the availability of more information typically exacerbates accuracy problems, while simultaneously increasing false confidence.
Heavy reliance on the internet and social media can plausibly create precisely such an information environment, which exposes a person to large amounts of information.Combining this with the original logic of Dunning and Kruger could suggest that the emergence of the social media environment for consuming political news may have given rise to a situation where particularly the poorly informed are likely increasingly suffer from sophistication overconfidence.

Materials and methods
The analysis uses repeated measures from two surveys conducted in Finland in 2008 (n = 1020) and 2020 (n = 1097).Both data were collected in-between general elections, as face-to-face interviews by trained interviewers from the same survey company.The political knowledge measure consists of nine identical items for both years, combined into an additive scale from 0 to 9 (see Appendix for details).
The objectively measured knowledge scores are contrasted with selfassessed sophistication, which in both surveys was asked before the knowledge items: How much would you say you know about politics and public affairs?(a) a very great deal; (b) a great deal; (c) a fair amount; (d) not very much; (e) very little.Since the self-assessment does not involve placing oneself in relation to others, but instead requires an evaluation of one's absolute level of sophistication, there is no straightforward way to combine it with the knowledge measure.Hence, the analysis uses two operationalizations.The broadly defined indicator includes those respondents who scored four or less out of nine on the knowledge scale, which puts the respondents into the bottom third of the population, but who nevertheless rate themselves as being "pretty well", "well" or "very well" familiar with political matters (see Appendix for variable information).The narrowly defined indicator includes those respondents whose score on the knowledge scale puts them in the bottom 15-20 percent in the population, but who nevertheless rate themselves as being "pretty well", "well" or "very well" familiar with political matters.
To capture the association between the Dunning-Kruger effect and political news consumption, the analysis relies on the item "How important are the following channels for you personally for receiving information on current social and political issues?",which was followed by a list of various media.In 2008, the social media era was only starting and only "Internet" was included as an option.In 2020, "social media" was added.Consequently, here the measures differ, but nevertheless capture the individual's reliance on the Internet or social media for political news versus reliance on traditional news media.This difference has been firmly linked to political learning and sophistication (Shehata and Strömbäck 2021;Van Erkel and Van Aelst 2021;Yamamoto and Yang 2022).Age, gender and education are included as controls in all analyses and post-survey weights are applied to both data to ensure representativeness of the voting-age population in terms of age, gender and education.

Results
Figure 1 demonstrates an increase in broadly defined overconfidence in political sophistication from 2008 to 2020.The difference of 3.3 percentage points is statistically significant at < .001-level. 1 The level of narrowly defined overconfidence remains essentially unchanged between the two measurements.
Figures 2 and 3 show the probabilities for broad and narrow overconfidence, respectively, for the different levels of internet / social media news reliance and for the two different years, when all controls are set to their means.
In both cases, the probability of overconfidence increases with internet / social media usage and the probabilities are consistently higher in 2020 than in 2008 (Tables A4 and A6 in the Appendix).The marginal probability of broadly defined overconfidence for people for who internet or social media was very important as a source for political news was 16 percent in  2008, while it was 29 percent in 2020.For the narrowly defined overconfidence, the corresponding increase was from 10 percent in 2008 to 12 percent in 2020, suggesting that only the broad definition was able to capture significant change in overconfidence.Although importance of internet / social media for political news is a statistically significant predictor of overconfidence in all analyses, its interaction term with year is not.It therefore seems although the probabilities for overconfidence are significantly higher for 2020 than for 2008 for heavy internet / social media reliance, it is unclear whether there is genuine over-time movement.
As a robustness check, the same analyses were run for newspaper readership instead of internet/social media.This allows assessing whether it is plausible that the increase in overconfidence could indeed be associated with internet/social media usage or whether it is equally likely for users of traditional media.The results with newspaper readership do not show the same patterns as observed above and all coefficients for year and newspaper use are statistically insignificant (Figures 1A and 1B in Appendix).This suggests increased overconfidence is associated with internet/social media usage, but not with newspaper readership.

Discussion
The findings demonstrate that, depending on the measure, overconfidence among low-sophistication individuals is slightly more widespread in 2020 compared to 2008.The most robust finding is that this tendency is associated with a high reliance on online and social media political news.This aligns with Yamamoto and Yang (2022) who report that social media use for political news is associated with overconfidence, while traditional media use is not.The finding is consistent also with the recent and growing literature suggesting that social media does not contribute to the learning of political facts (see e.g.Beckers et al. 2021;Shehata and Strömbäck 2021;Van Erkel and Van Aelst 2021;Amsalem and Zoizner 2023).This literature is starting to form a coherent picture of the more fundamental question regarding the impact of social media on political sophisticationand perhaps on representative democracy itself.Scholars persistently find that political learning from social media compared to traditional media is minimal at best.The current study adds to the burden by suggesting that particularly people who are not very politically competent are especially vulnerable to misestimating their own ability to understand politics in the contemporary online news milieu.
The strength of the current analysis lies in the comparability of the data across time.The identical measures and survey samples make it possible to observe the prevalence of the Dunning-Kruger effect in 2008 and 2020.The cross-sectional data do not allow assessing individual-level causalities, but they provide important circumstantial evidence suggesting that internet and social media reliance may in recent years have increased the Dunning-Kruger effect.
The evidence is not, however, certain.The increase was sizable and statistically significant at the aggregate-level, but at the individual-level the interaction term between online news reliance and year of observation was statistically non-significant.Nevertheless, there is a relationship between online news consumption and the broad operationalization of overconfidence, which included individuals who do not exclusively represent the most inattentive segment of the population, but the slightly more attentive one.The finding that reliance on online political news affects the self-appraisals of the modestly politically aware, rather than the apoliticals, is compatible with Zaller's (1992) model combining political awareness and opinion formation.As argued by Zaller, it seems that those who are completely politically unaware, are largely unreachable to political communications.They score low in political knowledge tests and probably do not mind much.The highly attentive also have highest exposure to political messages, but their level of sophistication helps them make sense of it and to make realistic assumptions about their own knowledgeability.But those who pay some, but not much, attention to politics, are most vulnerable.They get some exposure to political news but lack the ability to self-evaluate realistically the extent to which that exposure contributes to their political learning.
Overall, the findings imply that the contemporary social media may create an environment where low-sophistication individuals become more politically self-assured, but without a corresponding increase in actual sophistication.This invites the speculation that perhaps the logic of social media provides a platform where one can get confirmation for one's personal beliefs, which enhances political self-confidence regardless of actual sophistication levels.
The repeated cross-sectional data does not make it possible to deterwhether it is the online environment that is driving overconfidence or if individuals who are, for any reason, more overconfident to begin with or more prone to overconfident self-evaluations, are also more likely to prefer the online environment for political news.Based on this analysis, it is only possible to talk about an association between overconfidence in political sophistication and online news consumption.Secondly, the possible over-time increase is also a question mark.To the extent that existing evidence allows it, subsequent research should try to examine the direction of the developmenthow is the online news environment changing political sophistication?This analysis provides important circumstantial evidence, suggesting that it may be contributing to an increased and false sense of political competence among low-sophisticated individuals, but this is inconclusive.Future studies should also look into partisanship as a possible factor, which affects overconfidence that is attributable to media consumption habits (for partisanship and overconfidence, see Anson 2018).