In-school And/or Out-of-school Computer Science Learning Influence on CS Career Interests, Mediated By Having Role-models

Introduction Computer science (CS) is special among STEM subjects: it aims at an industry sector that has the most job growth but has a constant shortage in the workforce; it is a relatively young and burgeoning subject in K-12 education that has a shortage of classroom teachers; and it is one of a very few STEM subjects that large number of students can master by learning it completely out-of-school. To inspire the future generation to pursue the unfilled high-paying jobs in computing, and to fulfill a nation’s interest in innovation and technology, various groups of stakeholders called for both the expansion of in-school offering of CS courses and the investment in out-of-school CS experience opportunities. There is an unsettled debate, however, about the impact of in-school and out-of-school experience, and the comparison between the two. Most research that engaged in this discussion was limited by sample sizes or sampling biases. In this study, we sampled a national representative sample of 4,107 grade 5-12 students in the U.S. to investigate the effect of in-school, out-of-school or both two CS learning experiences on students’ CS career interests. Adopting an expectancy-value theory (EVT) framework, we are particularly interested in how role-model mediates this dynamic and if this dynamic is different for different gender or race/ethnicity.


Introduction
Computer science (CS) is special among STEM subjects: it aims at an industry sector that has the most job growth but has a constant shortage in the workforce; it is a relatively young and burgeoning subject in K-12 education that has a shortage of classroom teachers; and it is one of a very few STEM subjects that large number of students can master by learning it completely out-of-school.To inspire the future generation to pursue the unfilled high-paying jobs in computing, and to fulfill a nation's interest in innovation and technology, various groups of stakeholders called for both the expansion of in-school offering of CS courses and the investment in out-of-school CS experience opportunities.There is an unsettled debate, however, about the impact of in-school and out-of-school experience, and the comparison between the two.Most research that engaged in this discussion was limited by sample sizes or sampling biases.In this study, we sampled a national representative sample of 4,107 grade 5-12 students in the U.S. to investigate the effect of in-school, out-of-school or both two CS learning experiences on students' CS career interests.Adopting an expectancy-value theory (EVT) framework, we are particularly interested in how role-model mediates this dynamic and if this dynamic is different for different gender or race/ethnicity.

Literature
Theoretical framework.EVT has three key components: 1) the expected chance of success, 2) the task value for personal goals, and 3) the expected cost to achieve success.A person tends to pursue and persist in a task when the anticipated affordance of the task is aligned with personal values, and when the value and chances of success outweigh the estimated cost.The abovementioned factors are not fixed, they adjust with one's experience, which may come from within or outside of schools.
Learning CS in-school.Some scholars had concerns that in-school CS courses, downplaying the technical and mathematical nature of CS, might give students an over-simplified impression of a CS career (Beaubouef & Mason, 2005).The counter argument posits that in-school CS-although surveying the surface-offers a roadmap to the field and many chances for self-assessment.Such offerings not only help one to assess the expected chance for success and/or obstacles, but also boost one's confidence in achieving success (Armoni and Gal-Ezer, 2014).Moreover, the foundational courses in CS, such as AP CS Principles, connect CS with the society and students' lives, which may help students to decide if CS careers are aligned with their personal values and goals.
Learning CS out-of-school.CS profession is one of a kind STEM occupation in which a large number of the professionals is trained out-of-school (Stackoverflow.com, 2021).Scholars once worried that students learning coding out-of-school in a cowboy/cowgirl hacking style will end up consolidating bad programming habits (Kölling, 1999, p4).However, the out-ofschool offering of "geek out and mess around" may be beneficial in the long-run (Ito et al., 2009;Liggett, 2014), because when learners are situated in real-world challenges, they find it easy to decide if the task is manageable (in terms of difficulty) and relevant (in terms of personal goals).In addition, to mess around may reduce the perceived cost of errors or debugging.
Role models.The presence of a role-model helps students to assess if "people like me" can succeed, and if "people who share my values" can actualize their goals, in the professions of interests.Schoolteachers can usually serve as role-models: they give students career advice, and students see them as in-group persons (Lockwood, 2006).Alternatively, teachers can expose students to more CS role models by inviting guest speakers, arranging field trips, or watching movies (or reading biographies) about the great figures in the history of computing (Twarek, 2018).Arguably, all these approaches borrow some forms of out-of-school learning.

Minoritized groups in CS.
Women and racial/ethnic minorities are underrepresented in CS course enrollment or in AP CS exams (Code.org,2021).Ashcraft, et al. (2012) attributed it to the abstract schoolwork that focuses on independence and "innate" talent but downplay community and collaborations, which may disproportionately marginalize women and/or Black/Latinx students (Diekman et al., 2017;Tellhed et al., 2018).Students who have access to in-school CS classes may live in resourceful school districts, and students who attend out-of-school programs may have strong parental support.Seldom any research teased out these entangled confounding factors when considering the influencing of learning experiences on CS career interests.

Research question
Formally, we ask: RQ1: 1) What are the effects of in-school, out-of-school and both two CS learning experiences on grade 5-12 student's CS career interests, after we can balance on students' background variables?2) Do the effects of learning experiences vary by students' gender or race/ethnicity?RQ2: 1) What are the effects of learning experiences on students' likelihood of having CS role model(s)?2) Do these effects vary by student's gender or race/ethnicity?RQ3: What proportion of the effect of the effects of the learning experiences on CS career interests mediated by the students' having of role model(s)?

Methods
Sample and data collection.In June 2021, Gallop surveyed an U.S. national representative sample of 4,107 grade 5-12 students and their parents (sampling procedures will be explained in detail in my talk).The survey asked the parents about their family information (income, education, IT related profession, number of children, residence area, type of school, and access to computer devices or internet, etc., listed in Table 1), and asked children about their CS learning experience (4 categories: no-experience, learned CS in-school, learned CS outof-school, or learned CS both in and out-of-school), if they have CS role models (yes/no), and if they are interested in having CS/IT related careers (yes/no).
Analysis.We adopted the multinomial propensity score weighting technique that reduces the self-selection bias by balancing on the pre-treatment variables between multiple groups.In our analysis, the pre-treatment covariates were successfully balanced (not reported in this proposal due to space limit).To estimate the proportions of the treatment effects that were mediated by having role models, we follow the procedure for causal mediation analysis introduced by Imai, et al. (2010).The traditional path analysis approach is challenged by its violation of a series of assumptions that couple with the application of propensity score weighting (see explanation in detail from Jo, et al. (2011)), therefore, it was not adopted in our analysis.

CS career interests.
Table 1 showed the coefficients of the logistic regression that predict CS career interests as a function of CS learning experience.In M1.1, no weighting was applied.M1.2 applied multinomial propensity score weighting, and M1.3 applied the weighting while controlling for the covariates.Due to word limit, let's focus on M1.3 here.
----Table 1----The estimated effects of learning experiences in M1.3 were positive and significant, although smaller than the estimated effects from M1.1.Compared to none group, out-of-school group's odds in reporting CS career interest increased by 94% (β = 0.663, se = 0.055, p < 0.001, OR = 1.940), in-school group's odds increased by 40% (β = 0.339, se = 0.056, p < 0.001, OR = 1.403), and the both group's odds increased by 171% (β = 0.997, se = 0.054, p < 0.001, OR = 2.710).The out-of-school group's odds was 1.34 times of the odds of the inschool group (1.940/1.403= 1.34).Interestingly, the effect of both was nearly equal to the sum of out-of-school and in-school (on the logit scale), indicating that the effects of out-ofschool and in-school were additive to each other.Further, in M1.4,we found significant and positive interaction effects between learning experiences and female (vs.male), meaning the effects of learning experiences were stronger for female than for male students.For male students, out-of-school experience increased the odds of CS career interests by 72% compared to none (β = 0.541, se = 0.067, p < 0.001, OR = 1.718), in-school experience increased the odds by 16% (β = 0.149, se = 0.069, p < 0.05, OR = 1.160), and both experiences increased the odds by 145% (β = 0.897, se = 0.066, p < 0.001, OR = 2.451); post-hoc test showed that all of these three effects were significantly significant from each other.For female students, out-of-school experience increased the odds of CS career interests by 151% (exp(0.541+0.379)= 2.509), in-school experience increased the odds by 103% (exp(0.149+0.561)= 2.034), and both experiences increased the odds by 238% (exp(0.897+0.321)= 3.380); post-hoc test showed that all of the three effects were significantly different from each.We showed the interaction effects (with 95% confidence interval) in Figure 1 after calculating the predicted probability (converting from logit to the probability scale) of reporting CS career interests by learning experiences ✕ gender, after fixing other covariates at their means.We did not find an interaction effect between learning experience and race.

CS role model.
Table 2 showed the results of the logistic regressions that predicted students' reporting of having CS role models as a function of their CS learning experiences.M2.1 did not apply for weighting.M2.2 and M2.3 applied multinomial propensity score weighting.Focusing on M2.3, we found that out-of-school experience increase the odds of having CS role models by 462% (β = 1.726, se = 0.054, p < 0.001, OR = 5.621), and in-school increased the odds by 131% (β = 0.839, se = 0.051, p < 0.001, OR = 2.314), and experience both increased the odds by 410% (β = 1.630, se = 0.053, p < 0.001, OR = 5.103), compared to students who had none of the experiences.Out-of-school group had 2.429 times the odds of the in-school group (5.621/2.314= 2.429), and out-of-school was not significantly different from both.
We did not find significant interaction effects between learning experiences and gender or race.

Discussion
Summary.RQ1): For the effect on CS career interest: both > out-of-school > in-school > none.Female students had the lowest career interest, if without any CS learning experiences, but female students had stronger boost than male from their learning experiences, especially if that experience came from in-school.RQ2): For the effect on having CS role models: both = out-of-school > in-school > none.No significant interaction effect with gender or race/ethnicity.RQ3): Slightly more than half of the effect of out-of-school experience on CS career interest were mediated by having role models.Likewise, slightly less than half of the effect of inschool, and about one third of the effect of both, were mediated.

Limitation.
A major limitation (among others that were omitted due to word limit) of our study is not knowing what in and out-of-school learning encompass.Nevertheless, this is the first nationally represented sample study that showed that, even after we minimize the selfselection bias, we can still find the effect of in and out-of-school learning, whereas out-ofschool has stronger effect.This finding should motivate researchers to investigate the effective components and dynamics that make out-of-school CS special.

Implications (in brief).
When students learn CS out-of-school, such as online or in a bootcamp, they have lots of opportunities to interact with and learn from CS professionals.Many of the bootcamps were organized or supported by IT companies; most respondents to coding questions posted online are advanced programmers.As students have more exposure to the professionals in the industry (this can be done in-school but requires teachers to reach out to out-of-school resources), they may identify role models who can share insider perspectives, which may reaffirm the positive prospect of having a CS career.

Table 2 .
Logistic regreession models predicting CS role model as a function of CS learning experience .M2.1 did not apply any weighting; M2.2 and M2.3 applied multinomial propensity score weighting.
Figure 1.probability of CS career interests by learning experiences ✕ gender.