Evaluating the Impact of Educational Videos on Vaccine Science Knowledge Among Virginia High School Students

ABSTRACT Background Despite the overwhelming benefits of vaccinations, suboptimal public confidence in vaccines keeps immunization rates in the U.S far from ideal levels. Purpose To assess the efficacy of a high school video set curriculum, on vaccine history, science, and safety, in improving vaccine knowledge. Methods Sample included high school students in Virginia, who had completed required biology coursework. After being assigned to either the control or experimental group, their knowledge on vaccines was assessed before and after viewing the curriculum videos. Results Experimental group students showed greater knowledge increases after viewing the videos. Examined by individual question, twelve out of eighteen questions in the experimental group had significant improvement in accuracy/knowledge, compared to one out of eighteen among the controls. Discussion Our study demonstrates the value of using videos to convey vaccine-related knowledge to high schoolers. This knowledge can favorably impact rates of acute infectious disease as well as chronic conditions such as polio, cervical cancer, and long COVID syndrome. Translation to Health Education Practice Such educational videos may be integrated into the state curricula so high school students can understand the impact, safety, and efficacy of vaccines and make informed public health decisions as adults.


Background
Vaccines remain one of the most monumental public health achievements in human history 1,2 and their impact on human health became increasingly evident during the COVID-19 pandemic.Not only does vaccination lower morbidity and mortality in the short term but it also decreases health and socioeconomic disparities for the disadvantaged, as seen with the pneumococcal vaccine in the US and the measles vaccine in Bangladesh. 1 The introduction of vaccines during the COVID-19 pandemic averted 1.1 million additional COVID-19 deaths and more than 10.3 million additional COVID-19 hospitalizations in the U.S. by November 2021. 3 While the prevention and eradication of childhood communicable diseases are an evident benefit of vaccines, it is important to acknowledge that there has been a similar decrease in incidence of chronic diseases and morbidity from such diseases.For example, liver cirrhosis and cancer can be prevented by hepatitis B vaccination just as more than 95% of cervical cancer is preventable with HPV vaccination. 4,5Similarly, the lifetime neurologic consequences of measles or mumps can have a much more detrimental impact than the acute disease. 6,7Finally, up to 40% of Haemophilus Influenzae meningitis survivors may have lifetime neurological deficits. 8Thus, educating adolescents about vaccine science, during a most impressionable age, allows them to become educated consumers of vaccines as adults and parents thus decreasing the incidence childhood diseases as well as their chronic disease repercussions across the lifespan.
Beyond childhood, adults can also benefit from immunization if diagnosed with chronic illnesses or if the individual situation merits additional protection from infection, such as pregnancy, premature birth, cancer and other situations marked by immunocompromise. 9,10Also, adults can enhance and maintain immunity by receiving timely boosters of childhood vaccines and preventing seasonal yearly infection such as the flu, COVID, and pneumonia. 11espite the benefits of vaccinations to the public, suboptimal rates of public confidence in vaccines keeps immunization rates in the U.S far from ideal levels.Approximately, 1% to 3% of children are excused from school immunization requirements due to medical, religious, or philosophical exemptions, and in some communities the refusal rate can be as high as 20%. 12Parental concerns regarding vaccines are increasing 13,14 and stem from a variety of reasons including lack of confidence in vaccine safety or effectiveness, falsely low perception of disease risk, and lack of convenience or cultural context. 15ignificant efforts have targeted adults for messaging on vaccine safety and efficacy, as well as the devastating consequences of vaccine preventable illness, yet studies indicate that such messages may be powerless against belief systems that are formed well before adulthood. 16,17Most immunizations are given in childhood, yet the majority of people receive no systematic education that is designed specifically to increase knowledge of the purpose of these immunizations and their risks and benefits.Thus, children continue to rely on their family's perception of vaccines, which in turn may derive from media outlets where substantial content about vaccines contains anti-vaccination messages. 18he widespread use of digital media and smartphones by children provides an excellent opportunity to inform them about the scientific merits of vaccination, including the science behind vaccines as well as their safety and efficacy. 19Educational videos, in particular, can convey complex scientific information and are effective in modifying health behavior. 20Videos have also been shown to improve learning outcomes, 21 yet there is scarce research on the most effective ways to use videos to educate children on health topics.Prior work by this group showed that using a video-based online vaccine education platform resulted in positive shifts in vaccine-related knowledge, beliefs and attitudes among the children who participated. 22This project aims to fill a gap in science education regarding the essential role of immunizations in promoting public health.

Purpose
The purpose of this study was to evaluate a series of educational videos presented to Virginia high school students, who have typically completed a course in Biology, and assess their knowledge of vaccine science before and after viewing the videos with a computer or mobile device with internet capability.

Sample and study design
Study subjects were students in 10th, 11th, and 12th grade who had taken at least introductory Biology in high school.The study was widely advertised using social media, as well printed and electronic flyers.Specifically, we collaborated with the Virginia Junior Academy of Science (VJAS) whose membership includes middle and high schools across Virginia, and the study was publicized to student and teacher audiences in the VJAS quarterly newsletter as well as their social media pages (Instagram) and website.Student members of VJAS were in turn encouraged to share the study with their friends and peers.Promotional material indicated that completion of high school biology was a requirement and that is why the study focused on students in 10th grade and higher since biology is typically completed by students in 9th grade and it has a state mandated learning assessment associated with it.The proposed sample size was 150 students, based on funding available.The study was also conceived as a pilot project to gauge feasibility to recruitment, compliance, and response to such an educational intervention.
Electronic informed consent was obtained from the parent or guardian and assent was obtained from the participant.All e-mail communication with the study subjects also included the consenting parent.Subjects were compensated $60 for their time, and they received this after completing both study questionnaires and watching the videos assigned.The study was approved by the Hampden-Sydney College Institutional Review Board.
Once the student was enrolled in the study, he or she was asked to complete a pre-video questionnaire with a total of 18 items (17 True/False and 1 multiple choice).See below: (1) Polio was eliminated from the U.S. in 1990.
(2) There are two reasons to immunize: (a) To protect the individual directly (b) To protect those who cannot receive vaccines (3) Herd immunity occurs when enough people are made immune through vaccination to protect those who have not been immunized because they are not exposed to the germ.(4) All U.S. states have immunization requirements.(5) Vaccines are extensively tested for efficacy and safety before they are licensed.(6) Pertussis is called "whooping cough because of the sound victims make when trying to inhale.
(7) Pregnant women should not get vaccinated against pertussis in order to protect their newborn infants.(8) Cancer of the cervix is caused by a virus called human papilloma virus.(9) Males do not get cancer from human papilloma virus.(10) There is an effective vaccine to prevent meningococcal disease.(11) Two types of influenza (A and B) cause most diseases in humans.(12) Where do new strains of influenza often first appear each year?(13) Each year the U.S. loses over $90 billion due to Influenza.( 14) A pandemic is caused by a germ spreading all over the world.( 15) More children get sick from influenza each year than do adults.( 16) Immunizing pregnant women against influenza prevents influenza in their newborns.( 17) Measles has been eradicated worldwide.(18) There are effective vaccines to prevent COVID-19.
After submitting the pre-video questionnaire, subjects were randomly assigned to a control group watched educational videos of biology of cells for roughly 60 minutes or experimental group of educational videos about vaccines and received a link to one of the video sets as well as post-video-questionnaire which was identical to the pre-questionnaire.Viewing access of the videos was monitored.A total of 200 students initially enrolled in the study and 169 of them completed the pre-questionnaire.Of the 84 that were assigned to the control and 85 assigned to the experimental, 77 and 76 students completed the post-questionnaire, respectively.
The study period extended from 4/30/2022 to 10/26/ 2022 and took 80-90 minutes to complete the study.Note: One of the student's parents needed to give permission first and then the student would also need to sign a form agreeing to be involved in this study.

Videos
The 14 experimental videos used in this study were developed with expert consultants who have experience in formulating internet-enabled videos and leaders in vaccinology.The material was designed to be accessed at home, thereby minimizing time taken from class.It is intended to be part of the school curriculum (e.g., health, physical education, interdisciplinary programs, biology), and to be discussed in class.It is also postulated and hoped that the reach of the program will be extended to the students' family members.The control videos were a set of open access general science videos (See Appendix A for the videos).
Perceived susceptibility, severity, benefits, and barriers are sufficiently addressed in the videos.They emphasize the pervasive fatal consequences of illnesses such as meningococcal, influenza, COVID-19, HPV, and pertussis that leave unvaccinated individuals extremely vulnerable.Benefits of vaccines through the explanation of their stimulation of the immune system and concept of herd immunity motivates individuals to receive their vaccines.Finally, any concerns of the dangers of vaccines are put to rest through an extensive explanation of the safety mechanisms for vaccine approval and the reliability of credible sources such as pediatricians.

Analyses
Overall test scores were compared using the student t-test and group differences.In terms of the number of students who answered the questions correctly, data were analyzed using Pearson's chi-squared test for count data without the Yates' continuity correction.

Results
A total of 153 students completed the entire study, with 76 students in the control group and 77 in the experimental group (Table 1).While the study was advertised statewide, schools and students affiliated with the VJAS tended to be overrepresented including those in the northern Virginia region and those in urban settings.The control group had equal numbers of male and female students, while the experimental group had 7 more females than males.Most participants were Asian at 37.91%, followed by Caucasian (22.88%),African American (20.26%),Hispanic (15.69%) and the rest preferred not to say (3.27%).Approximately, a third of the participants were eligible for free or reduced-price lunch.
While the pretest scores in the control and experimental groups were extremely similar, the post-test scores for the two groups were notably different (p value 5.81 e-05 assuming equal variances) with students who watched the vaccine videos scoring much higher than those who did not watch the videos.
Table 2 delineates a detailed comparison of the two groups with regards pre-and post-test accuracy for each question.Among students who did not watch the vaccine videos, accuracy on only question number 12, which asks about the region of origin of influenza strains each year, significantly improved from the pretest to the post-test questionnaire.This was also the only non-True/False question and instead was in multiple choice format.For the experimental group, 12 out of 18 questions had significant improvement in accuracy when comparing the pre versus post-video questionnaires.

Discussion
This study evaluated the efficacy of online vaccine education videos which are meant to increase high school students' knowledge on vaccine history, science, and safety.This intent of the study was to help young people learn more about vaccines: what they are, what they do, and how they can help prevent diseases.
Students scored significantly higher on a vaccine science questionnaire after they watched a series of 14 assigned educational videos.The control group demonstrated increased knowledge as well, evidenced by an increased mean overall score.However, on an individual question basis only one question showed significant improvement among controls who did not watch the vaccine videos.
Although both the control and experimental groups increased their overall vaccine knowledge after watching the videos, the experimental group's mean difference in score was double that of the control group, rendering it more effective.The improvement in overall test scores in the control group was unexpected and could be due to various reasons.Students may have received education on vaccines through means external to the study during their participation, the pre-questionnaire spurred their curiosity and vaccine knowledge seeking behavior, or general science knowledge was helpful from the control video.
Within the experimental group the following questions did not show an increase in knowledge, based on pre and post-test scores: 1, 2, 5, 6, 9, 14, 15, 17, and 18.However, questions 6, 9, and 17 were approaching significance.The lack of improvement for the first question (Polio was eliminated from the U.S. in 1990) in the experimental group demonstrates a weakness in the curriculum and a need for greater focus on Polio.Questions 2, 5, 14, 15, and 18 showed little to no improvement in the experimental group most likely due to the high pre-video knowledge (>87%).It would be unreasonable, and in some cases, statistically impossible to have a significant improvement in knowledge.These results can inform future efforts to fine tune vaccine-related teachings based on established knowledge of high school students.
The most missed question in both groups was number 13 (Each year the U.S. loses over $90 billion due to Influenza -T/F), indicating a lack of knowledge on the severity of the financial repercussions of health-related expenses in the United States.On the other hand, question 18 (There are effective vaccines to prevent COVID-19.)had the highest correct response rate, indicating that students are aware of current events and their environments.
A prior study, evaluating an earlier version of the education intervention, conducted among a small group of middle school students in Virginia and Maryland, showed that children in the intervention group had significantly higher increases in knowledge compared to the control group. 22In addition, there was a significantly higher shift toward positive attitudes toward vaccines in the intervention group.But this study was small (only 17 students in the intervention group and 22 in the comparison group) compared to the 153 students participating in this study (76 in the control group and 77 in the intervention group).And the focus of the prior study was in middle school students in contrast to the current study which focused on high school students.But both studies support the general conclusion that the vaccine education project contributed significantly in improving knowledge and attitudes about vaccines.
The success of our video-based vaccine curriculum is consistent with prior literature that has examined the efficacy of technologies designed to enhance vaccine uptake.Digital technologies, including videos, have been shown to be effective in combating vaccine misinformation and can alleviate vaccine hesitancy, particularly when the digital intervention targets the specific needs and characteristics of a given population of interest. 11For example, vaccine videos aimed at parents caused significant reduction in vaccine hesitancy in Indonesia. 23When systematically interviewed, college students reported educational videos as their preferred source of health education on HPV and the HPV vaccine. 24he immediate and potential long-term educational and behavioral impacts of our vaccine videos could be due to the promising workings of eHealth and the health belief model.Research shows that low health literacy leads to poor health status. 25A systematic review of eHealth interventions, by Jacobs and colleagues, showed that the technology-based interventions had a significant impact for future positive outcomes regarding health literacy in a variety of settings, and for different diseases. 26eHealth technologies appear especially promising to encourage immunizations and increase vaccination rates. 27However, most studies have evaluated videos and other technology-based educational programs for adults/parents 28 as the audience and there is scarce data on increasing vaccine-related knowledge among high school students by using videos.Meanwhile research shows that when high school students use the internet to search for vaccine information, a high percentage of them take away misconceptions and inaccurate information, 29 underscoring the necessity for education through this curriculum.Yet children remain an understudied population with regard to vaccine science education and data on interventions to address vaccine hesitancy in this age group is extremely scarce.
The success of our intervention can be traced to its roots in the health belief model, a fundamental public health framework used to identify factors that influence a person's health behavior.These factors include modifying variables, perceived susceptibility, severity, benefits, barriers, cues to action, and self-efficacy. 30umerous studies have demonstrated a significant relationship between vaccine acceptability and its predictors based on the health belief model. 31In a study on COVID-19 vaccine uptake predictors, the perceived risks and benefits were assessed.People most commonly took the vaccine due to its effectiveness and ability to protect.On the other hand, it was most refused due to concerns regarding safety, effectiveness, and novelty. 32y underpinning this model in our intervention, we hope to change behavior and increase vaccine uptake.
Students' increased knowledge of vaccines has tremendous potential to create behavioral change with a long-lasting impact well into their adult lives.Extensive literature exists regarding the efficacy of educational interventions, geared toward children, that promote physical activity, 32 healthy nutrition, 33 obesity prevention, 34 and safer sex, 35 among other things.Most importantly, many of these interventions occur in school-based settings allowing for dissemination on a large scale.Finally, educating children also holds the potential for them to educate their parents and this has been described with regard to smoking cessation, 36 and sanitation and hygiene. 37

Strengths and limitations
The data collection performed here were performed with the help and support of the Virginia Academy of Science and Virginia Junior Academy of Science.As such, we were able to use the extensive contacts of these organizations to advertise our study to high school students from across the Commonwealth of Virginia and not just in specific schools.This enabled us to collect a dataset representing a diverse group of students with respect to race, socioeconomic background, and other factors that may contribute to the ways in which individuals perceive the scientific and medical communities.
Our study had some major limitations.While the videos adequately address several constructs of the Health Belief Model, the evaluation instrument directly assesses only knowledge, while some of the questions may also assess attitudes.This is a limitation of the current study and future studies with expanded cohorts should include all aspects of the HBM.Our intent with this initial study, and its evaluation instrument, was to assess the feasibility of recruiting students and having them successfully complete the pre-and postquestionnaires.We felt that knowledge assessment would most directly test whether or not the students had watched the videos.
While our study was effective in attracting a diverse group of students, it is also true that the personal connections of the authors within particular school districts resulted in the overrepresentation of students from certain parts of Virginia over others in our data pool.While we remain confident in our results and are pleased with the diversity of the student pool that we attracted, a follow-up study would benefit from both a larger pool of students and a stronger effort to ensure that all regions of the Commonwealth are equally represented.
The Virginia Department of Education was given $1 million in 2021 to study virus impact on learning.What if that money was spent on teaching about the impact of viruses so it will not impact the future.The Commonwealth continues to offer school health guidance, resources, and required training yet we still are reactive instead of proactive.
Prompted by the deficiency of vaccine education in Virginia's high school curriculum and benefits of childhood health literacy, this study was designed to address the gaps in basic knowledge of vaccine preventable communicable diseases.Educational videos have the potential to reduce vaccine hesitancy and can be used as health education tools among adolescents.In line with the rationale of the health belief model, educating students can lead to long-term benefits in their health behavior and improve public health worldwide, for future generations.

Translation to Health Education Practice
Although Health Education is traditionally considered the domain of community and public health professionals, physicians and allied health professionals, teachers, and educators may have an important role to play when it comes to educating special populations such as adolescents on their health behaviors.Advances in technology and access to mobile phones has created unprecedented opportunities for health behavior education and modification.To that end, our project which created and evaluated video-based curricula on vaccine science, for high school students, is a crucial public health initiative.Stakeholders and partners ranging from immunologists, pediatricians and science educators were involved and included in the design, planning, and implementation phases of this entire study.This plan aligns well with a number of priority areas of the National Commission for Health Education Credentialing (NCHEC) Inc. 38 including Area I: 1.1.2,1.1.3,1.1.5,Area II: 2.2.1, 2.1.2,and 2.1.3).
By focusing our efforts in Virginia, we have potentially laid the foundation for exciting future work that could make vaccination a more formal topic of discussion within the Commonwealth's STEM curriculum.Currently, there is no known curriculum in Virginia for adolescent education on vaccine-preventable disease and the benefits of vaccines.The current Virginia Standards of Learning have an entire objective dedicated to bacteria and viruses but only a cursory reference to vaccines or immunizations as proven means to protect us against certain communicable diseases. 39Thus, our research aligns well with Area V or Advocacy as outlined in the NCHEC 5.1 and 5.2.
The results shown here, combined with our continued collaboration with the Virginia Academy and Junior Academy of Science, provides a pathway for us to work with officials in the state's Department of Education to devise and implement an adolescent-level enhanced curriculum on the science behind vaccination and public health.Our research shows that such interventions, embedded within the science curricula, can be implemented by public and private educational settings across the United States thus aligning with NCHEC competencies Area IV: 1.1.9and 4.1.10.As well as 4.2.8 and 4.2.9.School nurses may be an underutilized resource for such initiatives and must be included in the planning and implementation of such programs.
While schools and science educators can be an invaluable resource in the dissemination of such digital interventions, they can just as easily be employed in health care settings like pediatricians' offices and other ambulatory healthcare settings.

Table 2 .
Differences in vaccine knowledge amongst students.