Do students with different personality traits demonstrate different physiological signals in video-based learning?

Abstract Past researches show that personality trait is a strong predictor for one’s academic performance. Today, mature and verified marker systems for assessing personality traits already exist. However, marker systems-based assessing methods have their own limitations. For example, dishonest responses cannot be avoided. In this research, the goal is to develop a method that can overcome the limitations. The proposed method will rely on physiological signals for the assessment. Thirty participants have participated in this experiment. Based on the statistical results, we found that there are correlations between students’ personality traits and their physiological signal change when learning via videos. Specifically, we found that participants’ degree of extraversion, agreeableness, conscientiousness, and openness to experiences are correlated with the variance of heart rates, the variance of GSR values, and the skewness of voice frequencies, etc.


The background of using personality traits to help students
Why do we need a method to assess students' personality traits? Existing literature has highlighted the relationship between personality traits and learning style or performance. For example, the research of Komarraju et al. pointed out that the Big Five (a popular model for the assessment of personality traits) together explained 14% of the variance in GPA while learning styles explained an additional 3% (Komarraju et al., 2011). Knowing students' personality traits in advance may help teachers assist students when they run into trouble in learning. Furthermore, utilizing personality traits as information can make it easier to adapt to students' preferred learning styles and may achieve better learning performance in turn. Among all learning mechanisms, we would like to limit the scope of this research to video-based learning. The spread of Covid-19, which is perhaps the most influential event in recent years, makes most college students in Taiwan experience video-based learning since mid-2021. In such a scenario, it is more difficult for lecturers to interact with students and hence figuring out a more efficient way to understand students becomes a more urgent need.
Learning via videos is not a completely new concept. Furthermore, since students learning via videos can learn at their own pace to some degree, it brings chances to researchers to understand students' learning behaviors. For example, the research results of Brinton et al. showed that for courses using the MOOC (Massive Open Online Courses) method, students' video-watching behaviors can be a good clue for figuring out when they need help from instructors (Brinton et al., 2016).
(1) Foshee, Elliott, and Atkinson pointed out that adopting an adaptive learning mechanism has significant positive effects on students' learning and academic competence (Foshee et al., 2016). In most scenarios, an e-learning environment is needed to adopt adaptive learning methods. Kerr defined adaptive learning as " . . . a way of delivering learning materials online, in which the learner's interaction with previous content determines . . . (Kerr, 2016)." As pointed out by Brinton et al., students' video-watching behaviors can be a good clue for figuring out when they need help from instructors (Brinton et al., 2016). The behaviors being analyzed in the research of Brinton et al. were play, pause, skip back, skip forward, rate-change fast, rate-change slow, and rate-change default.
According to Duff et al., learners with different personality traits tend to use different learning strategies (Duff et al., 2004). As a result, if we can assess students' personality traits via data recorded in video-based learning, we will have more information to adapt to students' needs.

Limitations of existing methods
Certainly, there were already mechanisms developed for personality traits assessment. A marker system which is a transparent format to quantify one's personality traits in terms of the degree of Extraversion, Agreeableness, Conscientiousness, Emotional stability, and Imagination was developed by Goldberg in 1992(Goldberg, 1992. However, using questionnaires does have some drawbacks. Perhaps, dishonest responding is one of the most severe problems (Zettler et al., 2015). Besides, filling and interpreting questionnaire results takes additional time. These issues limit the usage of existing methods.

The contribution and summarization of the proposed method
In this research, the goal is to develop a method to assess personality traits without questionnaires and marker systems. The design has several advantages: (1) it uses only physiological signals and thus does not have strong dependency on e-learning systems (2) there is no need to spend time filling questionnaires and doing analysis (3) there is a chance that physiological signals can reflect psychological status of human beings in a real-time manner (4) knowing students' personality traits helps to adapt to their needs as well as adjusting teaching pedagogies (5) the chance of dishonest responding is reduced In addition to the statistical results, we also developed a device prototype, a software for collecting the physiological signals, and a mathematics model to present the correlation between physiological signals and personality traits. The rest of this manuscript is summarized as the following: (1) section 2: summary of related works (2) section 3: explain the design of the experiment (3) section 4: describe the analysis process and results of the collected data (4) section 5: conclusion and future work

Related works
The goal of this research is to explore the relationship between personality traits and physiological signals and to figure out whether it is possible to assess students' personality traits via physiological signals. In this section, the survey of existing literature of personality traits, the connection between physiological signals and psychological status, and the relationship between personality traits and learning styles are presented.

Personality traits
The research of Goldberg proposed the Big Five factor structure (Goldberg, 1990). Later, a marker system which is a transparent format to quantify one's personality traits in terms of the degree of Extraversion, Agreeableness, Conscientiousness, Emotional stability, and Imagination was also developed (Goldberg, 1992). The Big Five factors has been adopted in many researches. The research of Barrick and Mount found that the Big Five dimensions are highly related with one's job performance (Barrick & Mount, 1991). Rothmann and Coetzer also pointed out that that Emotional Stability, Extraversion, Openness to Experience and Conscientiousness were related to task performance and creativity (Rothmann & Coetzer, 2003

Physiological signal and psychological status
Existing research results showed that physiological signals can be used as indicators for some psychological status. The research of Villarejo, Zapirain, and Zorrilla built a stress sensor based on Galvanic Skin Response (GSR; Villarejo et al., 2012). Lykken used GSR to build a lie detector (Lykken, 1959). The research reported a very high correction rate. Hamadicharef proposed a method to measure one's attention level from her/his electroencephalogram recordings (Hamadicharef et al., 2009). The research reported a classification accuracy rate of 89.4%. The research of Guo et al. used principal component analysis and SVM to classify emotion states based on heart rate variability and reached the accuracy of 71.4% (Guo et al., 2016). The research of Sriramprakash, Prasanna, and Murthy successfully developed a model to assess individual's degree of stress with electrocardiogram and GSR (Sriramprakash et al., 2017). Egger, Ley, and Hanke conducted a survey and found that physiological signals achieved 79.3% accuracy to assess individuals' emotion state and speech recognition achieved 80.46% for happiness and sadness specifically (Egger et al., 2019). Khan et al. proposed a novel deep neural network (DNN) architecture based on the fusion of raw heartbeat and breathing data for classifying and visualizing various emotion states and achieved 71.67% accuracy (Khan et al., 2021). Bastos developed several models for assessing personality traits via physiological signals. The signals used in the research were pupil, ECG, BVP (Blood Volume Pulse), and EDA (GSR) and there were totally 473 features extracted from the signals (Bastos, 2019).
Based on the survey of existing literature, it appears that the connection between physiological signals and psychological status has been established. Moreover, as pointed out by Anglim et al., one's personality traits and psychological status are correlated (Anglim et al., 2020), which indirectly shows that there is a possibility to figure out the relationship between one's personality traits and physiological signal change during some events. The research of Kocjan, Kavčič, & Avsec proposed very similar results (Kocjan et al., 2021). Besides, do we need medical-grade devices to achieve comparable results? According to Kyriakou et al., wearable devices were capable of detecting stress with 84% accuracy (Kyriakou et al., 2019), which suggests that we do not need medical-grade devices during the experiments. Komarraju et al. pointed out that the Big Five together explained 14% of the variance in GPA while learning styles explained an additional 3% (Komarraju et al., 2011). Köseoglu reported very similar results, but the ratio grew to 17% and 5% respectively (Köseoglu, 2016). Both researches showed that personality traits do explain one's academic achievements to a certain degree. Furthermore, Duff et al. found that people with a higher degree of extraversion and imagination tend to use the deep learning approach; people with a higher degree of neuroticism and agreeableness tend to use the surface learning approach; people with a higher degree of extraversion and conscientiousness and a lower degree of neuroticism tend to use the strategic learning approach (Duff et al., 2004). Although using a different personality model, the research results of Moreira et al. showed almost identical results. They found that students with a steady temperament or high character coherence tend to use the deep learning approach (P. A. Moreira et al., 2021). Moreira, Pedras, and Pombo conducted a survey and found that personality is the strongest predictor of academic performance (P. Moreira et al., 2020).

Experiment design
The goal of this research is to figure out the relationship between personality traits and changes in physiological signals in video-based learning. The hypothesis below was proposed: There is a relationship between participants' personality traits and changes in physiological signals in video-based learning.
If the relationship exists, we can use physiological signals to assess participants' personality traits. In the sections below, the experiment process to verify the hypothesis will be presented.

Collection of personality traits
In this research, the IPIP big five factor markers questionnaire 1 was adopted for the analysis of participants' personality traits as the ground truth. The questionnaire has 50 questions and each question has five possible answers: very inaccurate, moderately inaccurate, neither accurate nor inaccurate, moderately accurate, or very accurate. The questionnaire evaluates the scores of the five personality traits: extraversion, agreeableness, conscientiousness, emotional stability, and openness to experiences (imagination). The higher the score a participant received in a personality trait, the stronger the tendency the participant demonstrated in the personality trait. For convenience, we created a Web-based questionnaire application 2 to facilitate participants.

Collection of physiological signals
In this research, participants' voice frequencies and the physiological signals including hear rate values and GSR values were collected as features. We have built a device based on a Raspberry Pi for collecting the needed data. Figure 1 below demonstrates the prototype device:

Experiment process
The steps below were included in the experiment: (1) participants were randomly partitioned into two groups (2) participants watched a video; the first group of participants watched a video that introduces fundamental concepts of JavaScript while the second group of participants watched a cartoon (3) after the end of the video, participants were asked to summarize her/his understandings of the concepts included in the video (4) participants were asked to fill in the IPIP big five factor markers questionnaire

Description of variables
To capture the personality traits of participants, the IPIP big five factor markers questionnaire was adopted. Each item was then calculated according to the scoring instructions mentioned in the IPIP website, and the scoring results are numbers indicating the strength of each dimension. On the other hand, participants' voice frequencies, heart rates, and GSR values were also captured for correlation analysis. To simplify and analytical process, we did not use these values directly but used the variance, standard deviation, and skewness of them instead.
To sum up, there were two sets of variables in this research: (1) 9 independent variables:

Statistics results of personality traits and physiological signals
First, we followed the steps indicated by Goldberg to process the collected questionnaires (Goldberg, 1992). The questionnaire has 50 items and was designed to evaluate the tendency of five personality traits. The tendency of a personality trait is stronger if the calculated score is higher. Furthermore, several methods were adopted to collect voice frequency data, GSR values, and heart rates. Specifically, the javax.sound.sampled library was used for collecting the voice frequency data, the Grove GSR sensor 3 was adopted for collecting the GSR values, and the Grove

Figure 1. the device for collecting signals
Ear-clip heart rate sensor was utilized for collecting the heart rate values. 4 The list below shows the sampling rate of each signal: (1) voice frequency: 2 samples per second (2) GSR: 0.3 samples per second (3) heart rates: 1.1 samples per second Then, we grouped each signal by a 30 seconds time interval. For each group, the mean value of data contained in the group was calculated and used as the representative value. After the preprocessing step, each participant had roughly 40 records for each type of signal, since the experiment lasted for roughly 20 minutes. The only exception was the voice frequency data. The voice frequency data was only collected in the last 2 minutes of the experiment, so each participant has only 4 records of the corresponding grouped voice frequency data. Table 1 shows the statistics results:

Data analysis
The pearson correlation and the canonical correlation were adopted to figure out the correlation between physiological signals and personality traits. The results were shown in Table 2: As shown above, medium-level correlation was observed between some pairs of parameters. For example, the correlation coefficient between gsr_var and Extraversion was 0.5, which demonstrates medium-level correlation. The rcorr function defined in the Hmisc package was then utilized to further verify the significance of the correlation coefficients. Table 3 shows significantly correlated pairs: The following significant correlations have been observed: (1) The correlation between the personality factor "Extraversion" and the variance of heart rates is positive and significant; the strength is moderate.
(2) The correlation between the personality factor "Extraversion" and the standard deviation of heart rates is positive and significant; the strength is moderate.
(3) The correlation between the personality factor "Extraversion" and the variance of GSR is positive and significant; the strength is moderate.
(4) The correlation between the personality factor "Extraversion" and the standard deviation of GSR is positive and significant; the strength is moderate. (5) The correlation between the personality factor "Agreeableness" and the variance of GSR is positive and significant; the strength is moderate.
(6) The correlation between the personality factor "Agreeableness" and the standard deviation of GSR is positive and significant; the strength is moderate.
(7) The correlation between the personality factor "Conscientiousness" and the skewness of voice frequencies is positive and significant; the strength is weak.  The correlation between the personality factor "Openness to Experience" and the variance of GSR is positive and significant; the strength is weak.
Four of five personality factors were significantly correlated with at least one physiological signal. Besides, the results show that the variance of GSR alone is significantly correlated with "Extraversion", "Agreeableness", and "Openness.to.Experience".  However, "Emotional Stability" was not significantly correlated with any physiological signal. Additionally, the canonical correlation analysis was also performed. Since there were 9 physiological signals and 5 personality traits, at most five canonical dimensions could exist. Wilks' Lambda test showed dimensions 1-5, 2-5, 3-5, 4-5, and 5-5 were all not significant. The results are shown in Table 4: However, as shown in Table 5, Roy's Test showed that the first dimension was significant: Table 6 shows the standardized coefficients of the first dimension: To assess the importance of features, the distribution of the standardized coefficients was then analyzed. The absolute value of coefficients reflects the strength of the feature, so the quantile position of the absolute coefficients was used. For the x axis, the 75 th percentile was 1.0440 and for the y axis, the 75 th percentile was 0.8489. The absolute value of the standardized coefficients of gsr_sd, fr_var, fr_sd, Emotional.Stability, and Openness.to.Experience are larger than the third quartile, which draws to the conclusion that gsr_sd, fr_var, and fr_sd are significantly correlated with "Emotional Stability" and "Openness to Experience". The result shows that when combining with other personality traits, the relationship between "Emotional Stability" and physiological signals is still observable.
Based on the analysis results, we can say that the hypothesis proposed in section 3 holds, i.e., participants' physiological signals reveal their personality traits. Besides, the functionality of the device we designed is self-contained, which means adopting the proposed method and device in an existing lecturing environment will help lecturers figuring out students' personality traits without interfering with other technologies current deployed.

Effects of different types of materials
Since there were two groups of participants watching different types of videos, it appears there is a need to investigate the impacts of types of videos. T-test was performed to evaluate the difference of physiological signals between the two groups of participants. The results are shown in Table 7: According to the results, the difference of gsr_sk between the two groups is significant, while the differences of other signals are not significant. As a result, it appears reasonable to conclude that the result of this research is not affected by the different types of video materials.

Limitations of the results
The current results have several limitations: (1) The current results focus on figuring out the correlations between personality traits and physiological signals, not prediction models.
(2) In the experiment, participants watched a 20-minutes video, so the results may not be directly applicable in ordinary classrooms.

Conclusion and future work
The major contribution of this research is that we found that participants' physiological signal change during a video-based learning session reflects their personality traits. Specifically, we discovered the following (positive) correlations as shown in Table 8: Past researches have already shown that there are connections between one's personality traits and psychological status. According to Sriramprakash, Prasanna, and Murthy, individual's level of stress can be detected with electrocardiogram and GSR (Sriramprakash et al., 2017). Guo et al. reported that using heart rate variability alone can classify participants' emotion states (Guo et al., 2016). Crifaci et al. pointed out the variance of GSR between the stressed and unstressed groups was significantly different (Crifaci et al., 2013). Furthermore, Vollrath's research proved that people's personality affects how they react to stress (Vollrath, 2001). Compared with existing research results, our findings appear to be reasonably consistent. Moreover, compared with traditional personality traits assessment methods, which usually rely on questionnaires, the proposed method can be more objective and efficient since there is less human intervention.
Our results are partially consistent with Bastos' results (Bastos, 2019). Bastos found that one's degree of extraversion can be assessed by her/his ECG and EDA. ECG is strongly related to heart rates variance, 5 while EDA is identical with GSR. 6 The two research results appear to be similar in assessing one's degree of extraversion. Additionally, both research results showed that one's degree of openness to experience can be assessed by variance in GSR values. Besides, our method used a much smaller set of features (9 v.s. 145 for assessing extraversion).
In the future, the goal is to develop an assessment tool/device that can be used in classroom based on the current result. Knowing students' personality traits brings opportunities to improve their learning performance. The research of Komarraju et al. (Komarraju et al., 2011) and Köseoglu (Köseoglu, 2016) showed the connections between students' personality traits and their academic achievements. With such a tool, teachers may have a clue about the possible weaknesses in students' learning style and then support them before they fail. Furthermore, as pointed out by Gadekallu et al., the metaverse expands the scope of capabilities in social media (Gadekallu et al., 2022). As a result, with the security convern being carefully addressed, how users behave and interact in the metaverse can become another potential source of features for investigating their personality traits.