E-learning evaluation during the COVID-19 pandemic era based on the updated of Delone and McLean information systems success model

Abstract The Covid-19 pandemic makes the government adjust the learning system from face-to-face to e-learning amid restrictions on mass social contact. We develop the Updated Delone and McLean information systems success (D&M ISS) model by incorporating external factors. The newly added constructs include student capability, teacher capability, and social influence. We employed partial least squares structural equation modeling (PLS-SEM) on 427 respondents gathered from online survey questionnaires completed by undergraduate and postgraduate students during the pandemic. The study identifies that user satisfaction depends on improving the system’s quality, quality of information, teacher’s capability, students’ capability, and social impact. The study also finds a reciprocal relation between students’ satisfaction and net academic benefit. The e-learning system should be enhanced by improving the system’s quality and students’ satisfaction, quality of information, quality of service, and student and teacher capability that do not influence the use of e-learning. This paper provides valuable theoretical and practical implications for regulators and researchers to evaluate e-learning success.


Introduction
Covid-19 pandemic happens throughout the world, and it makes the government have to restrict social contact to prevent the virus from spreading. The social restriction makes the close government schools (Ebner et al., 2020;Karasan & Erdogan, 2021;Nariman, 2021) and changes the learning system from face-to-face to remote learning (Dhawan, 2020;Rizun & Strzelecki, 2020). Remote learning is done through a computer, smartphone, and other IT devices containing applications, such as (Raza et al., 2020), Zoom, Google Meet, Edmodo, Blog, Social Media, Webex, Coursera, Microsoft Team, or a specific platform used by the educational institution (Arpaci & Basol, 2020;Sukendro et al., 2020). The change in the learning system is expected to help students in face-to-face learning effectively.
Some studies have been conducted to evaluate e-learning during the pandemic (Ouajdouni et al., 2021;W. W. Cidral et al., 2020;Safsouf et al., 2020;Alam et al., 2021;Al-Fraihat et al., 2020). From the research, we infer that the current e-learning model is irrelevant for education in the pandemic era. At least there are four conditions to consider building an e-learning evaluation model during the pandemic, which is (1) e-learning is obliged for all levels of education (Alam et al., 2021;Lockee, 2021); (2) the lack of consideration for the teachers and students' preparedness within the education policy (IESALC-UNESCO, 2020; Tang et al., 2021), including the coverage of internet network and infrastructure (Lockee, 2021); (3) the sudden application of the policy makes teachers and students have inadequate preparation (Scherer et al., 2021); (4) the teaching and learning process changes the conventional learning model (Ghilay, 2017) or becomes the central teaching and learning model (Alam et al., 2021). It contradicts the previous condition where e-learning is an alternative or complementary to conventional learning. Within the background, the students and teachers should consider the readiness to use the technology of information and include the social factor as the determining factor in the success of e-learning.
In addition, the indicators for the success of e-learning are not only the satisfaction and benefits received but also the indicators that need to be expanded with the decisions of teachers and students to reuse e-learning. The indicator is essential for a country like Indonesia, which plans to continuously use e-learning after the pandemic (Bramasta, 2020). This system evaluation model aligns with the updated DeLone and McLean IS Success (D&M ISS) model (DeLone & McLean, 2016, 2003Petter et al., 2013).
Another factor that can make it easier for students to get the benefits of e-learning is teachers' ability to teach (Theresiawati et al., 2020). Moreover, the attitudes and characteristics of teachers play an essential role in e-learning (Al-Fraihat et al., 2020;Scutelnicu et al., 2019;Stickney et al., 2019;Yawson & Yamoah, 2020). The reason is that teaching through e-learning requires a different method from the traditional approach.
In addition to the teacher's ability, we are following (Romi, 2017) that the student's ability to use information technology must also be considered in explaining the success of e-learning. E-learning is a learning system that uses information systems. The ability of students to operate information systems will affect them in capturing usefulness and ease of use (Martins & Kellermanns, 2004) and will further affect the ease with which students take part in e-learning.
Then, we need to think about teachers' and students' social factors since their attitude depends on their social factors. The factors can reveal why they accept or reject the e-learning method. The social environment will shape culture and serve as a way of life for a society. This culture can be a belief in people's behavior and their relationship to the reality they live in (Aparicio et al., 2016). Environmental profiles shape teacher and student perceptions of the use and features of e-learning systems (Garfield & Watson, 1997), so culture influences the successful implementation of e-learning (Akano & Campbell, 2014;Bhogal & Campbell, 2015;Leidner & Kayworth, 2006). Thus, it is essential to include characteristics of the social environment in explaining the use of e-learning systems (Tam & Oliveira, 2017). This paper contributes to the expansion of previous studies in two ways. First, we developed an updated D&M ISS model, which is still limited to evaluating the success of information systems. Al-Fraihat et al. (2020), Yakubu and Dasuki (2018), and Safsouf et al. (2020) have used the D&M ISS model to evaluate the use of e-learning. However, they focus more on the quality of the system and the quality of information. They ignore the student's ability to use the system, the teacher's ability to teach through e-learning, and the social environment between teachers and students. This study extends the updated D&M ISS model approach by adding factors of ability and the social environment of teachers and students. Moreover, combining system characteristic factors and user identity/characteristics is still rarely used (Williams et al., 2014;Yawson & Yamoah, 2020).
Second, research expands on the success of e-learning by not being limited to the teacher/ student who decided to use e-learning but is expanding to the possibility of teachers/students reusing e-learning. The expansion of this indicator is substantial because e-learning is a learning system that can complement traditional learning methods. Several education regulators plan to continue using e-learning even after the pandemic ends (Bramasta, 2020).
We divide the findings into some parts. First, we provide the background of the research, the lack of previous studies, and the innovation in this research. Second, we present the theory and the development of the hypothesis. Third, we display the methods. Fourth, we present the findings and discussion. Fifth, we conclude this research, provide recommendations, and highlight the limitation of the study.

E-learning evaluation
Experts have studied the success of incorporating information systems in teaching and learning. The evaluation is critical since many entities develop information systems to implement the operation of education and improve the effectiveness and efficiency in achieving the entities' objectives. The development of the information system is required for the stakeholder to improve their performance. Hence, because of the requirement, the information system's stakeholder continuously develops its evaluation system (DeLone & McLean, 2016). Some evaluation models have been developed, such as Technology Acceptance Model (TAM; Davis et al., 1989), D&M ISS model (DeLone & McLean, 1992), Extending the Technology Acceptance Model (Dishaw & Strong, 1999), Unified Theory of Acceptance and Use of Technology (UTAUT) model (Venkatesh, 2003), and Updated D&M ISS model (DeLone & McLean, 2003).
The researcher also uses the evaluation model from the information system to evaluate the success of e-learning. We do this since e-learning uses the technology of information (Arpaci & Basol, 2020;Raza et al., 2020;Sukendro et al., 2020). Hence, the success of the information systems can measure the success of e-learning. Previous studies mention that the success of information systems can measure the success of e-learning, such as using (1) the D&M ISS model (Ouajdouni et al., 2021;Safsouf et al., 2020;Yakubu & Dasuki, 2018); (2) TAM model (Ouajdouni et al., 2021;Safsouf et al., 2020;Yakubu & Dasuki, 2018); and (3) UTAUT model (Ouajdouni et al., 2021;Safsouf et al., 2020;Yakubu & Dasuki, 2018). Ouajdouni et al. (2021) evaluate the success of e-learning for college student in Morocco using the D&M ISS model. They find that success depends on the use, perception, and students' satisfaction. W. W. Cidral et al. (2020) modify D&M ISS's model by adding students' learning orientation as a parameter within the evaluation system. W. W. Cidral et al. (2020) use the net benefit approach to measure e-learning success. In their research, W. W. Cidral et al. (2020) find that e-learning's success depends on students' use and satisfaction.
The evaluation model for the success of e-learning is different (Safsouf et al., 2020). The research combines TAM, expectation-confirmation model (ECM), D&M ISS, and self-regulated learning theory (SRL). The success is proxies by Safsouf et al. (2020) by the success of the students, and they find that the success of e-learning is influenced by the intention to use e-learning and selfregulation. Alam et al. (2021) use the factor of the system's quality, information, teachers, students, perception of use, and the use of e-learning as the factors that influence the students' academic success. Students' academic success is measured by the experience, the attainment of objectives, and the efficiency of the learning process. However, the study shows that students' quality is the sole influence on the students' academic success. The development of the evaluation model for e-learning is also done by Al-Fraihat et al. (2020). They developed a comprehensive e-learning evaluation model. The model measures success by seeing the benefits and emphasis within e-learning, including perceived satisfaction, usefulness, and use. The finding shows that e-learning's success depends on perceived satisfaction, usefulness, and use.
The e-learning evaluation models that the previous researchers develop focus on the benefit of the system yet neglect the success of the learning. The model focuses on attaining the objective and students' legal competence after finishing a particular subject. We see that only Alam et al. (2021) use the success of e-learning as a parameter that concerns students' academic success. The indicator is not specifically academic. Alam et al. (2021) use three indicators, which are: (1) students' happiness with e-learning; (2) the attainment of teaching goals; and (3) the effectiveness of learning. Therefore, e-learning aims to ensure an effective and sustainable education process (Theresiawati et al., 2020).

The updated D&M ISS model
The D&M ISS is an evaluation model for the system of evaluation developed in 1992. In the initial state, the model measures the success of a system by emphasizing the effectiveness or the success of the system of information (DeLone & McLean, 2016). The model uses system quality, information quality, user satisfaction, use, personal impact, and organizational impact as the success factors in the system of information (DeLone & McLean, 1992). In the D&M ISS model, the main factor in the system's success is the quality and information. The quality of systems and information will affect the use of information systems and user satisfaction (Al-Maroof & Salloum, 2021;Alsabawy et al., 2016). This use and satisfaction will increase individual performance and, subsequently institutional performance (W. A. W. A. Cidral et al., 2018). However, in the next step, the model gets critiques on the use factor. The factor is considered ambiguous, and it is eventually changed into the intention to use (DeLone & McLean, 2016).
The second critique of the D&M ISS model is that the model does not concern the service quality to the user as the success factor in the information systems (DeLone & McLean, 2016). Based on the critique, the D&M ISS model is updated. The updated model deems that the information system is not static, yet it continuously/dynamically develops. Users can or cannot obtain benefits from the system of information and it may impact the use or the satisfaction of the user (DeLone & McLean, 2016).
In the beginning, the D&M ISS model is used to evaluate e-commerce. However, this model can also evaluate the e-learning model (Al-Fraihat et al., 2020). Safsouf et al. (2020) also use the D&M ISS model to evaluate e-learning success in some state universities in Morocco. Pham et al. (2019) use the D&M ISS model and add loyalty as the indicator of e-learning success in Vietnam. Yakubu and Dasuki (2018) develop the D&M ISS model to evaluate e-learning success for students at a university in Nigeria. Meanwhile, Al-Fraihat et al. (2020) also developed D&M ISS to evaluate the success of Moodle at the University of Warwick, United Kingdom. From the research, we can conclude that the system information model must relate to the research objective and the context of the research. This is because the information system is dynamic, and the user continuously develops (DeLone & McLean, 2016). E-learning in the Covid-19 pandemic appears due to the change in education policy, which demands schools to close amid physical distancing (Ebner et al., 2020;Karasan & Erdogan, 2021;Nariman, 2021) and implemented for all levels of education (Alam et al., 2021;Lockee, 2021). The changing policy from conventional teaching to e-learning raised concerns about the lack of infrastructure and teachers' readiness (IESALC-UNESCO, 2020;Tang et al., 2021) and the lack of internet connection and other infrastructure (Lockee, 2021). In addition, teachers and students have limited time to replace conventional teaching methods with e-learning (Scherer et al., 2021), so they need social support (Dečman, 2015;Pynoo et al., 2011;Wang et al., 2009). An information system can help achieve learning objectives that success can apply to an academic one. E-learning aims to improve students' pedagogical skills and academic performance (Islam, 2013;McDowall & Jackling, 2006). We developed an updated D&M ISS Model in Figure 1.

Hypothetical development
The first indicator of the updated D&M ISS model is the system's quality ( Figure 1). The system quality regards the feasibility, functionality, reliability, data quality, portability, integration, and interest (DeLone & McLean, 2003). This means that the information system is easy to use, functions based on the user's needs, and is reliable, flexible, and trusted. This platform is used by the user and satisfies their needs. This is because the quality of the system focuses on the aspect of technology and the process of the system's characteristics (Yakubu & Dasuki, 2018) which later works on efficiency and cost reduction (Jang & Kim, 2006). Jang and Kim (2006), and Tam & Oliveira (Tam & Oliveira, 2017) have proven that the system's quality positively influences intention to use and student satisfaction. D&M ISS can prove that the system's quality influences user satisfaction (Yakubu & Dasuki, 2018;Al-Maroof & Salloum, 2021;Al-Fraihat et al., 2020;Safsouf et al., 2020;W. A. W. A. Cidral et al., 2018) and user's intensity (Al-Fraihat et al., 2020;Fianu et al., 2020Fianu et al., , 2018.
H1a: System quality influences student satisfaction.
H1b: System quality influences the intention to use E-learning.
Information quality relates to accuracy, timeliness, comprehension, relevance, and consistency of information in a system of information (DeLone & McLean, 2003, 1992. Previous studies found that information quality influence user intention to use information system and impact their satisfaction (Roky & Meriouh, 2015). D&M ISS evaluation model of e-learning ( Figure 1) shows that the quality of information influences the satisfaction of users ( H2b: Quality of information influences the intention to use E-learning The information system has two critical roles: providing information and producing services as a support to the user's development (DeLone & McLean, 2003). Service quality is defined as the system's overall quality (Fianu et al., 2020). DeLone and McLean (2003) states that service quality is related to assurance, empathy, and responsiveness. Experts who research e-learning evaluation with the D&M ISS model find that service quality influences the intention of using e-learning (Lee, 2010) and user satisfaction (Lee, 2010;Machado-Da-Silva et al., 2014;Yakubu & Dasuki, 2018). Based on these this framework (Figure 1), we develop a hypothesis: H3a: Service quality influences student satisfaction.
H3b: Service quality influences the Intention to Use E-learning E-learning is a learning model that alters the interaction between teachers and students through virtual platforms (Wang et al., 2009). The system demands teachers produce digital materials (Theresiawati et al., 2020). In e-learning, teachers and students do not directly interact, and they have communication problems and require much feedback. Yengin et al. (2010) conclude that there are three considerations to make e-learning effective active learning, positive reinforcement to students, and learning execution and evaluation. From this background, teachers should master both academic and technological skills for the success of e-learning (Aydın & Tasci, 2005). Romi (2017), and Pham et al. (2018) mention that teachers' quality is an essential element after system quality that impacts e-learning quality. The teacher can encourage students to continue their studies since they can handle e-learning and respond to the students' needs relevantly and eventually meet their expectations (Levy, 2007). Pham et al. (2019) find that instructor's quality constructs e-learning's quality and impacts students' loyalty to e-learning (Lwoga, 2012;W. A. W. A. Cidral et al., 2018). Based on these this framework (Figure 1), we develop a hypothesis: H4a: Teacher's capability influences student satisfaction.
H4b: Teacher's capability influences the Intention to Use E-learning E-learning is a computer-aided virtual learning approach. This approach requires teachers' preparedness and students' readiness. In e-learning, three dimensions determine its success: attitudes toward e-learning, students' trust in e-learning communication, and the ability to be involved in independent learning (Mutambik et al., 2020(Mutambik et al., , 2020Tang et al., 2021). Researchers find that the students' readiness can be measured with personal drive, self-efficacy, and technological ability (Junus et al., 2021). Apart from that, teachers are demanded to stimulate students to learn in e-learning. Therefore, teachers' quality impacts students' loyalty to e-learning (Lwoga, 2012). In this research, we see that our student's ability becomes the factor that impacts e-learning success. Based on these this framework (Figure 1), we develop a hypothesis: H5a: Student capability influences the Intention to Use E-learning H5b: Student capability influences the Intention to Use E-learning Siron et al. (2020), Raza et al. (2020), and Fianu et al. (2018) argue that the system's attitude is influenced by social impact. The social environment can encourage teachers and students can use or reject a system. Social influence is how far an individual trusts something and encourages other individuals to use a new system (Venkatesh, 2003). Social influence represents the subjective norm of TRA (Fianu et al., 2020;Venkatesh, 2003). Social influence has been proven to positively influence the use of e-learning (Dečman, 2015;Pynoo et al., 2011;Wang et al., 2009) and satisfaction (Safsouf et al., 2020). The sudden transition of the system from traditional learning to e-learning makes social support an essential factor that displays teachers' and students' attitudes toward the policy.
H6a: Social impact influences student satisfaction.
H6b: Social impact influences the Intention to Use E-learning DeLone and McLean (2003) opines that "use" and "satisfaction" are strongly correlated. The "use" should be prioritized, and the "satisfaction" should be defined as the process. The upbeat "use" experience leads to "satisfaction", which causes the "intention to use" (DeLone & McLean, 2003). The user feels satisfied if their needs have been fulfilled from the offered system. Students' satisfaction with e-learning can impact the students' intensity of using e-learning (Al-Maroof & Salloum, 2021;Theresiawati et al., 2020;Al-Maroof & Salloum, 2021;W. A. W. A. Cidral et al., 2018). Asides Levy (2007) finds that students' satisfaction influence how e-learning course is taken. Based on these frameworks (Figure 1), we develop a hypothesis: H7a: User satisfaction influences the Intention to Use E-learning.
The updated D&M ISS model (Figure 1) states that system user satisfaction will positively impact the benefits of using the system. Students' good experience using e-learning will positively impact academic performance (Al-Fraihat et al., 2020). After using e-learning, the students have experience using the system, which satisfies their needs. If the students are satisfied, they will get the benefit from e-learning. Hence, there is a positive value from students' satisfaction and the benefit that they get (W. W. Cidral et al., 2020;W. A. W. A. Cidral et al., 2018;Aparicio et al., 2017). We include the benefit of the e-learning policy as an indicator of this research.
H7b: User satisfaction influences academic benefit.
Before users get satisfaction with using the system, there is user experience. However, when the user is satisfied, they will decide to reuse the system (see, Figure 1). There will be a mutual correlation between intention to use and user satisfaction (DeLone & McLean, 2003). In the Brazilian context, W. W. Cidral et al. (2020) find that intention to use positively impacts student satisfaction. The increase in "user satisfaction" will lead to an increase in "intention to use (DeLone & McLean, 2003). Theresiawati et al. (2020) found that e-learning user satisfaction positively influence on e-learning user intentions.
H8a: Intention to Use influences student satisfaction.
E-learning is used as a medium to improve learning effectiveness. During the pandemic, e-learning is required to improve the learning process regardless of the closing of the school. E-learning can help students improve their knowledge and save time. For the teacher, e-learning helps them systematically manage the class (Al-Fraihat et al., 2020). Thus, e-learning impacts user satisfaction (Al-Fraihat et al., 2020; W. W. Cidral et al., 2020;W. A. W. A. Cidral et al., 2018). According to these frameworks (Figure 1), we develop a hypothesis: H8b: Intention to use influences net academic benefit.
Information system has been introduced to the user and gives continuous positive responses. The net benefits of using this information system will continue to be obtained by the user if the user uses it and will further increase the intensity of "use" and "user satisfaction" (DeLone & McLean, 2003). On the contrary, reinforcement might reduce the use of the information system. Adopting this framework (Figure 1), we argue an influence between net academic benefit, intention to use e-learning and student satisfaction.
H9a: Net academic benefit influences the Intention to Use E-learning.
H9b: Net academic benefit influences student satisfaction.

Methodology
This study focuses on 42,324 students at Universitas Negeri Semarang. The university has 38,514 undergraduate students and 38,514 postgraduate students. We focus on this university because: (1) education is carried out using e-learning for all courses, (2) this university has developed an information system as a medium for teachers to conduct e-learning, (3) committed to using e-learning even though there was no pandemic, and (4) committed to improving the e-learning system according to user needs.
This study aims to measure the success of e-learning by using the updated approach of the D&M ISS Model. Our target respondents are students at the State University of Semarang, Indonesia. We use this university because learning is done by e-learning with a platform that they are developing themselves and is committed to continuously updating the e-learning information system. The evaluation of the success of e-learning is important.

Participants
We distributed online questionnaires through social media (Facebook, WhatsApp) to student groups over two months. In this way, 427 respondents have been accepted with complete answers. The number of samples can be determined based on the number of variables observed. Comparing the number of samples with the minimum variable is five times that of the variables (Memon et al., 2020). Another opinion is ten times that of the variables (Yew et al., 2022). However, Hair et al. (2018) recommends a sample size of 15 times or 20 times the number of variables. Figure 1 shows the number of variables in this study is 9 variables. With 427 respondents, our sample size is 47.44 times of variables. Thus, our sample size exceeds the expert recommendation.

Research variables
Based on the modification the updated of D&M ISS Model, we find nine variables that as measured by indicators as we present in Table 1: We measured the variables using a 5-point Likert-type scale (1-strongly disagree, 2-moderately disagree, 3-slightly disagree, 4-neutral, 5-slightly agree, 6-moderately agree, 7-strongly agree).

Data analysis
Data were analyzed using the partial least squares structural equation modeling (PLS-SEM) approach with Warp PLS software. PLS-SEM is particularly well-suited for handling models that involve the use of second-level constructs or model development (Kosiba et al., 2022), focuses on estimation and prediction (Al-Adwan et al., 2022) and test the conceptualized model from the previous theoretical deduction ( Barrett et al., 2021) as is the case in our conceptual model developing the updated of D&M ISS Model. Barrett et al. (2021) used PLS-SEM to test the model developed from Davis' TAM model. Following Chhetri and Baniya (2022), and Barrett et al. (2021), we use PLS-SEM to analyze the data of this study.

Measurement model assessment
Before the distribution, we tested the instrument to evaluate its validity by piloting the questionnaire to 20 students. The results of the data test show that all constructs have internal consistency because each construct has a Cronbach alpha value ≥ 0.7 (Hair et al., 2019). Composite Reliability test results (Table 2) range between 0.788 and 0.936. The composite reliability value above suggested a recommended value of 0.7 (Faqih, 2022).
Construct validity was measured by the Average variances extracted (AVE) value, and discriminant validity were measured by the loading factor. The results of the AVE test (Table 2) resulted in the lowest score of 0.517 and exceeded the recommended minimum requirement of 0.5 (Faqih, 2022). Loading factor value (Table 5) resulted in the lowest score ranging from 0.702 to 0.918. The loading factor value is higher than the recommended value of 0.7, and is higher than the recommended value by Nunnaly (1979) of 0.7 (Devlin et al., 2014).
The variance inflation factor (VIF) test is used to see the correlation between variables. We present the VIF score in Table 3, where the largest score is 3.399. The tolerance score of VIF is 0.10 so that the model does not correlate between variables.
To ensure that our model is valid to answer the hypotheses; we assess the feasibility of the model, Table 3. Table 3 shows APC, ARS and AARS score with P-value <0.001. The ideal APC, ARS, and AARS values are ≤0.05. It shows that the goodness of fit model has been achieved. Table 2 also respectively shows the AVIF and AFVIF scores are 1.799 and 2.481. Both scores are ideal because it is over 3.3.

Descriptive of data
The description of the variable shows that the lowest score for the system quality is 5.28. The highest score is the provision of information (scored 6.05). The average score shows that the e-learning system has moderately good quality. Table 5 shows that the lowest average score for the information quality is on the indicator of the quality of information from e-learning (scored 5.28). The highest score is accessibility (5.82). The score assumes the students believe if that e-learning produces moderately good information. A similar finding can be seen in the variable of service quality. The lowest average score of this variable is 5.36 (reliability), and the highest is 6.04 (feature availability).
The external factor, which is student capability to use using computers, moderately agrees. The lowest indicator of this factor is independent learning (scored 5.31), and the highest is the positive attitude in using the technology of information (score 5.78). The score shows that the students have moderately good skills in using information systems. However, the teachers scored lower in student capability. Table 1 shows that teacher capability scored an average of 5.38. The lowest score in this variable is understanding the students' needs (scored 4.96), and the highest is the positive attitude to e-learning (scored 5.26).
E-learning with a moderately good information system, information production, and service has not effectively impacted the users. Table 1 shows that the social impact scored low in the total of 4.71(slightly). However, the variable of student satisfaction scored an average of 5.16. The score is relatively low, yet it is higher than the social impact. The result shows that the student satisfaction is at the level of slightly agree. A different finding can be seen on the NAB, which scored 5.56 (moderately agree). The highest score of NAB is in the indicator of encouraging task completion (scored 5.67). The lowest score is in the indicator of e-learning's ability to improve productivity (scored 5.31). The finding is strengthened by the variable of use e-learning (UE), which scored 6.16 (strongly agree). The highest score of this variable is the indicator of gathering information (6.31), and the lowest score is in the communication between lecturers and colleagues (score of 6.08). The result shows that the students frequently use e-learning as a medium to add knowledge and complete their homework.

Model test result
The hypothesis test result was summarized in Table 3. The coefficient that showed the relation between SQ and SS was 0.146, with a probability equal to 0.001. The coefficient of the SQ test result to UE was 0.121, with a probability equal to 0.006. Both probabilities were <0.01, which showed that system quality had a significant positive influence on student satisfaction and the use of e-learning.
The coefficient of INFQ test result to SS was 0.159 with a probability equal to <0.001. The probability was <0.01, which showed that information quality positively influenced student satisfaction. However, the coefficient that showed the relation between INFQ and UE was 0.012, with a probability equal to 0.405 (P > 0.01). Hypothesis H2b was rejected, and it was shown that information quality did not influence the use of e-learning.

SS-2
The ability of the system to fulfill students' needs.  Table 3 explained the coefficient that showed the relation between SERQ toward SS was −0.019 probability equal to 0.351. The coefficient that showed the relation between SERQ toward UE was −0.065, with a probability equal to 0.088. The probability score of SERQ toward UE was <0.01. It was a sign that service quality influenced the use of e-learning. However, it did not influence student satisfaction.
The coefficient that showed the relation between SC to SS was 0.205, with a probability equal to <0.001. It showed that student capability influenced student satisfaction. The coefficient that showed the relation between SC and UE was 0.042, with a probability of 0.194. Since the probability score was >0.01, the hypothesis was rejected. In conclusion, student capability did not influence the use of e-learning.
Table 3 also explained that the coefficient which showed the relation between TC toward SS was 0.240, with a probability equal to <0.001. The coefficient that showed the relation between TC toward UE was 0.029, with a probability equal to 0.275. This result showed that teacher capability influenced student satisfaction but did not influence the use of e-learning (Table 4).
The coefficient of the H6a test result (the relation of SI toward SS) was 0.201, with a probability equal to <0.001. It showed that social impact positively influenced student satisfaction. The coefficient of SI toward UE was 0.025, with a probability equal to 0.304, which explained that social impact did not influence the use of e-learning. Table 6 showed that the coefficient of SS toward UE (H7a) was 0.113 with a probability equal to 0.009. It proved that student satisfaction influenced the use of e-learning. The coefficient of UE toward the SS (H8a) test result was 0.024 with a probability equal to 0.308. It provided the opposite idea that e-learning did not influence student satisfaction. The coefficient of SS toward the NAB (H7b) test result was 0.716 with a probability equal to <0.001. In contrast, the coefficient of UE toward the NAB (H8b) test result was 0.056 with a probability equal to 0.121. This finding explained that net academic benefit was influenced by student satisfaction. However, it was proven that e-learning did not influence net academic benefit.
The test result of hypotheses 9a and 9b showed the coefficient of NAB toward UE was 0.028 with a probability equal to 0.028, while the coefficient that showed the relation between NAB toward SS was 0.72 with a probability equal to <0.001. It showed that net academic benefit positively influenced student satisfaction. However, the net academic benefit did not influence the use of e-learning.

Discussion
Model test results using SEM-PLS presented in Table 3 shows that system quality is in proportion to student satisfaction in e-learning. System quality is related to system characteristics (Yakubu & Dasuki, 2018). A qualified system is easy to use, practical, dependable, flexible, portable or easy to move, and integrated (DeLone & McLean, 2003). An e-learning system with those qualities enhances student satisfaction (Yakubu & Dasuki, 2018;Al-Maroof & Salloum, 2021;Al-Fraihat et al., 2020;Safsouf et al., 2020;W. A. W. A. Cidral et al., 2018) and attract people to be users (Al-Fraihat et al., 2020;Fianu et al., 2020Fianu et al., , 2018. Moreover, qualified systems focus on effective and efficient usage to enhance user satisfaction and the decision to use the system (Jang & Kim, 2006).
The research also shows that information quality is proven to enhance student satisfaction.
A qualified e-learning information system produces accurate, up-to-date, complete, relevant, and consistent information (DeLone & McLean, 2003, 1992. An e-learning system with those qualities enhances student satisfaction (Al-Maroof & Salloum, 2021;Safsouf et al., 2020;W. A. W. A. Cidral et al., 2018). However, our research shows that information quality does not affect the use of e-learning systems. This finding shows that an e-learning system that provides accurate, up-to-date, complete, relevant, and consistent information does not enhance the use of e-learning. This finding differs from Yakubu and Dasuki (2018), and W. A. W. A. Cidral et al. (2018) finding shows a relation between both aspects. The case probably results from the differences in characteristics between the use of e-learning systems during and before the pandemic. During the pandemic, it is mandatory to use e-learning at all levels of education (Alam et al., 2021;Lockee, 2021), and it has become the central learning system (Alam et al., 2021). This characteristic differs from the learning process before the pandemic, in which e-learning was only implemented for specific subjects and as an alternative or complement to face-to-face learning. The pandemic forces the students to use the e-learning systems provided by their institution, so they do not consider the information quality essential in using the e-learning systems.
Student satisfaction toward e-learning is not affected by the information quality. Besides, the research shows that service quality negatively influences the use of e-learning. Service quality is defined as the overall system quality (Fianu et al., 2020), including assurance, empathy, and responsiveness (DeLone & McLean, 2003). This result differs from Lee (2010), which states that service quality has a positive influence on the use of e-learning and is also different from Yakubu and Dasuki (2018), Lee (2010), and Machado- Da-Silva et al. (2014), that state service quality enhances user satisfaction. People used e-learning for a different reason during the Covid-19 pandemic. The government implemented a regulation to use e-learning systems as a replacement for face-to-face learning at all levels of education and in all subjects. Therefore, students do not have another choice but to follow the rule. As a result, service quality is mostly ignored.
The first external factor we used to explain the effectiveness of an e-learning system is teacher capability. It shows that teacher capability positively affects student satisfaction. Student satisfaction increases as teacher capability improve. However, we also find that teacher capability does not affect the use of e-learning systems. In e-learning, teachers are demanded to transform the learning material into digital learning material (Theresiawati et al., 2020) and conduct digital communication by giving feedback to students' responses (Yengin et al., 2010). Teachers who can do their tasks in the learning process will help enhance student satisfaction and use e-learning systems (Lwoga, 2012;W. A. W. A. Cidral et al., 2018). However, teachers' interest in stimulating the students to use e-learning systems is still low. As a result, teacher capability is failed to improve the use of e-learning (Lwoga, 2012).
The second internal factor is student capability. E-learning which uses information systems in a computer, demands students to learn independently (Mutambik et al., 2020;Priatna et al., 2020). Independence is related to cultural values where every person should take care of oneself before taking care of others (Aparicio et al., 2016). It means independent students will first focus on finding learning material from digital sources. This behavior should enhance student satisfaction. However, the research shows that student capability does not influence the use of e-learning. Students are forced to use an e-learning system during the Covid-19 pandemic. Therefore, the relation between those aspects cannot be found.
The third external factor is social impact. It is stated in the Theory of Planned Behavior approach that an individual's behavior is affected by their environment or is often called the subjective norm (Ajzen, 1991). In that approach, social factor is represented as social influences (Venkatesh, 2003). Venkatesh (2003) states that social influences are affected by the support from people around the user to use a new system. The research shows that social influence positively affects student satisfaction. This result supports Safsouf et al. (2020) finding that social influence enhances student satisfaction. The research does not show any relation between social influence and the use of e-learning. It is caused by the lack of support from their environment to use e-learning. Besides, we see that the use of e-learning during the Covid-19 pandemic is mandatory for all students.
The result shows that student satisfaction has a positive influence on the use of e-learning systems and the improvement of students' academic achievement. Students who are satisfied with the system will decide to use it again. Moreover, when students are satisfied with the e-learning system, they will take advantage of it. In conclusion, student satisfaction has a positive impact on the use of e-learning systems (W. W. Cidral et al., 2020;W. A. W. A. Cidral et al., 2018;Aparicio et al., 2017).
Unlike the previous hypothesis, we find that e-learning does not affect student satisfaction and academic achievement. There is a sign that an e-learning system is used unexpectedly (Scherer et al., 2021), with no preparation (IESALC-UNESCO, 2020; Tang et al., 2021), and applied to all subjects (Alam et al., 2021;Lockee, 2021) causes ineffectiveness to occur. Implementing e-learning is more difficult for subjects that require practice (Hamid et al., 2020). Sometimes, students cannot balance their job, family, and social life with their learning process through the e-learning system (Dhawan, 2020). Students may find problems like lousy internet connection, e-learning support systems, technology, and self-control when the institution is not well-prepared. In contrast, teachers may also face a lack of competence, operational skill, self-regulation, and isolation.
The research shows an enhancement or positive influence on student satisfaction for those who can take advantage of the e-learning system. It is also shown that the advantages offered by e-learning systems do not affect the use of e-learning systems. It supports the finding from DeLone and McLean (2003) that user satisfaction enhances student satisfaction. Since Net Academic Benefit (NAB) does not affect the use of e-learning systems, it supports the claim that the e-learning system is ineffective during the Covid-19 pandemic.

Conclusion
The Covid-19 pandemic causes a switch from face-to-face to the digital learning process. This regulation is implemented to minimize social contact. Due to the urgency, the action is taken without further consideration of facilities. Our research finds that user satisfaction depends on system quality, quality of information, teacher capability, student capability, and social impact. We also find a reciprocal relationship between student satisfaction and net academic benefit. The e-learning system should be enhanced with the improvement of the system quality and student satisfaction, quality of information, quality of service, and student and teacher capability that do not influence the use of e-learning.
Implementing e-learning for all subjects requires the teacher's capability to transfer the learning material into the digital one. Besides, the teacher is also demanded to use an effective learning method and develop a learning environment that stimulates students to use e-learning, supports students to learn independently, and be active in the discussion. The regulator and leader of the institution are expected to provide facilities to support the e-learning system. Moreover, the regulator should train teachers and students to use e-learning, develop an exciting learning method, and compose innovative digital learning material.
The limitation of this study is the low response of respondents (1.001%). However, the number of samples we obtained exceeded the minimum number of samples recommended by the scholar. In addition, this study only focuses on Semarang State University as the object of study, so the results of this study may differ from other researchers who use different research objects. Further researchers can expand their studies with various universities but still pay attention to the characteristics of the e-learning information system used by universities. The reason is that each university can run e-learning to use an information system that they have created themselves, so there may be differences between universities.
The researcher explains that implementing e-learning during the Covid-19 pandemic has a different background from the learning process before the pandemic. The previous researcher could compare the success of e-learning during and before the pandemic. The next researcher is also able to present cultural variables in explaining the success of e-learning.