The effectiveness of the STEMDISLEARN module in improving students’ critical thinking skills in the differential equations course

Abstract This study aims to examine the effectiveness of the STEMDISLEARN module on students’ critical thinking skills in the DE Course. This research design employed a quasi-experimental non-equivalent pre-test and post-test group. The samples amounted to 123 students, consisting of 64 students in the experimental group and 59 students in the control group. The experimental group used the learning treatment with the STEMDISLEARN module, while the control group employed conventional learning. Meanwhile, the t-test was used as the data analysis technique. Results showed that the two groups had no difference in the average scores of their critical thinking skills before treatment. After treatment, however, there was a difference in the average pre-test and post-test scores between the experimental and control groups.The average post-test scores for critical thinking skills in each group were better than the pre-test scores. However, the increase in the average scores for the experimental group was higher than in the control group. Thus, the STEMDISLEARN module is effective in improving students’ critical thinking skills in a DE course.


PUBLIC INTEREST STATEMENT
Differential Equation is one of the compulsory subjects in almost all universities, especially in the majors of mathematics, mathematics education, and engineering.This course is very useful in helping solve various real problems, not only in mathematics, but also in other fields, such as biology, chemistry, physics, engineering, economics, etc.In solving these problems, students need good skills in analyzing, evaluating, synthesizing, and interpreting (critical thinking skills).The STEMDISLEARN module is a differential equation learning module that is designed using the Science, Technology, Engineering, and Mathematics (STEM) approach, and is presented using discovery learning steps.STEM is a learning approach that is seen as highly relevant to 21st century skills, including critical thinking skills.Discovery learning is a learning model that can encourage students to find their own knowledge by using prior knowledge.The research finding provide suggestion for designing teaching materials that can improve critical thinking skills, especially in differential equations courses.

Introduction
Critical thinking is one of the most competencies that can be used for learning development to face the challenges of the 21st century (Ariyana et al., 2018;Battelle for Kids, 2019;WEF, 2016).It is useful for evaluating, analyzing, and/or synthesizing appropriate information to make arguments or conclusions, supported by evidence (Reynders et al., 2020).Dwyer et al. (2014) proposed that critical thinking is a metacognitive process, which includes analysis, evaluation, and inference.Promoting critical thinking skills are essential for successful in solving problem.Differential equation is one of the main courses in mathematics, science and engineering departments of higher education institutions (Kwon, 2020).Zeynivandnezhad et al. (2013) stated that the differential equation develops the knowledge of modeling, problem-solving, and interpretation of solutions to real problems.For a model formulation, it is necessary for students to be able to analyze problems by understanding information to determine its meaning.Furthermore, to solve and interpret a formulated model, they need to have the abilities to evaluate and synthesize information by determining the relevance and reliability of information and relating or integrating the information to develop an argument or make a conclusion.These abilities are a domain of critical thinking skills, as suggested by Reynders et al. (2020).However, there have little study exists on students' critical thinking skills in solving differential equations.
Various learning approaches are recommended for promoting students' critical thinking skills (Apriliana et al., 2019;Bensinger, 2015;Dolapcioglu & Doğanay, 2022;Plummer et al., 2022;Shanta & Wells, 2022).STEM education is one of learning approaches currently considered a very suitable learning skill development in the 21 st century (Peters-Burton & Stehle, 2019;Sen et al., 2018).Tambunan and Yang (2022) proposed that STEM education has six values for mathematics education, which are application, integration, interest, performance, usefulness, and implementation.There are several advantages of the learning process using STEM, namely developing active communication and teamwork, integrating learning into various themes of real life, increasing critical thinking and problem-solving skills, increasing self-confidence as a bridge between education and career planning, and preparing students to make technological innovations in life (Morze et al., 2018).
Discovery learning is a learning model that refers to the use of inquiry in integrated STEM (Mystakidis et al., 2022;Thibaut et al., 2018).Schneider (2014) proposed discovery learning as a learning model that involves students in learning through discovery for the deep learning process by developing meta-cognitive skills and enhancing students as active participants in learning.The use of the discovery model can encourage students to learn actively, train independently, and increase their enthusiasm to obtain various learning sources (Chusni et al., 2021).Vajravelu (2018) stated that differential equation is a central part of bachelor's degree programs in the science and engineering.This course introduces students to the fields of science, engineering, and mathematics.Aisha et al. (2017) argued that differential equation is closely related to the science, social science, and technology fields.The differential equation course is essential because of its wide use in not only mathematics but also in other fields such as physics, chemistry, biology, engineering, economics, etc (Agarwal et al., 2021;Boyce et al., 2021).Based on the aforementioned, STEM approach based discovery is one of prospective learning model in differential equation course to promote critical thinking skills.
One of learning resources in higher education that can be used to present the learning material is a learning module (Kemristekdikti, 2019).Mohd Noah and Ahmad (2017) stated that the module is a learning material unit that discusses a specific topic systematically and sequentially.In relation with STEM approach, Barrett et al. (2014) stated that the interdisciplinary STEM module can develop students' content knowledge in the learning process.Previous studies have shown that learning module development by using the STEM approach had a positive implication for learning (Barrett et al., 2014;Chen & Huang, 2020;Jajuri et al., 2019;Siew & Ambo, 2018;Toma & Greca, 2018;Uğraş, 2018).Furthermore, STEM-based learning modules can improve students' critical thinking skills (Retnowati et al., 2020;Yaki, 2022).On a similar note, Savran Gencer and Dogan (2020) also stated that the STEM module is able to develop students' critical thinking skills in sciences, such as interpretation, analysis, and inference.Kariman et al. (2019) described that learning with modules based on the discovery learning model was effective in increasing student learning outcomes.Moreover, some previous studies show that learning modules using the discovery learning model can improve students' critical thinking skills (Julita et al., 2019;Wahyudi et al., 2019;Yuliani & Saragih, 2015).The predominance of module based on STEM and discovery learning to improve the thinking skills and learning outcomes is potential to be integrate.However, modules using the STEM approach based on discovery learning models have not been found in differential equation courses (R. P. Khotimah, Adnan, Ahmad, et al., 2021).In response, Khotimah et al. (2022) has developed the STEMDISLEARN module, which is a STEM-based discovery learning module in the topic of differential equations.

Research question
Based on the background, this study was conducted to answer the following research questions.
(1) Is there any significance difference in the average scores of the students' critical thinking skills in the experimental and control groups before treatment in differential equations course?
(2) Is there any significance difference in the average scores of the students' critical thinking skills in the experimental and control groups after treatment in differential equations course?
(3) Is there any significance difference in the average scores of the students' critical thinking skills in the experimental group before and after treatment in differential equations course?
(4) RQ-3: Is there any significance difference in the average scores of the students' critical thinking skills in the control group before and after treatment in differential equations course?

Objective of the study
The current study aims to test the effectiveness of the module in enhancing the college students' critical thinking skills.In particular, the objectives of the study are testing whether there is a difference: 1) in the average scores of critical thinking skills between the experimental and control groups before and after treatment; and 2) in the average pre-test and posttest scores of critical thinking skills between the experimental and control groups.

Critical thinking skill
Critical thinking is one of the essential skills in the 21 st century (Battelle for Kids, 2019; Griffin & Care, 2015;UNICEF, 2012).In the current study, critical thinking skills involve analyzing, evaluating, synthesizing relevant information to form an argument or reach a conclusion supported with evidence.Analyzing is interpret the information to determine meaning and to extract relevant evidence.Evaluating is determine the relevance and reliability of the information that might be used to support a conclusion or argument.Synthesizing is connect or integrate the information to support an argument or reach a conclusion.Forming arguments is make a claim and provide evidence to support it (Reynders et al., 2020).These indicators were employed because they were very relevant to STEM-based courses at the undergraduate or bachelor's degree level.

STEMDISLEARN module
STEMDISLEARN Module is a learning module which developed by STEM approach and discovery learning model in the topic of differential equations.The STEMDISLEARN module contains three learning units, namely Discovery I which includes the introduction of differential equations, Discovery II which includes first-order differential equations, and Discovery III which includes first-order linear differential equations.The module material is not given directly at the beginning of the module because the students will find the concept of the material themselves.Each material is presented in the form of an introduction to the material and learning activities.Learning activities begin with providing a stimulus and problems to be resolved (stimulation and problem oriented phases; the concepts of science, mathematics in STEM).Students use technology to gather information, collect data, and process data in solving the problems given (data collecting and data processing phases; the concepts of technology, engineering, mathematics in STEM).Furthermore, students carry out verification to check the answers to the problems obtained by using the concept of material that has been found or by using the Maple Program (verification phase; mathematics, technology, and engineering concepts in STEM).The next activity involves the lecturer and the students drawing conclusions from the learning that has been implemented (generalization phase; the concept of mathematics in STEM) (R. P. Khotimah, Adnan, Ahmad, et al., 2021).

Research design
This study utilized the quantitative research method with a quasi-experimental design.This design was employed because the samples were not randomly distributed into control and experimental groups equally Chua (2016).Chreswell (2012) suggested that random sample distribution into two groups equally is difficult because it can disturb a learning process in class; therefore, this study applied a quasi-experimental design.In this study, the types of the quasi-experimental design used were the non-equivalent group pre-test and post-test design (see Table 1).

Participants and research sample
The population of the study amounted to 204 third-year students of the Department of Mathematics Education of Muhammadiyah University who were taking the Differential Equation course.The research took place during the 2021/2022 odd semester.Sampling used a cluster sampling technique by selecting the samples in groups (Idris, 2013) human research ethics approval from the institution.Hence, all the participants were treated ethically based on the predetermined standards.
The experimental group received treatment using the STEMDISLEARN module while the control group was exposed to the treatment without the STEMDISLEARN module (conventional learning).All the participants in this research are Indonesian students.Hence, the STEMDISLEARN Module and instrument were developed in Indonesian Language

Instrument
The research aims to test the effectiveness of STEMDISLEARN Module on the student critical thinking skill in the differential equation course.So, the research instrument used was a critical thinking skill test.The test was developed in the form of essay test of four questions, referred to the learning materials of differential equations, course learning outcomes, critical thinking skill indicators as well as the assessment rubrics.The assessment of critical thinking skills employed the critical thinking assessment rubrics adapted from Reynders et al. ( 2020) with under permission.
The instrument was tested for validity and reliability through a pilot study involving 30 students outside the research sample.The validity test result using moment product correlation and the reliability result using Cronbach's Alpha are presented in Tables 2 and 3 respectively.
By using a significance level of 5% for N = 30, the value of r table = 0.361 is obtained.From the results of the calculation of four items (corrected item-total correlation column), the four items are more than 0.361.Hence the instrument is valid.
Based on the calculation result in Table 3, it is obtained that α = 0,804, so that the instrument is reliable.The following is an example of the critical thinking test instrument used in this study.
In English

Research procedure
This research aims to test the effectiveness STEMDISLEARN Module on the college student's critical thinking skill.It involves two variables, which are dependent variable (critical thinking skill) and independent variable (teaching and learning using STEMDISLEARN Module in the experimental group and traditional teaching and learning or conventional in the control group).
The research instrument used in this study is a critical thinking test, which has measured its validity and reliability through a pilot study.In the actual study, the two groups were given a pretest before the treatment.The treatment in the experimental group was teaching and learning differential equations using STEMDISLEARN Module.Meanwhile, the treatment in the control group was traditional (using lectures, questions and answers, and assignments).This treatment was given in seven weeks, and they were given a post-test after the treatment.The pre-test and posttest results in the experimental group were compared with the results in the control group.The data were analyzed to see the effectiveness of the STEMDISLEARN Module on the college students' critical thinking skill.
The research procedure is shortly described in Figure 1.

Data analysis
Critical thinking skills data were analyzed by using t-test statistics because the study is only comparing two groups or conditions (Pallant, 2011).The t-tests used were the independentsamples t-test and paired-samples t-test.The independent-samples t-test was used to compare the average scores of the students' critical thinking skills in two different groups (experimental and control), while the paired-samples t-test was used to compare the average scores of the students' critical thinking skills before and after treatment in the same group.

Description of research results data
The data of the students' critical thinking skills before and after the treatment in the experimental and control groups are reported in Table 4 below.Overall, all of the statistical figures in both groups were higher after treatment than before treatment.Before treatment, almost all in the experimental group were higher than the control group, while the standard deviation of the control group (6.463) was nearly the same as the experimental group (6.289).Referring to the minimum scores, both the experimental group and the control group reached the minimum extreme scores (0.00) before treatment.After treatment, both groups increased in the minimum score, but the increased score in the experimental group (58.33) was higher than that of the control group (52.08).As regards to the maximum score, the experimental group had a higher score than the control group before treatment.After treatment, the maximum score for both groups experienced an increase, but the increased score of the experimental group (70.83) was higher than that of the control group (60.41).
Regarding the average scores, both groups had obtained low scores before treatment.After treatment, the average scores increased very drastically.However, the increased average score of the experimental group (70.524) was higher than that of the control group (61.370).As regards to the standard deviation scores, the control group had a higher standard deviation score than that of the experimental group before treatment.After treatment, both groups increased in standard deviation, but the standard deviation of the experimental group was higher than that of the control group.
The data distribution of students'critical thinking skills in the pre-post test is presented in Figure 2.
Figure 2 compares the two groups' critical thinking skills data distribution before and after treatment.The line in the middle of the box plot for the experimental group is almost the same as the control group before treatment.However, it is higher than the control group after the treatment.It means the students who taught using STEMDISLEARN Module had a higher median score after treatment.Therefore, the average score of students' critical thinking skills in the   treatment group is higher than in the control group.Besides, the box plots for the experimental group are almost as long as the control group before and after the treatment.Hence, the critical thinking skills scores are spread out among students in the two groups.All the lines in the middle of the box plots are close to the center of the box, which means the distribution of scores has slight skew at all.

Normality test result
Before data analysis, a normality test was first performed as a prerequisite for the analysis test.
The results of the normality test in both groups using the Kolmogorov-Smirnov are presented in Table 5.
By using a significance level of 5%, based on the calculation results of the CT pretest data in the control group, the probability score (Sig.) was 0.33 > 0.05, while the probability score (Sig.) of the CT post-test data in the control group was 0.35 > 0.05.Therefore, it can be concluded that the samples of the control group were derived from a normally-distributed population.
Based on the results of calculating the CT pretest data for the experimental group, the probability score (Sig.) was 0.076 > 0.05, while the probability score (Sig.) of the CT post-test for the experimental group was 0.314 > 0.05.Thus, it can be concluded that the samples of the experimental group were also derived from a normally-distributed population.Due to the fulfillment of these normality test criteria, a statistical test to determine the effectiveness of the STEMDISLEARN module on students' critical thinking skills can be performed.

Students' critical thinking skills before treatment
An independent sample t-test was used to compare the average scores of the students' critical thinking skills in the experimental and control groups before treatment, the results are reported on Table 6.
The calculation results in Table 6 by using a significance level of 5% were t obs ¼ À 0:595<t 0:025;121 ð Þ ¼ 1:9798.Therefore, a null hypothesis was confirmed.It also proved that the probability score (sig.) was 0:553 > 0:05; so a null hypothesis was confirmed.Thus, there is no difference in the average scores of the students' critical thinking skills between the experimental and control groups before treatment.

Students' critical thinking skills after treatment
An independent sample t-test was used to compare the average scores of the students' critical thinking skills in the experimental and control groups after treatment, which are presented in Table 7.
The calculation results in Table 7 by using a significance level of 5% were t obs ¼ À 7:249< À t 0:025;121 ð Þ ¼ À 1:9798.Therefore, a null hypothesis was rejected.It was also proven that the probability score (sig.) was 0.000 < 0.05; so a null hypothesis was rejected.Thus, there was a significant difference in the average scores of the students' critical thinking skills between the experimental and control groups after the treatment.

Students' critical thinking skills in the experimental group
A paired sample t-test was used to compare the average scores of the students' critical thinking skills in the experimental group before and after treatment, with the results presented in Table 8 below.
Based on the calculation results in Table 8, it was found that t obs ¼ 55:934>t 0:025;63 ð Þ ¼ 1:998.Thus, a null hypothesis was rejected.Thus, there was a significant difference in the average score of students' critical thinking skills in the experimental group before and after treatment.It was evidenced that the probability score (sig.) was 0.000 < 0.05, so the null hypothesis was rejected.Therefore, there was a significant effect of the STEMDISLEARN module used in improving the students' critical thinking skills on the learning material of the Differential Equation course.

Students' critical thinking skills in the control group
A paired sample t-test was used to compare the average scores of the students' critical thinking skills in the control group before and after treatment, which are reported in Table 9 below.
Based on the calculation results in Table 9, it was found that t obs ¼ 46:077>t 0:025;58 ð Þ ¼ 2:002.Therefore, a null hypothesis was not accepted.This means that there was a significant difference in the average scores of the students' critical thinking skills in the control group before and after treatment.It was also proven that the probability score (sig.) was 0.000 < 0.05, so a null hypothesis was rejected.Therefore, there was a significant effect of the non-STEMDISLEARN module (conventional learning) in improving the students' critical thinking skills on the learning material of the Differential Equations course.
The average scores of the students' critical thinking skills in the experimental and control groups before and after treatment can be visualized in Figure 3.
Figure 3 shows that the average scores of the student's critical thinking skills in the experimental and control groups increased by 70.524, and 61.370 respectively.

Discussion
There was no treatment for the two groups when the pre-test was carried out; thus, there was no difference in the student's critical thinking skills.This is in line with the previous results which found that there was no difference in the critical thinking skills between the experimental and control groups before treatment (Topsakal et al., 2022;İ ̇smail & Arslan, 2021).This also similar to the research results of Yaki (2022) and Sidiq et al. (2021), which discovered that there was no significant difference in the critical thinking skills scores between the experimental and control groups in the pre-test results.Thus, the two groups were balanced before treatment.
Based on the calculation results in Table 4, the average score of the experimental group's critical thinking skills (78.695) after the treatment were higher than the control group (68.856).This is in agreement with the previous research result, that said the critical thinking's score in the experimental group was higher than in the control group (Sulistyanto et al., 2023).In the current study, the critical thinking skills of the students who used the STEMDISLEARN module were better than that of the students who did not use the module (conventional learning).This result is in line with those of Hacioglu and Gulhan's (2021) which argue that STEM had a positive effect on students' critical thinking skills.In addition, Yaki (2022) also found that the average score of critical thinking skills in the experimental group (STEM-based learning group) was higher than that of the control group.
The teaching and learning using the STEMDISLEARN module can increase the average scores of the students' critical thinking skills by 70.524.The results of the calculations in Table 4 show that teaching and learning without using-STEMDISLEARN module (conventional learning) can also increase the average score of critical thinking skills by 61.370.However, this was lower than using the STEMDISLEARN module in the Differential Equation course (70.524).This is relevant to the research of Abdurrahman et al. (2019) which reported an increase in students' critical thinking skills in the control group, but the increase was lower than those in the experimental group.These results supports the result of Listiaji et al. (2022)  the experimental and control groups, however the increase in the experimental group was higher than the control group.
The STEMDISLEARN module used by the students in the experimental group employed a discovery learning-based STEM.Topsakal et al. (2022) explained that a learning process designed with the STEM approach can develop students' critical thinking skills.The previous research results had also argued that STEM-based learning modules were effective in improving students' critical thinking skills (Savran Gencer & Dogan, 2020).Several other research results also described that the critical thinking skills of students using STEM before and after the test were significantly different from when using the non-STEM approach (Asigigan & Samur, 2021;Duran & Sendag, 2012;İ ̇smail & Arslan, 2021).
Besides STEM, the use of discovery steps in the module can encourage students to construct their knowledge based on prior knowledge.Thus, the critical thinking skills of the students who applied the STEMDISLEARN module were significantly different from those who did not use it.This is relevant to the research by Julita et al. (2019), which found that the critical thinking skills of the students who used the discovery learning-based module and those who did not were significantly different from one another.Several past studies have also discovered that students' critical thinking skills were better by using a discovery-based approach compared to using a non-discovery learning approach (Rahmawati et al., 2021;Wahyudi et al., 2019;Yaiche, 2021).Pahrudin et al. (2021) stated that students' critical thinking skills before and after learning with different STEM-based inquiries as well as a discovery-based STEM learning model can improve their skills.Meanwhile, Jamaludin et al. (2022) argued that there was an increase in critical thinking skills based on the pretest and post-test scores in the experimental group, namely those who received treatment using guided discovery learning.A discovery-based learning model is able to improve students' critical thinking skills (Chusni et al., 2022;Wahyudi et al., 2019).
The discovery learning involves stimulation, problem statement, data collecting, data processing, verification, and generalization phases (Ariyana et al., 2018).In the teaching and learning using STEMDISLEARN Module, students actively participated in each discovery learning stage.This coincide with the result of Wartono et al. (2018), in inquiry-discovery based learning, students are more active since they are facilitated to acquire their own knowledge through direct experimentation.
The STEMDISLEARN Module has encouraged the students to use their prior knowledge in developing a new knowledge by inquiry or discovery stages.This is in line with the result of Hong (2017), who that revealed students were facilitated in discovery learning to construct their knowledge on their own.This is also consistent with the findings of Shieh and Yu (2016) that the discovery learning process is to reconcile, integrate, and organize the new and old information to enable students to self-construct knowledge.

Conclusion and recommendations
This study showed that the STEMDISLEARN Module effectively enhances the college student's critical thinking skills in the topic of differential equations.The STEMDISLEARN Module has encouraged the students to use their prior knowledge in developing a new knowledge by inquiry or discovery stage.STEMDISLEARN Module can improve the student's critical thinking skills in analyzing, evaluating, synthesizing, and forming arguments.This study has limitations; namely, the differential equation material developed in the module is still limited to the introduction of differential equations, first order differential equations, and first order linear differential equations.In addition, the dependent variable in this study is still limited to critical thinking skills.Thus, further research can develop the STEMDISLEARN Module on other topics in Differential Equations.The module's effectiveness should be tested on the other variables in the teaching and learning of differential equations.3. Berikut adalah salah satu soal latihan persamaan diferensial yang diberikan oleh dosen pada mahasiswa untuk dikerjakan di rumah.

Figure 3 .
Figure 3. Improvement of average scores of critical thinking.

Table 1 . Non-equivalent group pre-test and post-test design
. The amount of the sample comprised of 123 students, consisting of 64 students in the experimental group and 59 students in the control group.Before participating in the research, they get an explanation in advance about the research objectives and voluntary participation in this research.Researchers have obtained Chua (2016)uivalent respondent group M1 : Critical thinking -pre in the control group M2 : Critical thinking -pre in the experimental group M3 : Critical thinking -post in the control group M4 : Critical thinking -post in the control group X1 : Teaching and learning with the conventional approach X2 : Teaching and learning with the STEMDISLEARN module.Source: Adapted fromChua (2016).

Table 9 . Paired sample T-Test for control group Paired Differences
; there was an increase in critical thinking skills in