Development and validation of a null curriculum questionnaire focusing on 21st century skills using the Rasch model

Abstract Curriculum development has always been dealt as one of the most important areas of language learning and teaching by educational institutions. Null curriculum has been viewed as one of the significant kinds of curriculum types which is important due to its absence, being left out or overlooked but should not be disregarded. For this reason, a 48-item questionnaire for null curriculum in ELT with a special focus on twenty-first century skills was developed and validated by the application of the Rasch model. The Rasch model was employed to decide whether the scores of an instrument are purposive, meaningful and significant. Rasch analysis using Winsteps software version 3.73 was used to determine the dimensionality, use of response category, sample appropriateness and reliability of the scale. The test was administered to 80 university instructors teaching at BA level in different universities in Iran. The results revealed that the Rasch model fits the test after deleting six items from the scale and a suitable reliability was confirmed. This indicates that the questionnaire is potentially valid and can be used as a tool for measuring aspects of null curriculum focusing on twenty-first century skills.


PUBLIC INTEREST STATEMENT
Living and surviving in todays' world is not simply performing academically well. There are some special skills and abilities that are vital for all graduates who enter workplace and society. These skills are known as twenty-first century skills. The present study tried to design, develop and validate a questionnaire for measuring various aspects of null curriculum with a special focus on twenty-first century skills. Null curriculum is viewed as one of the significant kinds of curriculum types which is important due to its absence, being left out or overlooked. It refers to that which is not taught at schools, but this omission sends a message to students that these are not important enough to study. When special subjects are not included in the written curriculum and form part of the null curriculum, students may believe these subjects have minimal value or are less important than the other subjects in educational experience or social life. So they are significant because of their absence.

Introduction
Educational institutions have always been dealing with curriculum development as one of the most important areas of language learning and teaching. Curriculum can be regarded as a connective bridge between teacher and students. Curriculum can cover different meanings depending on the situation and context in which it is used. In some situations, curriculum is used as a framework for external settings for the learning process; while in other contexts learning problems are diagnosed and the connection between the students and teachers is restored by the use of the curriculum (Gholami et al., 2016).
The idea of null curriculum was first presented by Eisner (1985) and later developed and studied by other scholars and researchers. It refers to that which is not taught. It influences by its absence. Sometimes some courses or topics are not considered as possibilities and at other times, they are discarded as less important than that which is selected as part of the intended curriculum. The neglected courses would be like the courses related to twenty-first century skills such as collaboration, team working, critical thinking, digital literacy, creativity, entrepreneurship in ELT, problem solving, managing language systems, etc. that advocates believe schools and universities need to teach to help students thrive in today's world.
The null curriculum refers to the things that students do not have the chance and opportunity to learn. In this regard, learners learn something based on the absence of certain experiences, interactions and discourses in the classroom. For instance, if students are not taught to question, talk about and call out sexist language in books, they are learning that this thing is not an essential thing for them to be exposed to (Milner, 2017). In other words, what is absent from curriculum is actually present in what students are learning.
The null or nonexistent curriculum, in directing focus on what is not present, brings to the field of curriculum studies an important theoretical tool for considering that which is not offered to students, and the potential educational significance and effect of such neglect. Being an important educational issue, null curriculum should be considered as a major curricular reform and should be taken into consideration more specifically. A curriculum universe should be specified so that null curriculum can be identified. A null curriculum cannot be described completely just by examining an existing curriculum (Flinders et al., 1986). Providing a complementary list that specifies the null curriculum is not possible unless an initial, complete list that is representative of a curriculum universe is available. The notion of null curriculum has different worthwhile applications in special areas of curriculum development. Like many other concepts and terms in the field of education, null curriculum cannot be defined with operational care and exactness unless we are ready to take a risk. Null curriculum is not evidently and immediately seen but this does not mean that it should be disregarded.
The importance of null curriculum has made scholars and researchers around the world open a new window in curriculum development and reconsider this issue and study the case in various levels. Abid et al. (2014) investigated the null curriculum in university education system in Islamic Republic of Iran. They believed that the null aspects of curriculum are evident in the curriculum development. They investigated the causes of the identified curriculum and ignored it; they focused on the needs of the scientific community. They stated that the public education system should consider the needs of scientific activities. In this way, designers need to consider the criteria and standards. Gholami Pasand (2013) investigated the MA curriculum of TEFL in Iran, paying special attention to courses. This survey aimed at finding the gaps in the courses. Gonzalez-Ocampo et al. (2015) studied about the curriculum question in doctoral education. Their focus was on explicit curriculum analysis to be able to bring about some changes in the processes and outcomes of doctoral education. Keshtiaray et al. (2012) worked on designing an experiential curriculum based on a phenomenological approach. They tried to decode and analyze the text messages using library sources and design a new model. Ilie (2013) presented a model for curriculum development make use of conceptual framework of hyper rationalistic pedagogy. Two theoretical concepts are considered as important here: theory into practice and mereologic reasoning. This was the first draft of hyper rationalistic model of curriculum development. Banoobhai (2012) explored the role of critical reflection as a tool for curriculum implementation and innovation. Critical reflective classroom practices attempt to help teachers to change policies and classroom needs. Teachers should try to change learning and teaching context with strong focus on critical reflective practices.
Living and surviving in today's world is not simply performing academically well. There are some special skills and abilities which are vital for all graduates who enter workplace and society. There are some core competencies that are needed to be taught by schools and universities to help students live in today's world. P21which is the partnership for twenty-first century skills, a consortium created by various organizations including business leaders, departments of education and research organizations, was a pioneer to develop a framework for learners to be ready and prepared for success in twenty-first century life (Trilling & Fadel, 2012, as cited in Campbell & Kresyman, 2015. According to this consortium's research the following are the modern skills and knowledge needed for all students in twenty-first century.
• Collaboration, team work and communication The twenty-first century learning environments should create learning practices, human support and physical environments that will support the teaching and learning of twenty-first century skill outcomes. It should also support professional learning communities that make educators able to collaborate, share their ideas, share the best practices and integrate twenty-first century skills into classroom practices. Students should be enabled to learn in a relevant and real world context and have access to quality learning tools, technologies and resources. These environments should provide twenty-first century architectural designs for groups and team learning besides individual learning. Both face-to-face and online interaction and learning should be supported.
As a result, as null curriculum has been shown to have a crucial role in learners' academic life and surviving in twenty-first century has been regarded as significant and vital, the researchers in the present study aimed to develop and validate a null curriculum questionnaire by the application of the Rasch model using Winsteps 3.73.
Rasch model, named after the Danish mathematician and statistician Georg Rasch, could be considered as a prescriptive probabilistic mathematical concept. Two properties of invariance and interval scaling have made it incredible. These two properties are achieved when the basic assumption of unidementionality underlying the model is met which means that the data fit the model.

Study design
As the purpose of the present study is to search for the aspects of null curriculum in Iranian ELT context in BA level with a special focus on twenty-first century skills and then design and develop a questionnaire for measuring these aspects utilizing both qualitative and quantitative processes, the researchers have deemed the design of the study to be a mixed-method one.
Mixed methods are the best choice for this research because qualitative methods allow the researcher to listen to the views of research participants and support the data gathered in the quantitative part of the research. Data can be collected directly from the source; we can have a rich narrative description of the data and participant perspectives will define what is real. The qualitative part of the study is needed since qualitative methods are unsurpassed for research problems where the variables are unknown and unclear and need to be explored (Creswell, 2005). According to Patton (1990), qualitative studies permit the researcher to approach the issue without being limited by some predetermined categories of analysis which contributes to the depth, openness and detail of qualitative works. The quantitative part of the research will support the data gathered in the qualitative part and increase the trustworthiness of the study. In fact this study design helps the researchers to increase the chance of double-checking and verifying the obtained results.

Purpose and research questions
The purpose of the present study is to gain insight about various aspects of null curriculum in Iranian EFL context in BA level with a specific focus on twenty-first century skills. The key research questions guiding this inquiry are: Q1: What are the probable aspects of null curriculum in TEFL curriculum at BA level in Iranian context? Q2: What are the perceptions of Iranian EFL university instructors towards null curriculum in TEFL curriculum at BA level in Iranian context? Q3: What are the perceptions of Iranian EFL university instructors towards twenty-first century skills?

Context of the study
A detailed description of the setting of the present study is elaborated in this section. The setting of this research is universities in Iran. The reasons for choosing the setting for this research are varied. First, null aspects of educational curriculum is under investigation and this is a broad concept and category, thus all educational settings including public schools and private institutes cannot be taken into account and a more limited and specified setting is needed. Therefore, just universities are selected. Second, an important aim of the present study is to evaluate the university programs and curriculum of ELT at BA level to be able to modify course books and curriculum, to solve the probable problems and challenges that arise at colleges and universities. Third, the curriculum used in different educational situations that is university, institutes and schools are not the same; therefore, we need to take just one into account. Fourth, the researcher's accessibility to all EFL teachers and learners is not possible, so we need to exclude some.
BA program in Teaching English as a Foreign Language (TEFL) in Iran is among the most popular programs in the country. In BA level, learners are offered with some general proficiency English courses such as reading, writing, speaking and listening courses, some specific teaching courses such as syllabus design, coursebook evaluation, teaching principles, theories of language learning and teaching, teaching methodology, language testing, study skills, research and linguistic courses. As some of the participants of the study were not familiar with null curriculum which is the focus of the present study, in the questionnaire they were told that: "Educational institutions have always been dealing with curriculum development as one of the most important areas of language learning and teaching. Null curriculum has been viewed as one of the important kinds of curriculum types which is important due to its absence, being left out or overlooked, but it should not be disregarded. This questionnaire attempts to find out the instances of null curriculum (with a focus on twenty-first century skills) in Iranian ELT context".

Measures
A 48-item questionnaire consisted of various items for investigating aspects of null curriculum focusing on twenty-first century skills in BA level was designed and used in the present study. The questionnaire was developed according to the underlying theories, research and available literature with a special focus on two already made questionnairesthe first was developed by Ravitz (2014) and the second was designed by Ashraf et al. (2016).
Generally items were classified into 10 different constructs namely: 1. Critical thinking and problem solving (including 5 statements such as "in my opinion in TEFL BA classes students' decision making skills are developed"); 2. Collaboration skills (including 5 statements such as "in my opinion in TEFL BA classes, students learn how to listen actively to other members and team working is developed"); 3. Communication skills (including 5 statements such as "in my opinion in TEFL BA classes, students learn how to answer questions in front of others); 4. Creativity and innovation skills (including 5 statements such as "in my opinion in TEFL BA classes, students' curiosity and risk taking skills are developed); 5. Self-direction skills (including 5 statements such as "in my opinion in TEFL BA classes, students' self-direction skills are developed); 6. Technological literacy (including 5 statements such as "in my opinion in TEFL BA classes, students' technological literacy in learning is developed); 7. Global and local connection skills (including 5 statements such as "in my opinion in TEFL BA classes, students learn how to make local and global connections"); 8. Economic and financial literacy (including 5 statements such as "in my opinion in TEFL BA classes, students learn how to make, manage and invest money"; 9. Business and entrepreneurial skills (including 4 statements such as "in my opinion in TEFL BA classes, students learn cooperative education and increase their entrepreneurial thinking through different programs"; and 10. Media literacy (including 4 statements such as "in my opinion in TEFL BA classes, students learn how to use appropriate sources of information when needed". Each item of the questionnaire was rated on a five-point Likert scale from (not really) to (to a very great extent). The whole questionnaire took about 10 minutes to be completed. Table 1 shows the Cronbach's α internal consistency reliability coefficient of the scales.

Participants
A total number of 80 university instructors teaching at BA level in different universities in Iran participated in the present study. Their native language was Persian with English as a foreign language. The participation was voluntary and participants were provided with a simple explanation about null curriculum and twenty-first century skills before their cooperation. All the gathered data remained anonymous and confidential and the participants were told not to put their names or any other identifying information on the survey. All the survey was done in a paper-based format. All the participants were university instructors teaching at BA level and the subject matter of the courses they were teaching was not regarded as a determining factor. Table 2 displays the demographic information of the participants.

Data collection
This study used a survey method to gather information about the aspects of null curriculum in Iranian ELT context in BA level with a special focus on twenty-first century skills; therefore, a questionnaire consisting of 48 items (10 different constructs) and three sections was designed and developed. The first part of the questionnaire dealt with bio data, which elicited personal information of the participants. The second part consisted of a 5-point Likert scale questionnaire in English with choices ranging from "not really" to "to a very great extent". The third part was an open ended question asking the respondents to write any further idea they have about null curriculum aspects in BA level that haven't been mentioned in the inventory.
The validity of the questionnaire was checked by 10 TEFL experts, expert validation and then Rasch was utilized using Winsteps software version 3.73 (Linacre, 2009a) to confirm the construct validity of the questionnaire. Also the reliability was estimated by Cronbach Alpha (.98). The overall data collection procedure ran from June 2018 to March 2019.

Data analysis
For the data analysis, first normality of the data was checked through employing the Kolmogorov-Smirnov test. The reason for using this test was that the sample size was more than 50 (Wayne, 1990). In order to assure the construct validity of the questionnaire, the data were subjected to Rasch analysis using Winsteps software version 3.73 (Linacre, 2009a). For checking the reliability of the designed questionnaire with Likert type scale, Cronbach Alpha was employed.

Findings
The data were subjected to Rasch analysis using Winsteps software version 3.73 (Linacre, 2009a), in order to confirm the construct validity of the "Questionnaire for Null Curriculum in ELT: Focusing on 21st Century Skills" questionnaire.

Individual item characteristics
The items are set from difficult to easy. As Table 3 shows, Item 40 (FL5: Financial Literacy 5) is the most difficult item on the questionnaire while the easiest item is Item 12 (Comm 2). Therefore, the difficulty of item 40 is estimated to be 1.45 logits with the standard error of 0.18. This means that can be 95% sure that the true value for the difficulty of this item situated somewhere between 1.09 and 1.81 logits, i.e., two SEs below and above the observed measure.
Following the criteria proposed by Bond and Fox (2007), the results of Infit MNSQ columns, signify items 40, 39, 36 is not within the acceptable range of 0.60-1.40, moreover, Infit and Outfit ZSTD show items 40, 39, 36, 16, 3 and 1 out of the acceptable range of −2 to 2. So these misfit items should be either deleted or modified because of lack of fit to the model. It needs to be mentioned that all the items fit the Rasch model when all the items' outfit and infit mean square values (MNSQ) are within the acceptable range of 0.60-1.40 and outfit and infit (ZSTD) within the acceptable range of −2 to 2.
The analyses of the items yield an item difficulty range of −1.19 to 1.45 logits with separation reliability of .94. Rasch person estimates ranged from −4.69 to 1.98, with separation reliability of .98. Thus, the study revealed to have a high-reliability value.

Response scale analysis
The category statistics for the 5-point scale are shown in Table 4. Responses were categorized almost with a similar proportion except for the fifth category. It indicated that comparing to categories 1, 2 and 3, categories 4 and 5 were chosen by the smaller proportion of the respondents, thus it is better to merge these two categories.
The category observed average is listed in the third column in Table 4. This is the mean of all persons in the sample who chose that category. It is supposed that observed averages to increase with category values which is the pattern observed with these data.   The infit mean-squares and outfit mean-squares for each category level are the average of the infit and outfit mean-squares associated with the responses in each category. The expected values for all categories are 1.0; and the values above 1.50 are considered problematic (Linacre, 2009a). As it is shown in Table 4, all categories were within acceptable limits. Moreover, thresholds column was shown to be in order (−2.50, −0.77, 0.76, 2.51). Figure 1 provides a graphic representation of the probability curves for all response categories. Each category must have a peak on the curve which means that each category shows a particular section of the measured construct. In this study, all the categories had a peak on the curve, presenting a range of hills; hence, each category represents a unique section of the measured construct.   ideally be located along the whole scale to meaningfully measure the 'ability' of all persons" (Bond & Fox, 2007).
The person-item map revealed that the persons are mainly clustered toward the top of the scale, which means few items target respondents with the great and very great extent of agreement. It seems that merging the 4th and 5th scales leads the questionnaire to show a wider range of abilities (see Figure 3).

Examination of unidimentionality
In order to check the unidimensionality of the scale, global fit statistics were checked by studying patterns in the residuals. The smaller the residuals, the better the data fit the model. It is supposed that the residuals are randomly distributed and uncorrelated (Linacre, 2009a). Principal components analysis (PCA) on standard residuals has been used to check unidimensionality. If the latent trait explains all the information in the data and the residuals describe random noise, the data will fit the Rasch model. On the other hand, if a factor is extracted from the residuals, the test is not unidimensional (Baghaei & Cassady, 2014).
To decide whether the factor extracted from the residuals can be safely ignored or not, the size of its eigenvalue should be considered. The size of the eigenvalue in the first factor is a degree of unidimensionality or overall fit of data to the Rasch model (Smith, 2002). Unidimensionality of the data can be examined through the row "unexplained variance in the 1st contrast" in Table 5. Raiche (2005) has suggested that secondary dimensions which have the strength of at least two items (eigenvalue = 2) are a sign of concern.
Therefore, the eigenvalue of 2.8 for the first contrast in the present study shows that the test is not unidimensional. Moreover, PCA of the standardized residuals (Table 3) showed that the Rasch dimension is as big as 65.3, which explains 88.3% of the variance; 23.4% is explained by item measures, and 64.8% is explained by person measures. In all, 11.7% of the variance remains unexplained.

Follow-up analysis
In a follow-up analysis, categories 4 and 5 were merged into one category and items 1, 3, 16, 36, 39 and 40 were omitted to reanalyze the scale. The result revealed that the 42 remaining items had an acceptable outfit mean-square and infit mean-square fit statistics. PCA of standardized residuals indicated that measures explain 88.5% of the variance and the eigenvalue in the first factor was reduced to an acceptable size which confirms the unidimensionality of the scale.
As Table 6 shows, threshold estimates after deleting the six items and combining the last 2 response scales were −1.81, 0.17 and 1.65. Moreover, the infit and outfit mean squares for each category level were within accepted limits.

Discussion
Since the concept of null curriculum has opened a new window in the area of language learning and teaching which is not suggested by other curricula to students and instructors, the present study aimed to design, develop and validate a questionnaire for null curriculum in ELT focusing on twenty-first century skills using the Rasch model. The analyses using Winsteps software version 3.73 (Linacre, 2009a) addressed the identification of evidence for a unidimensional structure for the questionnaire and confirmation of the efficacy of a 5-point response scale. Findings of the study and examination of item characteristics and response scale quality confirmed that the Rasch model fits the "null curriculum in ELT: focusing on twenty-first century skills" questionnaire after deleting six items from the original 48-item questionnaire, and the internal validity of the test was confirmed.
The 42-item null curriculum questionnaire focusing on twenty-first century skills had an acceptable person separation reliability of .98 and item separation reliability of .94. Moreover, threshold estimates after deleting the eight items were shown to be within the accepted range.
Furthermore, the hierarchy of item difficulties provides important information considering the subscales of the questionnaire and more specifically about null curriculum and twenty-first century skills. As the items were set from difficult to easy, it was revealed that item 40 which is related to financial literacy (i.e. "students learn how to balance their budget and manage financial risks") was the most difficult item in the questionnaire; however, item 12 related to communication skills (i.e. "students learn to answer questions in front of others") is revealed to be the easiest.
Generally, the researchers believe that language curriculum development is an important issue which needs to be reconsidered by policy makers and stake holders. Null curriculum needs to be cared for and neglected or absent courses ought to receive more attention as they could be really effective and beneficial to assist students thrive in today's world.
To sum up, the overall findings of the study confirmed that the 42-item null curriculum questionnaire which has a special focus on twenty-first century skills is an effective unidimensional representation for evaluating twenty-first century skills as aspects of null curriculum.

Limitations and further directions
One of the limitations of the present study could be the fact that the researchers considered only the university instructors in different universities in Iran; thus, other studies could work on teachers and instructors in various contexts such as private institutes and schools in other countries and with different cultures to be able to expand the validation of the questionnaire with a broader examination and as a result generalize the findings. Moreover, the validation process could be conducted with EFL learners in various learning contexts to come up with new insights and probably different results. The present study has had BA level into consideration; therefore, probably other studies could be done taking MA or PhD levels and programs into account. So, aspects of null curriculum in higher education can be taken into account. Other studies could possibly be done using instructors in MA level or PhD level as higher education programs participants. In addition, future researches may try to assess whether the current survey could predict future job success of learners while controlling for all known predictors or not. Besides, the present questionnaire focuses on aspects of null curriculum concentrating on twenty-first century skills; therefore, future studies may focus on other determining factors and skills that are important in the life of people in society.