Development of a questionnaire to assess student behavioral confidence to undertake interprofessional education activities

ABSTRACT Measurement of the impact of interprofessional education (IPE) is the golden chalice educationalists chase. We undertook the development of a scale to measure IPE Academic Behavioral Confidence (IPE-ABC) in allied health, nursing, and social work pre-registration students. This work formed part of the evaluation of a large IPE framework embedded across two Scottish universities. General ABC has been shown to influence student perceptions of study experiences and it is thus reasonable to postulate that ABC could influence student perceptions of IPE. This research developed a questionnaire to ascertain health and social care students’ confidence to engage in IPE, utilizing a mixed method approach. Fifteen different professional groups of pre-registration students (n = 565) participated in the assessment of the 38 item questionnaire. Exploratory factor analysis identified three factors: 1/interprofessional teamwork, 2/behaviors underpinning collaboration, and 3/interprofessional communication collectively accounting for 38.2% of the variance. Internal consistency of the overall scale (Cronbach’s α = .93) was very good with subscales demonstrating very good internal consistency, 1 (α = .89), or respectable consistency 2 (α = .78) and 3 (α = .79). We conclude the IPE-ABC questionnaire could be utilized to enhance and assess the success of IPE related activities.


Introduction
Interprofessional Education (IPE) aims to enable different health and social care professional groups to develop a mutual understanding, trust and respect whilst countering ignorance, prejudice and rivalry by preparing students for collaborative practice (Barr et al., 2017). Literature demonstrates that students react positively to IPE experiences, develop more positive attitudes toward other team members, reduce negative stereotypes of other professions, and become ready to engage in IPE and collaborative practice (Reeves et al., 2016). However, the literature also identifies that these changes do not occur consistently across all professional groups (Reeves et al., 2016).

Factors known to impact student learning
Some of the differences observed within the IPE literature may relate to the process of student learning and the known interaction between student personal characteristics (presage factors), the teaching environment (processes) and student outcomes (product) (Biggs, 1989). Factors that could influence student reactions to, and engagement with IPE are identified in Figure 1.

Self-concepts, self-efficacy and confidence
Self-concepts are key components of academic motivation which impact on the effort students undertake relating to learning interventions and study engagement. Sanchez and Roda (2003) define self-concept as the set of perceptions or reference points that a subject has about one's self. The academic self-concept refers to an individual's knowledge and perceptions about themselves in achievement situations and the views students develop and believe about themselves and their academic capabilities are key determinants in their success or failure within the academic environment (Sander & Sanders, 2006a). Academic self-concepts in specific subject areas affect student intrinsic motivation, persistence and achievement. The academic self-concept combines self-evaluation of perceived academic abilities (self-efficacy) and general feelings of self-worth associated with perceived academic self-confidence (Sander & Sanders, 2006a). Academic selfconcepts could contribute to differing student attitudes toward, perceptions of, and engagement with IPE interventions.
Self-efficacy can be defined as an individual's belief about their capabilities to produce effects and efficacious individuals have confidence that "challenges" are tasks that can be mastered and overcome (Bandura, 1994). Self-efficacy in the academic environment refers to the beliefs individuals hold in relation to their perceived capabilities to self-regulate their learning by utilizing strategies that are systematically oriented toward their learning goals (Usher et al., 2008). Students' selfefficacy is related to motivation and achievement and influences student engagement, persistence and effort in academic learning activities (Williams et al., 2017).
However, student motivation to undertake academic tasks is not only related to beliefs regarding capacity to complete the task successfully, but also the value placed on the task initially. Expectancy-value theory suggests that study motivation is underpinned by student beliefs about not only how well they will do on an activity, but also the extent to which that activity is valued (Sander & Sanders, 2006a).
Confidence is an important element in behavioral master but is situational specific and someone may exhibit confidence in a familiar setting, though may lack confidence in a new environment. Sander (2009) suggests understanding student confidence in the academic setting could be beneficial in creating a more positive learning environment. The construct of academic behavioral confidence (ABC) is underpinned by constructs of selfefficacy, expectancy values, and self-concepts as ABC relates to students' beliefs that they can perform competently in a particular learning situation and the perceived value they place on the task (Sander & Sanders, 2009). ABC is grounded in the general notion of exploring how students differ in the extent to which they have a strong belief, firm trust, or sure expectation of how they will respond to the demands of university. Sander and Sanders (2003) proposed that ABC is important for making sense of student expectations in teaching, learning and assessment. Generic ABC can be assessed using the ABC scale which measures confidence relating to general academic study. The ABC scale was developed in students undertaking general academic study in large lecture-based courses and was developed to measure global academic performance (Ochoa & Sander, 2012). Sanders and Sander (2007) undertook a study using the validated ABC in medical and psychology students and identified that medical students had a significantly higher score on the ABC. These findings were duplicated showing significant differences in ABC between medical, psychology and health care students with the medical students demonstrating significantly more confidence (Sander & Sanders, 2009). Berben (2008as cited in Sander, 2009) found students with low ABC preferred formal lectures and disliked student presentations. Furthermore, gender differences are apparent with male students reporting higher levels of confidence within ABC subscales of grades, verbalizing, and studying (Sander et al., 2011). It is apparent that ABC impacts on students' expectations relating to teaching methods. However, the current ABC scale is designed to measure global academic performance rather than IPE specific proposed outcomes and activities.

Scales to measure confidence in IPE
Validated instruments to measure confidence within IPE related activities are limited. Mann et al. (2012) developed a validated scale to measure health profession students' IPE self-efficacy beliefs in isolation which offers a limited view to student perceived confidence to undertake IPE related activities. The questionnaire also includes items relating to specific modular requirements around experiential learning on clinical practice. Hermansen-Kobulnicky et al. (2018) developed a scale for confidence based on the Interprofessional Education Collaborative sub-competencies. The scale was developed to monitor changes and although it suggested that student confidence increased post IPE intervention limited information was provided in relation to questionnaire development and psychometric properties of the tool.

IPE-ABC scale construction/item development
The purpose of this research was to develop a psychometrically robust questionnaire to measure student confidence to undertake academic based IPE activities. An exploratory sequential mixed method design was employed for scale construction.
Ethical approval for the study was received from Glasgow Caledonian University, School of Health and Life Sciences Ethics Committee.
The researchers explored learning outcomes and competencies that should be achieved by students through an academic IPE interaction. A literature review identified key IPE learning outcomes defined by Thistlethwaite and Moran (2010)  , 2010). A mapping process was undertaken identifying common competencies that underpinned IPE interventions undertaken in the academic setting. These related to teamwork, reflection, communication, roles, responsibilities and ethics. These were deemed as potential constructs that should underpin the IPE-ABC scale.
Contextual relevance of the proposed constructs were explored by conducting a small focus group with three physiotherapy students undertaking IPE modules. Thematic analysis identified students were facing: ethical dilemmas, challenges in teamwork, an evolving understanding of their own and other professional roles and a fluctuating level of confidence to engage in IPE activities. The results suggested the constructs identified within the literature were contextually relevant and had face validity. Item generation was undertaken by the key researcher who explored the literature in relation to the constructs of interest, utilizing existing questions from previously developed tools (Boynton & Greenhalgh, 2004). Student comments were also considered in item generation to optimize question saliency (McColl et al., 2001). An alignment check, mapping questions to the constructs of interest was undertaken (Cox & Cox, 2008) and a self-reported 35 item IPE-ABC scale was developed. The Likert scale used in the pilot study is presented in Figure 2.

Pilot study
A pilot study was undertaken utilizing the tool with 179 first year health and social care students. Face and content validity were assessed by evaluating questionnaire acceptability, missing data and response patterns. Eleven percent of the questionnaires had at least one missing data response but there were only 21 in total. Three questions demonstrated a small number of skipped responses and were found to contain double items. These questions were disaggregated from three to six items. Analysis of the pilot study data suggested students reported different levels of perceived confidence related to professional group.
Content validity was assessed via an expert review. Ten academics rated the scale items as very relevant, somewhat relevant or not relevant. Content validity ratio (CVR) analysis was undertaken to exclude the potential for agreement on items occurring via chance. A critical value of .6 was used to determine if items should be retained and no items were removed. (Lawshe, 1975).
Based on the outcomes of the pilot study and expert review a second 38 item iteration of the IPE-ABC scale was developed and used in the final study.

Final study
The final 38 Item scale was administered to a convenience sample (N = 629)of first year nursing, allied health and social work students. Students rated their perceived level of confidence to undertake IPE related behaviors using the Likert scale as identified in Figure 2.

When working in interprofessional teams on a scale of 1 (not at all confident) to 4 (Very confident), how confident do you feel in your ability to undertake the following:
Not at all confident Very confident Unsure 1. Rely on students from a different professional group to undertake allocated team tasks.

Data analysis
All data was inputted into IBM SPSS v 21.0. Data were checked during this process with a systematic random sample of questionnaires screened for incorrect completion at the time of data entry. All incomplete cells were screened against the original paper questionnaires to ensure the empty cell was a correct response from a respondent and not related to incorrect data entry. Post data entry completion, 10% of questionnaires were randomly selected and screened against their SPSS entry and found to be 98.3% accurate. All errors identified were corrected and the sample had a small enough data input error to be regarded as accurate and fit for further analysis.

Construct validity
Exploratory factor analysis (EFA) was undertaken to identify constructs as the research team had no expectations regarding the number and nature of variables.

Sample adequacy
A Kaiser-Myer-Okin (KMO) and Bartlett's Test of Sphericity were undertaken. The KMO yielded a value of .939 indicating close patterns of correlation within the data signifying EFA should identify distinct and reliable factors. Bartlett's Test of Sphericity yielded a significant result (<.001) indicating items correlated in some way. EFA was undertaken using a maximal likelihood method [MLM] (Costello & Osborne, 2005) allowing for the generalization of findings to whole populations (Field, 2013). Eigenvalues were used to aid factor extraction and factors were retained using the Kaiser criteria (Field, 2013). Factors were retained if they demonstrated an eigenvalue of greater, or equal to one (Beavers et al., 2013). Scree plots were also used to aid factor extraction. Oblimin oblique rotation was undertaken to simplify data and maximize and minimize high factor loadings within the data set (Beavers et al., 2013)

Identification of factors
The factor solution with the simplest structure identifiable within the pattern matrix was the factor solution retained and was established using the following predetermined criteria: Items within the questionnaire were considered a good indicator of a factor if they demonstrated a high loading (>.3) (Hof, 2012), did not significantly cross load onto another factor with values of > .32 (Costello & Osborne, 2005) and each factor also needed to have at least three items that loaded onto it to give it a meaningful interpretation.

Reliability
Cronbach's alpha statistic was used to establish internal consistency of the IPE-ABC scale. Minimally acceptable values were determined to be between 0.65 and 0.7 (Field, 2013). The value of α is dependent on the number of items within a scale and as the number of items increase then so does α. Therefore, corrected item-total correlations were reviewed for complete questionnaire and subscales within the questionnaire as these illustrate how well individual items correlate within a scale. Values above 0.3 were deemed to demonstrate a fair level of correlation and items demonstrating values of less than 0.3 would be considered for deletion (Field, 2013).

Questionnaire analysis confidence
Complete IPE-ABC scale mean scores were reviewed to identify the mean level of Total Confidence for the questionnaire. Mean subscale scores were reviewed to identify the mean level of confidence within each subscale. Mean total confidence and subscale scores were analyzed on the basis of gender, age and professional group to identify difference across groups. The normality of data was analyzed using an assessment of skewness and kurtosis measurement. For normally distributed data parametric testing was undertaken (Independent T test and Analysis of Variance [ANOVA]). Non-parametric (Kruskal Wallis and Mann-Whitney) testing was undertaken to analyze non normally distributed data. Statistical significance was set at p = .05 (two-tailed), with Bonferroni adjustments made for pairwise comparisons of post-hoc tests as necessary. Cohen's d and eta square (η 2 ) were produced as measures of effect size.

Response rates
568 students completed the questionnaires (90% response rate). Three questionnaires were excluded from analysis with one student not completing any of the items in the questionnaire and two not meeting study inclusion criteria.

Missing data
Only 114 cells within SPSS were incomplete. Due to the small number of cells that had no coded input, imputation was undertaken as a statistical technique to deal with missing data. A median replacement method was undertaken so that the value imputed was a whole number and could be interpreted within the remit of the questionnaire response scale. Data was imputed at item level rather than at total score level, as no participant had large numbers of missing data and this allowed the participants valid responses to be included.

Study sample
Of the 565 students included in data analysis, 12.9% of students were male and 87.1% female; 64% of the students were aged 16-21, and 36% of the students were over the age of twentyone. Fifteen professional groups took part in the study. Data for all nursing students irrespective of their branch equated to 251 students accounting for 44.4% of the study sample. Allied health professional students (n = 261) accounted for 46.2% of the study sample. BA (Hons) social work students (n = 53) comprised 9.4% of the population.

Construct validity
An initial EFA was run to obtain eigenvalues for each factor in the data. Six factors had eigenvalues over Kaiser's Criterion of greater than 1 (10.846, 2.199, 1.474, 1.279, 1.200, and 1.103) and in combination accounted for 47.63% of the variance in the correlation matrix. The scree plot showed an inflexion that justified retaining three factors accounting for 38.2% of the variance.
The MLM identified thirty-seven items that loaded onto factors with values of greater than .3. Item 6 demonstrated a loading value of .262. Four items double loaded with loadings greater than .3 (items, 22, 28, 29, 31) however, only questions 28 & 29 loaded onto two factors to a level of greater than .32 and both demonstrated moderate loading on one factor and minimal acceptable loading on the second factor (Table 1) suggesting that questions 6, 28, and 29 should be considered for removal or rewording in further iterations of the questionnaire. Beavers et al. (2013) advocate that items are explored for conceptual significance prior to their removal from a factor. This was undertaken for question 6 (Table 1) and the research team concluded that this result occurred due to the timing of the questionnaire. Students completed the questionnaire in their first IPE tutorial prior to any group interaction and may not have understood the need for rapport. Questions 28 and 29 (Table 1) demonstrated a greater loading on factor two when compared to factor one. However, they did not cross load at a value of > .4 suggesting a moderate loading on two factors and were therefore retained. Items 28 and 29 sat conceptually in factor two as they could be defined as behaviors needed to enhance collaboration and this decision was supported by the fact they loaded onto factor 2 at more than a 0.5 difference.
These results suggested that the IPE-ABC scale had three subscales reflecting the latent variables underpinning the constructs of interprofessional related activities (Figure 3). These subscales were named; • Interprofessional Team Work (ITW).

Internal consistency
The  Factor one Rotated Factor Loadings 1 2 3 1 Rely on students from a different professional group to undertake allocated team tasks.
.305 2 Engage in debate with students from a different professional background.
.388 3 Reflect on your individual performance in relation to team working .479 16 Discuss your understanding of your professional roles and responsibilities with students from different professional groups.
.473 17 Communicate effectively with other students.
.452 .222 19 Use common terminology when communicating with students from different professional backgrounds.
.524 20 Assist interprofessional team members at times of difficulty.
.323 .230 21 Feedback to fellow students in a way that promotes team development when undertaking interprofessional activities.
.481 22 Engage fully in activities undertaken as part of interprofessional team work.
.313 .300 24 Reflect on experiences of working in interprofessional teams .534 25. Recognize interprofessional learning activities with equal importance to your discipline specific learning activities.
.379 .243 30 Recognize if stereotypical judgments are being made in relation to other professional groups.
.411 .245 31 Undertake fully your agreed roles when working with fellow students from different professional backgrounds.
.463 .317 32. Recognize and observe the limitations of your own professional role.
.617 34. Perform effectively in an interprofessional team as a leader when undertaking an academic task.
.546 −.202 35. Recognize the competence of other professions in relation to your own.
.535 37. Feed back to fellow students in a way that promotes positive team interactions when undertaking interprofessional activities.
.666 38. Utilize interprofessional experience to guide your future learning.
.440 Factor two 1 2 3 4 Listen to the views of students from different professional backgrounds.
.395 5 Demonstrate respect for the skills of fellow students from different professional backgrounds.
.498 12 Demonstrate you value the views of fellow students, even if they differ from your own.
.368 23 Demonstrate respect for the roles and responsibilities of other professions in relation to your own.
.530 26 Recognize students from different professional backgrounds as equal to students from your own professional background.
. .377 13 Resolve conflict effectively when working with fellow students, even when conflict becomes personal.
.497 14 Verbally confront fellow students who express negative judgments against other professional groups.
.451 15 Identify issues that have led to team breakdown.
.514 total questionnaire analysis, supporting its removal. However, when considering question 5 on a subscale basis its corrected item-total correlation value was .363 suggesting it demonstrated internal consistency within the Behaviors Underpinning Collaboration subscale and supported its inclusion. Furthermore, removal of the item from the subscale yielded a α = .77 value, which did not serve to improve the Behaviors Underpinning Collaboration subscale reliability score (α = .78).

Questionnaire analysis
Scoring criteria were developed for the IPE-ABC Questionnaire. Respondents could obtain a maximal score of 152 and a minimal score of 0, for their total self-reported perceived confidence to undertake IPE related activities. In relation to subscales, the "Interprofessional Team Work" subscale contained twenty items whereas both the "Behaviours Underpinning Collaboration" and the "Interprofessional Communication" subscale contained nine items each. Respondents on the Team Work subscale could therefore obtain a maximal score of 80 and a minimal score of 0 for their perceived confidence to undertake IPE team work related behaviors. For the other subscales the maximal score that could be obtained was 36 with a minimal score of 0. Scores and descriptors for the total confidence and confidence subscales used within the current study are presented in Table 2.

Results: total confidence (TC)
The data obtained demonstrated for the total sample an overall mean TC score of 120.6 (95% confidence interval [CI] 119.40--121.82) suggesting students overall demonstrate a "confident" level of self-perceived confidence (Table 3). This pattern was evident irrespective of age or gender. Thirteen out of 14 professional groups also identified as being confident. Only Radiotherapy and Oncology students rated themselves as "moderately confident". No significant differences were identified in TC scores based on gender, age or professional groups. Table 3 The pattern of students reporting to be "confident" was also evident within the ITW subscale ( -45-30.82). No significant differences were identified within IC subscale scores based on age. Significant differences were evident for gender with male students demonstrating higher levels of confidence when compared to female students and professional groups with a weak to moderate effect size. A Tukey's post hoc analysis revealed that statitiscally significant differences occurred between the speech and language pathology and social work students at the p < .05 level, however once Bonferroni adjustment was made for multiple pairwise testing against a p < .00055 threshold this relationship was non-significant (Table 3).

Discussion
We have reported the process and results of the development of a psychometrically robust scale to measure Pre-Registration student self-reported confidence with regard to the performance of IPE academic related activities. Data reduction techniques resulted in the identification of three factors underpinning the IPE-ABC Questionnaire: Interprofessional Team Work, Behaviors Underpinning Collaboration and Interprofessional Communication. The questionnaire demonstrated internal consistency and results suggest the scale may be able to discriminate between gender differences on a subscale basis. The distribution of scores across the scale was looked at in detail as part of questionnaire analysis. Thirty-five items in the study demonstrated highest frequencies in the 3.00 and 4.00 end of the scale, a positive skew of responses toward the "very confident" end of the scale. The potential of measurement error due to social desirability response bias [SDRB] and acquisition bias cannot be fully excluded (Rattray & Jones, 2007;Van De Mortel, 2008). SDRB can occur for a variety of reasons, the individual may self-deceive, or present a positive answer to conform to socially acceptable values, avoid criticism or gain social approval (Van De Mortel, 2008). SDRB is most likely to occur in response to socially sensitive questions and as some questions within the IPE-ABC scale may be classed as sensitive it is possible that students may feel they cannot disclose a lack of confidence in certain behaviors. However, the questions did draw responses in the slightly confident end of the scale and the questionnaire was a self-reported scale which is acknowledged as a method to decrease SDRB (Van De Mortel, 2008) The use of counterbalancing for items within a questionnaire is advocated to minimize the chance of acquisition bias. As the IPE-ABC Questionnaire was measuring behaviors this was considered, however, was not undertaken in order to optimize questionnaire simplicity and minimize questionnaire length (Rattray & Jones, 2007).
Other factors such as over-confidence bias could contribute to the results. A similar pattern was identified by Mann et al. (2012) when measuring self-efficacy (a construct underpinning ABC) in relation to IPE with mean values of items falling within the higher part of the scale of their 0-10 scale.
Over-confidence bias was a factor within the original ABC scale (Sander, 2009;Sander & Sanders, 2003) with students reporting high levels of confidence on the commencement of study at University which then decreased over time. Sander and Sanders (2009) hypothesize that the decreasing trend in scores over time was related to unrealistic student expectations about their ability at the start of university, which dropped due to adverse course experiences. This has been observed in other IPE studies looking at students' readiness to engage within IPE (McFadyen et al., 2010), attitudes toward IPE (Coster et al., 2008) and Professional Identity (Tunstall-Pedoe et al., 2003). More recently Williams et al. (2017) noted differences in selfreported self-efficacy dependent on the year of study.
Within the IPE literature students tend to assess themselves positively in relation to their attitudes toward IPE and their ability to undertake IPE interactions, on commencement of training which is consistent with the findings in the current study. However, these positive perceptions in ability and attitude appear to decrease over time (McFadyen et al., 2010;Williams et al., 2017). As students' progress through more complex and challenging aspects of the curriculum with a greater exposure to interprofessional working this may impact on their self-concept and self-efficacy and thus over all confidence. Further research should establish the longitudinal patterns in relation to student confidence trends.
Male students demonstrated significant greater levels of self-perceived confidence within the Interprofessional Communication subscale consistent with the findings of Sander and Sanders (2009) and Williams et al. (2017). The original ABC questionnaire within health and social care students, demonstrated that male health and social care students were more confident within three of the four ABC subscales (Sander & Sanders, 2009). Williams et al. (2017) found male students demonstrated greater levels of selfefficacy, in relation to interprofessional evaluation and feedback. Literature suggests male students seem to be more prone to over-confidence bias (Sander & Sanders, 2006b). Sander and Sanders (2009) hypothesize that this is due to males being more self-centered and less attuned to social interaction issues. This could account for why male students within the current study demonstrate higher perceived levels of confidence in relation to team work which is reliant on social interactions.

Limitations
Response scale construction within questionnaire development can be a source of measurement error (Artino et al., 2014). We anchored only the end points of the questionnaire, possibly increasing measurement error as it is difficult for the respondents to interpret the unlabeled options. We also chose a fourpoint scale to mitigate the chance of central tendency bias and this could have forced respondents to choose higher ends of the scale in relation to confidence increasing the chance of measurement error.
The other limitation of this study relates to the ability of a single measurement to determine the stability in the factor structure and reliability underpinning the questionnaire. These problems have been highlighted as flaws in one of the most common measures used within IPE, the RIPLS. The RIPLS scale seems to lack a stable factor and reliability structure, with studies demonstrating inconsistent findings in relation to factor stability and also weak and inconsistent internal consistency (Schmizt & Brandt, 2015). Issues with stability may be related to the instrument itself or due to differences in populations being surveyed, or the timing and administration of the tool. Thus, at the current time although the results look favorable for the IPE-ABC Questionnaire, more psychometric testing is needed to confirm the above findings.

Conclusion
Confidence is a factor that is known to impact on intrinsic motivation to study. Confidence is underpinned by the constructs of self-efficacy, the self-construct and expectancy and value theories. It is not known if confidence impacts on students' engagement with IPE. This work has developed a psychometrically robust tool to measure self-reported confidence for IPE in academic settings, and will be useful in determining the impact of IPE in the academic environment.
EFA identified 3 factors that underpin the scale which relate to team working behaviors, behaviors underpinning collaboration and communication. The tool demonstrates good levels of internal consistency overall and good or respectable levels of internal consistency for the subscales within the IPE-ABC Questionnaire. The questionnaire demonstrates some level of discriminative ability in relation to differences seen based on gender in the Interprofessional Communication subscale. It may however also demonstrate an element of SDRB, acquisition or over-confidence bias, but it is difficult to determine if this is a result of measurement error, or is reflective of student naïve perceptions of their ability to undertake IPE related behaviors. More work is needed to confirm the repeatability of the current findings in different cohorts and over longitudinal timeframes.