Chinese doctors’ views on workplace-based assessment: trainee and supervisor perspectives of the mini-CEX

ABSTRACT Purpose: This study investigated whether the mini-clinical evaluation exercise (mini-CEX) has been successfully integrated into the Chinese context, following its introduction as part of the national general training programme. Materials and methods: Online questionnaires (N = 91) and interviews (N = 22) were conducted with Year 1 trainee doctors and clinical supervisors at a cancer hospital in China to explore users’ experiences, attitudes and opinions of the mini-CEX. Results” Trainees were more likely than supervisors to report understanding the purpose of the mini-CEX and agree that it encouraged reflection and helped improve overall performance. Both trainees and supervisors felt that it provided a framework for learning, that it was useful in identifying underperformance, and that it informed learning progression. Groups were equally positive about the commitment of their counterpart in the process and valued the focus on detailed feedback. It was perceived as cultivating the learner–teacher relationship. Overall, both groups felt they ‘bought in’ to using the mini-CEX. However, concerns were raised about subjectivity of ratings and lack of benchmarking with expected standards of care. Conclusions: Chinese trainees and supervisors generally perceived the mini-CEX as an acceptable and valuable medical training tool, although both groups suggested enhancements to improve its efficacy.


Introduction
The nationwide implementation of the Standardised Resident Training Program in 2017 [1], introduced a range of workplace-based assessments (WBAs) into Chinese postgraduate medical education [2]. The 'mini-CEX' (mini clinical evaluation exercise) is one such assessment [3,4], involving direct observation of a trainee-patient encounter, with ratings and feedback given by the supervisor across domains such as 'history-taking', physical examination, communication, clinical judgement, professionalism and organisation. With strong theoretical validity, the mini-CEX is used internationally [5,6].
When implemented properly, the mini-CEX has positive educational value in clinical training [7,8], but outcomes can be constrained by trainee and supervisor knowledge and attitudes [9]. Acceptance of the mini-CEX differs among users [10,11], but few studies have examined concordance of trainee and supervisor perspectives [12,13].
Furthermore, the acceptability of educational innovation may determine the utility of the mini-CEX in new cultural contexts [14,15]. Studies in southeast Asia have indicated variability in shared understanding and acceptance of the mini-CEX [16,17]. Little is known about how the mini-CEX is implemented in China, as few studies have discussed its validity and reliability [18,19]. As the mini-CEX was designed for use in Western healthcare education, this highlights a need for further investigation into its acceptability and use in practice.
This study therefore aimed to investigate perceptions of the mini-CEX among Chinese trainees and supervisors, particularly considering users' understanding of this assessment and attitudes towards it.

Setting
The host institution was a teaching hospital specialising in the radiological, surgical and medical treatment of adult cancer situated in one of the most developed cities in mainland China. This served a mixed urban and rural population comprising demographically and ethnically diverse communities. Since 2017, The institution introduced WBAs as supplement to the annual exam, which consisted of a knowledge test and Objective Structured Clinical Examination [OSCE). The teaching faculty was therefore offered training of WBAs -including mini-CEX -on an annual basis.

Measures
A 15-item questionnaire was designed to examine trainees' and supervisors' experiences, attitudes and understanding of the mini-CEX, including questions with high reliability adapted from [20,21]. The questionnaire showed good internal consistency, achieving Cronbach's alpha of 0.95 for both groups [22].
Semi-structured schedules including 9-12 questions for a 30-min interview were devised to explore participants' perspectives in more depth [9]. Pilot testing of measures was conducted with 12 volunteers (6 trainees, 6 supervisors).

Procedure
Participant recruitment was conducted through advertisements (via all-staff email, teaching sessions and staff noticeboards) providing a link to the online questionnaire. An invitation to participate in the interviews was included at the end of the questionnaire; volunteers were recruited until the requisite number was reached. Interviews were conducted in a meeting room at the host institution, scheduled as convenient for the participants and audio-recorded with permission.
Qualitative analysis of the interview data was undertaken using principles described by [23]. The interviews were transcribed verbatim by a researcher (YL) to capture the richness of the data set, and read and re-read to identify latent themes reflecting participants' perceptions, interpretations and attitudes. Initial codes were extracted and free coded in NVivo, before the data were actively searched for themes, and codes grouped into themes by consensus (YL and LN).

Ethics and data processing
The project was approved by UCL Ethics Committee and the Ethics Committee of Affiliated Cancer Institution and Hospital of Guangzhou Medical University. All participants confirmed informed written consent. Questionnaires were returned anonymously. Audio-recorded interview data were deleted immediately upon transcription. Analysed data contained no identifiable information.

Trainee and supervisor experience of the mini-CEX
The majority of trainees (35/43, 81%) indicated a maximum of two previous assessments with the mini-CEX, whilst supervisors had had more experience (Table 2).

Trainee and supervisor perceptions of the mini-CEX from the questionnaire
Trainees and supervisors differed in their responses (Mann-Whitney U, z = 3.6, p < 0.001). Trainees were more likely to report a clear understanding of the purpose of the mini-CEX (60% 'agree' or 'strongly agree') compared to supervisors (48%) ( Table 2). On a practical level, trainees were also more likely to report that supervisors initiated the mini-CEX (79% trainees, 48% supervisors) and that there was sufficient time for each assessment (70% trainees, 48% supervisors). The groups showed similar levels of agreement that the patient encounters were directly observed (65% trainees, 67% supervisors) and that feedback was provided after each encounter (70% trainees, 69% supervisors), although supervisors were more likely to report that feedback was immediate (75% supervisors, 65% trainees). The majority of participants in both groups reported that feedback was specific (81% trainees, 75% supervisors) and highlighted weaknesses in performance (74% trainees, 69% supervisors). Trainees were more likely to report that the mini-CEX encouraged them to reflect (77% trainees, 60% supervisors) and helped to improve their performance (74% trainees, 56% supervisors). The majority of both groups felt that the score reflected the quality of the trainee's performance (65% trainees, 69% supervisors), that the mini-CEX was useful in identifying underperformance (77% trainees, 70% supervisors), and that it informed learning progression (74% trainees, 69% supervisors). The groups showed similar levels of agreement that their counterpart in the process showed commitment during the assessment (67% trainees, 69% supervisors) and that overall, they 'bought in' to the use of the mini-CEX (70% trainees, 69% supervisors).

Trainee and supervisor perceptions of the mini-CEX from the interviews
Themes related to perceptions of the mini-CEX, its perceived educational impact and attitudes towards its acceptability ( Table 4).

Perceptions of the mini-CEX
Participants reported diverse perceptions of the purpose of the mini-CEX, in terms of its various roles as an aid to learning, trainee progression or administrative monitoring. Trainees and supervisors expressed concerns about the reliability of scoring, noting the subjectivity of ratings, and that overly generous scores may be given as a means of encouraging trainees. Interviewees commonly highlighted the need for further training and benchmarking of scores to expected standards.
There was general consensus that initiation of assessment by mini-CEX was the responsibility of the clinical supervisor, although there were diverse views about its timing. This partly reflected supervisors' views on its formative role, e.g. as a periodic check on progress, or to review the standard of performance attained towards the end of an attachment. However, service pressures also presented challenges to supervisors' availability.
Almost every interviewee confirmed that trainees were always observed and that observations were used to generate feedback. While supervisors used observation to identify specific areas to improve, trainees felt that being watched helped to create a sense of learning.
Similarly, almost all interviewees agreed that feedback was given after each patient encounter, and all confirmed that feedback was honest. The majority of interviewees reported that feedback was detailed and specific, often providing explicit examples. It was accepted that a key aim of feedback was to correct incorrect practice, in the context of a positive and encouraging approach.

Educational impact of mini-CEX
Interviewees consistently described the mini-CEX as a framework for learning and teaching, particularly noting its comprehensiveness and value in ensuring that all domains were covered. Positive benefits were identified in terms of its effects in stimulating reflection, strengthening the learner-teacher relationship, providing a sense of personal achievement, and aligning learning with clinical practice.

Acceptability of mini-CEX
No trainees showed negative attitudes towards the assessment, although one reported that some supervisors were 'sloppy' in their approach. Two supervisors had misgivings about its structured approach and focus on a single encounter. Generally interviewees reported 'engaged commitment' and expressed a willingness to 'try it more'. Four interviewees expressed a neutral stance, feeling the mini-CEX was generally acceptable, but that some refinement (such as standardisation) was needed.

Attitudes towards the mini-CEX
The present study found a strong positive attitude towards the mini-CEX among the majority of trainees and supervisors. Trainees valued the mini-CEX more than supervisors, perhaps due to the direct impact on their training, which echoes some earlier studies [20,24,25]. Although some investigators have discussed poor engagement with the mini-CEX [21,26], the majority of participants perceived their counterparts as showing a committed attitude. This may be dependent on who is chosen as the assessor [12,21]. In this context, the clinical supervisor is the default assessor, and the pre-existing relationship may enhance commitment. Traditional Chinese culture values a close bond between the learner and teacher [27]. The present study found that participants viewed organising and leading the mini-CEX as the supervisors' responsibility, with the trainees' job to learn and correct mistakes. Such notions create a consensus, resulting in mutual commitment in teaching and learning. Paradoxically, cultures that value doctor autonomy and independence may be less conducive to observation and constructive feedback between members [28,29].

Mediated tool for teaching and learning in the workplace
The majority of trainees and supervisors perceived the mini-CEX as meaningful, and helpful in avoiding oversights. Teaching and learning in clinical settings are often unstructured due to clinical demands [30]. Any tools that clarify the processes for both learners and teachers are, therefore, essential for learning [31]. The mini-CEX may be particularly helpful for clinical supervisors because the majority are not professional teachers. Its utility in recognizing strengths and underperformance in a number of domains suggests it has value in identifying teaching opportunities across a broad spectrum [32]. The structured approach also helps with assessment, aiding supervisors in recalling their observations and translating their impressions into ratings [33]. The mini-CEX promotes learning through handson experience with real-world tasks [25,34]. This is likely to be the greatest advantage perceived by Chinese trainees, as hands-on opportunity is often limited by the strict power distance in social norms as well as the low-trust relationship between Chinese patients and junior doctors [35].

Making sense of assessment through interaction
This study confirmed previous findings that observing performance allows supervisors to identify learning needs [36] and generate feedback that promotes learning [37]. Supervisors were identified as active information processors and goal directors [33,38], sharing their expectations with trainees following observation [39,40].
Trust is pivotal when analysing performance. Observation offers trainees feedback on the emotional climate of the doctor-patient encounter [39,41]. Trainees also know that supervisors will help if needed. For supervisors, direct observation allows them to ensure patient safety while gaining insight into the trainee's progress [39].
Both trainees and supervisors valued the element of feedback, which is consistent with evidence that feedback is a powerful tool for learning [42,43]. 'Task-oriented' feedback dominated, consistent with previous studies showing that for trainees, contentspecific feedback is a 'shared conversation', essential to making sense of learning goals and aiding understanding [44].
Most published literature criticises the 'polite feedback culture' for withholding honest and constructive feedback in order to 'save face' [45]. However, this study indicated that users valued feedback focusing on identifying weaknesses to improve performance. This could reflect cultural norms regarding roles in learning, in which teachers are leaders, guides and masters [27]. This finding also resonates with recent research in East Asian countries that support the educational value of the mini-CEX [15,16]. Traditional teacher-centred Chinese and medical cultures highlight a power differential [15,27,29,46]. Within this context, a supervisor's expectations and goals represent parts of the curriculum [41] and influence the trainee's motivation for learning. Consequently, feedback serves as a mediating tool about expectations of acceptable performance [47].
Study participants noted the subjectivity of mini-CEX scoring, which can compromise its utility [17,21]. This highlights a tension between criteria that focus on authenticity and trustworthiness rather than psychometric models that strive to objectify performance and yield reproducible, generalisable judgements [2,10]. Users appeared to operate from differing assumptions of how 'standardised' assessments should be [48]. While many trainees felt confident about the mini-CEX rating standard, fewer supervisors felt the same. This divergence may reflect the implicit perception that ratings are summative [2]. This poses a dilemma for supervisors, who also felt a need to encourage trainees at their formative career stage with lenient scoring. It is possible that supervisors held beliefs that were more consistent with traditional principles of scientific measurement [48]. Historically, medicine and science have reflected a positivist paradigm, in which scientific measurement aims to find objective truth [49]. Consequently, it may be unsurprising that supervisors are concerned about the 'robustness' of using narrative text for the final judgement [2]. However, the majority of trainees admitted that they were not as concerned about the score because feedback was more direct and useful. Previous studies have not adequately discussed this, possibly because participants believe that judgement in performance assessments such as the mini-CEX is 'in situ' and, therefore, can only be understood in context [37]. Trainees may see variations in rating as a source of information about their performance, interpreted as a reflection of supervisors' experiences [50]. Thus, trustworthiness and authenticity become key players in constructing the meaning and criteria of assessment [2,51]. These are the two most important attributes in Chinese cultural norms for teaching and learning [46], which may mediate users' concerns with 'objective ratings'.

Challenges
The mini-CEX is predicated on the notion that assessment drives learning [8,52]. However, the current study indicated that participants did not fully understand the formative purpose of the mini-CEX, particularly supervisors, a fact that has been acknowledged in numerous studies [8,12,21]. [53], noted that stakeholders' beliefs in the assessment shape the way they use it. The current study suggests that participants see the mini-CEX as serving mixed purposes. As the majority of mini-CEX evaluations occur towards the end of a rotation, both groups were aware that the assessment outcome could impact the trainee's progression. Despite these worries, both parties used the mini-CEX as a diagnostic tool and means of identifying underperformance.
The current study identified that time constraints could undermine the perceived educational benefits of the mini-CEX. Trainees may challenge the value of a mini-CEX conducted retrospectively or without direct observation, a repeated finding in global studies of mini-CEX implementation [54].

Implications and future research
The present study indicated that Chinese participants held positive views about the educational value of the mini-CEX, but there were unresolved concerns about its purpose (formative/summative) and the subjectivity of scoring. Further research could examine how users navigate the potentially competing aims of this process. Future studies may consider other healthcare settings in China [14,55] to examine generalisability of these findings in this new cultural context.

Conclusion
This study echoed previous research suggesting that formative purpose, direct observation and real-time feedback are crucial components for ensuring effective utilisation of the mini-CEX. Overall, the mini-CEX was welcomed and accepted by Chinese users, although users did not completely understand the notion of formative purpose. The educational value of the mini-CEX appears to be strongly associated with the prevalent cultural norm of learner-teacher relationships based on trust. This study showed that direct observation creates a sense of safety for learning, and moreover, observation is bidirectional as it allows trainees to observe supervisors illustrating corrections.
Some unique cultural attributes about feedback characteristics were revealed, such as preference for mentioning weaknesses and demonstrating correction.
Adequate training for all participants and sufficient time allocation for the mini-CEX were highlighted as implementation needs.

Practice points
• The utility of mini-CEX is dependent on its acceptability among users and shared understanding of its purpose. • Overall, the mini-CEX is welcomed and generally accepted in China, with trainees somewhat more positive than supervisors. • A trustworthy learning-teaching relationship enhanced the educational value of the mini-CEX. • Faculty training and ensuring time allowance can optimise the full educational potential of the mini-CEX.