A qualitative analysis investigating competence assessment of undergraduate nursing students

Background This study reports on the process of competence assessment in undergraduate nursing programmes in New Zealand. Aim To identify current competence assessment practice and determine how competence assessment is constructed in order to reflect student development. Design This research utilised a qualitative exploratory-descriptive design. Participants Nurse educators from nine tertiary institutions opted to participate in the research. Methods Semi-structured interviews were conducted and data were analysed using a thematic analysis approach. Results Three themes are reported in this article; clinical assessment pedagogy, measure of competence, and relational assessment practice. There was significant variation between and within institutions in conducting competence assessment while scaffolding competence assessment throughout the degree programme was highlighted as a challenge. Conclusions This study highlights the need for an assessment process that allows for the developing competence of nursing students and competencies that are designed to adequately assess students in each year of the nursing programme.


Impact statement
There is a high degree of variation in the competence assessment of undergraduate nursing students and a need for appropriate scaffolding of clinical assessment in order to appropriately reflect nursing student development.

Plain language summary
Interviews have been conducted with participants from tertiary institutions that offer the undergraduate nursing degree in New Zealand to ascertain how they undertake clinical assessment.Nursing students are assessed in each clinical placement throughout the nursing degree in order to determine if they are safe and competent to progress.As all nursing students are

Introduction
In New Zealand students in nursing programmes typically undertake clinical placements within the areas of primary care, medical/surgical, and mental health.Over the three-year undergraduate degree programme students are required to undertake a minimum of 1100 h clinical practice.There are two main models of clinical teaching and learning in New Zealand; the Preceptorship model and the Dedicated Education Unit model which are discussed in further detail in Borren & Harding, 2020.There is currently no standardised method of assessing clinical competence for nursing students in Aotearoa New Zealand.Many institutions use the Nursing Council of New Zealand (NCNZ) competencies for registered nurses as a framework for clinical assessment, (Nursing Council of New Zealand, 2007), however, this framework was not designed for the purpose of assessing nursing student competence.Clinical Managers (or equivalent) from 16 out of 17 institutions offering the Bachelor of Nursing programme in New Zealand were invited to take part in a semi-structured interview to investigate current competence assessment practice, (the final institution could not be invited due to the first author being an employee of this institution at the time, which provided a conflict of interest).This article reports on phase 1 findings of a 3-phase study investigating competence assessment in the Bachelor of Nursing programmes in Aotearoa New Zealand.Determining the suitability and consistency of undergraduate clinical assessment practice is paramount to patient safety in view of the significant and expanding role of registered nurses in the health care system both nationally and internationally.Understanding how clinical competency assessment operates in practice is relevant for nursing education to ensure that students who are assessed as competent, are in fact competent, as this has implications for patient safety and healthcare outcomes.In New Zealand, the government announced a restructuring of the Institutes of Technology and Polytechnics (ITP) sector in 2018.Te Pūkenga, a single national vocational education provider was established in 2020 merging New Zealand's ITP and transition Industry Training Organisations (ITO) into one national tertiary institution by 01 January 2023 (Te Pūkenga, 2022).As 13 of the undergraduate nursing degree programmes in New Zealand are delivered through ITPs this has prompted a shift from a largely competitive to a more collaborative model in the delivery of undergraduate nursing education and this has implications for future Bachelor of Nursing curriculum design in New Zealand.Looking at clinical competency assessment is relevant in cases where standardisation of curriculum, clinical practice or merger of educational providers is occurring.

Literature review
Key topics identified in review of the literature included the impact of workload and resource pressure in conducting competency assessment, inconsistency in assessment practice, and relationship building between nursing students and assessors.A number of challenges in conducting competency assessment were identified in the literature and many studies discussed the difficulty for preceptors of conducting the assessment due to time limitations and workload pressure (Bradshaw et al., 2012;Burke et al., 2016;Cassidy et al., 2012;Hallin & Danielson, 2010;L'Ecuyer et al., 2018;McCarthy & Murphy, 2008;Webb & Shakespeare, 2008).Preceptors reported struggling to find dedicated time to focus on the clinical assessment while still being responsible for a Contemporary Nurse patient load and this was found to adversely impact the quality of the assessment (Helminen et al., 2016).This pressure was intensified by nursing staff being concurrently responsible for multiple students who were working at different levels in the programme (Cassidy et al., 2012).Nursing preceptors have reported performing clinical tasks themselves as opposed to providing students with learning opportunities, due to time constraints, potentially impacting the quality of the clinical placement and the assessment process for the student (Hallin & Danielson, 2010).
Inconsistency in clinical assessment has been reported and students describe the expectations regarding preparation for clinical assessment as being vastly different between preceptors (Bradshaw et al., 2012;Burke et al., 2016;Butler et al., 2011).This could be reflective of the varying levels of education and preparation for the preceptorship role (Burke et al., 2016;Butler et al., 2011;Engstrom et al., 2017;Franklin & Melville, 2015;Hallin & Danielson, 2010;Lovrić et al., 2017;L'Ecuyer et al., 2018), and highlights the subjective nature of the competency assessment process (Cassidy et al., 2012;Franklin & Melville, 2015;Helminen et al., 2016).The clinical assessment process can be inconsistent and often affected by the personal characteristics of each of the individuals involved; student, preceptor, and clinical lecturer (Helminen et al., 2016).Desirable personality traits in students, such as kindness and confidence can influence competence assessment (Burke et al., 2016).Subjectivity in clinical assessment impacts on reliability and validity of the assessment (Franklin & Melville, 2015;Helminen et al., 2014;Helminen et al., 2016;McCarthy & Murphy, 2008).There is also ambiguity regarding what could be considered incompetent practice (Levett-Jones et al., 2011;L'Ecuyer et al., 2018).
Forming effective relationships between students, preceptors and clinical educators was identified as crucial to clinical assessment (Cassidy et al., 2012;Hallin & Danielson, 2010;L'Ecuyer et al., 2018;Webb & Shakespeare, 2008).Students have identified the success of a clinical placement as being highly dependent on the strength of the relationship that is formed with their preceptor (L'Ecuyer et al., 2018).Preceptors and clinical lecturers should not underestimate the impact that they have as role models on students, Lovrić et al. (2017), with the onus on developing positive relationships often falling to the student (Hallin & Danielson, 2010).Webb and Shakespeare (2008) describe the investment of 'emotional labour' that is required by students to build a positive relationship with their preceptor to promote a successful outcome when clinical assessment is undertaken.

Aim
To identify current competence assessment practice and determine how competence assessment is constructed in order to reflect student development.

Design and participants
This research utilised a qualitative exploratory-descriptive design.Nine tertiary institutions out of the 16 invited participated in the research resulting in 10 semi-structured interviews (one institution had two people in a relevant role, who opted to be interviewed separately).This included a range of large urban and smaller regional institutions.Interviewees held roles that were directly related to the management of and/or direct involvement with the undergraduate competence assessment process, although their actual job titles and responsibilities varied.Interview questions were designed in collaboration with highly experienced nursing educators and covered the structure, design, and delivery of clinical competency assessment, as well as the quality assurance and expected levels of student evidence of clinical competence.The participants were emailed the schedule of interview questions a week prior to the interview.Interviews were conducted face-to-face at the respective institution, only the participants and the first author were present.Interviews ranged from 45 to 90 min.

Ethical considerations
Ethical approval for the research was obtained from the University of Canterbury Human Ethics Committee (HEC 2018/93).Participants received the information sheet as part of the recruitment email, and upon expressing willingness to participate were sent both the consent form and the interview questions prior to the interview taking place at the participant's institution.Audio recordings of the interviews were deleted once an electronic transcription had been created.Transcripts were checked by all participants for accuracy prior to analysis with minor amendments requested by one participant.Individual comments were deidentified and institutions were grouped by size and whether they were regional or urban, and not identified individually.

Data analysis and interpretation methods
Thematic analysis was undertaken to analyse the interview data (Braun & Clarke, 2013).The first phase involved a process of total immersion in the collected interview data.The audio recordings were listened to (prior to being deleted), and the typed transcripts were read and reread for further familiarisation with the data and notes were taken.The transcripts were transferred to NVivo 12 by the first author and systematically coded in relation to the significance of the data to the research questions.There were 234 initial codes identified.Thematic maps were created to illustrate how codes were aligned with identified sub-themes and finally, thematic clusters were created to illustrate the relevance of specific themes to the research questions resulting in key themes being derived from the data.To reduce the risk of overinterpretation and unconscious bias, all maps, results, and interpretations of the first author (a nurse educator) were checked by the other authors, one of whom has no professional background in nursing.

Results
Three themes that emerged from the semi-structured interviews are reported in this article.These were clinical assessment pedagogy, measure of competence, and relational assessment practice.

Clinical assessment pedagogy
There was substantial variation between institutions when it came to determining and recording the outcome of competence assessment.In particular, variation in assessor practice was identified as a key challenge and this was evident from within and between institutions.
The one thing about assessment of competence is that it is very much the standard of the individual nurse, whether she's a preceptor or a clinical educator, determining what is an acceptable level, I think … .I just think that that always feels like it's a challenge, because I don't know that all nurses have the same sense of what is competence.
Participants identified challenge in relation to multiple institutions placing multiple levels of students out in clinical areas at the same time, with varying scopes of practice, methods, and tools for assessing their competence.

Contemporary Nurse
… it's very difficult for our clinical placement providers, being asked to often work now with multiple tertiary programmes coming into their clinical setting.It's very difficult for staff to get their minds around that and they're often working not just with different programmes but with different levels of students, because of different stages of the programme that they're coming into the setting, for whatever reason.
It was felt that orientation to the placement area was important in enabling the nursing student to feel a greater part of the healthcare team and that part of the orientation involves introducing the assessment expectations so that students, (and staff), are aware of how competence in the area develops and can be met.It was recognised by participants that education for preceptors and clinical nurse educators was an important factor in helping nursing students apply their theoretical knowledge within the clinical environment, and to quality assure the clinical competency assessment process.
I think more integration with our RN preceptors so that they understand how the student's learning and what our expectations of that student learning is.Because some sign without much thought.I think they just sign because the students put them in front of them.So I suppose it's about us educating the preceptors about the standard that is expected.
There also appears to be some disjoint between the outcomes of the competency assessment (i.e.students being signed off as competent to practice), and being ready for the nursing workplace.There can be an expectation from clinical areas that nursing students are prepared in their undergraduate degree to the extent that they can 'hit the ground running' when they enter the nursing workforce.There appears to be some conflation between being 'competent' and being 'experienced' as a nurse.This work-ready concept was questioned whereby it was felt that this mindset places an unrealistic expectation on newly qualified nurses.
I think we need to look at having some agreement of what constitutes workplace readiness, because we hear a lot about that.I get really fascinated by conversations about graduates not being workplace-ready, yet they complete so many clinical hours in the very workplaces that they're going to be employed into.And I think, well how can we not be getting it right that they're not perceived as being workplace ready?Time constraints were identified as an issue related to the impact of clinical placement length on nursing student experience.One participant questioned the validity of an assessment when there is limited time to work alongside a student with few opportunities to view examples of meeting competence and queried whether the assessment process is addressing a sustained level of competence.
But the part, and this is probably overall with the whole competencies a bit of an issue, but I think it's probably exacerbated with students is that because you're trying to find this one great example you feel like with some students you're actually just finding the one and only example of that student meeting competency.

Measure of competence
Some BN programme providers are expecting the nursing students to provide evidence of practice to support meeting the competencies as part of their competence assessment tool, while other providers are expecting the clinical lecturing staff to provide examples of how competency is being met.Some providers have developed a tool that requires all of the competencies to be met for each clinical placement while some institutions do not require that they all be met until the end of the three-year degree programme.There is a reported lack of consistency and standardisation related to providing evidence of competence from within and between institutions.The documentation that is used to record the assessment evidence and outcome varied between institutions.
For each student placement, they have to write the eight pages of the Nursing Council document for every student at each placement, apparently.Which seems bizarre, because there's quite a difference between the institutions.I think this obviously is going to be fantastic findings for Nursing Council, but I think as you'll go around and speak to people from different institutions, the expectation on how they do their competency tool is really different isn't it.I definitely think that we need to have a nationally consistent approach to competency assessment with all our tertiary BN providers and some standardisation of the clinical competency assessment tool.Because, again, it's very difficult for our clinical placement providers, being asked to often work now with multiple tertiary programmes coming into their clinical setting.

Relational assessment practice
The importance of developing effective relationships between all individuals involved in the competence assessment process was highlighted.Participants discussed the potential challenge of relationships with key stakeholders such as the Nursing Council of New Zealand which accredits the BN programmes and sets the competency assessment requirements, and New Zealand Qualifications Authority which regulates the educational quality of degree programmes.
That's what I think is one of our biggest challenges, is we serve two masters when we look at undergraduate health care programmes.We've got to satisfy NZQA, so that they can be awarded a degree, but we've got to satisfy the nursing profession as well, because it's a professional qualification.Sometimes the two don't play nicely together.What we've got to do is find a certain degree of defendable harmony between the two.
Three key roles were identified as being directly involved in competence assessment.Firstly, the nursing student, secondly the clinical lecturer employed by the tertiary institution, and finally the registered nurse whom the student is working alongside within the clinical environment.The accountability for the ultimate decision in regard to whether a student passed the assessment varied.
I think in some areas the students will do the self-assessment for examples and the preceptors will make their comments and then the lecturer will make their comments.In other areas the preceptors don't have anything to do with the competency tool: they complete a feedback form each week and the lecturer is the one that completes the competency tool.Whereas here some papers the preceptors will complete it, some papers they don't.So the lecturer completes it.And so there's an inconsistency in that … One participant spoke about the vulnerable position that students can find themselves in when working in an environment that creates a power differential between the registered nurses and students due to the RNs involvement in the assessment.
I just think that it's important as a profession that we're very aware that when we're deciding that somebody's competent and it's a professional noticing of that … .So as an experienced professional nurse, I can decide what I think's important, and therefore it gives me power and influence.
Different institutions allocated varying amounts of clinical lecturer contact time per placement area and there appeared to be flexibility regarding how the clinical lecturers managed this time.This may be related to differences of opinion on who is responsible for collecting the written evidence of student clinical competency and what is considered appropriate support for students.While one institution allocated specific clinical time for nursing student paperwork others discussed the pressure on students to complete the required documentation and the impact on them in the clinical setting.
Another thing that I haven't mentioned is that we don't expect the students to have to be writing furiously in practice around their examples of their competencywe actually give them clinical time to do that because they're writing up their clinical reflections or captures of their clinical practice that they need time to do that.We don't want to take them away from the clinical experience to do that.
Clear communication with nursing students throughout the clinical placement was identified as an important element of the competence assessment process.Helping students to identify when and how their practice meets competency and how they can provide evidence for this was illustrated by participants.
What works well: there's no surprises, I think.If there's an area that they're clearly not performing in, then it is fairly well addressed in quite a timely mannerit's alerted in quite a timely manner so that we can, with the student, develop a plan of how we're going to make this work.There's no surprises.There's transparency here, yeah.

Discussion
In line with international literature (Burke et al., 2016;Sweet et al., 2018), standardisation through development of a national assessment tool was identified as a potential method of addressing the variability in assessment practice and a way to promote the validity and reliability of the assessment.Standardising assessment paperwork could provide clarity in regard to expectations for both the clinical and education providers as well as the nursing students (Sweet et al., 2018), and ensure that safety for patients, students, and staff is maintained through attaining a prescribed and understood level of competence throughout the programme.In Australia, the Australian Nursing Standards Assessment Tool (ANSAT) has been developed as a standardised method of assessing undergraduate nursing student competence (Ossenberg et al., 2016).The instrument has been validated through a pilot study that demonstrated that the tool is sensitive to assessing differing year levels of nursing students within different practice setting environments (Ossenberg et al., 2016) This research supports the view that standardisation may also alleviate the logistical challenges that were identified in the current study related to managing different levels of students from different education providers with various assessment practices out in clinical practice at the same time.Consulting with all parties involved in the assessment process would be essential in order to ensure that all elements of the assessment and how this relates to clinical practice experience have been considered (Immonen et al., 2019).
The impact of resourcing was discussed and in particular the time constraints in clinical practice that adversely impact on the ability of registered nurses to adequately provide clinical placement opportunities for nursing students and assess competence.Time constraints and workload when precepting nursing students could also impact on patient, staff, and student safety (Wu et al., 2016).Collaboration between nursing student, clinical lecturer, and preceptor was evident in the assessment in each of the institutions and the relationships formed between the individuals in these roles directly influence the student assessment experience and outcome.In line with research conducted by Engstrom et al. (2017), building constructive working relationships between staff employed within tertiary institutions and clinical placement staff is paramount to the assessment process whilst strong relationships at the strategic management level between education providers and clinical placement areas is critical for optimal delivery of the BN programme (Borren & Harding, 2020), in particular given the need to manage the different expectations set by the different regulatory bodies that govern the BN programmes.
Clinical placement availability can at times impact on the ability of programmes to deliver optimal clinical learning experiences for students.In addition, the power differential between assessors and nursing students in the decision of what constitutes competence can impact the student experience in placement and could constitute a safety issue, both for patient outcomes and the student.Clear guidelines and expectations for students and assessors scaffolded to the level of the student would aid in clarifying expectations for all individuals involved.Regular feedback throughout the placement (not just at the time of the assessment), is required to help inform the assessment and enhance student development (Almalkawi et al., 2018).Further education related to assessment expectations is needed to minimise assessor confusion (Tuomikoski et al., 2020).
Participants in the study reported the competence assessment process as being largely subjective aligning with previous studies (Helminen et al., 2016;Watts & Walker, 2018).In some institutions, the students were expected to provide evidence of competence and in other institutions, the lecturers or clinical staff are providing the evidence.There was variation regarding what constitutes sufficient evidence and the time that is taken to provide this.There is also variation in the tools used by each of the institutions to assess the competence of nursing students in New Zealand and competencies that were designed to reflect a registered nurse standard of practice are being used as a basis for assessing the developing competence of nursing students.

Limitations
This article reports on data attained from interviews with clinical educators, consequently, the views and perspectives are that of the employees of the tertiary institutions whilst the views of nursing students and clinical preceptors have not been included.At the time of data collection and analysis the first author was employed as an academic in a BN programme and this may have contributed to an unconscious bias although doctoral supervision throughout the research design and analysis process aided to alleviate this.

Conclusion
This study has identified key factors impacting on competence assessment of undergraduate nursing students in Aotearoa New Zealand and highlighted the implications of variation in competence assessment practice.Participants discussed standardisation of assessment process between institutions as a means of promoting safety and enhancing clarity.Consultation with stakeholders involved in competence assessment (including nursing students and clinical staff), is recommended in the development of a standardised assessment tool to ensure that the assessment of nursing students is aimed at the appropriate level of the student.Further recognition and review of workload allocation for clinical staff working and educating nursing students in placement areas would be of benefit.Development of guidelines that identify clear expectations of how competence can be achieved at each level of the undergraduate nursing programme would be useful.Further research is required to gain a broader understanding of the experience of competence assessment from the perspective of preceptors, nursing students, and academic staff.This study highlights the need for a process that allows for the developing competence of nursing students and appropriate competencies that are designed to meet the need of each year of the nursing programme to be developed.