Prompting medical students to self–assess their learning needs during the ageing and health module: a mixed methods study

ABSTRACT Understanding our learning needs is fundamental for safe, effective and knowledge-based medical practice and facilitates life-long learning. A mixed methods study investigated fourth-year medical students’ self-perceived understanding of their learning needs using 1] a visual scale, before and after a four-week module in Ageing and Health (A&H) and 2] through focus group discussions. During 2013–14 academic year, all students (252) were invited to use a Visual Analogue Scale (VAS) tool to self-assess their learning needs that were linked to Ageing and Health curriculum learning outcomes. Assenting students (197 at pre-self-assessment, 201 at post-assessment) returned anonymous Visual Analogue Scales, self-assessing history-taking skills, examination skills, knowledge of medication use, co-morbidity, nutritional and swallowing assessment responses, before and after the A&H module. Three student focus groups explored whether completion of the VAS self-assessment had prompted improved self-awareness of their learning needs. The VAS responses increased for each curriculum domain with significant differences between the pre-and post responses – for the student-year-group. Nutritional and swallowing knowledge showed the greatest improvement from a self-assessed low baseline at entry. Focus-group students generally viewed the VAS tool positively, and as an aid for prompting consideration of current and future clinical practice. Some students recognised that ‘a need to be ready-for-work’ focused engaged learning; others demonstrated self-regulated learning through self-motivation and an action plan. The Visual Analogue Scale quantitative responses showed increased student-self-perceived learning for each curriculum domain at fourth-year completion of the A&H module, suggesting that prompting self-assessment had increased students’ knowledge and skills. Focus group students saw the VAS tool as useful for prompting awareness of their current and future learning needs. Additional educational strategies should be explored to enable all students to self-reflect and engage effectively on their learning needs, to gain the skills for the maintenance of professional medical competence. Abbreviations: A&H: Ageing and Health Module; e-portfolio: an electronic version of an evidence portfolio, which allows medical students and graduates to reflect and document learning and competencies; F1: year1 of post-graduate medical clinical training; GMC: General Medical Council-the regulation organisation for maintaining standards for doctors in UK; Logbook: usually a written document which can be used to record procedures and attendance at clinics or case-based discussions and can be used to set learning outcomes and to structure teaching in clinical settings for medical students and doctors; PDP: personal development plan is used to plan future learning and skills needs for work and education with an plan for action/s outcome; SPSS: Statistical Package for the Social Sciences; VAS: Visual Analogue Scale is a visual method of describing an experience.


Background
Self-directed learning is a core concept that is fundamental to adult education. In its simplest form, selfdirected learning means that the learner exerts control over their own learning and directs strategies to manage their learning tasks. Garrison [1] elaborated a more complex model in which there is the integration of self-management (contextual control) together with self-monitoring (cognitive responsibility), and motivation (entering and task), as dimensions that reflect a meaningful and worthwhile approach to self-directed learning. In their consideration of selfassessment Eva & Regehr [2] suggest that the learner requires to recognise both their weaknesses and strengths, in a way that enables them to set challenging but reachable goals. Pisklakov et al., 2014 [3] in their review of self-evaluation and self-assessment in medical student and medical resident education was of the view that self-evaluation was the ability of learners to judge the quality of their work, based on evidence, for the purpose of self-improvement.
Self-assessment has been argued to enhance learning, including deep and lifelong learning [4]. In their study, Sharma et al. 2016 [5] examined the selfassessment by undergraduate medical students on their subsequent academic progress and concluded that self-assessment increased the interest and motivation of students leading to enhanced learning and better academic performance, and helping them in the development of critical skills for analysis of their own work. Sitzmann and colleagues [6] in their metaanalysis to assess the validity of self-assessments of knowledge in the broad area of education and workplace training, suggested that self-assessments of knowledge are moderately related to cognitive learning, motivation and satisfaction.
Student self-assessment has been studied previously [7][8][9] but is known to vary across individuals and context [2,10,11]. Although self-assessment is used to help learners identify their own learning needs, the available literature suggests there are conflicting results about reliability and validity [2,12,13]. Self-evaluation does not come naturally and requires training [14], with some evidence to show that learning and reflection were improved when learners practised self-assessment and received good feedback. Therefore, self-assessment has remained an important learning and educational developmental tool for both students and teachers [6].
The General Medical Council (GMC) UK, in its 'Assessment in Undergraduate Medical Education; Overview a clear strategy' (2011), advised that educational strategy should cover formative as well as summative assessment, and that 'Students should become accustomed to seeking maximum benefit from feedback, self-assessment, reflection and the development of lifelong learning skills' [15]. The current GMC consultation 'About the Outcomes for Graduates 2017ʹ (GMC 2017) [16], again advises that medical students should commit to lifelong learning, and take responsibility for their own learning.
In order to further explore the concept of selfassessment, we invited medical students at Queen's University Belfast to assess their self-perceived learning needs during their fourth-year clinical attachment in Ageing and Health, by using a Visual Analogue Scale (VAS) [17]. The Visual Analogue Scale (VAS) used in this study, measured a characteristicselfperceived learning need-that cannot be easily quantified by ordinal scales, such as the Likert scale. The VAS has been used widely in clinical practice and in nursing teaching programmes [17][18][19]. The student anonymous returns allowed us to evaluate any change in self-perceived learning and skills for six main curriculum domains in the Ageing and Health (A&H) module. As an adjunct to the quantitative analysis of the Visual Analogue Scale (VAS) returns, three student focus groups were invited to discuss issues around student understanding of the use of self-perceived assessment as a means of improving insight into their current in-module learning needs and for their future medical practice.

Setting
We conducted the study at Queens University Medical School, which offers a five-year curriculum with clinical medicine integrated from year one. The clinical curriculum has a modular structure; the sequence of the modules is identical for all students. The Ageing and Health Module (A&H) in the fourth year of the clinical medicine, comprises four weeks of clinical Elderly Care Medicine the first week of lectures, seminars, case-based study, and three weeks of clinical teaching sessions under the supervision of speciality consultants. Six groups of students (40)(41)(42) approximately) rotate through the curriculum and through Ageing and Health module in consecutive blocks -A, B, C, D, E and F during the academic year.

Participants
During 2013-14 academic year all medical students entering the Ageing and Health module were invited to participate in a teaching evaluation of their selfperceived learning needs, for six areas of the curriculum learning outcomes with respect to elderly patientshistory-taking skills, examination skills, medication use, co-morbidity, and nutritional and swallowing assessments. There were no selection or exclusion criteria. Participation was voluntary and anonymous students assenting to participate returned completed Visual Analogue Scale (VAS) responses, at the beginning and end of their A&H Module.

Visual Analogue Scale tool (VAS)
The Visual Analogue scales (VAS) [17,18] is a graphic scale in which the respondent places a mark at a point along a line, with endpoints of 1 and 10 [18], with 1 representing very little knowledge or skills and 10 representing very good knowledge or skills. The student mark on the VAS scale was measured from the endpoint 1 and this distance calculated arithmetically as a percentage of the total measurement between the endpoints 1 and 10. The percentage was changed to a score out of 10. The VAS tool scale was completed for the six clinical domains in the A&H curriculum ( Figure 1).

Quantitative statistical analyses
The anonymous pre-and post VAS responses were measured with responses calculated out of 10 and analysed during a 6-week summer research studentship (GK). Comparison of pre-and post-module responses were made for the six knowledge and clinical domains, for the whole 2013-14 student year group and for the six individual blocks of students for A&H module, using two-tailed unpaired t-tests and non-parametric tests Mann-Whitney U and Kruskall Wallis as appropriate and SPSS analysis. A p-value <0.05 considered significant. Data sets are a v a i l a b l e a t h t t p s : / / f i g s h a r e . c o m / s / 5469e44e08092002e41c. Doi: 10.6084/m9.figshare. 7294922

Focus groups and qualitative analysis
A general invite for interested students (4)(5)(6)(7)(8) in the academic year 2013-14 was made for focus group participation. Three focus groups took place the first focus group early in academic year 2013-14, the second group late in the academic year 2013-14, and the third group early in the academic year 2014-15. Participating students gave written consent to take part in a semi-structured discussion facilitated by student researcher (GK) who had appropriate training and experience. Students were advised that the session (~30 min) would be recorded and transcribed but that no contributor would be identified in person. All focus groups used the same semi-structured interview ( Table 1), but the second and third focus groups, used in addition, the thematic outputs from the first focus group, to further refine and probe the emerging thematic issues towards saturation. The transcribed discussions were read and re-read individually by each of the three authors and grounded theory qualitative methods [20] were used to establish emerging themes and linked concepts. Students were given Information leaflets and assenting students provided written consent. The Visual Analogue Scale questionnaire within the project was part of a curriculum development of the teaching course providing the students with the opportunity Table 1. Focus group semi-structured questions.
The focus group will take the form of an opening introductory question followed by 4-6 semi-structured questions.

Visual Analogues Scale (VAS) analysis
There were a total of 398 anonymous completed VAS returns for self-perceived learning needs for the 2013-14 year group of medical students-197 at pre-self-assessment and 201 at post-assessment after completion of the A&H module. This represented an approximate 80% return from the 250 students who commenced fourth year medical studies. The numbers for completed and returned VAS scores in Groups A, B, C, D, E and F were as follows;  Table 2 shows the mean, variance, standard deviation and standard error for the pre-and postaggregated VAS scores for the Total Student Year for each of the six A&H clinical domains. There are increases in history taking skills, examination skills, medication use, co-morbidity, nutritional assessment and swallowing assessment in the post-student compared to the pre-student scores which are highly significant when compared by unpaired two-tailed student t-test. Figure 2 demonstrates significant increases in the post-VAS scores compared to the pre-scores for self-perceived learning needs for the aggregated whole year group, shown as box-plots, medians and 25% and 75% ranges, using Mann Whitney U test statistic. Both nutritional and swallowing assessments, comparatively new areas of clinical knowledge introduced to the fourth year students in the A&H module showed an increase of approximately three points in the post-VAS score. There was also a wider range between pre-and post-VAS scores with many students showing low pre-module scores in comparison to more familiar areas of history taking and examination skills ( Figure 2). The means for pre-and post-VAS scores for each of the six student class groups A, B, C, D, E and F and for each of the six curriculum domains were analysed by unpaired two-sided t-test are shown in Table 3. The post-student self-assessment VAS scores demonstrate increases in self-rating self-assessment response for post-VAS for each student class group for the clinical domainsexamination skills, medication use, comorbidity and nutrition and swallowing assessment that are significant ( Table 3). The history taking skills for student class groups A and D did not show a change postself-assessment from pre-self-assessment scores that was significant, and it is considered that many students in fourth Year medicine would have already gained a good standard of history taking skills when commencing the A&H module. Figure 3 shows box-plots, medians and 25% and 75% ranges for the individual student class groups (ABCDEF) for nutrition, a clinical skill where students showed a three-point post-self-assessment score increase. Pre-VAS self-assessment scores are shown as (A-1, B-2, C-1, D-1, E-1, F-1) and post-VAS scores as (A-2, B-2, C-2, D-2, E-2, F-2) and demonstrate a significant increase across all the class groups, using Mann Whitney U test statistic.

Focus groups and themes
Six, 4 th year medical students participated in the first focus group, five, 4 th /5 th year medical students in the second focus group, and four 5 th year medical students in the third focus group. All participating students were recruited during the 2013-14 academic year, but because of irreconcilable timetabling for student and facilitator availability, the third focus group took place when students entered 5 th year medicine.

Findings
The four key themes emerging from the three focus groups with respect to the students' understanding and use of the Visual Analogue Scale as an adjunct to their learning are outlined under the main headings in Figure 4.

Easy to Use
In this sub-theme students generally saw the VAS as easy to use, one student advising 'Yeah it was grand; its easy to use, very straight forward'.

Useful to structure learning in e-portfolio and Personal Development Plans (PDPs)
In this second theme, students commented on several aspects of their future learning. One student said 'I think (VAS) would give you some structure for setting out your PDPs [personal development plan]', and in considering e-portfolio use, a student commented 'If you were (to) have those kind of scales and it was more focused, the e-portfolio would be more beneficial '.

Improving deeper levels of Understanding
Students could see beyond 4 th year undergraduate studies and to their future medical practice commenting 'I think I knew in the future, we'd have to write reflections, [so] I actually made a list of stuff that I need to learn how to do'. There was recognition that skills should not remain static as in 'It's all about improving our skills and improving'.

Focused Learning for entering Workplace
Students recognised a different learning approach between 4 th and 5 th year as the reality of being a doctor in charge of patients became closer, saying 'I supposed we're very F1-focused now [year1 post-graduate medical training] …whereas in 4 th year, I don't know ….. that I'd have got my head into that space'. Another student volunteered -'It [VAS and self-assessment] is something I'm only really seeing the importance of now… now that we're actually faced with the world of work'.

Logbook or on-line modification
The students were ready with suggestions to improve the VAS tool and thought an on-line or logbook version for self-assessment should be considered -'if you could do it on-line and [if] you did it at the start and at the end of a six week module and then it was able to give feedback'.

Mid-Module Reflection Opportunity
A related sub-theme suggested that 'it [VAS] should have a mid-point reflection-then you have the educational opportunities that you can tailor to what you need'.

Binary competent/non-competent for some Clinical Skills
A comparison of the binary terms 'competent' or 'non-competent' rather than the VAS scale was discussed as perhaps more useful for assessing clinical knowledge, as in-'I would be more in favour of "competent" or "not competent" rather than a Likert scale'. Another student agreed with this idea but also recognised that 'different skills should be assessed differently as in '(For) a nutritional assessment…that's a 'yes' or 'no', but 'history-taking is so varied; it's not an assessment where it's either" yes" or 'no'.

Self-Reflection
In this theme, some students spoke of recognising their strengths and weaknesses as in 'there's so much to know that it's really important to be able to identify where you are weaker'. Recognising that each person will reflect differently on their own learning needs, others commented 'I think it depends how critical you are of yourself. So one person's nine could be another person's five'.

Self-Motivation
Motivation was recognised as important in addressing learning needs, with a student explaining 'the VAS probably does make you think "I'd need to have a look at that" but whether or not you have the selfdrive to do that is another question'. A number of students acknowledged that personality was part of the self-motivation response with one student saying 'I think it depends on the person; some people are going to care about it more than others. It's really subjective'.

Acting on Learning Needs
There were different approaches to acting on learning needs. One student told how he/she had used the VAS during the A&H module saying 'there was a question about your drug knowledge so I knew I needed to concentrate on that. During attachment I was looking at the [drug] kardex a lot'. Another recognised the role of personality as in 'You have to make that decision whether you want to improve on that area'.

Discussion
In this study of self-perceived assessment of learning needs, we found that the class year of 4 th year medical students' evaluation of their self-assessed learning needs, showed increases for each of the six core curriculum domains in Ageing and Health (A&H) that were significant, when comparing the pre-and post-module responses, using the visual analogue scale (VAS). A number of studies have looked at 'before and after' assessment in teaching programmes. In a longitudinal study, Arnold et al. [21], asked medical students to complete self-evaluation at the close of each rotation during a six-year curriculum. The authors found that both students' self-evaluations and faculty members' ratings of students' performance increased year by year, but conversely the relationship between both decreased each year. Despite these dissonant findings, both groups considered that self-evaluation encouraged learning and professional development. In the pre-and post-self-assessment of learning outcomes in one hundred and forty-five, fourth-year students at Göttingen Medical School study,  Schiekirka et al. [2013] reported that there was good agreement between the performance-gain derived from the formative examination data and performance gain derived from the student self-assessment data in the 33 specific learning objectives, in a cardiorespiratory module [22]. In a Stop Think approach to encourage self-assessment of learning, a research study invited students to rate their feeling of difficulty before and after completing an anatomy or physiology concept map exercise [23]. The differences in the results between the students' estimation of their preand post estimated difficulty score correlated with students' objective-class examination scores, suggesting that this method appeared to be successful in stimulating students to self-assess their learning activity. In other research such as a small study of videotaped communication skills, there was a lack of concordance between undergraduate medical students' own assessment of self-efficacy and trained observers' assessment at two different monitoring points [24]. Similarly, Gruppen and colleagues found a very low correlation (0.25) between the observer and self-assessed student rating in 10 areas of curriculum [25].
The Visual Analogue Scale (VAS) used in this study, measures a characteristicself-perceived learning need-that cannot be easily quantified by ordinal scales, such as the Likert scale. However, it could be argued that translating the change in students' self-perceived learning need after the Ageing and Health module, into a numeric evaluation may or may not be a true reflection of the metacognitive nature of their learning. Against this suggestion, the VAS has been used widely in clinical practice [17,18], in social and behavioural research [26], in business studies [27], in pain assessment medicine [28,29] and in nursing teaching programmes [30]. A benefit of this type of visual analogue scale as opposed to a Likert scale with 5 or 7 degrees of agree/not agree, is that students can visually estimate their learning needs and later comparisons are considered a truer reflection of change. Celenza & Rodgers [19] described the VAS tool as reliable and a valid alternative to a Likert-type scale (LTS), in their evaluation of perceptions of a bedside-clinical-teaching programme. In reviewing the benefits and shortcomings of the VAS in measuring clinical phenomena, Wewers & Lowe [31] argued that the VAS was of most value when comparing change within individuals or groups over time, but was likely of less value for comparisons at one time-point. In the present study, the VAS tool was used to produce interval data between the pre-and post-estimations of student-perceived-learning needs for the Student Year or Student class-group, which according to these authors is more likely of greater value and reliability [31]. Similarly, in a commentary on self-assessment, Lam (2009) argued that aggregated self-assessments were likely more useful in comparison data [32].
But irrespective of which method, scale or analysis is used for self-assessment, most studies suggest that it is difficult to self-assess accurately. In investigating undergraduate students' self-assessment ability in comparison with their instructor's judgement, both Mattheos et al. [33], and Rees & Shepherd [34] reported that students tended to overestimate their competence compared to teachers. Similarly, Davis and colleagues [35] in a meta-analysis comparing the accuracy of physician self-assessment compared to externally assessed competencies found that physicians had a limited ability to accurately self-assess. A number of studies have reported that females perform better than males in observed examinations although their self-assessment scores tended to be lower [24,33]. There is limited evidence to suggest that students improve their self-assessment skill over time [21,33,36,37], though self-assessment is considered a learnable skill [38].
What did the focus groups tell about student self-assessment?
In the first and second themes, students saw the VAS tool as a potential aid for identifying their current and future self-directed learning needs, and this was encouraging. However, a few negative comments suggested that some students had no apparent understanding of the importance of developing self-directed learning skills. But as students reached final year, the need to be ready-for-work focused their learning, similar to the work-place-imminent-learning behaviour recognised by Billet [39].
A further theme reflected students' readiness to make suggestions for improvements to the VAS. They suggested the VAS for self-assessment should be incorporated into an online version of the logbook to support self-reflection. These important studentled suggestions for the VAS have led to developmental changes being incorporated into the A&H logbook with monitoring of learning outcomes. Students also needed to be reassured that self-assessment was being used as a student self-development skill for workplace learning and not for use by Faculty [40,41].
Students demonstrated different learning styles from some able to self-regulate their learning to others who seemed less likely to act on their weaknesses. These thematic outputs prompt teachers, clinicians and educators to consider the many different student learning-styles, the complex and changing clinical environments in which students are expected to learn [42] and the embodied tensions of selfassessment-'between wanting to know and afraid to know' - [43,44].

Strengths
This was a mixed methods study providing useful insights from using a combined quantitative and qualitative approach. We used six of the main learning objectives from the A&H course for the selfassessment study so covering the most important clinical and knowledge learning outcomes of the four-week A&H attachment. The study was simple to set up, easy to implement and was not time-consuming for either participating students or the research supervisor. There was a high student response rate in study and we did not use incentives to encourage students to take part in the study, hence reducing bias. This particular method is valuable as it did not require individual labelling of students and was conducted completely anonymously. The VAS took initial student performance into account and the performance gain regarding specific objectives was calculated from aggregated data at the group level and did not require extensive pre-and posttesting.
The qualitative aspect of the project gave some encouraging insights into the students' engagement, use and value of the process of self-assessment. It allowed the students' views to be heard and studentled ideas to become incorporated in the processes of self-assessment in teaching and learning outcomes in the A&H module. The whole process became a tool for empowering students by giving them a sense of shared decision-making in course improvement and demonstrating that self-assessment could enhance the quality of their learning as students and thorough out their medical careers.

Limitations
The study's findings have a number of limitations. Since participation was voluntary and anonymous, only students who completed the VAS were included in the quantitative analysis and the results are presented as aggregated student-year and the rotating six student-group results. It is not known whether some of the same students joined the focus groups. The emerging results and themes may therefore not be representative of the whole year group and so the outcomes should be considered exploratory. The effect of either student gender or graduate status could not be assessed because student responses were anonymous. The study involves only one university and may not have wider validity. In addition, the focus groups were relatively small, although numbers alone should not argue against the quality of the output obtained.
The observation from other studies that increased scores or performance gains did not necessarily mean that actual learning had taken place, equally applies to our research findings. In addition, the body of evidence showing that students tend to overestimate their competence in relation to the judgement of their instructors and that increased self-confidence can compromise patient safety is highly relevant [21,[33][34][35] and is also applicable.

Summary comments
Self-assessment of clinical knowledge and accuracy of skill performance is considered essential to the practice of medicine and self-directed life-long learning and best practice should be encouraged in order to improve and advance the skill [40,45]. Medical practitioners are responsible for their own learning, through self-assessment (reflection, self-monitoring, external information seeking) [46].
Undergraduate medical students or junior doctors in-training often do not take time to actively think about their learning needs before they enter a new learning module or clinical environment rotation. Here we have used a simple VAS tool to aid students to self-assess their learning their needs. We demonstrated a significant 'performance gain' in their self-perceived knowledge and skills after the Ageing and Health module with evidence that the students had used self-assessment to improve their knowledge and skills. The qualitative aspects of the project demonstrated that students engaged in the process and saw self-assessment as beneficial. It also gave students a sense of shared decisionmaking and allowed them to advance student-led suggestions for improvements in the A&H module. Finally focus group discussions demonstrated that students recognised how self-assessment was a tool that could be used to improve their current and future medical knowledge and skills.

Conclusions
• Students' self-perceived knowledge and skills increased in domains linked to learning outcomes after the A&H module. • In focus groups, students generally viewed the VAS tool positively, and as an aid for identifying current and future learning needs. • New educational strategies should be developed in the Medical School Curriculum so that all students can maximise their self-reflective and self-assessment skills.