The evaluation of an e-learning prescribing course for general practice

ABSTRACT Prescribed medication may lead to significant morbidity or mortality as a result of these medications causing adverse events, or because of a prescribing error. E-learning is a common tool used in supporting training in prescribing. This paper describes the development of an e-learning course and the subsequent evaluation undertaken by the users with the aim of obtaining an effective e-learning course for prescribing. The e-learning course was developed by general practitioners and pharmacists and focussed on the principles of good prescribing, examined the common reasons for prescribing errors, and was evaluated using self-reported quantitative and qualitative measures. Scores significantly increased on an assessment given before and after the course. The majority of respondents reported that the e-learning course had a positive impact on prescribing knowledge, skills and attitudes, with medication reviews the top area where a change in prescribing practice was reported. Over 90% of the respondents agreed that the e-learning course was easy to use and a useful part of their continuing professional education. This study shows that clinicians recognise the on-going need for training in prescribing, but the lack of training is one of the factors contributing to errors, which suggests that more education is needed, not just for GPs in training, but for qualified GPs as well.


Introduction
Prescribing continues to be an important intervention for general practice patients. In 2019, a total of 1.12 billion prescription items were dispensed in the community in England, costing £9.08 billion [1]. For some patients, prescribed medication may lead to significant morbidity or premature mortality as a result of these medications causing adverse events or because of a prescribing error, which in turn further increases costs [2][3][4].
Prescribing in general practice occurs in the context of an increasingly elderly population (and therefore an increase in treating patients with comorbidities), which has resulted in a growth in polypharmacy. This creates complex medication regimens, with the potential for drug interactions and side effects [4,5]. A systematic review (undertaken as part of the PRACtICe study) revealed a paucity of research on prescribing errors in the community [3]. The review identified only one relevant study [4]. This analysed the prescriptions of 256 care home residents in 55 homes, demonstrating that 39.1% of patients had one or more prescription errors per day (8.3% of all prescriptions). The most common (37.9%) errors were due to incomplete information on the prescriptions, 23.5% due to unnecessary drugs and 18.4% were monitoring errors, the majority (90.6%) being due to the complete lack of monitoring [4]. In 2012, the PRACtICe study, a large study commissioned by the General Medical Council (GMC) provided greater insight into the nature and prevalence of errors in general practice. This study examined in detail 6,048 unique prescriptions for 1,777 general practice patients in England and found that 5% of the prescriptions reviewed were associated with an error [3]. Of these, a small proportion (1 in 550), were classified as severe in nature. Communication from the Medical Defence Union (MDU) revealed that between 2011 and 2015, the majority of cases relating to prescribing errors involved general practitioners [2]. The MDU stated that many of the errors were due to checks: such as drug-drug interactions or allergies, not being performed, as well as issues with selecting the correct drug, correct dose and correct delivery method. The contributing role of clinical systems on prescribing errors, especially in the environment of general practice with fluctuating workload, staff shortages, missing records, distractions and time pressures has been articulated. Poor quality prescribing has contributed to some of the current public health challenges: inappropriate polypharmacy, opioid dependence and antimicrobial resistance [2].
The PRACtICe study included a root cause analysis that helped establish a set of recommended strategies that may improve prescribing. The need to invest further in education and training was identified by all GPs, irrespective of their stage or experience of prescribing [3]. Although the call for more education and training is not new, a specific gap that was identified by the PRACtICe study was for more resources to support continuing professional development in the area of prescribing [3]. In the undergraduate environment, this recently led to the convening of a Safe Prescribing Work Group by the GMC and the Medical Schools Council, including introducing the Prescribing Safety Assessment for final year medical students prior to graduation [6]. However, postgraduate education is often limited by clinical schedules, difficulty in supervision and competition with other training requirements, despite the importance of prescribing, as the skills are used in everyday practice [6,7]. It is important to highlight that education and training initiatives need to be inclusive of postgraduate prescribers, something that has been shown in the GMC Outcomes for Graduates, which emphasised that doctors in the profession today will experience major changes in medical practice, which will necessitate the need to engage in lifelong learning [8,9]. E-learning is a common tool used to support postgraduate (and to some extent undergraduate) training in prescribing [1]. Many advantages of e-learning have previously been described in the literature [10,11] including, but not limited to: enhanced accessibility, teacher accountability, costeffectiveness, and reduced administrative burden. A further advantage is that learners are able to choose content, order, pace, location and timing of the instruction with the ability to update and amend content at any time, which is important given that prescribing education changes rapidly. The e-learning package was developed to address these concerns. In the UK community setting, the Royal College of General Practitioners (RCGP) has an important role in hosting a wide variety of e-learning courses, including the e-learning prescribing course described in this paper [10]. This paper describes the development of the course and the subsequent evaluation undertaken to explore its impact on prescribing and acceptability as perceived by users.

The e-learning prescribing course
The e-learning course was developed by a team, which included GPs and pharmacists, in partnership with the National Institute for Health Research (NIHR) Greater Manchester Primary Care Patient Safety Translational Research Centre and the Medical Defence Union. It was based on lessons learnt from the PRACtICe study, which focused on the principles which inform good prescribing and examined the common reasons for prescribing errors (see Figure 1) [3,[13][14][15][16][17][18]. The e-learning course was launched on the Royal College of General Practitioners (RCGP) website in January 2014 and was advertised by the RCGP in their regular email bulletin to members. However, the e-learning course was made available to both members and non-members [19].

Evaluating the e-learning course
The e-learning course was evaluated using self-reported quantitative and qualitative measures (including qualitative interviews not presented in this paper) and captured different dimensions of interest as presented below.

Pre-course and post-course assessments
The pre-course and post-course assessment were an optional component of the course and included 13 multiple choice questions intended to be completed at the beginning and end of the e-learning course. The resulting score aimed to provide some indication of baseline knowledge of prescribing and subsequent learning achieved by individual users. The scores of these two data sets were analysed in IBM SPSS® Statistics, Version 25.0.0.1 [20] using the paired t-test. No demographic data were linked to these scores.

Evaluation survey
The evaluation survey was presented as an optional part of the e-learning course and used a combination of 5-point Likert scales and free text fields to collect information in three main areas (i) demographic dataprofession, whether GPs were MRCGP qualified, number of e-learning courses completed in the past year, special interest, gender and age; (ii) what was gained by completing the e-learning course -knowledge/learning, skills, attitudes towards developing strategies for safer prescribing, and awareness of important aspects of prescribing covered by five individual lessons; and (iii) the e-learning course -ease of use and usefulness as part of continuing professional development (CPD).
The Likert scale responses were analysed using descriptive statistics in IBM SPSS® Statistics, Version 25.0.0.1 [20]. Due to high ceiling effects, responses were presented as dichotomous responses (Strongly Agree and Agree were coded as positive comments; Neutral, Disagree and Strongly Disagree were coded as negative comments). For the free text answers, the main themes that emerged were coded and agreed on by at least one other researcher [21]. The frequency with which a theme occurred was calculated as a way to summarise these free text data.

Pre-course and post-course assessment
A total of 3,423 people engaged with the e-learning course between January 2014 and November 2018. Of these, 2,164 (63.2%) completed the assessment before and after completion of the five lessons and were included in the analysis. The mean pre-course and postcourse scores were 60.4% (SD = 13.4%) and 77.8% (SD = 11.8%) respectively. A paired t-test demonstrated there was a significant difference between the pre-and post-test scores (t (2163) = 59.48, p < 0.001) and on average, the post-test scores were 17.4 percentage points higher than the pre-test scores (95% CI [16.8, 18.0]).

Participant characteristics
In total 765 people completed the evaluation survey between January 2014 and September 2017, which was the period these data were available for use, and represents 22.3% of the people who originally engaged with the survey. As shown in Table 1, the majority were GPs (90.1%) which included 89 GPs (11.6%) who identified as being GPs in training. Other professions represented included those in the fields of pharmacy (4.4%) and nursing (3.1%). Most of the GPs (83.8%) held the MRCGP qualification. A high proportion of respondents completed 10 or fewer e-learning courses (80%) over the past year and over half were female (59.2%). Nearly an equal proportion of respondents fell into the 40 and under and over 40 age ranges.

What was gained from completing the e-learning prescribing course (Likert scale questions)
The majority of respondents reported that the e-learning prescribing course had positively impacted their prescribing knowledge, skills and attitudes (Table 2). Similarly, they indicated that the course had positively highlighted important aspects of prescribing.

Changes to prescribing practice following completion of the e-learning prescribing course (free text responses)
One free text question was included in the evaluation survey to capture details of what aspects of prescribing had changed, or the respondent would like to change, as a result of completing the e-learning course. Most (82.4%) of the participants who completed the evaluation survey shared one or more traits in their prescribing practice they would change. As demonstrated in Table 3, medication reviews was the top area where change in prescribing practice was reported (24.0%) with giving clear drug dosing instructions (16.4%) and choosing the right dosing (12.4%) also featuring in the top three changes mentioned.

Experience and suitability of the e-learning prescribing course
The course was well received with 93.2% of the respondents agreeing/strongly agreeing that the e-learning course was easy to use and an even higher proportion (98%) agreeing/strongly agreeing that the e-learning course was a useful part of their CPD.

Discussion
This e-learning course was developed in response to an identified need to reduce prescribing errors in primary care in the UK following the PRACtICe study, which identified that 1 in 20 of all prescription items contained an error [3]. This is the first study of its kind in primary care, developing a way to improve accuracy of prescribing in primary care. Analysis of the scores of the 13 multiple-choice questions completed both before and after the e-learning course, demonstrated that the e-learning course significantly improved scores on this assessment, with the percentage correct increasing from 60.4% to 77.8%. This is shown in the analysis of the comments in the evaluation survey, with a large majority of participants stating that the course was of use to them and increased their awareness and knowledge of prescribing. Analysis of user data from the e-learning course gives strong support of the efficacy of the learning course, although it was not possible to provide evidence of improved patient outcomes.
One previous small-scale study showed that providing complete dose instructions and monitoring were areas needing improvement [4], as did the PRACtICe study [3]. Participants thought their prescribing habits had changed in both of these areas as a result of the e-learning course. This demonstrated that the e-learning course does address problems previously identified in primary care. In the present study, providing effective medication reviews was identified as the most important aspect of prescribing that should be improved, which is in agreement with the PRACtICe study [3]. The PractICe study [3] found that about half of the prescribing and monitoring errors that were identified involved repeat prescriptions, which in turn, could be reduced by effective medication reviews [22]. This finding is supported by Cullinan [23] who identified the difficulties doctors had in safely prescribing to older people who have complex repeat prescriptions. As mentioned above, the main strength of the e-learning package is that it is the first of its kind in primary care. It is unique, with the majority of alternative available courses being targeted towards medical students [10], most of these courses tend to be case-based as this package was [10]. As noted by Ruiz [12], e-learning does not replace traditional training, but rather serves to complement the learning of prescribing skills presented by an instructor. This package addresses an important problem, prescribing errors, in a rigorous way which can be adapted and used in the future for continuing professional development. The main limitation is the loss of participants at each stage of the course. Of the 3,423 participants who took part in the e-learning prescribing course, 2,164 (63.2%) completed both the pre-and post-test, but only 765 (22.3%) carried out the evaluation survey. The large drop-out rate is consistent with other studies [23,24]. However, with over 2,000 participants, we had sufficient power to demonstrate a significant difference between the pretest and the post-test scores. In future studies, to reduce attrition, it may be useful to make the posttest and evaluation survey compulsory in order to gain a certificate and contribute to CPD hours. This approach may improve the learning experience by allowing participants to evaluate their own learning using the post-test score and reflecting on their learning in the evaluation survey. The evaluation survey was not compulsory, and this may have affected the comments as only the more motivated participants may have provided feedback.
There was no method of analysing if participants were unique or repeat users. Furthermore, it was not possible to link the test scores with the evaluation surveys to determine whether test scores correlated with themes from the evaluation survey, which would have allowed us to target 'low scoring' prescribers and identify their characteristics. It was unknown if those who did not complete the evaluation survey felt less positive about the e-learning package, which may have made it less likely that they would complete the survey. Furthermore, there were limitations to the data collected -there was a high ceiling effect, which affected our statistical approach. Another weakness of this study was there was no examination of the longer-term impact of having completed the e-learning course on an individual's prescribing practice. We have completed telephone interviews with seven individuals months after the course, but this has not been included in the evaluation due to the small numbers of interviews conducted. In the future, more interviews could be performed after the e-learning has been completed in order to improve our knowledge of the long-term effectiveness of the e-learning package. One study that addressed the issue of a sustained effect was conducted by Gordon [24]. Paediatric junior doctors were randomly allocated to an intervention group, which received an e-learning course, or a control group. The prescribing skills of doctors in the intervention group were significantly higher at three months, and they manifested greater confidence. However, the authors point out that the effect of this on the number of adverse events is unknown. Another aspect of current prescribing practice that could be addressed by a future e-learning course concerns prescribing for older adults with multiple comorbidities, which often requires changing prescribing practices, and training that is specifically geared to older adults [25]. The population of older adults with multiple co-morbidities is growing, requiring education focusing on the appropriateness of medication in this group, especially with regards to medicine reconciliation and review. Evidence has shown that the use of structured tools to shape prescribing practice, especially in older adults, increases the appropriateness of prescribing decisions.
In this regard, a recent study of hospital doctors found that an e-learning module, focusing on geriatric pharmacology, had a positive effect on prescribing knowledge and confidence [26]. The participants who took the e-learning module obtained significantly higher test scores at four weeks, and maintained them at 12 weeks, with a higher percentage of participants in the intervention group also considering themselves 'confident' when prescribing for older patients [26]. A study of paediatric prescribing [26] identified increased knowledge following an e-learning programme, 97.8% of participants reported increased knowledge.
As noted earlier, one thing that needs to be addressed by future studies is whether improved knowledge, as measured by test scores, influences outcomes for patients. Franchi [27] conducted a randomised control trial in which wards in a hospital were assigned to an intervention, consisting of e-learning focusing on drug prescribing among older patients, or a control group, who did not receive e-learning. No differences in potentially inappropriate medications at discharge or potential drugdrug interactions at discharge were found between the two study groups. However, improvement in knowledge could not be measured in this study as it did not use a prepost design [27].
Prescribing is the most common intervention offered in primary care, and therefore, getting it right is vital. This study shows that prescribers recognise the ongoing need for training in prescribing and the positive engagement with this course demonstrates the suitability of e-learning to provide part of this education. As the Health Foundation [28] has pointed out, although training can help medical students feel more confident about their prescribing, the longer-term impact of training in reducing errors is less clear. However, reducing prescribing errors requires a multifaceted approach, of which ongoing training and competence testing is an important component.
From the data presented here, as well in other literature, it is clear that the lack of pharmacology and prescribing training is one of the factors contributing to errors in prescription items from UK general practice. This suggests that more education is needed in this area, not just for GPs in training, but for qualified GPs and other non-medical prescribers.