Patient satisfaction with out-of-hours GP cooperatives: A longitudinal study

Objective. For over a decade, out-of-hours primary care in the Netherlands has been provided by general practitioner (GP) cooperatives. In the past years, quality improvements have been made and patients have become acquainted with the service. This may have increased patient satisfaction. The objective of this study was to examine changes in patient satisfaction with GP cooperatives over time. Design. Longitudinal observational study. A validated patient satisfaction questionnaire was distributed in 2003 – 2004 (T1) and 2007 – 2008 (T2). Items were rated on a scale from 0 to 10 (1 (cid:2) very bad; 10 (cid:2) excellent). Setting. Eight GP cooperatives in the Netherlands. Subjects. Stratiﬁ ed sample of 9600 patients. Response was 55% at T1 (n (cid:2) 2634) and 51% at T2 (n (cid:2) 2462). Main outcome measures. Expectations met; satisfaction with triage nurses, GPs, and organization. Results. For most patients the care received at the GP cooperative met their expectations (T1: 86.1% and T2: 88.4%). Patients were satisﬁ ed with the triage nurses (overall grade T1: 7.73 and T2: 7.99), GPs (T1: 8.04 and T2: 8.25), and organization (overall grade T1: 7.60 and T2: 7.78). Satisfaction with triage nurses showed the largest increase over time. The quality and effectiveness of advice or treatment were given relatively low grades. Of all organizational aspects, the lowest grades were given for waiting times and information about the cooperative. Conclusion. In general, patients were initially satisﬁ ed with GP cooperatives and satisfaction had even increased four years later. However, there is room for improvement in the content of the advice, waiting times, and information supply. More research is needed into satisfaction of speciﬁ c patient groups.


Introduction
Around the year 2000, Dutch general practitioners (GPs) started reorganizing out-of-hours primary care from small rotation groups into large-scale GP cooperatives.In these cooperatives, 50 to 250 GPs take care of populations ranging from 100 000 to 500 000 citizens [1,2] (see Box I for more characteristics).Other Western countries have also made this shift, for instance Denmark and the United Kingdom [3 -5].Reasons for this reorganization include the low personal commitment of GPs to be on call, the increasing workload with many nonurgent requests for help, and the shortage of GPs in some countries [1 -3,6 -8].
In order to provide patient-centred care, GP cooperatives should include patient experiences in their quality policies.Previous studies have shown

Study design and setting
We conducted a longitudinal observational study into patient satisfaction with GP cooperatives providing out-of-hours primary care.The study was performed in a convenience sample of eight GP cooperatives, located across the Netherlands.Using a patient questionnaire, patient satisfaction was measured in two time periods : 2003 -2004 (T1) and 2007 -2008 (T2) in different patient groups.At T1, the GP cooperatives had existed for one to four years, while T2 was four years later.A stratifi ed sample of 9600 patients was invited to participate in the study.Stratifi cation was based on type of contact: the questionnaire was sent to an equal number of patients who had received telephone advice, a health centre consultation, or a home visit.

Questionnaire
A previously developed and validated questionnaire was used to measure patient satisfaction [17].The questionnaire consisted of background items, an item on care expectations, and items on patient satisfaction.The items on patient satisfaction were divided into three sections: telephone triage nurse, GP, and organization.These sections consisted of different items measuring various aspects of the contact and one item measuring overall satisfaction.The sections regarding the triage nurse and GP consisted of items on communicative and medical skills.
Patients were asked to rate each item on a scale from 1 to 10 (grade 1 ϭ very bad; grade 10 ϭ excellent).For patients who received telephone advice only, the items on satisfaction with the GP were not scored.

Data collection procedure
For each cooperative and at both T1 and T2, 600 patients received the questionnaire equally spread out over three contact types: telephone advice (200), health centre consultation (200), and home visit (200).The questionnaire was sent to the patient ' s home between two and four days after the GP cooperative contact, including a return envelope.In the case of non-response, a reminder was sent after 10 days [17] Administrative contacts and deceased patients were excluded.

Statistical analysis
Mean scores, standard deviations (SD), and 95% confi dence intervals (95% CI) were calculated for each item.In addition, the percentage of negative evaluations (grades Յ 6) was calculated, with corresponding 95% CIs.
Differences in patient and contact characteristics between T1 and T2 were analysed with a t-test (for gender) and chi-squared tests (for age group and contact time).To compare grades at T1 and T2, Mann -Whitney U-tests were used, with differences being considered statistically signifi cant at p Ͻ 0.05.Patients with more than 50% of all grades Յ 6 were indicated as dissatisfi ed patients.Three multivariate logistic Patient experiences are an important element of quality of care.This study showed that a few years after the onset of GP cooperatives in the Netherlands, patients were satisfi ed with this out-of-hours primary care service.On most aspects, satisfaction had even increased four years later.However, there is still room for improvement in the content of the advice, waiting times, and information supply.
Box 1. Features of general practitioner (GP) cooperatives in the Netherlands [1].
• Circa 125 GP cooperatives in the Netherlands • Out-of-hours defi ned as daily from 5 p.m. to 8 a.m. and the entire weekend • Population of 100 000 to 500 000 patients • Distances to GP cooperative not exceeding 30 km • GP cooperative usually situated in or near a hospital • Access via a single regional telephone number, meaning the fi rst contact is mostly with a triage nurse (only 5 -10% walk in without a call in advance) • Telephone triage by nurses supervised by GPs: contacts are divided into telephone advice, centre consult, or GP home visit • Most triage nurses are GP assistants, who received intermediate vocational training (80%); the remaining 20% are nurses with a bachelor ' s degree • Participation of 50 -250 GPs per cooperative with a mean of four hours on call per week • GP shift lasts 6 -8 hours, with a fi xed salary of about 65 euros per hour • Per shift GPs have different roles: home visits, centre consults, and supervising telephone triage • Drivers in identifi able GP cars that are fully equipped (e.g.oxygen, intravenous drip equipment, automated external defi brillator, medication) • Information and communication technology (ICT) support including electronic patient fi les, online connection to the GP car, and sometimes connection with the electronic medical record in the GP daily practice regression analyses were performed to examine whether age, sex, contact time, and type of contact could predict patient dissatisfaction, using all items at T1 and T2.Data were analysed using SPSS 16.0.

Patient expectations
For most patients the care received at the GP cooperative met their expectations: T1: 86.1% and T2: 88.4% (not in table ).The percentage of mismatches of expectation and perceived care (e.g.patients who expected to receive a health centre consultation but who received a telephone consultation) was 13.9% at T1 and 11.6% at T2.

Satisfaction with triage nurse
Satisfaction with the telephone triage nurse significantly increased on all aspects between T1 and T2.
The triage nurse received an overall grade of 7.73 at T1 and 7.99 at T2.At both T1 and T2, patients gave the highest grades on the communication skills " Taking patient seriously " (T1: 7.86 and T2: 8.12) and " Taking time to talk " (T1: 7.86 and T2: 8.07), whereas the medical items " Effectiveness of advice " (T1: 7.13 and T2: 7.47) and " Quality of advice " (T1: 7.32 and T2: 7.67) were given the lowest grades.The percentage of scores Յ 6 per item ranged from 11.3% to 28.8% at T1 and from 10.4% to 23.2% at T2. Between T1 and T2 the percentage of scores Յ 6 signifi cantly decreased on the aspects " Professionalism " , " Taking patient seriously " , " Understanding problem " , " Reassurance " , " Feasibility of advice " , and " Effectiveness of advice " (Table II).

Satisfaction with GP
There was a signifi cant increase in satisfaction with the GP on all items.Patients gave the GP an overall grade of 8.04 at T1 and 8.25 at T2.The GP received the highest grades on the communication skills " Taking patient seriously " (T1: 8.16 and T2: 8.29) and " Friendliness " (T1: 8.12 and T2: 8.27) and the lowest grades on the medical skill " Effectiveness of advice or treatment " (T1: 7.69 and T2: 7.93).The percentage of scores Յ 6 ranged from 8.1 % to 17.2% at T1 and from 7.0% to 15.4% at T2.There were no signifi cant differences in the percentage of scores Յ 6 between T1 and T2 (Table III).

Satisfaction with organization
There was a signifi cant increase in satisfaction on the items " Information about GP cooperative " , " Time between contact and consultation " , and the overall grade and a slight decrease in " Accessibility by telephone " and " Time in waiting room " .Patients gave the organization an overall grade of 7.60 at T1 and 7.78 at T2.They gave the highest grades on " Accessibility by telephone " (T1:7.82 and T2:7.78).The lowest grades were given on " Information about GP cooperative " (T1:7.12 and T2:7.36) and " Time in waiting room " (T1:7.42 and T2:7.27).The percentage of scores Յ 6 ranged from 13.8% to 28.1% at T1 and from 15.3% to 24.3% at T2. Between T1 and T2, the percentage of scores Յ 6 signifi cantly decreased on " Information about GP cooperative " and the overall grade (Table IV).

Predictors of dissatisfi ed patients
Of all patients, 11.6% (n ϭ 569) expressed their dissatisfaction on more than half of the items regarding the triage nurse, 14.6% (n ϭ 712) regarding the organization, and 8.2% (n ϭ 278) regarding the GP.Patients older than 65 years were less often dissatisfi ed with the triage nurse, GP, and organization (OR ϭ 0.55, OR ϭ 0.58, and OR ϭ 0.66 respectively).Parents of patients under the age of fi ve were less often dissatisfi ed with the triage nurse (OR ϭ 0.68).Male patients were less often dissatisfi ed with the GP (OR ϭ 0.69).Patients who had received telephone advice were more often dissatisfi ed with the triage nurse (OR ϭ 1.78) and organization (OR ϭ 2.05).Patients with home visits were more often dissatisfi ed with organizational aspects (OR ϭ 1.61; Table V).

Principal fi ndings and interpretation
A few years after the onset of GP cooperatives, patients expressed satisfaction with the triage nurses, GPs, and organization of the cooperatives, and on almost all aspects satisfaction even increased four years later.
There were two negative trends: satisfaction with the time in the waiting room and accessibility by telephone showed a slight, but non-signifi cant, decrease between the two measurements.
The lowest scoring items were the time in the waiting room and information about the cooperative.The highest grades were given for the communicative skills of the triage nurses and GPs, whereas medical skills -the quality and effectiveness of the advice and treatment -received lower scores.
Patients who received telephone advice were more often dissatisfi ed with the triage nurse and the organization than patients who had a consultation at the cooperative.This might be related to the nurses ' gate-keeping function on the phone, as Dutch patients access the GP cooperative mostly via a regional telephone number (see also Box I).The triage nurses can provide self-care advice, even if the patient was expecting a GP consultation, Previous research has shown that a mismatch between expectations and actual care is associated with a negative evaluation [9].
There were fewer dissatisfi ed respondents among (parents of) young children, the elderly, and male patients.As there were more elderly patients at T2, this could have contributed to the increase in satisfaction scores.
The aim of our study was to examine patient satisfaction with GP cooperatives over the years.We did not compare the results with the situation before the reform of out-of-hours care.Patients could have been even more satisfi ed in the former system of small rotation groups, but the reform was mainly introduced to reduce the workload of GPs and to manage the shortage of GPs [1].The considerably high and even increasing satisfaction scores in our study seem to show that patients have accepted the reform.We cannot conclude that satisfaction is optimal, as it is unknown what maximum level is achievable.
Over the past years, patients have been informed about the working routines of GP cooperatives and they have got used to the changed situation.This may have positively infl uenced patients ' opinions, especially regarding triage nurses.Initially, there were concerns about the role of triage nurses [18].Patients were anxious about their ability to describe symptoms over the telephone, or to understand and follow up the advice received [19].In the course of the years, patients have become acquainted with the function of the triage nurse, and the positive experiences may have resulted in an increased appreciation.Moreover, over time, the clinical experience of triage nurses has increased and implementation of certifi ed education for triage nurses started in 2007.During T2, implementation of this programme was still going on, increasing the number of certifi ed nurses.On the other hand, satisfaction is a subjective measure: satisfaction may increase if expectations decrease due to social or other changes.

Strengths and weaknesses
A large number of patients participated in this multicentre longitudinal study, contributing to the  8.5 -11.4) 9.0 (7.6 -10.4) Clear explanation 8.02 (1.73) (7.94 -8.11) representativeness of the results.The response rates were similar to response rates in other patient survey studies in this health care sector [5,9,10, 20 -22].However, we did not perform a non-response analysis.Therefore we do not know if patients who responded to the questionnaire differ from patients who did not.An extensive non-response analysis in a previous study, which partly used the same questionnaire, did not reveal any important differences between the response and non-response groups.Also, no relation was found between the response rate of the participating GP cooperatives and the satisfaction scores [17].We do not have information on the group of non-responders in our study.Compared with the total group of patients contacting a GP cooperative, reported for one region in the Netherlands [23], there were more elderly people in our study (about 30% in our study versus 19%) and fewer children (about 20% versus 28%).The gender distributions were almost the same (about 47% male patients in our study versus 43%).
Although most of our results were statistically signifi cant, the actual differences in grades between the two measurements were small: sometimes only one or two tenths of a point on a 10-point scale.This reduces the clinical relevance of our fi ndings.However, because there was an increase in scores on almost all aspects at the second measurement (T2), we believe we have found evidence that patient satisfaction has improved over time.
Furthermore, the timing of the fi rst measurement (T1) varied from one to four years after the inception of the GP cooperative.It is unclear how this infl uenced the results.A study by Christensen and Olesen showed that satisfaction decreased just after the onset of GP cooperatives, but it began to increase again as people got used to the service [4].Perhaps the differences  between T1 and T2 in our study would have been even larger if all GP cooperatives had had their fi rst measurement only one year after the start of the cooperative.Finally, the questionnaire contained items on a broad range of topics relevant to GP cooperatives.Our study showed high satisfaction on these themes.Nevertheless, patients may be less satisfi ed with factors and conditions which have not been asked about in the questionnaire.

Comparison with other studies
One study, by Christensen and Olesen, also reported longitudinal results on patient satisfaction with GP cooperatives [4].One year after the onset of GP cooperatives in Denmark, 73% of the respondents were (partly) satisfi ed.Similar to our results, they found an increase in satisfaction three years later (81% (partly) satisfi ed).Four years after the onset of GP cooperatives in Denmark, 86% of the patients received the type of service they had expected and these fi ndings are comparable to our results [4].
Other studies, using cross-sectional study designs, have also shown comparable results.In a study by Giesen et al.,conducted in 26 GP cooperatives in the Netherlands, which included the T1 data of telephone consultation patients from our study, the percentage of negative evaluations (scores Յ 6) ranged from 12% to 33% for aspects concerning triage nurses and from 19% to 35% for aspects concerning organization [9].Moreover, Van Uden et al. reported an overall satisfaction rate of 76% for telephone advice and 80% for GP consultation or home visit in Dutch GP cooperatives [19].In the United Kingdom, various studies have shown that patients are satisfi ed with the GP cooperative, especially with regard to health centre consultations and home visits by GPs [5,21,24].

Implications for future research and clinical practice
The results of our study indicate that in general patients are satisfi ed with the GP cooperative, but there is room for improvement concerning the medical skills of the professionals (i.e. the quality and effectiveness of the advice and treatment), as these skills received lower scores than communicative skills.Future studies should examine in more detail which elements of the medical skills can be improved, and education of professionals should focus on these skills.Moreover, the waiting time at the GP cooperative is an important issue to tackle in quality improvement initiatives, as patients gave low absolute scores on this item and the scores even decreased over the years.Finally, professionals should be trained to make expectations of care more explicit during the fi rst contact with the GP cooperative, to improve patient satisfaction.
The second measurement in our study was made about fi ve years ago.In recent years, additional developments have taken place in out-of-hours care and GP cooperatives, such as increasing collaboration with hospital emergency departments with a shared entrance, increasing education and certifi cation of triage nurses, continuing implementation of computerized triage systems to support decisions, and use of supervising telephone GPs.We recommend further measurements of patient satisfaction with GP cooperatives, to obtain up-to-date results which take recent developments into account and to examine whether satisfaction still increases, stabilizes, or decreases.
Finally, more research is needed into the satisfaction of specifi c patient groups, such as the elderly, chronically ill, and non-natives, preferably using both quantitative and qualitative research methods.In this way, the effect of specifi c measures can be evaluated and optimal patient-centred care can be achieved.

Table I .
Patient and contact characteristics.

Table II .
Satisfaction with the triage nurse: mean score and percentage of negative evaluations on T1 and T2.

Table III .
Satisfaction with the GP: mean score and percentage of negative evaluations on T1 and T2.

Table V .
Predictors of dissatisfaction with telephone nurse, GP, and organization.Ref ϭ reference group.1ORϽ 1 indicates that the odds of being a dissatisfi ed patient are smaller compared with the reference group; OR Ͻ 1 indicates that the odds are larger.2Patientswho received telephone advice from the triage nurse did not score any of the items on satisfaction with the GP.* * p Ͻ 0.01; * * * p Ͻ 0.001.

Table IV .
Satisfaction with the organization: mean score and percentage of negative evaluations on T1 and T2.