|Year : 2011 | Volume
| Issue : 3 | Page : 539
Remote and rural: Do mentors enhance the value of distance learning continuing medical education?
K Butterworth1, B Hayes2, M Zimmerman3
1 Patan Hospital, Lagankhel, Kathmandu, Nepal
2 National Academy of Medical Sciences, Bir Hospital, Kathmandu, Nepal
3 Nick Simons Institute, Jhamsikhel, Lalitpur, Nepal
|Date of Submission||25-Aug-2010|
|Date of Acceptance||01-Nov-2011|
|Date of Web Publication||09-Dec-2011|
Patan Hospital, Lagankhel, Kathmandu
Source of Support: None, Conflict of Interest: None
Context: Experts suggest that distance learning continuing medical education (CME) is only effective when there is the opportunity for two-way discussion and reflection. The value of on-line mentoring has been mainly studied in the West.
Objectives: We examined the benefits and practical implications of providing mentors for distance learning CME in a low technology setting.
Methods: We conducted a randomized controlled trial with qualitative and quantitative analysis of the impact of mentoring on completion of CME and quality of reflective learning.
Results: Twenty-six of 64 doctors completed all four CME modules. Non-completers were interviewed by telephone. Odds ratio analysis suggested that mentored doctors were three times more likely to complete their CME; however, this did not reach statistical significance (p=0.07, 95% CI 0.89-10.57). Being in rural practice (p=0.05) and younger in age (p=0.005) were significantly associated with completion of CME. Mentored doctors seemed to show a higher quality of reflection on learning. Contact between mentors and mentees was difficult. Both mentors and mentees felt that optimal use of the system was not made.
Discussion: Despite mentors' perceptions that they had little impact, mentored doctors did appear to be more likely to complete CME. Work is needed to increase the quality of interpersonal and educational interaction between mentors and mentees.
Keywords: Continuing medical education, distance learning, mentoring
|How to cite this article:|
Butterworth K, Hayes B, Zimmerman M. Remote and rural: Do mentors enhance the value of distance learning continuing medical education?. Educ Health 2011;24:539
One of the challenges for providers of distance learning continuing medical education (CME) materials is how to encourage doctors to use what they learn, to the patients’ benefit. Interactive learning, with opportunities to practice, is key to changing what doctors actually do rather than solely improving their knowledge1.
Experts suggest that distance education programmes are only successful if there is some provision for two-way communication in the educational process with guidance, counselling and timely feedback on assignments2. Almost all of the research on distance learning and e-mentoring has come from the West. However, there is huge potential for implementation of distance learning in low resource settings, using simpler technology.
In 2007, an extensive evaluation of the CME needs of doctors working in rural Nepal3 showed that over 60% of rural doctors had access to a computer, but not internet. Many respondents expressed the desire for someone to guide them through a CME programme. In this study, we aimed to define the benefits and practical implications of providing mentors for distance learning CME, in a low technology setting for doctors in rural Nepal.
We used a stratified, randomized control trial over a six-month period, combining quantitative and qualitative measures.
Participants: We were particularly interested in targeting junior doctors and postgraduate general practitioners (GPs) working in remote, rural areas. We advertised the CME programme widely in the national press, at a national GP conference and with posters in rural hospitals. While there are approximately 8,000 doctors in Nepal, there are only about 200 postgraduate GPs in the whole country.
Many junior doctors act as generalists in small hospitals without any specific training or support. We accepted all doctors (excluding those with an Ayurvedic background and health assistants) who applied to participate in the study during the six weeks prior to the start of the programme.
Although we were targeting remote and rural doctors, we accepted as participants all those who responded to the advertisement. Approximately one-half of the applicants were currently working in Kathmandu Valley. There is a great deal of mobility amongst junior doctors and we reasoned that some of these doctors might later also move to rural practice. Those doctors expressing willingness to participate in the project were subdivided into Kathmandu-based and rural practitioners, and assigned a number. A computer-generated random number list was used to place physicians into an intervention (receiving a mentor) or a control group (no mentor). All doctors agreeing to participate in the study signed an informed consent form.
Programme: Project participants could choose four CME modules out of seven: Emergency Medicine; Paediatrics; Obstetrics and Gynaecology; Chronic Medicine; Psychiatry; Dermatology; and Infectious Disease. Eight general practitioners acted as mentors; three expatriate GPs with extensive experience in Asia and five senior Nepali GPs. There was a short one-half-day workshop on mentoring for those with no previous mentoring experience. Each mentor was assigned four mentees, two urban and two rural.
A facilitative, reflective model of mentoring was promoted (see Table 1).
Table 1: Model of mentoring used for the study intervention group
Evaluation: For each module, there were evaluation forms designed to enhance mentees’ reflections on learning, including a three-month reflection at the end of the study. After six months, a structured questionnaire was also sent to mentees regarding their experiences during the mentoring process.
Results from written feedback were analysed independently by two authors, looking for key themes, and then discussed as a team. This information was then used to inform areas and topics to explore in-depth during a focus group discussion (FGD), held in Kathmandu. All participants were invited to attend.
The FGD was recorded on tape, transcribed and key themes were extracted independently by two of the authors. Themes were discussed and further analysed by the rest of the team, consisting of all three authors and our Nepali secretary. Identified themes were triangulated with feedback from mentors and quantitative data from those completing CME.
Mentors met monthly during the study period, and also completed a semi-structured questionnaire at the study close. Additionally, for participants failing to complete the entire CME, considerable effort was made to contact them, and a semi-structured interview was conducted by phone.
Analysis: Data from participants completing CME and the evaluation forms were analysed related to age, gender, qualification (MBBS only or postgraduate qualification) and place of work, as well as mentoring status. We used simple logistic regression to model whether or not a doctor had completed an evaluation as well as whether they had completed all four modules. Covariates in the model included age, gender and the doctor’s qualifications. Our main exposures were whether or not the doctor was mentored and whether they were currently Kathmandu Valley residents.
Overall, 64 doctors were enrolled into the project. Most of the doctors (58; 90.6%) were either junior doctors, without postgraduate qualifications, or GPs. A few specialists also applied – mainly specialist graduates but currently working in a generalist setting. Thirty-four were working in the Kathmandu Valley, while 30 were located in rural practice. In total, 16 doctors in the Kathmandu group and 16 doctors in the rural group were assigned to mentors.
Between-groups analysis by location, for both intervention and control groups, revealed that those outside Kathmandu were older by about 6.3 years (p=0.06) and were less likely to be women (p=.003). The number of years since qualification was only slightly higher outside of Kathmandu (p=0.06), but participating doctors in Kathmandu were less likely to have a postgraduate qualification (p=0.05). Analysis comparing the mentored with non-mentored groups showed no significant differences in the demographic characteristics of overall years of experience, age and postgraduate status.
During the study period, 26 of 64 doctors (40.6%) completed all four CME modules. Of these, 17 (65.4%) were mentored. Computation of an odds ratio suggested that mentored doctors were three times more likely to complete their CME; however, this did not reach statistical significance (p=0.07, 95%CI 0.89-10.57), possibly due to small numbers. Doctors working outside the Kathmandu Valley, as well as younger doctors, were more likely to complete all four CME modules. However, completion did not vary by gender, years since internship or postgraduate qualification.
Qualitative review of the reflection in evaluation forms showed that, in general, doctors who were being mentored gave more detailed and specific comments on their CME experiences. Every CME participant was asked to comment on what they had learned from each module they completed, and based on this new knowledge, how they would practice differently in the future. For example, upon completion of a module on asthma, a mentored participant stated: 'Will use 8 puffs of MDI in place of two that I’m using at this time during acute attacks of asthma.' On the other hand, a non-mentored participant simply wrote: 'Management.' As a second example, after a module on fungal nail infections, the mentored physician response was: 'Current recommendations for fungal nail treat with fluconazole 150mg weekly 3-6 months,' while that of the non-mentored participant was 'proper oral therapy'.
Only twelve three-month reflection forms were collected, three from non-mentored doctors and nine from mentored. Although odds ratio estimates suggested that mentoring, younger age and rural practice were associated with completion of reflection, results were not statistically significant.
Only five doctors participated in the focus group discussions. On the other hand, seven mentors gave written feedback and participated in regular meetings where key feedback was provided. Most mentors initially contacted their mentees every one to two weeks. However, due to the difficulty of being in contact and often lack of response, several mentors later reduced the frequency of contact to every two to four weeks. Mentors expressed frustration at the difficulty in contacting their mentees. Communication by cell phone was not reliable and not all mentees had easy access to email. Most mentors used both phone and email, but found email easier. Additionally, during the study period there was an average of 18 hours of power cuts per day in Kathmandu, with variable power cuts in other parts of the country. This made it difficult for participants to complete the CME and for communication between mentors and mentees by email.
Seven mentors indicated that they generally enjoyed the personal contact with mentees but felt they did not add much to mentee learning. Mentees mostly asked about process issues. There was little reflection on CME content, or opportunity to apply the content to clinical practice. This was confirmed in the qualitative data from mentees.
Four mentees said they found having a mentor very useful, three somewhat useful and one not useful. Several mentees mentioned that they would value face-to-face meetings with their mentor, particularly at the end of a module. One of the main roles of the mentor was seen as someone to encourage mentees to complete the modules.
FGDs brought out several key themes regarding mentoring. First, mentees did not make use of the mentoring system. This was indicated by both mentees and mentors to be due to lack of knowledge about the concept. Second, mentors felt that most of the functions they completed could just as easily have been done by an administrator. Third, some mentees did not seem interested in having a mentor. Yet, there was feedback from those in the non-mentored group that they would have appreciated more interaction with the CME provider.
Finally, we were able to contact 29 of 38 (76.3%) of the doctors who did not complete the CME, and conducted a phone interview to get their feedback. Nine doctors were lost to follow-up. Six had completed all four CME modules, but after the end of the study period. Three had completed two or three modules, 12 completed one module and eight never completed any. The main reason for failing to complete the CME was lack of time. Several doctors moved their workplace during the study period, and had difficulty either accessing a computer or had left the country. Lack of electricity was mentioned by three individuals.
While results suggested that mentored doctors were more likely to complete the CME, the low completion rate in this study failed to show a statistically significant impact of mentoring. Yet, lack of supervision has been found to be a key factor in poor motivation of participants in distance learning participation for rural medical officers in Kenya4.
Participants were asked to record three new things that they learned and how they might change their practice. Grounded in learning theory, this was taken as a surrogate measure of the depth of the learning experience. Mentored doctors tended to give more specific and detailed responses to these questions, perhaps suggesting better reflection.
To improve the quantity and the quality of interpersonal interaction between mentors and mentees, level of social comfort needs to be addressed5. Several participants said they would value face-to-face meetings with their mentor. More research is needed on how best to facilitate good rapport and trust within the mentoring relationship in the context of remote and rural medicine.
The mentors in this study were disappointed in the level of interaction with mentees. There was limited reflection on actual clinical practice. Mentors were interested in aiding this level of analytic reflection but found that mentees did not respond. It may be necessary to provide further faculty development training in how to facilitate on-line discussion5
Overall, in answer to the study question of provision of mentors enhancing the value or impact of CME, we would answer a qualified 'yes.' Mentored doctors appeared to be more likely to complete CME. Mentoring may also improve completion of tools designed to encourage greater depth of learning and implementation of knowledge.
Finally, lessons learned from our experience suggest adding the following: an administrator to provide regular reminders to complete CME; participants being allowed to choose the option of a mentor; education of mentees on the purpose of mentoring; an increase in the level of social comfort felt by the mentee; interactive tasks as part of the CME that require more reflection on learning and practical implementation in the workplace; and training mentors in how to increase the educational value of the mentor-mentee interaction.
We would like to thank Ms. Deepa Mukhia for her hard work in gathering data and conducting telephone interviews, and Mr. Bob Gerzoff for his support in statistical analysis.
1. Mazmanian P, Davis D. Continuing medical education and the physician as a learner: Guide to the evidence. Journal of the American Medical Association. 2002; 288 (9):1057-1060.
2. Knebel E. The use and effect of distance education in health care: What do we know? Operations Research Issue Paper 2 (2) Quality Assurance Project 2001.
3. Butterworth K, Hayes B, Zimmerman M, Knoble S. Needs assessment for continuing medical education amongst doctors working in rural Nepal. SE Asian Journal Medical Education. 2010: 4(1):34-42.
4. Nyarango PM. Distance learning for rural medical officers in Kenya: The first pilot project. East African Medical Journal. 1991: 68(9):741-743.
5. Sargeant J, Curran V, Jarvis-Selinger S, Ferrier S, Allen M, Kirby F, Kendall H. Interactive on-line continuing medical education: Physicians’ perceptions and experiences. Journal of Continuing Education in the Health Professions. 2004; 24:227-236.