|ORIGINAL RESEARCH PAPER
|Year : 2010 | Volume
| Issue : 3 | Page : 389
Innovative Method of Needs Assessment for Faculty Development Programs in a Gulf Medical School
BV Adkoli, KU Al-Umran, MH Al-Sheikh, KK Deepak
College of Medicine, King Faisal University, King Fahd Hospital of the University, Al-Khobar, Saudi Arabia
|Date of Submission||23-Aug-2009|
|Date of Acceptance||01-Oct-2010|
|Date of Web Publication||30-Nov-2010|
B V Adkoli
P.O. Box 2208, Al-khobar 31952
Source of Support: None, Conflict of Interest: None
Background: Faculty development lays the foundation for the quality enhancement in medical education. However, programs are not always based on the needs of the participants, and there is dearth of information on methods to derive faculty's needs. The Medical Education Unit at the University of Dammam, Saudi Arabia, carried out an innovative method to identify and prioritize faculty needs in order to plan future activities.
Methods: A questionnaire was designed, pilot-tested and administered to all faculty members (N=200). The respondents rated the perceived importance (high, moderate, low) and their performance (good, average, poor) on twelve competencies described in the literature. The ratings of perceived importance - high/moderate, and self-rated performance- average/poor, were summed up to determine priority rankings for continuing education. The respondents' rating of various continuing education activities, their willingness to participate and commit time, and their suggestions for strengthening faculty development were also analyzed.
Results: All the twelve competencies were perceived as 'highly important' by the subjects. They felt most confident in teaching in large and small groups, attitudes and ethical values, and decision making skills. The competencies prioritized as "gaps" were knowing how to develop learning resources, plan curriculum, evaluate courses and conduct research. The prioritized activities were specialized courses, orientation workshops for the new faculty, and training in educational research skills. This implied a multi-phased approach to faculty development. A majority (62.4%) were willing to devote 2.2 hours per week to faculty development. Respondents suggested initiatives that should be undertaken by the Medical Education Unit and the broader institution.
Conclusion: We demonstrated a participatory approach to needs assessment by identifying the gaps between "perceived importance" and "self-rated performance", as criteria for determining priorities. Findings also demonstrated the need for adopting a comprehensive approach to faculty development in which both departmental and organizational initiatives are required. Our findings are applicable to the Gulf Region context and our methodology can be applied anywhere.
Keywords: Need assessment, faculty development, medical education, continuing medical education
|How to cite this article:|
Adkoli B V, Al-Umran K U, Al-Sheikh M H, Deepak K K. Innovative Method of Needs Assessment for Faculty Development Programs in a Gulf Medical School. Educ Health 2010;23:389
Faculty development (FD) has assumed a great significance in meeting the diverse roles and responsibilities of a medical educator as a clinician, researcher, administrator, and educational leader1. The main driving forces for FD are public accountability, the changing nature of health care delivery and the need to sustain personal academic vitality2.
A volume of literature on faculty development has accumulated over the past two decades replete with useful lessons. It has been shown that FD programs should be tailored to the needs of institutions, departments and individuals, take a systematic approach in their planning, implementation and evaluation, utilize self-directed learning and participatory education approaches, and contribute to both professional and personal development of the faculty3-4. Of these, the initial step of needs assessment is perhaps most crucial because it helps all teachers realize their fullest potential and enables program organizers to optimize and prioritize their activities.
Medical schools in the Gulf Region have witnessed a rapid increase in the enrolment of students, leading to a shortage of faculty. Many schools have established Medical Education Units (MEU) to organize faculty development5-6. The Medical Education Unit of the College of Medicine, University of Dammam was established in 2005 with the objective of enhancing faculty skills in teaching, assessment and educational research. The college follows a conventional curriculum and is operated by Saudi faculty and expatriates of diverse backgrounds. A logical step for those of us directing the Medical Education Unit was therefore to conduct a needs assessment to inform us in planning our activities for the future.
Our principal objectives were to use a participatory process to prioritize the competencies on which to focus and the functional activities to be undertaken by the unit. In our development efforts we also wanted to estimate the willingness of faculty to participate and the time they would commit, so we could target a realistic set of activities. Finally, we sought to elicit suggestions from the faculty for ways to strengthen our FD initiatives.
Previously published faculty needs assessment efforts of schools have employed a variety of tools and techniques, including questionnaires7-12, Likert scales13-14, focus groups15, interviews16, and Delphi techniques17, generally to assess continuing medical education (CME) needs in various settings. A few studies have captured the differences between perceptions of ‘what an ideal CME should be’ and ‘what is actually practiced’18. There were two studies particularly relevant to our study. One attempted to assess the difference between faculty participants’ ‘current ability’ and ‘ideal ability’ in their various roles19 and the other study dealt with the difference between ‘perceived importance’ and ‘self-rated level of competence’ as the basis for prioritizing faculty development needs20. Another important development in the field is the effort of Hesketh et al.21 to identify twelve competencies expected of a medical educator.
Our study considers these twelve competencies as the basis for a FD needs assessment in our school and takes the view that, ideally, the focus of FD programs should be prioritized on the basis of the difference (gap) between expected competencies and actual performance of faculty. The greater the gap between expectations and performance, the greater the need for these areas to be stressed in FD efforts.
We drafted a semi-structured, partly open-ended and partly closed-ended questionnaire. Central in the questionnaire was a three-point Likert scale in which respondents were asked to rate the perceived importance (high, moderate, low) and their perceived current performance (good, average, poor) on each of twelve competencies derived from the work of Hesketh et al.21 (see Appendix).
Other important issues addressed in the questionnaire were the perceived functional activities and services provided by the Medical Education Units. The subjects were asked to rate the activities in terms of relevance and usefulness to bolstering their performance. These items were derived from the literature (Davis et al.3) and the experience of the researchers. Subjects were also asked to indicate if they were willing to participate in FD programs and the amount of time that they could devote to this purpose. An open-ended question was used to elicit comments and suggestions from the respondents for strengthening FD. A draft questionnaire was created and reviewed by the research team for its content validity, then pilot-tested with ten faculty member volunteers to assess the instrument in terms of time required for completion, language and user-friendliness. Based on feedback, the questionnaire was modified and finalized. It was administered through the chairpersons of 25 departments to the entire faculty of College of Medicine (N=200). Completion was voluntary and respondents were reassured that responses would be confidential.
Method of analysis: The analysis of numerical data was carried out using Microsoft Excel. The ratings made by the participants with respect of the ‘perceived importance’ versus ‘self-rated performance’ for each of the twelve competencies were tabulated in the form of a 3 x 3 contingency table, as illustrated in Table 1.
Table 1: Illustrative contingency table showing how ratings made by the participants on “Perceived Importance” and “Self-rated Performance” for a given competency (teaching in large/small groups in this example) are combined to calculate a priority score.
We counted the number of responses for perceived importance as well as self-rated performance for each of the twelve competencies. For identifying the priority scores, we summed the four cells pertaining to Importance (High/Average) and Performance (Poor/Moderate). This sum represents the gap between importance and performance, or the ‘training deficit’. Based on the deficit scores, the priorities of competencies were ranked.
For prioritizing the functional activities expected of the Medical Education Unit, we relied on response counts and ranked them accordingly. Similarly, we counted the number of subjects who were willing to participate as teachers or resource persons and calculated the mean time and range to quantify this information.
The qualitative analysis of the open-ended comments and suggestions for strengthening FD was initially carried out by one researcher who listed all comments, identified general themes and sub-themes and then grouped all comments within them. This work was then checked independently by a second researcher as a verification of the themes and the proper grouping of all the comments under the appropriate theme. Theme assignment differences were resolved through discussion.
A total of 109 questionnaires were returned, yielding a response rate of 54.5%. Table 2 describes the response rate by gender, faculty rank and by pre-clinical versus clinical department appointment. All the 25 departments of the undergraduate course (MBBS) were represented among respondents except Microbiology. Sixty-seven (61.5%) had more than 15 years of experience on the faculty. Expatriate faculty outnumbered Saudi nationals 55% to 45%. Response rates were not different among faculty ranks and between clinical and pre-clinical departments. However, the response rate was higher in males versus females.
Table 2: Survey participation and response rate by gender, faculty rank, and pre-clinical versus clinical department appointment
Prioritization of Competencies
Table 3 shows the respondents’ rating of perceived importance, self-rated performance, the priority scores and the ranking of priorities arrived for each of the twelve competencies. All twelve competencies were perceived as important by the faculty, as revealed by their ratings which ranged between 72 and 86. However, their self-rated performance received highest ranking on teaching in large/small groups (81), attitudes and ethical values (80), and decision making skills (74). The faculty felt least confident in developing learning resources (42), planning curriculum (46), and evaluating courses and conducting research (47).
The priority scores derived (as per the procedure outlined in Table 1) from the sum of ratings of four cells – perceived importance high/average and self-rated performance poor/average are shown in the Table 3. Developing learning resources (54) received highest priority followed by planning curriculum (52) and evaluating courses and conducting research (49). Decision-making skills (23), attitudes and ethical values (16), and teaching in large/small groups (16) received lowest priority.
Table 3: Respondents' rating of perceived importance, self-rated performance, priority scores and ranking with respect to twelve faculty competencies
Ranking of Suggested Activities For Faculty Development
The activities prioritized by the respondents from the given list (as provided in the questionnaire) are shown in Table 4. The table includes additional activities suggested by the respondents. Organization of specialized workshops, for example: 'How to frame Multiple Choice Questions?' (91.7%), orientation program for newly recruited faculty (91.7%), and training in educational research (88.9%) top the priority list of activities and services. The lowest priority was assigned to consultation services to the faculty, e.g., how to lead a small group discussion (70.6%), assisting faculty in pursuing educational projects (68.8%), and sponsoring a few interested faculty for higher degree programs in education (67.9%).
Table 4: Ranking of suggested activities for faculty development
Faculty Development Need Assessment Survey (2009) College of Medicine, University of Dammam
Willingness to Participate in the MEU Activities
A majority of the participants (62.4%) expressed willingness to participate in the activities of MEU and ability to spare an average of 2.2 hours per week (range 1 – 8) for this purpose. Those willing to join as facilitators or resource persons (37.6%) indicated that they can spare 2.4 hours per week. Nineteen percent indicated that they have no time for FD activities.
Comments and Suggestions for Strengthening Faculty Development
The comments and suggestions offered by the respondents fell within two categories: initiatives to be addressed by the school and those to be undertaken by the MEU (Table 5). The initiatives expected at the school level were to make a policy decision for participation in FD to be mandatory for faculty, strengthen faculty development infrastructure and facilities, and provide incentives and recognition. The initiatives expected of the MEU were to assume more power and responsibilities to monitor course delivery, prepare an annual calendar of activities, distribute learning resources, and emphasize practical aspects of faculty development.
Table 5: Comments and suggestions given by the respondents for strengthening faculty development and functioning of school’s Medical Education Unit
Our study demonstrates a participatory approach to a medical school’s needs assessment in which the entire faculty is involved in identifying FD needs and priorities. We have borrowed a framework from earlier studies and applied a method of arriving at the priority areas based on the gap between the skills that faculty perceive as most important for their roles and how they rate their own performance in these skills.
The needs identified and prioritized by our respondents can be explained on the basis of their background experience and local context. The respondents in our study are mostly senior faculty with more than 15 years of experience. It is natural for them to think that their teaching skills are satisfactory, but they need support in producing learning resources, planning curriculum and conducting research, especially in view of the challenges faced in handling new technologies like e-learning and sustaining leadership towards the end of their career. Decision-making seems to have received low priority by a high proportion of our predominantly expatriate faculty, who do not hold many administrative responsibilities in our context. Attitudes and ethical values are often thought to be "self-learned behaviors", rather than "molded" by faculty development. Contrary to our expectation, the competence in learner assessment was given only average priority. This might be explained by the training the faculty already received through a series of workshops in this area conducted by the MEU during the last four years.
The activities prioritized for faculty development through identifying gaps between perceived importance and current performance should be considered along with the faculty’s suggestions for strengthening FD initiatives. Our respondents expect the MEU to tailor its activities for faculty at different levels of experience. Accordingly, activities should be phased in such a manner that all new faculty are sensitized through an orientation program. Most would attend basic instructional workshops, a few would opt for specialized workshops on different aspects, pursue research and assume educational leadership. This has been previously described as multi-phased22 or tiered approach to faculty development23. Low priorities assigned to providing consultation to the faculty on specific issues (e.g., how to conduct a small group discussion), assisting the faculty in pursuing educational projects, and sponsoring interested faculty for higher degree programs abroad (e.g., Masters in Health Professional Education) indicate that most faculty are not yet ready for long-term programs leading to educational leadership.
A major development in the field of FD is the premise that the success of FD initiatives does not rest on the efforts of the few individuals who organize them, but depends upon organizational variables including the institution’s infrastructure and facilities, overall leadership, faculty incentives and recognition for contributions to FD24-28. This is reflected in our respondents’ suggestions for strengthening FD and the operations of the MEU, which have been categorized as initiatives needed by the MEU and those needed by the institutional leadership. Preparing a calendar of activities and distributing learning resources are within the purview of MEU. On the other hand, the steps to be taken for making the faculty participation a mandatory requirement, providing incentives, standardizing Multiple Choice Questions, monitoring the quality of courses, and strengthening college-wide infrastructure rest on policy choices made by the top school administration.
A valuable addition to our study is the data obtained on the willingness of the faculty to participate in FD and the time they are willing to commit as either a participant or a resource person. This has not been highlighted in the earlier studies. We plan to create a database of those willing to join as participants and resource persons, to be used in scheduling future programs. Those who are not willing can be advised to pursue alternative career pathways as core scientists, clinical specialists or researchers.
Our study has some limitations. The data generated through the questionnaire, especially self-ratings of performance, is likely subject to desirability bias and over-represents the opinions of male faculty. The faculty may not accurately recognize the limitations of their skills as teachers. Respondents likely over-reported their willingness to commit time to FD, and their actual cooperation with future activities stemming from this survey may well be less than they promise. Moreover, what we have gathered is preliminary information. Further assessments are needed to validate our findings by incorporating other tools such as Delphi technique, focus groups, interviews and external consultation. Though the findings of our study are immediately applicable in our setting, the methodology should be replicable in any setting worldwide.
Through our needs assessment survey, we demonstrated a participatory approach which is simple, feasible and useful in the Gulf Region context and should also be in other regions. We focused on identifying the gap between faculty’s perceived importance and self-rated performance in twelve skill areas, as criteria for prioritizing FD content. Our study also led to a database for identifying participants who are motivated and willing to give time to FD as facilitators and learners. This study also highlighted the need for adopting a multi-phased approach in delivering FD tailored to faculty at each career stage, and emphasized the roles of both medical education units and schools in strengthening faculty development.
1. Bligh J, Brice J. Further insights in to the role of medical educator. Academic Medicine. 2009; 84:1161-1165.
2. Gruppen LD, Simpson D, Sealer NS, Robins L, Irby DM, Mullan PB. Educational fellowship programs: common themes and overarching issues. Academic Medicine. 2006; 81:990-994.
3. Davis M, Karunathilake I, Harden RM. AMEE Education Guide no. 28: The development and role of departments of medical education. Medical Teacher. 2005; 27:665-675.
4. Mclean M, Cilliers F, Van Wyk JM. AMEE Guide No. 36. Faculty Development: Yesterday, today and tomorrow. Medical Teacher. 2008; 30:555-584.
5. AL-Wardy NM. Medical Education Units: History, functions and Organization. Sultan Qaboos University Medical Journal. 2008; 8:149-156.
6. Hamdy H, Telmesani AW, Al Wardy N, Abdel-Khalek N, Carruthers G, Hassan F, Kassab S, Abu-Hijleh M, Al-Roomi K, O’malley K, El Din Ahmed MG, Raj GA, Rao GM, Sheikh K. Medical Teacher. 2010; 32:219-224.
7. Mann KV. Not another survey! Using questionnaires effectively in needs assessment. The Journal of Continuing Education in the Health Professions. 1998; 18:142-149.
8. Laidlaw TS, Mc Leod H, Kaufman DM, Langille DB, Sargent J.. Implementing a communication skills program in medical school: needs assessment and programme change. Medical Education. 2002; 36:115-124.
9. Maltais P, Goulet F, Borduas F. Educational skills and knowledge needed and problems encountered by Continuing Medical Education providers. The Journal of Continuing Education in the Health Professions. 2005; 20:91-96.
10. Wood TJ, Marks M, Jabbour M. The development of a participant questionnaire to assess continuing medical education presentations. Medical Education. 2005; 39:568-572.
11. Leite P. Faculty Development Needs Assessment Survey. 2007. Kansas State University’s College of Technology and Aviation accessed from http://www.sal.k-state.edu/facultystaff/mission.htm
12. Farley H, Caseletto J, Ankel F, Young KD, Hockberger R. An assessment of the Faculty Development needs of junior clinical faculty in Emergency Medicine. Academic Emergency Medicine. 2008; 15:664-668.
13. Robinson BE, Barry PP, Resnick N, Bergen MR, Stratos GA. Physician confidence and interest in learning more about Geriatric topics: A need assessment. Journal of the American Geriatric Society. 2001; 49:963-967.
14. Bauer TA, Sanders J. Needs assessment of Wisconsin primary care residents’ and faculty interest in global health rating. BMC Medical Education. 2009; 9:36.
15. Drickamer MA, Levy B, Irwin K, Rohrbaugh RM. Perceived needs for Geriatric Education by Medical Students, Internal Medicine Residents and Faculty. Journal of General Internal Medicine. 2006; 21:1230-1234.
16. Crandall SJ. Using interviews as a need assessment tool. Journal of Continuing Education in the Health Profession. 2005; 18: 155-162.
17. McLeod PJ, Steinert Y, Meagher T, McLeod A. The ABCs of pedagogy for clinical teachers. Medical Education. 2003; 37:638-644.
18. Moore DE. Needs assessment in the new health care environment: Combining discrepancy analysis and outcomes to create more effective CME. Journal of Continuing Education in the Health Professions. 2005; 18:133-141.
19. Amin Z, Eng KH, Gwee M, Hoon TC, Rhoon KD. Addressing the needs and priorities of medical teachers through a collaborative intensive faculty development programme. Medical Teacher. 2006; 28:85-88.
20. Wallin DL, Smith CL. Professional development needs of full-time faculty in technical colleges. Community College Journal of Research and Practice. 2005; 29: 87-108.
21. Hesketh EA, Bagnall G, Buckley EG, Friedman M, Goodall E, Harden RM, Laidlaw JM, Leighton-Beck L, McKinlay P, Newton R, Oughton R. A framework for developing excellence as a clinical educator. Medical Education. 2001; 35:555-564.
22. Benor DE. Faculty development, teacher training and teacher accreditation in medical education: twenty years from now. Medical Teacher. 2000; 5:503-511.
23. Amin Z, Burdick WP, Supe A, Singh T. Relevance of Flexner Report to the contemporary Medical Education in the South Asia. Academic Medicine. 2010; 85:333-339.
24. Rubeck RF, Witzke DB. Faculty development: A field of dreams. Academic Medicine. 1998; 73:S 32-37.
25. Steinert Y. Faculty development in the new millennium: key challenges and future directions. Medical Teacher. 2000; 22:44-50.
26. Steinert Y, Mann K, Centeno A, Dolmans D, Spencer J, Gelula M. A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Medical Teacher. 2006; 8:497-526.
27. Bland CJ, Seaquist E, Pacala JT, Center B, Fistad D. One school’s strategy to assess and improve the vitality of its faculty. Academic Medicine. 2002; 77:368-376.
28. Adkoli BV, Sood R. Faculty Development and Medical Education Units in India: A survey. The National Medical Journal of India. 2009; 22:28-32.