|ORIGINAL RESEARCH PAPER
|Year : 2007 | Volume
| Issue : 1 | Page : 11
The "Culture OSCE" - Introducing a Formative Assessment into a Postgraduate Program
L Aeder, L Altshuler, E Kachur, S Barrett, A Hilfer, S Koepfer, H Schaeffer, SP Shelov
Maimonides Medical Center, Brooklyn, NY, USA
|Date of Submission||09-Mar-2007|
|Date of Web Publication||18-Apr-2007|
1301 57th Street, Brooklyn, N.Y. 11219
Source of Support: None, Conflict of Interest: None
Background: There is a growing need for appropriate training models in the area of cultural competence. An Objective Structured Clinical Exam (OSCE) format is ideal for this endeavor, since it allows for skills practice and feedback. As a result, we designed the first formative Culture OSCE at Maimonides Medical Center and have been implementing it since 1999.
Program Development: An interdisciplinary committee developed the OSCE as a formative assessment. Stations were designed based on a review of the literature and real situations experienced in the hospital. A two-hour workshop introducing the concept of cultural competence precedes the OSCE. The emphasis is on skills that are generalizable to encounters with any culture. Standardized patients are recruited from the relevant cultural groups or are trained to understand specific cultural issues. Costumes and props are utilized to enhance the authenticity of the encounter. Faculty, recruited and trained to observe encounters, gives constructive feedback, thus enhancing faculty development in this area as well. A rating scale was developed which incorporates communication and cultural skills as two separate dimensions of the encounter.
Program Evaluation: Written feedback is obtained from residents, the trained faculty observers and the standardized patients. Resident feedback has demonstrated good face validity. A post-OSCE debriefing session allows residents an opportunity to consolidate learning and give oral feedback.
Conclusion: The Maimonides Medical Center Pediatrics Department designed the first Culture OSCE. This is deemed to be a valuable training tool, and serves to highlight the importance of cultural competence within the Department.
Keywords: OSCE, Culture, Cultural Competence
|How to cite this article:|
Aeder L, Altshuler L, Kachur E, Barrett S, Hilfer A, Koepfer S, Schaeffer H, Shelov S P. The "Culture OSCE" - Introducing a Formative Assessment into a Postgraduate Program. Educ Health 2007;20:11
|How to cite this URL:|
Aeder L, Altshuler L, Kachur E, Barrett S, Hilfer A, Koepfer S, Schaeffer H, Shelov S P. The "Culture OSCE" - Introducing a Formative Assessment into a Postgraduate Program. Educ Health [serial online] 2007 [cited 2017 Aug 17];20:11. Available from: http://www.educationforhealth.net/text.asp?2007/20/1/11/101637
The role that culture plays in the provision of healthcare has finally been recognized over the last several years. This has been as a response to the growing multicultural nature of society and the continuing disparities in healthcare between people of different socio-cultural backgrounds (Institute of Medicine, 2002). It is becoming increasingly clear that in order to achieve patient satisfaction and compliance, a physician must demonstrate cultural competence. This skill includes eliciting and being sensitive to the patient’s ethnic and cultural values (Kleinman et al., 1978). In addition, culture must be taught throughout the continuum of medical education (Altshuler & Kachur, 2004).
At the Maimonides Medical Center (MMC), a not-for-profit hospital in Brooklyn, New York, the Pediatrics Department developed a formative assessment in cultural competence for the residents at the post-graduate trainee level. MMC is situated in a highly diverse community. Together with its affiliates, MMC services large Jewish, Hispanic and Chinese communities as well as growing numbers of East European and other Asian immigrants (Fig. 1). In 1998 the Pediatrics Department decided to completely revise the curriculum to address cultural competence. To plan the content, a multidisciplinary committee was convened consisting of members of the Pediatrics, Psychology, Psychiatry, Social work and Child life departments. Teaching culture was not carried out systematically prior to 1998, and given these demographics it was considered a priority. The question was how to proceed.
Fig. 1 – Ethnicity and Religious distribution of patients at Maimonides Medical Center
One proposed strategy was to teach it via a series of lectures. However, concerns were expressed that this approach requires the learning of each individual culture and may lead the residents to assume that everyone from a specific culture would react similarly to a particular situation. According to Julliana Newman, Culture is “a dynamic, constantly evolving process. Culture is what we do and how we do it; culture is what we consider ‘normal’ thinking and behavior” (Newman, 1998). Instead of teaching specifics, the goal was to teach the trainees to consider and explore differences in perspectives and beliefs, and to negotiate across value systems.
Cultural care has a large skills component that cannot be imparted in a lecture. The ideal was to develop cultural competence while promoting self-assessment and active learning consistent with adult learning theory (Knowles, 1990). Additional instructional methods were sought for evaluating and developing those skills. Many institutions in the USA were using Objective Structured Clinical Examinations (OSCEs) for evaluating clinical skills at the graduate level. OSCEs seemed to be ideally suited for these learning goals because they provide an opportunity for active participation and learning (building on already acquired skills and knowledge). They provide immediate feedback and can be designed to have clear goals and objectives. Furthermore, trainees would be exposed to multiple and varied clinical situations with people from different cultures at a rapid speed, giving them a unique opportunity to develop flexible approaches to patients from different backgrounds. Finally, trained observers could assist the residents to engage in self-reflection.
OSCEs have been used for clinical education since 1975 (Harden et al., 1975). In 1998, at the outset of the project, committee members reviewed many articles describing the application of OSCEs to various medical subspecialties and allied healthcare fields. OSCEs had been designed to teach and assess more general skills such as communication (Hodges et al., 1996) and ethics (Singer et al., 1993). However, no publication describing their use in teaching cultural competence was found.
The process of station development began with a review of the literature to identify relevant competencies (Like et al., 1996; Carrillo et al., 1999). Clinical situations were sought from committee members’ experiences. We also invited hospital staff to share clinical situations presenting cultural challenges. These contributions were important for validating the exercise. The idea was to develop scenarios that would teach cultural awareness and demonstrate the profound impact culture has on healthcare. The goal of each station was to assist the residents in exploring the patient’s point of view and compare and contrast how their viewpoint differed from that of the physician. The cases would also highlight the competencies identified in the literature search. To ensure that they were an accurate reflection of the cultures showcased, the scenarios were submitted to colleagues from the pertinent culture for review.
After considering the cost, time constraints, space available, and benefits of additional stations, the faculty determined that five stations would be adequate to sample the subject. Table 1 lists the stations that have been developed since 1998. Each station includes an overview, a brief set of instructions for the resident, a detailed set of instructions for the standardized patients (SPs) and a feedback protocol for the trained observers. The station overview describes the objectives and the logistics involved in setting up the station. A short set of teaching points is included in the feedback instructions. Two minutes are allowed for transition and reading of instructions, 10 minutes for completing the task and 5 minutes for feedback.
Table 1 - OSCE Stations
Actors are hired to play the roles of SPs as described in the scenarios (Fig. 2). When representatives of a designated culture are not available, intensive education is given to the actors, enabling them to portray cultural beliefs. Appropriate costumes and props are added when relevant. The faculty is recruited to observe the residents’ performance at the stations, complete the rating form, provide structured feedback and augment SP commentary.
Fig 2 – Standardized Patients in the Lost in Translation Station
A two-hour workshop is held prior to the OSCE (Table 2). The workshop utilizes presentations, group discussion, and self-reflection as a means of increasing cultural awareness. During this time residents are exposed to the principles of cultural competency. They are invited and encouraged to reflect on their own culture as well as their attitudes and prejudices on the subject. The focus is to show how one’s own personal cultural beliefs might affect clinical encounters, particularly with patients from different cultural groups. Methods for addressing these differences are explored.
Table 2 – Workshop Teaching Points
The debriefing session is conducted immediately after the OSCE. Residents are encouraged to detail their impressions and difficulty with each station while learning points are highlighted. This is an essential part of the educational process as it allows residents to integrate what they learned during the workshop and the OSCE, and to generalize their learning to their clinical work. Key reading articles are distributed for each station to enhance further learning.
Rating Scale Development
Specific competencies for each scenario were identified from the literature during rating scale development in order to augment the face validity of the exercise. When reviewing the content of these items it was noted that they could be classified as either communication skills or cultural competence. There is not always such a clear line between the two, and yet their purposes are quite distinct. Communication skills function to establish a rapport and to develop a relationship with the patient. Culture-specific skills demonstrate an ability to bridge differences between the physician and the patient – differences in value systems and ways of understanding. The key elements of cultural competency included are: eliciting the patient’s cultural perspective (items 10 and 11 in Table 3); communicating the physician’s theory about what is wrong and what should be done (items 12 and 13); and differentiating that view from that of the patient and negotiating those differences successfully (items 14 and 15). A four-point Likert scale rating is utilized ranging from “not done” to “done exceptionally well”. In addition, a global score is given to each resident based on the overall quality of the encounter (Table 3). A shortened rating form was designed for the use of the SPs which asks for global ratings of communications skills and cultural competence, and provides ample space for comments.
Table 3 – Evaluation form
Administrative support is obtained to ensure that the maximum number of residents will be present for the exercise. The cohort of residents is divided into two groups with the OSCE running on two separate days in order to prevent creating a problem with hospital and ambulatory coverage. The residents are requested not to discuss the content of the OSCEs with each other. However, given the complexity of each station, prior knowledge of the case would not necessarily help the residents’ performance in each station.
All stations were pilot tested before the OSCE early in 1999 to assure that they would achieve our goals smoothly and successfully. The stations were videotaped during the trial run so that they could be used for future training. Recording the stations prior to subsequent OSCEs also allowed for standardization of the observers. By having the faculty jointly view and compare resident/SP encounters on tape, the intra and inter-rater reliability of the exercise were enhanced. This process also helped refine the rating sheets and scoring system.
At the end of the debriefing session residents are asked to complete a program evaluation form to assess station-specific and overall teaching effectiveness and satisfaction. Even at the outset of the program their reaction was positive. The results for the first two years of the program are shown in Figure 3. There was a 100% response rate. Sixty-six percent of the residents felt that they definitely learned something new while 26% participants thought they “possibly” did. Only 8% felt that the activity was not beneficial. The majority of the residents felt they received valuable feedback and were evaluated fairly. The resident’s evaluation of the educational value of the individual stations is shown in Figure 4. Overwhelmingly, the residents gave all the stations moderate to high rankings demonstrating the face validity of the OSCE. Other outcome measures on the culture OSCE have been previously published (Altshuler et al., 2003).
Fig. 3 – Overall Evaluation of Culture OSCE
Fig. 4 Educational Value of Stations
During the debriefing session the residents offered additional verbal comments about each station. The focus of the discussion is not on understanding a single culture, but on the skills that could be learned and generalized to all cultural encounters. This also helps to avoid the stereotyping inherent in a didactic lecture or even after viewing OSCE scenarios in isolation. In addition, the combination of oral and written resident feedback allows for future modification of existing stations and development of new ones to closely suit educational goals. Faculty observers and SPs are also debriefed.
An unexpected benefit of the implementation of this exercise has been to provide faculty development about cultural issues. The trained observers in each OSCE station consist of interdisciplinary faculties who have agreed to volunteer their time for this teaching exercise. All observers are provided with information about general issues of cross-cultural communication in addition to the specifics of each case. Attention is given to the provision of effective feedback and incorporating the residents’ self-assessments and SPs into the discussion. Over the eight years that this program has been in existence, a cohort of interested and informed healthcare professionals has been recruited to promote this awareness in other ambits of resident education and supervision.
Achieving cultural competence requires adequate opportunities for learning and integration. The culture OSCE is a valuable tool in this process. Since this project was initiated, the Accreditation Council for Graduate Medical Education (ACGME) in the United States has redefined its competency-based model to include six areas: patient care, medical knowledge, interpersonal and communication skills, professionalism, practice-based learning and improvement, and systems-based practice (ACGME, 1999). Physicians are required to “demonstrate sensitivity and responsiveness to a diverse patient population.” Cultural competency is too complex to be taught by didactic lectures alone. The OSCE is an ideal vehicle for observing the residents’ skill in negotiating cultural differences through challenging cultural scenarios. Another positive aspect is that it promotes residents’ self-evaluation and provides constructive feedback from multiple trained observers and SPs. This enhances and maximizes the learning value of the exercise. The MMC residents’ response to the exercise has been enthusiastic, and confirms that the Culture OSCE is viewed as a positive learning experience.
The MMC Pediatrics Department is the first institution to have designed an OSCE solely for the purpose of teaching cultural competence (Altshuler & Kachur, 2001). Robins et al. published a report describing the inclusion of one cultural competence station in the context of a comprehensive clinical skills OSCE (Robins et al., 2001). The article concludes that ethnic background influences student performance. However, since the study was limited to one station, the results are not readily generalized. We feel that an OSCE solely devoted to culture emphasizes the importance of the subject. Multiple stations facilitate the residents’ exposure to a wider variety of cultural encounters and perspectives. The feedback in the Robins study was provided only by the SPs. Although SP feedback is essential, it is not sufficient to analyze the full encounter. An SP must concentrate on acting out the role, whereas a trained observer has ample time to observe and document specific examples of each resident’s reaction. This documentation is a key element of subsequent feedback. Together, SPs and observers provide a more comprehensive overview of each resident’s performance. In addition, faculty learning is enhanced when they serve as trained observers. Using faculty in this manner promotes the value and significance of cultural competence.
Rosen et al. published a report of a workshop which included different didactic methods developed to teach medical students cross-cultural communication skills (2004). Using an OSCE-type format in one of them, groups of 8 students were exposed to cultural scenarios with one student interviewing the standardized patient. Outcome measures consisted of pre and post workshop surveys submitted by the students. Since this format allowed for large group discussions, it required more time to elaborate each case. Our design facilitates individual participation and feedback and fosters a more rigorous assessment of individual resident performance.
Setting up an OSCE is a labor intensive exercise. However, the majority of the effort takes place before the examination so that on the day of the OSCE the time is used very efficiently (Harden et al. recognized this as far back as 1975). The OSCE committee at MMC has been meeting regularly for the last eight years to update the contents and format of the stations using the information obtained from the residents’, SPs’ and observers’ evaluations to maximize the learning value and efficiency. We take evaluation comments very seriously, using this data to improve the quality of the educational exercise over time. Many new stations have been developed since the program’s inception which has created great versatility. The OSCE has now become part of the culture of our residency program.
The Pediatrics Department at MMC designed the first OSCE that is entirely devoted to cultural competence. Scenarios were developed that effectively evaluated residents’ skill levels, including assessment of health beliefs and negotiation of sometimes opposing viewpoints in the clinical encounter. These scenarios can be produced without stereotyping or simplifying issues. A rating scale was also developed that differentiated communication skills from cultural competence. Not only has a purely cultural OSCE highlighted the importance of the subject to the residents, but it has also become an important faculty development tool. Programs interested in setting up a similar assessment should use scenarios that reflect their own populations while generalizing the concepts of cultural competence by emphasizing the core skills.
ACCREDITATION COUNCIL FOR GRADUATE MEDICAL EDUCATION (ACGME) (1999). Outcome Project: Enhancing residency education through outcomes assessment. Retrieved September 14, 2004, from http://www.acgme.org/outcome/comp/compMin.asp
ALTSHULER, L., & KACHUR, E. (2001). A Culture OSCE: Teaching residents to bridge different worlds. Academic Medicine, 76, 514.
ALTSHULER, L., SUSSMAN, N.M., & KACHUR, E. (2003). Assessing changes in intercultural sensitivity among physician trainees using the Intercultural Development Inventory. International Journal of Intercultural Relations, 27, 387-401.
CARRILLO, J.E., GREEN, A.R., & BETANCOURT, J.R. (1999). Cross-cultural primary care: a patient-based approach. Annals of Internal Medicine, 130, 829-34.
HARDEN, R.M., STEVENSON, M., DOWNIE, W.W., & WILSON, G.M. (1975). Assessment of clinical competence using objective structured clinical examinations. British Medical Journal, 1, 447-51.
HODGES, B., TURNBULL, J., COHEN, R., BIENENSTOCK, A., & NORMAN, G. (1996). Evaluating communication skills in the OSCE format: reliability and generizability. Medical Education, 30, 38-43.
INSTITUTE OF MEDICINE (2002). Unequal treatment: confronting racial and ethnic disparities in health care. Washington (DC): National Academy Press.
KACHUR, E., & ALTSHULER L. (2004). Cultural competence is everyone’s responsibility! Medical Teacher, 26, 101-5.
KLEINMAN, A., EISENBERG, L., & GOOD, B. (1978). Culture, illness, and care: clinical lessons from anthropologic and cross-cultural research. Annals of Internal Medicine, 88, 251-8.
KNOWLES, M.S. (1990) The Adult Learner, A Neglected Species. Houston: Gulf Publishing Company.
LIKE, R.C., STEINER, R.P., & RUBEL, A.J. (1996). STFM core curriculum guidelines: recommended core curriculum guidelines on culturally sensitive and competent health care. Family Medicine, 28, 291-7.
NEWMAN, J. (1998). Managing cultural diversity: the art of communication. Radiologic Technology, 69, 231-242.
ROBINS, L.S., WHITE, C.B., ALEXANDER, G.L., GRUPPEN, L.C., & GRUM, C.M. (2001). Assessing medical students’ awareness of and sensitivity to diverse health beliefs using a standardized patient station. Academic Medicine, 76, 76-80.
ROSEN, J., SPATZ, E.S., GAASERUD, A., ABRAMOVITCH, H., ET AL. (2004). Medical Teacher, 26, 126-32.
SINGER, P.A., COHEN, R., ROBB, A., & ROTHMAN, A. (1993). The ethics objective structured clinical examination. Journal of General Internal Medicine, 81, 23-8.