|ORIGINAL RESEARCH PAPER
|Year : 2007 | Volume
| Issue : 1 | Page : 7
DREEM and beyond; studies of the educational environment as a means for its enhancement
SR Whittle, B Whelan, DG Murdoch-Eaton
University of Leeds, Leeds, United Kingdom
|Date of Submission||24-Feb-2007|
|Date of Web Publication||18-Apr-2007|
S R Whittle
Faculty of Biological Sciences, University of Leeds, Leeds LS2 9JT
Source of Support: None, Conflict of Interest: None
Context: Educational environment makes an important contribution to student learning. The DREEM (Dundee Ready Educational Environment Measure) questionnaire is a validated instrument for assessing educational environment, but used alone it has little value for identifying means of remediation of poor aspects of environment.
Aims: This study used qualitative analysis, in association with the DREEM questionnaire, to evaluate the educational environment across all five years of a large undergraduate medical school, and identify areas for change to enhance student experience.
Method: The DREEM questionnaire was administered to 968 undergraduate students, together with an open question asking for suggested changes to current medical school practices. Items of concern highlighted by this study were further defined through qualitative analysis, using focus groups, email questionnaires and introduction of Stressful Incident reporting.
Results: Through responses to the open question, two items with low scores on the DREEM questionnaire were identified as requiring remediation. Focus groups and email questionnaires were used to define the underlying causes of poor scores, which varied by student year group. Stress resulting from experiences on clinical placement was highlighted by some students, but on closer investigation found to be rare. Remedial steps to improve student support are described.
Conclusions: The qualitative data have substantially enhanced questionnaire interpretation, and allowed actions to address common causes for student dissatisfaction to be undertaken. This combined methodology is recommended to other institutions wishing to improve the educational environment, and thus the overall quality of educational provision.
Keywords: educational climate; educational environment; student perceptions; student support; course evaluation; quality assessment
|How to cite this article:|
Whittle S R, Whelan B, Murdoch-Eaton D G. DREEM and beyond; studies of the educational environment as a means for its enhancement. Educ Health 2007;20:7
There is growing recognition of the importance of educational climate/environment in underpinning effective student learning. Students’ perception of the environment within which they study has been shown to have a significant impact on their behaviour, academic progress and sense of well-being (Genn, 2001, Pimparyon et al., 2000, Audin et al., 2003).
Various methodologies have been utilised to investigate educational climate. Recent studies include qualitative approaches (Seabrook, 2004) or the use of questionnaires (Audin et al, 2003, Roff et al., 1997, Sobral, 2004). Of these, only the DREEM (Dundee Ready Educational Environment Measure) questionnaire (Roff et al., 1997) is specific to the unique environment experienced by students on medical and healthcare-related courses. This instrument was developed by an international Delphi panel, and has been applied to a number of undergraduate courses for health professionals worldwide (Roff, 2005). However, it is clear that such questionnaires cannot tell the whole story. While they may be valuable in pinpointing areas of concern shared by a majority of students, they give no insight into the underlying reasons for these responses.
Till (2004) notes that the instrument has been used mainly to compare different medical schools, and that reports in the literature “gave no indication of what institutions do after they get the results of a climate study.”
This study combines quantitative and qualitative methods to investigate students’ attitudes towards the educational environment, across the complete 5 years of the undergraduate medical course. It identifies reasons underlying poor scores, and describes remedial actions taken where indicated.
The DREEM questionnaire consists of 50 items, each scored 0-4 on a 5-point scale. Negative statements are scored in reverse, so that high scores on these items indicate disagreement, i.e. a positive result. The questionnaire generates an overall “score” for the course. The statements may also be subdivided to provide an indication of student perceptions of 5 separate elements of the educational environment:
- ‘Perceptions of learning ’ (PoL)
- ‘Perceptions of teachers’ (PoT)
- ‘Academic self-perception’ (ASP)
- ‘Perceptions of atmosphere’ (PoA)
- ‘Social self-perceptions’ (SSP)
The questionnaire was administered to students in all five years of the undergraduate medical course. The questionnaire was available in both paper and web-based formats. Students were made aware of the aims of the study and the importance of high levels of participation, by email and announcements at lectures. In addition to the questionnaire, students were also asked to respond to an open-ended question: “If you could change three things about the medical school, what would they be?”1. The teachers are good at providing feedback to students
Responses were analysed, and those giving an overall score across all year groups of 2.0 or below were investigated further, as the questionnaire developers advise (McAleer & Roff, 2002).
After discussion with members of the Medical Students’ Representative Council (MSRC), focus group interviews for years 1-3 and an email questionnaire for years 4-5 based on the focus groups, were undertaken by senior MSRC members to explore these areas. Ten to fifteen students who were randomly selected from class lists attended the focus groups. Fifteen students in each of years 4 and 5 responded to requests for additional feedback.
Investigating causes of student stress during placements
The Stressful Experience report form, developed to encourage students to report and seek support after distressing experiences during clinical placements, was designed in collaboration with an experienced researcher in Psychiatry and a BACP-accredited counsellor. It is sent electronically, with an explanatory letter, every month to each student who is undertaking a clinical placement. Forms may be returned anonymously if preferred and the Director of Student Support investigates reports as appropriate offering follow-up to students who request it. The nature and frequency of these reports was monitored in order to provide insight into the stressful incidents experienced by students on clinical placement.
A total of 968 students completed the questionnaire (Y1=212, Y2=225, Y3=194, Y4=167, Y5=170) giving an overall response rate of 86%.
Responses to all items are shown in Table 1 and the three highlighted items showed an overall response across the entire medical school of below 2, which was regarded as negative.
Table 1:Responses to DREEM questionnaire items for each year of the course
Analysis of responses to the question “If you could change three things about the medical school, what would they be?” showed good correlation with most questionnaire scores. Feedback to students and the student support system were highlighted as a source of dissatisfaction by students from all years. Neither the workload, nor the amount of material to be learnt was identified as an issue.
Focus groups/e-mail questionnaires
Of the three items averaging than 2.0 in the questionnaire, two were felt to require additional information in order to interpret the responses and to consider possible remediation. Both these areas were explored more fully through focus group interviews and email questionnaire responses.
This was the most negative response that indicated dissatisfaction throughout the course. Responses to the open question and from focus groups, revealed that in the early years of the course, students’ concerns centred on material completed as self-directed study, which was not individually checked.
Student A: “give the answers to work sessions and anatomy at the end of the year so when you’re revising you know you are revising the correct answers”
Student B: “especially in histology … I never know if I’m doing the right thing or learning the right name of a particular thing”
Student C: “could have some way of marking anatomy worksheets…. online?”
Student D: “or have sessions to go through answers of self directed learning” (Year 1 focus group)
In later years, dissatisfaction centred around feedback from examinations;
“(need) an opportunity to go over the papers highlighting the questions that were done badly by the group, especially in the final years when you realise that you need to know the stuff in the exams that you weren’t sure about” (Year 5)
2. There is a good support system for students who get stressed
Over 40% gave an “undecided” response to this question, thus it was unclear from the questionnaire whether students were reporting that they had not needed to seek support, or were unsure of the value or accessibility of the existing systems.
Responses from the open question and focus groups suggested that year 1 students were aware of appropriate routes through which to seek academic or pastoral support, having been given this information during induction. Year 2 students however, appeared unaware of most of this information. Students in higher years suggested that they would use support systems outside the university, e.g. family, friends or their GP.
Concern was expressed in all year groups that the style of teaching and large numbers of teaching staff made it difficult to get to know anyone well enough to confide in them during times of academic or emotional stress:
“No-one knows who I am or what my strengths and weaknesses are” (year 4)
Some students felt that personal tutors should give academic, as well as pastoral support:
“have a personal tutor who is within the medical school so that they know more about the course…” (year 2)
In years 4 and 5, students highlighted the emotional stress of dealing with difficult situations during their clinical placements, e.g. terminal illness, child abuse.
“I would have appreciated a sort of supervision service where I could talk about stuff that I’ve seen or how I feel about seeing suicidal people or women who are beaten by their husband to name a few things” (year 4)
Investigating causes of student stress during placements
This observation led to the design and introduction of a Stressful Experience report form, where students could report distressing incidents experienced on clinical placement and seek support. Analysis of reports received in the first 6 months of use suggested that these events are rare (16 reports received from 950 students). The majority of stressful experiences reported related to teaching style, e.g. students feeling inappropriately criticised (either ‘unfairly’ or in the presence of others) and only 2 reports related directly to distressing clinical experiences. Student feedback on the introduction of this initiative has been strongly positive.
This is the first study to report results from the complete undergraduate cohort of a medical school. The DREEM questionnaire has provided an overview of student opinion throughout the medical school and allowed areas of concern to be highlighted. The items that were given an average score below 2.0 across the whole medical school were examined in depth. The qualitative data studies have provided very valuable insights into the results obtained from the questionnaire. Of the 50 items, 3 scored less than 2 on the 0-4 scale. The response to the item “I am able to memorise all I need” has scored below 2 in all published reports (Al-Hazimi et al., 2004, Till, 2004, Bassaw et al., 2003, Roff et al., 2001), which might suggest that the volume of information requires further reduction in many medical curricula. However, it is significant that neither workload, nor the volume of material to be learned feature in responses to the open question. This is in sharp contrast to the other two poorly scoring items, where free responses underlined student concerns over support and feedback. This suggests that responses to this item are not critical of the courses, but rather demonstrate students’ understanding of the enduring need to seek out new information throughout their studies and subsequent careers. We conclude that remediation is not required in this instance.
Two items remained therefore that gave cause for concern and required further investigation. Both these items score poorly at most other reporting institutions (Al-Hazimi et al., 2004, Till, 2004, Bassaw et al., 2003, Roff et al., 2001), suggesting that these are the most difficult areas of educational environment to support.
Further investigation of responses to the item “The teachers are good at providing feedback to students” showed that underlying concerns differ depending on the year group of the students questioned. The main worry for students in the first two years of the course relates to self-directed learning. Students appear to lack confidence in their ability to complete exercises independently. The fact that this source of concern disappears in higher year groups would confirm that the lack of self-confidence underlies these requests. This may correlate with the observation that some items within Academic Self Perception (ASP) scores tend to be lower for students in the early stages of the course. Increased reassurance alone may improve student self- confidence during the early part of the course. This finding has been fed back to teaching staff involved with students in their first 2 years of study.
In later years, the underlying concern relates to insufficient feedback after examinations. At first sight, this may appear to be a continued lack of confidence. However, the observation that students can identify areas of uncertainty during examinations suggests that they can reflect accurately on their own strengths and weaknesses, but are not yet fully prepared to accept responsibility for resolving their weaknesses independently. Curricular changes introduced by Tomorrow’s Doctors (GMC, 1993) require students to develop the skills necessary to become independent learners. These results suggest that students are not gaining these skills to the extent required. It may be necessary to provide students with additional opportunities to take responsibility for their own learning, allowing them to enhance their skills and attitudes in this area.
Further insight was gained from detailed responses to the item “There is a good support system for students who get stressed.” It is apparent that students quickly forget where to access support systems. The nature of medical courses ensures that students learn in both academic and clinical environments and have access to specialist teachers from many scientific and clinical disciplines. However, the course structure makes it difficult for personal tutors to have a working knowledge of the entire course and students need to establish a pattern of seeking advice from a range of sources. Our study shows that it is insufficient to put this information in course books, or induction sessions. Knowledge regarding sources of information/support must be available to students in an immediately accessible form throughout the course. One outcome of this study has been the design and production of a student support poster, which is displayed in all buildings where students are taught. These ensure that contact details of key staff within the medical school and information relating to other services, e.g. counselling service, chaplaincy, are readily available to students at all times. Feedback from the MSRC confirms that the distribution of these posters has been well received by students.
Results from the Stressful Incident reports suggest that very few students may find difficulty in managing the transition from the ‘safety’ of the classroom to clinical teaching. Analysis of the reports received allowed us to investigate the type of incidents that caused stress when students are in clinical placements. While analysis suggests that distress caused by clinically related experiences on placement is very rare, distribution of report forms on a monthly basis has been continued, owing to strong support for this initiative from students. They clearly appreciate the opportunity to report their concerns directly to the Director of Student Support and an additional route through which to seek support. Adverse experiences during medical training have been reported to have long term effects on students (Wilkinson et al., 2006), therefore this is an important addition to the student support system.
In this study the qualitative analysis was limited to those items that scored poorly (2.0 or below on a 0-4 point scale) in the DREEM survey. It may be that similar analysis could profitably be applied to other items, to gain a better insight into student understanding of their environment. In particular, this might be a valuable tool to investigate low scores in particular year groups, or in longitudinal studies to monitor changes in student perception as they progress through the course.
The combination of quantitative and qualitative methods to investigate educational environment shows several advantages over the DREEM questionnaire alone. The DREEM questionnaire has been used to date for a variety of purposes, including profiling institutional strengths and weaknesses, particularly during curricular changes (Till, 2005), comparing student perceptions between different institutions and cohorts, and predicting academic achievement (Roff, 2005). DREEM creates a snapshot of student perception of their study environment, but cannot provide information about the concerns underlying poor scores. Generation of a better understanding of these concerns through qualitative analysis allows identification of areas that require remediation and provides a mechanism for improving the educational environment.
The authors would like to thank Suesanne Samara and Louise Cripps for assistance in running the focus groups, and Mr Alan Butler and Mrs Elizabeth Mitchell for assistance in designing the Stressful Experience report form. Approval for this study was given by Leeds School of Medicine Learning & Teaching Board.
AL-HAZIMI, A., ZAINI R., AL-HYIANI, A., HASSAN, N., GUNAID, A., PONNAMPERUMA, G., KARUNATHILAKE, I., ROFF, S., MCALEER, S., & DAVIS, M. (2004). Educational environment in traditional and innovative medical schools: A study in four undergraduate medical schools. Education for Health, 17(2), 192-203.
AUDIN, K., DAVY, J. & BARKHAM, M. (2003). University Quality of Life and Learning (UNIQoLL): an approach to student wellbeing, satisfaction and institutional change. Journal of Further & Higher Education, 27(4), 365-382.
BASSAW, B., ROFF, S., MCALEER, S., ROOPNARINESINGH, S., DE LISLE, J., TEELUCKSINGH, S. & GOPAUL, S. (2003). Students’ perspectives on the educational environment, Faculty of Medical Sciences, Trinidad. Medical Teacher, 25(5), 522-526.
GENERAL MEDICAL COUNCIL (1993). Tomorrow’s Doctors; Recommendations on undergraduate medical education. London: General Medical Council.
GENN, J.M. (2001). Curriculum, environment, climate quality and change in medical education – a unifying perspective. Medical Teacher, 23(5), 445-454.
MCALEER, S. & ROFF, S. (2002). Part 3; A practical guide to using the Dundee Ready Education Measure (DREEM). In, J.M. GENN (Ed), AMEE Medical Education Guide No.23 Curriculum, environment, climate, quality and change in medical education; a unifying perspective. Dundee, UK: Association of Medical Education in Europe.
PIMPARYON, P., ROFF, S., MCALEER, S., POONCHAI, B., & PEMBA, S. (2000). Educational environment, student approaches to learning and academic achievement in a Thai nursing school. Medical Teacher, 22(4), 359-365.
ROFF, S. (2005). The Dundee ready Educational Environment Measure (DREEM) – a generic instrument for measuring students’ perceptions of undergraduate health professions curricula. Medical Teacher, 27(4), 322-325.
ROFF, S., MCALEER, S., HARDEN, R.M., AL-QAHTANI, M., AHMED, A.U., DEZA, H., GROENEN, G. & PRIMPARYON, P. (1997). Development and validation of the Dundee Ready Education Environment Measure (DREEM). Medical Teacher, 19(4), 295-299.
ROFF, S., MCALEER, S., IFERE, O.S. & BHATTACHARYA, S. (2001). A global diagnostic tool for measuring educational environment: comparing Nigeria and Nepal. Medical Teacher, 23(4), 378-382.
SEABROOK, M.A. (2004). Clinical students’ initial reports of the educational climate in a single medical school. Medical Education, 38, 659-669.
SOBRAL, D.T. (2004). Medical students’ self-appraisal of first year learning outcomes: use of the course valuing inventory. Medical Teacher, 26(3), 234-238.
TILL, H. (2004). Identifying the perceived weaknesses of a new curriculum by means of the Dundee Ready Education Environment Measure (DREEM) Inventory. Medical Teacher, 26(1), 39-45.
TILL, H (2005). Climate studies; can students’ perceptions of the ideal educational environment be of use for institutional planning and resource utilisation? Medical Teacher, 27(4), 332-337.
WILKINSON, T.J., GILL, D.J., FITZJOHN, J., PALMER, C.L., & MULDER, R.T. (2006). The impact on students of adverse experiences during medical school. Medical Teacher, 28(2), 129-135.