|ORIGINAL RESEARCH ARTICLE
|Year : 2016 | Volume
| Issue : 1 | Page : 16-24
Psychometric properties of the dundee ready educational environment measure in a sample of Ghanaian Medical Students
Victor Mogre, Anthony Amalba
Department of Health Professions Education and Innovative Learning, School of Medicine and Health Sciences, University for Development Studies, Tamale, Ghana
|Date of Web Publication||18-Mar-2016|
Education Unit, School of Medicine and Health Sciences, University for Development Studies, P. O. Box: TL 1883, Tamale
Source of Support: None, Conflict of Interest: None
Background: The Dundee Ready Educational Environment Measure (DREEM) has been widely accepted and recognized for the assessment of the educational environment in a variety of health professions education programs. Concerns regarding the psychometric properties of the DREEM have been raised. This study evaluated the psychometric properties of the DREEM in a sample of Ghanaian medical students following a problem-based learning curriculum. Methods: A sample of 234 second to fourth year medical students of the University for Development Studies, School of Medicine and Health Sciences were invited to complete the DREEM questionnaire. Psychometric measures employed included Cronbach's alpha analysis, confirmatory factor analysis and principal component analysis with varimax rotation. Results: The internal consistency for the overall DREEM was 0.92. Apart from two subscales that had Cronbach's alpha values < 0.70, the remainder had values ranging from 0.73 – 0.78. Confirmatory factor analysis did not support the original five factor structure of the DREEM. Large significant correlation coefficients were found between the five factors raising concerns about the independency of the individual subscales. Exploratory factor analysis suggested various factor solutions ranging from 4 to 14 factors. The four factor structure was interpretable and was maintained. All of the four factors achieved eigenvalues > 1, and in total they accounted for 37.6% of the variance. Alpha values for the subscales of the new four factor structure ranged from 0.618 – 0.915. Discussion: The overall internal consistency of the DREEM was found to be excellent. The internal consistency of the individual subscales was variable, as two subscales had alpha values < 0.70 and the remainder exceeding 0.70. The original five factor structure of the DREEM was not supported. Exploratory factor analysis suggested a four factor solution as a possible alternative to the five factor structure of the DREEM.
Keywords: Dundee Ready Educational Environment Measure, Ghana, learning environment, medical students, psychometric properties
|How to cite this article:|
Mogre V, Amalba A. Psychometric properties of the dundee ready educational environment measure in a sample of Ghanaian Medical Students. Educ Health 2016;29:16-24
| Background|| |
A school's educational environment encompasses student-teacher relationships, teaching and learning strategies, students' psychosocial and emotional needs, as well as the physical structures and facilities provided of the institution. A productive and conducive learning environment is provided if the higher education institution is able to provide all of these. It is generally recognized that the learning environment affects students' competencies, academic achievement and success., It is therefore not surprising that the educational environment has been identified as one of the targets for the evaluation of medical education programmes by the World Federation for Medical Education.
Several tools have been developed to assess a school's educational environment over the last four decades.,, Although several of these tools have been shown to have robust validity and high measurement reliability, the most widely used and accepted one in medical education is the Dundee Ready Educational Environment Measure (DREEM), developed by Roff et al. It measures students' perceptions of the educational environment in five areas: Teaching, course organization, academic self-perception, atmosphere and social self-perception. The DREEM has been used for several purposes including to identify weaknesses in the curricula to inform change,,,, to compare students' expectations with actual experiences to identify gaps that requires improvement  and to compare the learning environments at different training sites , and at various stages  of their training. The DREEM also provides an opportunity to compare the educational environment across higher education institutions internationally , allowing medical schools to benchmark their educational environments.,
The use of the DREEM for both within-country and cross-national comparisons makes it pertinent to continuously assess its psychometric properties  because variations across countries in its reliability and validity can affect the meaning of comparisons. The few studies that have assessed the psychometric properties of the DREEM have reported mixed results that do not fully support the validity and reliability of its original structure.,,,,,,
In 2007 the University for Development Studies, School of Medicine and Health Sciences (UDS-SMHS) adopted and implemented PBL (Problem-based learning) for the training of its medical students through a curriculum referred to as the Problem-based learning, Community-based Education and Service Curriculum. Out of Ghana's five medical schools, the UDS-SMHS is one of two that uses the PBL methodology. After graduating two annual cohorts of students through this curriculum, a curriculum review has led to changes. Given these changes, it is important to assess students' perceptions of the educational environment to serve as baseline data for the local curriculum review. Furthermore, the educational environment has been accorded little attention in Ghana and other countries in sub-Saharan Africa. Therefore the data generated from this study will supplement the limited research on the educational environment in sub-Saharan Africa through the application and validation of the DREEM for the first time here.
This study aims to investigate the psychometric properties of the DREEM in a sample of Ghanaian medical students.
| Methods|| |
Setting and participants
This cross-sectional study was conducted among preclinical medical students of the University for Development Studies, School of Medicine and Health Sciences (UDS-SMHS). The UDS-SMHS medical programme is seven years during which students complete their preclinical training in years one through four, and subsequently move to clinical training in years five through seven. In year one, students follow a classical teaching and learning methodology as a preparatory year for subsequent years. The curriculum for years two through seven follows a PBL methodology. To allow for uniformity of teaching and learning methodologies and for convenience, only students from years two to four were eligible to participate in this study, which was voluntary. This study was approved by the Ethics Committee of the University for Development Studies, School of Medicine and Health Sciences, Ghana.
The DREEM questionnaire as well as three questions on demographic characteristics (age, gender and year of study) was self-administered to all eligible students.
The Dundee Ready Educational Environment Measure
Developed by Roff et al. through a Delphi approach involving a range of health professional educators in different settings and various countries, the DREEM is a 50-item questionnaire used for assessing the educational environment in health professions education programs. Each item is rated on a five-point Likert scale: 0 is strongly disagree, 1 is disagree, 2 is neither agree or disagree, 3 is agree and 4 is strongly agree. With a maximum score of 200, the 50 items are categorized into five major domains: 1) perceptions of learning (max. score = 48), 2) perceptions of teachers (max. score = 44), 3) academic self-perceptions (max. score = 32), 4) perceptions of atmosphere (max. score = 48), and 5) social self-perceptions (max. score = 28). These five domains were identified through the initial psychometric assessment of the DREEM by Roff et al. Nine items (Items 4, 8, 9, 17, 25, 35, 39, 48 and 50) are negatively worded and are recoded prior to calculating total and subscale scores. The entire DREEM questionnaire was employed for this study with a few minor changes.
Data collection procedure
The DREEM questionnaire was administered to all students following an exam given at the end of a curriculum block. Although voluntary, all students agreed to participate. Students spent typically 10 to 15 minutes to complete the questionnaire.
All data were manually entered into IBM SPSS version 21 (IBM Corp., USA) and later saved as a Microsoft Excel 2010 file. Descriptive statistics of mean, standard deviation and frequencies were used to describe the study sample. Basic psychometric properties including reliability and validity were tested. Prior to analyses, the data was assessed for its suitability for factor analysis. With a Kaiser–Meyer–Olkin measure of sampling adequacy of 0.857 and a statistical significance (P < 0.001) of Bartlett's test of Sphericity, the data and sampling were deemed to be adequate and suitable for validity and reliability tests. Internal consistency was tested through Cronbach's alpha. In addition, corrected item-total correlation (CITC) and Cronbach's alpha values if items are deleted (CAID) were also calculated. Both total and subscale Cronbach's alphas were calculated. Ideally, alpha values should range between 0.7 and 0.9 to demonstrate acceptable reliability, but values as low as 0.6 are sometimes considered acceptable. Alpha values below 0.6 suggest high heterogeneity and values above 0.9 suggest that the items may be too similar (item redundancy). To test the original five factor structure of the DREEM using confirmatory factor analysis (CFA), we used AMOS Version 21 (IBM Corp.) and the Maximum Likelihood Estimation method. CFA investigates how the data fits into a predetermined and constructed model by presenting the relationship between the data in the model and estimation of errors. An array of model fit statistics is produced to describe the relationship between the data and the model. Assessment of model fit of the data was done using model Chi-square goodness-of-fit and approximate fit indices. A non-significant (P > 0.05) Chi-square goodness of fit indicates a model fit. The approximate fit indices employed included: Goodness-of-fit index (GFI), adjusted goodness-of-fit index (AGFI), normed fit index (NFI), relative fit index (RFI), incremental fit index (IFI), Tucker-Lewis fit index (TFI) and comparative fit index (CFI). The rest were root mean square error of approximation (RMSEA) and root mean squared residual (RMR) in which values less than 0.05 would signify reasonable model fit., Values greater than 0.9 arising from the GFI, NFI, RFI, IFI, TFI and CFI indicate model fit,, whereas values of 0.85 and greater represent acceptable model fit. RMSEA values of 0.08 and less (preferably less than 0.05) and RMR values of 0.05 and less indicate a reasonable model fit., These descriptive fit indices are very sensitive to sample size, hence we employed several indices. In our efforts to further assess the factor structure of the DREEM an explorative factor analysis was conducted. Factor loadings were obtained after varimax rotation. Calculating corrected item-to-subscale, item-to-total correlation coefficients, and the respective Cronbach's alpha coefficients assessed internal consistency. We used Kaiser's eigenvalue criterion (>1) and a scree plot  to determine the number of factors to extract.
| Results|| |
Two hundred and sixty five questionnaires were distributed to students, and 234 complete questionnaires were returned to yield a response rate of 88.3%. Of the 234 respondents, 118 were male and 81 were female (35 students did not indicate their gender). Respondents had a mean ± SD age of 22.2 ± 2.1 ranging from 18 to 34 years of age (16 students did not indicate their age).
The basic psychometric properties of the DREEM in our sample are as shown in [Table 1]. The internal consistency for the overall DREEM score was 0.92. All subscales had Cronbach's alphas ranging from 0.73 – 0.78 except for the social self-perception and the academic self-perception subscales, which had Cronbach's alphas < 0.70.
|Table 1: Basic psychometric properties of the original Dundee Ready Educational Environment Measure questionnaire (n=234)|
Click here to view
The mean total score of the DREEM was 115.57. The corrected item-total correlations and Cronbach's alpha if item deleted of the items according to the subscales of the DREEM are presented in [Table 2].
|Table 2: Corrected item-total correlations and Cronbach's Alpha if item deleted of the items of the Dundee Ready Educational Environment Measure according to subscales (n=234)|
Click here to view
All items had corrected item-total correlations of 0.2 or greater except items 4, 10, 17, 39, 48, and 50. Item 48 (The teaching is too teacher-centred), item 17 (Cheating is a problem in this school), item 50 (The students irritate the teachers), and item 4 (I am too tired to enjoy this programme) had extremely low corrected item-total correlations of -0.003, -0.051, 0.07 and 0.097 respectively. Generally, the deletion of the items with corrected items-total correlations < 0.2 improved the Cronbach's alphas of their individual subscales [Table 2].
A confirmatory factor analysis was conducted on the original DREEM questionnaire to determine its construct validity [Table 3]. The present data did not fit Roff et al.'s five-factor structure of the DREEM questionnaire. Standardized regression weights (factor loadings) ranged from 0.024 to 0.732 indicating that some items did not represent the constructs being measured. A low goodness-of-fit was indicated by a significant Chi-square test (P < 0.001), GFI (=0.70), AGFI (=0.67), CFI = (0.67), TLI = (0.65), NFI (=0.52) as well as RMR (=0.094). Only RMSEA (=0.068) showed quite an acceptable fit. Apart from seven items, all items had standardized regression weights ≥ 0.3. These items were 4, 10, 17, 25, 39, 48 and 50. Items 17 and 48 had extremely low standardized regression weights.
|Table 3: Results of the confirmatory factor analysis of the original Dundee Ready Educational Environment Measure questionnaire (n=234)|
Click here to view
Correlations between the factors from the confirmatory factor analysis are presented in [Table 4]. The correlations between the factors are very large raising issues about the independence of the individual factors.
|Table 4: Factor correlation matrix of the original Dundee Ready Educational Environment Measure (n=234)|
Click here to view
Exploratory factor analysis using principal component analysis with varimax rotation was conducted. Various factor solutions were provided ranging from 4 to 14 factors. However, the four factor structure was found to be both statistical and theoretical logical. Explaining 37.6% of its variance, the four factors had eigenvalues > 1 with items having communalities that ranged from 0.089 to 0.621 [Table 5]. Item 17 had an extremely low communality (0.089). Three items (13, 17 and 50) had communality values < 0.2 [Table 5]. Except items 17 (Cheating is a problem in this school), item 14 (I am rarely bored on this programme) and item 13 (The teaching is student-centred) which had factor loadings < 0.3; all items had loadings ≥ 0.3. Each factor was reviewed and named according to the content of the items loading unto it [Additional File 1 [Additional file 1]]. The review of the individual items loading onto the factors indicated that some items were classified illogically; these items were moved to appropriate factors. Items 28, 42 and 46 were moved from factor 2 to factor 3. In a similar fashion, items 10 and 41 from factor 3 to factor 2. The decision to make the changes was borne out of the fact that the content of the items were either appropriate for the new factor (10, 46, and 42), and/or the items loaded similarly to both factors (28 and 41). Cronbach's alpha analysis of the new four factor solution yielded values ranging from 0.619 to 0.915 indicating an acceptable reliability. Corrected item-subscale correlations were used to determine the adequacy of each item within a factor. The first factor had corrected item-subscale correlations that ranged from 0.376 to 0.714; from 0.323 to 0.590 for the second factor; from 0.245 to 0.537 for the third factor and from 0.195 to 0.385 for the fourth factor. Six items (25, 17, 48, 50, 10 and 14) had corrected item-subscale correlations below 0.3.
|Table 5: Principal component analysis with varimax rotation for the Dundee Ready Educational Environment Measure questionnaire (n=234)|
Click here to view
| Discussion|| |
This study considered the psychometric properties of the DREEM in a sample of Ghanaian preclinical medical students. In this sample the overall DREEM scale had an excellent internal consistency. Even though the Cronbach's alpha of three of the subscales were above 0.70 indicating an acceptable internal consistency, two subscales had values below 0.70. The five factor structure of the original DREEM was not supported, raising concerns about its construct validity. Exploratory factor analysis suggested a four-factor solution, which was found to have acceptable internal consistency and construct validity.
The total Cronbach's alpha of the DREEM was 0.920, quite similar to those reported previously ,,, but higher than that reported by Hammond et al. and lower than that of Wang et al. among nursing students in China. The subscales had Cronbach's alpha values that ranged from 0.54 to 0.78. Generally, alpha scores over 0.7 are accepted as indicating good internal consistency.,, Similar to several other studies,,,,, the academic self-perception and the social self-perception subscales had Cronbach's alpha values below this threshold, indicating variable internal consistency of these subscales. The variation in the Cronbach's alpha scores suggests the sample-dependent characteristic of the statistic and warrants the call to continue to evaluate the psychometric properties of the DREEM.
Reviewing the individual items, items Q4, Q10, Q17, Q39, Q48 and Q50 had both low correlations and low factor loadings. Interestingly, most of these items are negatively worded (Q4, Q17, Q39, Q48 and Q50) and have been previously reported ,, to reduce the reliability of the subscales they represent. In the present study the removal of these items improved the Cronbach's alpha values of their individual scales as demonstrated by the Cronbach's alpha values if item is deleted. This suggests that the internal consistency of the subscales could be improved by revising and restructuring these items. Still, the low correlations and low factor loadings of these items could also be as a result of the misinterpretation of these items by the students. High correlations were also observed between the subscales indicating there were significant overlapping and lack of discrimination between them. These findings also suggest there are repetitions of items measuring similar constructs that compromise the construct validity of the DREEM. Continued efforts should be made to revise and establish the psychometric properties of the DREEM, as suggested by Vaughan et al. and Hammond et al. Authors using the DREEM should endeavour to always report its psychometric properties.
In agreement with previous findings ,,, and as demonstrated above, our data did not support the original five-factor structure of the DREEM. With this in mind we conducted an exploratory factor analysis. After several factor solutions were suggested, a four-factor solution was arrived at, different from the original five-factor structure of the DREEM. Except factor 4, all factors in the new model had Cronbach's alpha values ranging from 0.750 to 0.915 indicating a good level of internal consistency of the subscales.
In the development of measurement scales, items with corrected item-subscale correlations <0.20 is usually rejected. Only item 17 (Cheating is a problem in this school) had a corrected item-subscale correlation of 0.195 compared to six items on the original DREEM. In a study that assessed the internal consistency of the DREEM among Chinese nursing students, Wang et al. also found item 17 to have an item-subscale coefficient <0.20. In agreement with the suggestion of Wang et al. this item could be modified or suppressed to improve the internal consistency of the subscales of the DREEM.
Taking into account items with factor loadings ≥ 0.3, the four-factor solution had fewer items that had factor loadings below 0.3 compared to the five-factor structure. Whereas the original DREEM had seven items (14% of the items), the new four-factor solution had three items (6% of the items). These were items 13, 14 and 17. Item 17 has been reported in several studies to have low factor loadings., It stands to suffice that to obtain a reliable and valid DREEM, some factors would have to be revised and restructured. We reluctantly suggest the removal of these factors, as this should be done with caution. Items that might appear to be inappropriate in a particular population might not be so in another. Also the original validity and reliability of the DREEM as demonstrated by Roff et al. was derived from theoretical and qualitative procedures. Evidently, the new four-factor structure has not been proven to be a superior measurement model compared with the original, but could be seen as an alternative structure which can be explored/tested further.
Strengths and weakness of the study
This study is not without limitations. The study constituted medical students from a single institution, with a non-random sample, limiting the generalizability of its findings. Conceding to the requirement of a sample size of 5-10 per item, with a minimum of 100 participants for factor analysis regardless of the number, our sample size of 234 cannot be said to be adequate for factor analysis for the DREEM instrument with 50 items.
| Conclusion|| |
This is the first study to assess the psychometric properties of the DREEM among medical students in Ghana and probably anywhere in Sub-Saharan Africa. The data generated from this study could be useful to other health professions institutions in other developing countries. To stakeholders of health professions educations, the DREEM could be used to assess the educational environment to identify issues and challenges from the perspective of the student.
In these Ghanaian medical students, the overall internal consistency of the DREEM total score was found to be excellent, as it was also for three of its subscales. Internal consistency of two subscales was not good, with Cronbach's alpha values below 0.7. Confirmatory factor analysis did not support the original five-factor structure of the DREEM. Several factors had factor loadings < 0.3. Exploratory factor analysis suggested a four-factor solution as a possible alternative to the five-factor structure of the DREEM. Since it is generally difficult to prove validity of a measurement scale in any absolute sense, we advocate for a comprehensive application of the DREEM in future studies across institutions and different nations to establish its psychometric properties.
Authors wish to thank Mr. Suara Sufyan for assisting in the data collection. Authors are also grateful to students for their voluntary participation in the study.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Harden RM. The learning environment and the curriculum. Med Teach 2001;23:335-6.
Genn JM. AMEE Medical Education Guide No 23 (Part 2): Curriculum, environment, climate, quality and change in medical education-a unifying perspective. Med Teach 2001;23:445-54.
Roff S, McAleer S. What is educational climate? Med Teach 2001;23:333-4.
International standards in medical education: Assessment and accreditation of medical schools' – Educational programmes. A WFME position paper. The Executive Council, The World Federation for Medical Education. Med Educ 1998;32:549-58.
Rothman AI, Ayoade F. The development of a learning environment: A questionnaire for use in curriculum evaluation. Acad Med 1970;45:754-9.
Marshall RE. Measuring the medical school learning environment. Acad Med 1978;53:98-104.
Feletti GI, Clarke RM. Review of psychometric features of the Medical School Learning Environment Survey. Med Educ 1981;15:92-6.
Soemantri D, Herrera C, Riquelme A. Measuring the educational environment in health professions studies: A systematic review. Med Teach 2010;32:947-52.
Roff S, McAleer S, Harden RM, Al-Qahtani M, Ahmed AU, Deza H, et al
. Development and validation of the Dundee Ready Education Environment Measure (DREEM). Med Teach 1997;19:295-9.
Byrnes JM, Goldstein S, Venator B, Pollicino C, Ng SK, Veroff D, et al.
The impact of population-based disease management services for selected chronic conditions: The Costs to Australian Private Insurance – Coaching Health (CAPICHe) study protocol. BMC Public Health 2012;12:114.
Al-Hazimi A, Al-Hyiani A, Roff S. Perceptions of the educational environment of the medical school in King Abdul Aziz University, Saudi Arabia. Med Teach 2004;26:570-3.
Till H. Climate studies: Can students' perceptions of the ideal educational environment be of use for institutional planning and resource utilization? Med Teach 2005;27:332-7.
Bouhaimed M, Thalib L, Doi SA. Perception of the educational environment by medical students undergoing a curricular transition in Kuwait. Med Princ Pract 2009;18:204-8.
Demirören M, Palaoglu O, Kemahli S, Ozyurda F, Ayhan IH. Perceptions of students in different phases of medical education of educational environment: Ankara university faculty of medicine. Med Educ Online 2008;13:8.
Carmody DF, Jacques A, Denz-Penhey H, Puddey I, Newnham JP. Perceptions by medical students of their educational environment for obstetrics and gynaecology in metropolitan and rural teaching sites. Med Teach 2009;31:e596-602.
Bennett D, Kelly M, O'Flynn S. Are the bigger hospitals better: DREEM on? Ir J Med Sci 2010;179:515-9.
Roff S, McAleer S, Ifere OS, Bhattacharya S. A global diagnostic tool for measuring educational environment: Comparing Nigeria and Nepal. Med Teach 2001;23:378-82.
de Oliveira Filho GR, Schonhorst L. Problem-based learning implementation in an intensive course of anaesthesiology: A preliminary report on residents' cognitive performance and perceptions of the educational environment. Med Teach 2005;27:382-4.
Dimoliatis ID, Vasilaki E, Anastassopoulos P, Ioannidis JP, Roff S. Validation of the Greek translation of the Dundee Ready Education Environment Measure (DREEM). Educ Health (Abingdon) 2010;23:348.
Aghamolaei T, Fazel I. Medical students' perceptions of the educational environment at an Iranian Medical Sciences University. BMC Med Educ 2010;10:87.
Hammond SM, O'Rourke M, Kelly M, Bennett D, O'Flynn S. A psychometric appraisal of the DREEM. BMC Med Educ 2012;12:2.
Vaughan B, Mulcahy J, McLaughlin P. The DREEM, part 2: Psychometric properties in an osteopathic student population. BMC Med Educ 2014;14:100.
Jakobsson U, Danielsen N, Edgren G. Psychometric evaluation of the Dundee Ready Educational Environment Measure: Swedish version. Med Teach 2011;33:e267-74.
Yusoff MS. The Dundee Ready Educational Environment Measure: A confirmatory factor analysis in a sample of Malaysian medical students. Int J Humanit Soc Sci 2012;2:313-21.
Wang J, Zang S, Shan T. Dundee Ready Education Environment Measure: Psychometric testing with Chinese nursing students. J Adv Nurs 2009;65:2701-9.
de Oliveira Filho GR, Vieira JE, Schonhorst L. Psychometric properties of the Dundee Ready Educational Environment Measure (DREEM) applied to medical residents. Med Teach 2005;27:343-7.
Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika 1951;16:297-334.
Nunnally JC, Bernstein IH, Berge JM. Psychometric Theory. New York: McGraw-Hill; 1967.
Stevens JP. Applied Multivariate Statistics for the Social Sciences. Abingdon: Routledge; 2012.
Brown TA. Confirmatory Factor Analysis for Applied Research. New York: Guilford Publications; 2014.
Schermelleh-Engel K, Moosbrugger H, Müller H. Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods Psychol Res Online 2003;8:23-74.
Streiner DL, Norman GR, Cairney J. Health Measurement Scales: A Practical Guide to Their Development and Use. England:Oxford University Press; 2014.
Vaughan B, Carter A, Macfarlane C, Morrison T. The DREEM, part 1: Measurement of the educational environment in an osteopathy teaching program. BMC Med Educ 2014;14:99.
Stewart TJ, Frye AW. Investigating the use of negatively phrased survey items in medical education settings: Common wisdom or common mistake? Acad Med 2004;79 10 Suppl: S18-20.
Ostapczuk MS, Hugger A, de Bruin J, Ritz-Timme S, Rotthoff T. DREEM on, dentists! Students' perceptions of the educational environment in a German dental school as measured by the Dundee Ready Education Environment Measure. Eur J Dent Educ 2012;16:67-77.
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5]