|ORIGINAL RESEARCH ARTICLE
|Year : 2016 | Volume
| Issue : 2 | Page : 75-81
Factors influencing medical students' self-assessment of examination performance accuracy: A United Arab Emirates study
Sami Shaban1, Elhadi H Aburawi2, Khalifa Elzubeir3, Sambandam Elango1, Margaret El-Zubeir1
1 Department of Medical Education, College of Medicine and Health Sciences, UAE University, Al Ain, UAE
2 Department of Pediatrics, College of Medicine and Health Sciences, UAE University, Al Ain, UAE
3 Department of Psychiatry, Norfolk and Suffolk NHS Foundation Trust, Woodlands Hospital, Ipswich, Suffolk, UK
|Date of Web Publication||19-Aug-2016|
Department of Medical Education, College of Medicine and Health Sciences, United Arab Emirates University, P. O. Box 17666, Al Ain
Source of Support: None, Conflict of Interest: None
Background: Assessment of one's academic capabilities is essential to being an effective, self-directed, life-long learner. The primary objective of this study was to analyze self-assessment accuracy of medical students attending the College of Medicine and Health Sciences, United Arab Emirates University, by examining their ability to assess their own performance on an MCQ examination. Methods: 1 st and 2 nd year medical students (n = 235) self-assessed pre and post-examination performance were compared with objectively measured scores (actual examination performance). Associations between accuracy of score prediction (pre and post assessment), and students' gender, year of education, perceived preparation, confidence and anxiety were also determined. Results: Expected mark correlated significantly with objectively assessed marks (r = 0.407; P < 0.01) but with low predictability (R 2 = 0.166). The average objectively determined mark was 69% and the average expected mark was equivalent to 83%; indicating that students significantly overestimate their examination performance. Self-assessed pre-examination score range was significantly different between males and females (P < 0.05) with females expecting higher marks. Preparation and confidence correlated significantly with actual examination score (P < 0.05; r = 0.459 and 0.569 respectively). Discussion: Gender, self-reported preparation and confidence are associated with self-assessment accuracy. Findings reinforce existing evidence indicating that medical students are poor self-assessors. There are potentially multiple explanations for misjudgment of this multidimensional construct that require further investigation and change in learning cultures. The study offers clear targets for change aimed at optimizing self-assessment capabilities.
Keywords: Anxiety, confidence, medical students, preparation, self-assessment accuracy
|How to cite this article:|
Shaban S, Aburawi EH, Elzubeir K, Elango S, El-Zubeir M. Factors influencing medical students' self-assessment of examination performance accuracy: A United Arab Emirates study. Educ Health 2016;29:75-81
|How to cite this URL:|
Shaban S, Aburawi EH, Elzubeir K, Elango S, El-Zubeir M. Factors influencing medical students' self-assessment of examination performance accuracy: A United Arab Emirates study. Educ Health [serial online] 2016 [cited 2017 Nov 21];29:75-81. Available from: http://www.educationforhealth.net/text.asp?2016/29/2/75/188688
| Background|| |
Self-assessment and critical self-reflection are considered important aspects of professional medical practice. , Self-assessment has been defined as "the involvement of learners in making judgments about whether or not learner-identified standards have been met"  and is used in the metacognition literature to refer to the judgments an individual makes on the basis of self-knowledge. Hence, self-assessment can involve an individual considering and judging whether he/she is capable of or has achieved what is necessary to complete tasks successfully. Such judgments about assessment outcomes can be made by students on their own essays, reports, projects, presentations, clinical skills, dissertations, and examination scripts. By self-assessing, students develop skills to critically appraise themselves and become more effective life-long learners, which are essential for practicing physicians in identifying areas for further professional development. ,,,
This complex, multidimensional activity is nevertheless, not easy. For students, self-assessment skills are not learned spontaneously; rather, training, practice and good quality teacher feedback are required. Moreover, questions have been raised as to whether students can objectively estimate their performance and investigations of the construct has been criticized in the medical education literature as flawed on methodological grounds. ,,
While some studies have found congruence between students' and their teachers' assessments of performance,  several others have shown that students gauge their abilities and performance poorly. For example, average performing medical students thought they performed better on a required clinical exercise than their supervisors thought they did,  and length of time in medical school, anxiety and gender have been shown to influence accuracy of self-assessment. ,,,, Indeed, regarding gender differences, the literature indicates that female medical students are more often less confident and underestimate their performance than males. , Less frequently evaluated is consistency of self-assessment from one time period to another, such as pre and post-examination, as well as self-assessment comparisons for students of different years of medical education.
Understanding self-assessment and the influences of gender, stage of medical education, perceived preparation, anxiety and confidence could assist in development of specific interventions. The interventions could be in the form of faculty feedback to students, training in self-assessment, assisting students in making realistic judgments about their preparedness for examinations and general enhancement of a learning culture which support development of self-assessment capabilities among medical students.
The present study had two objectives. The first was to determine self-assessment accuracy of medical students in the first two years of study of a six-year post-secondary medical education program. The second objective was to identify characteristics that potentially influence self-assessment accuracy, including year in medical school, gender, perceived preparation, confidence and anxiety. Based on study outcomes we aimed to make recommendations for curriculum interventions designed to assist students in the development of self-assessment. This study allowed us to highlight factors affecting junior medical students' self-assessment accuracy in a Middle Eastern medical school environment, wherein there exists a paucity of published literature on the topic.
| Methods|| |
Context and participants
The College of Medicine and Health Sciences (CMHS) of the United Arab Emirates University is a large public medical school in United Arab Emirates (UAE). The medical curriculum is of six years duration and is divided into three phases, each lasting two years, followed by a one year internship. Students are selected for entry into medical school after finishing secondary school. Females represent approximately 75% of our annual intake. Admission to the CMHS is restricted to UAE nationals.
The language of instruction is English. In the first two years of medical school, students learn the basic sciences. This premedical phase of the curriculum is comprised predominantly of didactics and lab based learning. Problem based learning from the third year of the curriculum provides a learning environment that requires students to develop learning objectives and implement self-directed learning skills to achieve objectives and engage in self-regulated learning. During years three and four, they learn to apply biomedical knowledge and skills to the study of organ systems. In the fifth and sixth years students study various aspects of clinical medicine divided into six to ten week rotations in several clinical disciplines. At the end of each module a summative examination is conducted. For this study the final exams of two courses were chosen for each of year one and year two medical students during the 2012-2013 academic year. These final exams are conducted electronically in the Computer Assessment Center of the CMHS.
A short electronic survey of four Likert-scale questions was developed to assess students' self-assessment of their preparedness for the exam, confidence in passing the exam, prediction of mark (score) on the examination, and anxiety level both before and after taking the exam. [Table 1] shows the pre examination survey. The post examination survey is similar with past tense wording. In question 3 of the survey we asked the student to predict a range of expected mark out of a choice of ranges. We then compared the actual mark with the predicted range. We considered congruence in predicted scores and actual scores if the actual mark fell within this predicted range.
The answers to the survey were collected and stored electronically along with the results of the exam. Data were analyzed using SPSS for descriptive statistics, Pearson's correlation coefficient calculations, regression analysis, and t-test.
Ethical approval to conduct the survey was obtained from the College's Research Ethics Committee.
| Results|| |
The total number of students taking the four exams was 471, of which 235 (50%) chose to participate in the pre-examination survey and 59 (12.5%) chose to participate in both the pre and post-examination surveys.
There was no statistically significant difference between pre and post-examination survey scores for those who completed both surveys (59 students). In fact, a high correlation was observed between pre and post-examination survey results for all four questions. The number and percentage of students per question with same, higher and lower response values on the pre vs post examination surveys are shown in [Table 2].
|Table 2: Number and percentage of students per question with same, higher and lower responses on the pre- versus post-examination surveys (n=59)|
Click here to view
Correlation between students' response to expected mark (pre-examination) and actual mark was significant (r = 0.407; P < 0.001). The average objectively determined mark of this group of students (both years combined) was 69% and the average response to expected mark was 3.73 on a scale from 1 (<60%) to 5 (90-100%) which is equivalent to an average expected mark of 83%. Comparing this with the resulting actual average mark of 69%, it is clear that students significantly overestimate their examination performance. [Figure 1] shows the regression line with significant correlation but low predictability.
There were also significant correlations between reported preparedness and expected mark (Pearson's Correlation = 0.459, P < 0.001) as well as between reported confidence and expected mark (Pearson's Correlation = 0.569, P < 0.001). Reported anxiety and expected mark were not significantly correlated. [Table 3] contains all correlations (n = 235).
|Table 3: Correlations between actual mark, preparedness, confidence, expected mark, and anxiety (n=235)|
Click here to view
Taking only the pre-examination survey responses (235 students), we looked at differences in predicted performance between gender and year of study groups. Self-assessed pre-examination score range was significantly different between males and females (t-test P < 0.05) with females expecting higher marks. Correlation between actual mark and self-assessed expected mark for males and females show that males had a higher correlation (Pearson›s Correlation = 0.478, P < 0.001) than females (Pearson's Correlation = 0.339, P < 0.001). Also of note, older students (2 nd year vs 1 st year of study) expressed higher levels of anxiety although this did not reach statistical significance (t-test, P = 0.07). [Table 4] shows correlations between actual mark vs expected mark, preparedness, confidence, and anxiety by gender and year of study group (n = 235).
|Table 4: Correlations between actual mark versus expected mark, preparedness, confidence, and anxiety by gender and year of study groups (n=235)|
Click here to view
Finally, we were interested in determining the average mark of those who took the pre-examination survey vs those who chose not to take this survey as one means of identifying possible participation bias. [Table 5] presents the mean mark of each group with significance difference in the mark indicating that the group that took the pre-examination survey did significantly better on the examination compared to the group that did not take this survey.
|Table 5: Comparison of average mark for those who took the preexamination survey versus those who chose not to take this survey|
Click here to view
We also compared the marks of those who took only the pre-examination survey with those who took both the pre and post-examination surveys. There was no significant difference in their marks (t-test, P = 0.071).
| Discussion|| |
Despite a difference between numbers of students completing the pre and post-examination surveys, results have revealed several areas of interest. Findings are discussed under headings below.
Self-assessment accuracy, gender and year in medical school
The present study considered two variables which could explain differences in accuracy of self-assessment: Gender and year in medical school (i.e., maturity and experience of the curriculum). Pre-examination survey responses by gender revealed that female students significantly generally expected higher marks than male counterparts [Table 4]. Past research has indicated that female students at different stages of the medical education continuum tend to underestimate performance; suggesting less confidence than men. ,,, Similarly, a previous study of self-assessment in our own institution revealed that female self-evaluations in a PBL course were statistically significantly lower than males.  The authors speculated that females' adoption of less self-confident approaches to self-evaluation was possibly related to Arab cultural and social beliefs. Our findings revealed that males in our study were more accurate in predicting their marks than females and suggest that while female medical students are possibly more confident about their academic performance they are nevertheless less accurate than their male counterparts in evaluating it, in that they expect higher marks.
Findings of no significant differences in self-assessment of first and second year students challenged our expectations of the influence of our PBL learning environment to improve skills in self-assessment. Two meta-analyses , revealed that students in advanced courses appear to be more accurate assessors of performance than those in introductory courses. There is nevertheless general agreement that similar to all skills, acquisition is developmental and context specific.  Expanding and supporting ability to self-monitor learning and performance across the curriculum and not just for those in their formative years of medical education is probably called for. Improvement can be realized through reflection, seeking out and using feedback from teachers, peers and other sources. , In this regard, we would concur somewhat with Eva and Regehr  that "self-assessment as a mechanism of ongoing monitoring must take precedence over self-assessment as a mechanism for identifying and redressing gaps" (pS53).
Institutional support for principles of self-monitoring and assessment for learning in contrast to assessment of learning is crucial  and we are endeavoring to align assessment to feedback and reflection. At the end of PBL and Clinical skills sessions, students are encouraged to reflect on their performance. Self-reflection on performance is also a part of their portfolios. These activities are important contributors to a learning culture that creates opportunities and conditions for self-assessment and self-monitoring of learning. We would further suggest that other often overlooked aspects of ongoing monitoring are the interrelated mechanisms of self-awareness and emotional intelligence (EI). Benbassat and Baumal  indicate that medical students and practicing doctors can be taught to be aware of inadequate competencies and performance in specific domains. Given increasing awareness of the role emotional competence plays in individual performance, medical educators could do more to facilitate medical students' mastery of emotional intelligence. ,
Self-assessed mark, preparation, confidence and anxiety
There were notable similarities in mean ratings for preparation, confidence and anxiety among those who completed pre and post-examination surveys. Additionally, consistent with other studies, we observed an over-estimation of expected mark in contrast to objectively determined mark, indicating that our medical students have poor self-assessment accuracy and tend to be overly optimistic when judging their performance. ,,,,
We further observed that despite over optimistic estimations of examination performance, surprisingly, a statistically significant correlation between expected and actual marks of 0.407 existed and this was in excess of a weighted mean correlation of 0.210 reported by other researchers as a useful point of reference.  This indicates that although limited in accuracy, our students have a self-assessment capability which is probably amenable to improvement. Improved self-assessment accuracy has been demonstrated in situations where the task is objectively measured and students have specific information about assessment criteria and standards. 
Overestimation of expected marks in our context could thus be explained by a combination of factors. Self-assessment requires identification of criteria or standards applicable to one's performance and making judgments about the extent to which ones performance is likely to meet those criteria.  Medical students in our College are well aware that a score of 75% is applicable to all end course assessments and therefore could be assumed to be able to assess their preparation and performance relative to this minimally accepted institutional standard. Students also undertake regular formative assessments aimed at providing opportunities to gauge progress in mastering the material. However, students may be focusing primarily on the overall score achieved and not enough on reflection of strengths and weaknesses and/or teacher provided feedback. Our findings suggests a possible lack of specific, constructive, supportive feedback post formative and summative examinations and pre-assessment sharing with students the success criteria for each assessment activity. , Information rich assessment data, structured opportunities for reflection and facilitated self-assessment using a portfolio approach can be designed to support development of habits of reflection and self-assessment. ,, It should also be remembered that our students are high school entrants to medicine and consequently probably have less self-assessment experience, are less well equipped to seek, interpret and learn from feedback than their more mature, graduate entry counterparts in other countries. Clearly the gaps between high school and medical school experiences and expectations have to be closed. If students are to be able to monitor and assess the quality of their own work, provision of substantial evaluative experiences not as an extra but as a strategic part of curriculum design is necessary.
In the present study, students' rated themselves as being "somewhat" confident in passing the examination and this remained stable pre and post-examination. Overconfidence and inaccurate evaluation of preparation are other potential sources of self-assessment misjudgment. There was however, a significant positive correlation between reported confidence, preparation and actual performance. Confidence describes a judgment about one's ability to achieve a desired level of performance (self-efficacy) in the examination and was presumed to be based on known levels of preparation. Mavis  found that performance in clinical skills and biomedical science curricula were related to perceived anxiety, self-efficacy and preparedness. Nevertheless, several authors have highlighted how by its very nature, self-assessment can never be objective or free from the beliefs and values individuals hold about themselves.  Using pre and post examination self-assessment accuracy as a marker of self-assessed preparedness and performance in the examination could have been confounded by a basic human tendency to preserve a sense of self-efficacy.
Our study further revealed that students were "somewhat" anxious and self-reported mean ratings of anxiety was similar pre and post-examination although not correlated with actual marks. Depending on context and level (Yerke-Dodson law), test-anxiety can be detrimental to or beneficial to academic performance and self-assessment. , Strategies including cognitive behavior therapy, peer coaching and courses are reported to help alleviate excessive anxiety, optimize learning strategies, confidence and performance. ,
Survey respondents' versus non-respondents mean marks
Finally, we examined average marks of those who took the pre-examination survey vs those who chose not to and observed that the group that took the pre-examination survey performed significantly better on the examination compared to the group who did not. Some studies indicate that low achieving students appear to lack insight into quality of their performance.  Our finding may also be indicative of lower achieving students' lack of appreciation and understanding of the rationale for performance self-assessment. In such circumstances, a clear explanation of the purpose of self-assessment followed by repeated self-assessment practice, immediate constructive feedback and discussion of potential discrepancies between objective and student self-assessments are likely to be useful.  It is increasingly recognized that specific opportunities to practice self-assessment skills is useful and should be provided in the curriculum as a prerequisite to engagement in self-assessment activities. 
Limitations and future directions
Obvious limitations of the study are low response rate and incompleteness of the data in terms of pre and post-examination self-assessments. Nevertheless, we believe that pre-examination survey data is substantial enough to indicate recommendations for faculty improvement of assessment feedback, student self-reflection, -awareness, -monitoring and -assessment skills practice via e.g., use of reflective diaries, portfolios, PBL and small group discussions.
Other limitations are that the study relied heavily on quantitative data and used a self-assessment rating scale which has not been validated. We would therefore recommend that future studies adopt a mixed method approach and where possible utilize validated instruments.
The present study results support observations that individuals require multiple interrelated competencies when making judgments of performance and therefore we would suggest a focus on a cluster of salient variables in future studies. For example, it is probable that self-assessment, preparation and confidence are supported by competencies not measured in this study. These and other competences could be evaluated along a continuum of mastery to determine at what point there is an impact on accurate prediction of performance. Additionally, since a self-assessing organizational culture is more likely to nurture and facilitate self-assessment in individuals, enhancing learning environments to support a group of valued self-assessment practices and capabilities will have a positive impact.
| Conclusions|| |
Self-assessment accuracy of students in the first two years of medical school is associated with gender, perceived confidence and preparation. Both perceived self-assessment significance and capabilities are potentially associated with individual and institutional learning culture influences. There are however, additional contributors to competence of this multidimensional, context bound construct that requires further investigation. Our study supports observations that individuals require multiple competences when making judgments about their performance and therefore a reformulation of the research agenda that includes focus on mastery of a cluster of competences rather than on self-assessment per se is called for. Undergraduate education and training programs may find these data useful when designing interventions aimed at improving self-assessment capabilities among medical students.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
General Medical Council. Tomorrow's Doctors: Outcomes and Standards for Undergraduate Medical Education. London: General Medical Council; 2009.
General Medical Council. Good Medical Practice. London: General Medical Council; 2013.
Eva KW, Regehr G. Self-assessment in the health professions: A reformulation and research agenda. Acad Med 2005;80 10 Suppl: S46-54.
Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. JAMA 2006;296:1094-102.
Deakin Crick R. Learning how to learn: The dynamic assessment of learning power. The Curriculum Journal 2007;18:135-53.
Ward M, Gruppen L, Regehr G. Measuring self-assessment: Current state of the art. Adv Health Sci Educ Theory Pract 2002;7:63-80.
Eva KW, Cunnington JP, Reiter HI, Keane DR, Norman GR. How can I know what I don't know? Poor self assessment in a well-defined domain. Adv Health Sci Educ Theory Pract 2004;9:211-24.
Eva KW, Regehr G. Knowing when to look it up: A new conception of self-assessment ability. Acad Med 2007;82 10 Suppl: S81-4.
Fitzgerald JT, Gruppen LD, White CB. The influence of task formats on the accuracy of medical students' self-assessments. Acad Med 2000;75:737-41.
Gruppen LD, Garcia J, Grum CM, Fitzgerald JT, White CA, Dicken L, et al.
Medical students' self-assessment accuracy in communication skills. Acad Med 1997;72:1126.
Blanch DC, Hall JA, Roter DL, Frankel RM. Medical student gender and issues of confidence. Patient Educ Couns 2008;72:374-81.
Blanch-Hartigan D. Medical students' self-assessment of performance: Results from three meta-analyses. Patient Educ Couns 2011;84:3-9.
Colbert-Getz JM, Fleishman C, Jung J, Shilkofski N. How do gender and anxiety affect students' self-assessment and actual performance on a high-stakes clinical skills examination? Acad Med 2013;88:44-8.
Colthart I, Bagnall G, Evans A, Allbutt H, Haig A, Illing J, et al.
The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME Guide no 10. Med Teach 2008;30:124-45.
Minter RM, Gruppen LD, Napolitano KS, Gauger PG. Gender differences in the self-assessment of surgical residents. Am J Surg 2005;189:647-50.
Das M, Mpofu D, Dunn E, Lanphear JH. Self and tutor evaluations in problem-based learning tutorials: Is there a relationship? Med Educ 1998;32:411-8.
Edwards RK, Kellner KR, Sistrom CL, Magyari EJ. Medical student self-assessment of performance on an obstetrics and gynecology clerkship. Am J Obstet Gynecol 2003;188:1078-82.
Lind DS, Rekkas S, Bui V, Lam T, Beierle E, Copeland EM 3 rd
. Competency-based student self-assessment on a surgery rotation. J Surg Res 2002;105:31-4.
Falchikov N, Boud D. Student self-assessment in higher education: A meta-analysis. Rev Educ Res 1989;59:395-430.
Boud D. Avoiding the traps: Seeking good practice in the use of self-assessment and reflection in professional courses. Soc Work Educ 1999;18:121-32.
Kruidering-Hall M, O'Sullivan PS, Chou CL. Teaching feedback to first-year medical students: Long-term skill retention and accuracy of student self-assessment. J Gen Intern Med 2009;24:721-6.
Brookhart SM. Successful students' formative and summative uses of assessment information. Assess Educ Princ Policy Pract 2001;8:153-69.
Benbassat J, Baumal R. Enhancing self-awareness in medical students: An overview of teaching approaches. Acad Med 2005;80:156-61.
Arora S, Ashrafian H, Davis R, Athanasiou T, Darzi A, Sevdalis N. Emotional intelligence in medicine: A systematic review through the context of the ACGME competencies. Med Educ 2010;44:749-64.
Chew BH, Zain AM, Hassan F. Emotional intelligence and academic performance in first and final year medical students: A cross-sectional study. BMC Med Educ 2013;13:44.
Gordon MJ. A review of the validity and accuracy of self-assessments in health professions training. Acad Med 1991;66:762-9.
Langendyk V. Not knowing that they do not know: Self-assessment accuracy of third-year medical students. Med Educ 2006;40:173-9.
Dannefer EF. Beyond assessment of learning toward assessment for learning: Educating tomorrow's physicians. Med Teach 2013;35:560-3.
Buckley S, Coleman J, Davison I, Khan KS, Zamora J, Malick S, et al.
The educational effects of portfolios on undergraduate student learning: A best evidence medical education (BEME) systematic review. BEME Guide No 11. Med Teach 2009;31:282-98.
Earl L. Assessment as Learning: Using Classroom Assessment to Maximize Student Learning. Thousand Oaks, CA: Corbin Press; 2003.
Mavis B. Self-efficacy and OSCE performance among second year medical students. Adv Health Sci Educ Theory Pract 2001;6:93-102.
Chemers MM, Hu L, Carcia BF. Academic self-efficacy and first year college student performance and adjustment. J Educ Psychol 2001;93:55-64.
Stewart RA, Hauge LS, Stewart RD, Rosen RL, Charnot-Katsikas A, Prinz RA; Association for surgical education. A CRASH course in procedural skills improves medical students' self-assessment of proficiency, confidence, and anxiety. Am J Surg 2007;193:771-3.
Ferguson E, Semper H, Yates J, Fitzgerald JE, Skatova A, James D. The 'dark side' and 'bright side' of personality: When too much conscientiousness and too little anxiety are detrimental with respect to the acquisition of medical knowledge and skill. PLoS One 2014;9:e88606.
Chapell MS, Blanding ZB, Silverstein ME, Takahashi M, Newman B, Gubi A, et al
. Test anxiety and academic performance in undergraduate and graduate students. J Educ Psychol 2005;97:268-74.
Zick A, Granieri M, Makoul G. First-year medical students' assessment of their own communication skills: A video-based, open-ended approach. Patient Educ Couns 2007;68:161-6.
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5]