|ORIGINAL RESEARCH PAPER
|Year : 2012 | Volume
| Issue : 2 | Page : 105-110
Teaching Medical Students Neonatal Resuscitation: Knowledge Gained and Retained from a Brief Simulation-based Training Workshop
NM Lai1, CF Ngim1, PD Fullerton2
1 Department of Paediatrics, Monash University Sunway Campus, Clinical School Johor Bahru, Johor, Malaysia
2 Monash University, Clayton Campus, Wellington Road, Clayton, Victoria 3800, Australia
|Date of Submission||06-Aug-2011|
|Date of Decision||21-Feb-2012|
|Date of Acceptance||25-Apr-2012|
|Date of Web Publication||14-Nov-2012|
N M Lai
Department of Paediatrics, Monash University Sunway Campus, Clinical School Johor Bahru, JKR 1235, Bukit Azah, Johor Bahru, 80100, Johor
Source of Support: None, Conflict of Interest: None
Context: Despite being an essential clinical skill, many junior doctors feel unprepared to perform neonatal resuscitation. We introduced a neonatal resuscitation training workshop in 2009 for our final-year medical students. Objectives: We assessed the effectiveness of our workshop in improving knowledge immediately post-training and at the end of the year. Methods: We retrospectively analysed the data of our students who attended the workshops during their Paediatric posting in small groups. The workshop was adapted from the American Academy of Paediatrics (AAP) Neonatal Resuscitation Provider (NRP) programme, and included overview lectures, practical simulation, interactive video scenarios and assessments (pre- and post-tests), which comprised 21 multiple-choice questions covering evaluation, practical actions and theory. We repeated the assessment in the final week of the students' medical training ("final test"). We analysed the data using paired t-test, analysis of variance (ANOVA), linear regression and Friedman's test. Findings: All 56 students attended the workshops. Their mean scores (out of 21) were 11.7 (SD 2.5) (pre-test), 16.2 (SD 1.9) (post-test) and 13.6 (SD 2.3) (final test) (P ≤ 0.001 for all pair- wise comparisons). The workshop's timing (earlier or later in the year) had no relationship with the students' final test scores (P = 0.96). In the final test, 68.5%, 67.4% and 60.6% on average answered correctly questions on practical action, theory and evaluation, respectively (P = 0.03). Discussion and Conclusions: Our workshop produced a modest gain in student knowledge on neonatal resuscitation at the end of their medical course. The students' overall gain in knowledge was below our expectation, and evaluation appeared to be their weakest domain. Further research should evaluate strategies to enhance longer-term knowledge retention with practical performance.
Keywords: Neonatal resuscitation, training, undergraduate
|How to cite this article:|
Lai N M, Ngim C F, Fullerton P D. Teaching Medical Students Neonatal Resuscitation: Knowledge Gained and Retained from a Brief Simulation-based Training Workshop. Educ Health 2012;25:105-10
|How to cite this URL:|
Lai N M, Ngim C F, Fullerton P D. Teaching Medical Students Neonatal Resuscitation: Knowledge Gained and Retained from a Brief Simulation-based Training Workshop. Educ Health [serial online] 2012 [cited 2020 Apr 2];25:105-10. Available from: http://www.educationforhealth.net/text.asp?2012/25/2/105/103457
| Context|| |
Neonatal resuscitation is an essential skill for obstetric and neonatal care providers, especially for junior doctors who are often the first persons to attend to a newborn in need of resuscitation. The Neonatal Resuscitation Provider (NRP) training course, jointly developed by the American Academy of Paediatrics (AAP) and the American Heart Association, offers a comprehensive and systematic training programme that has been adopted in 130 countries worldwide.  Reports from different countries attributed the improvements in neonatal mortality and morbidity over the past two decades partly to the implementation of a systematic neonatal resuscitation training programme. ,,,, However, reports also showed that many involved in the care of newborn infants felt unprepared to perform neonatal resuscitation. ,,,,,
In Malaysia, the NRP training programme has been introduced in hospitals nationwide since 1996, with nearly 15000 practicing health care providers trained over eight years under this programme.  However, this programme targets mainly practicing health care providers rather than undergraduate health care students. For instance, new interns in the Paediatrics and Obstetrics undergo the standard two-day training programme at the commencement of their postings. This two-day programme covers theory as well as practical demonstration and includes written and practical assessments. Given the depth of knowledge and the complexity of skills covered within a short period, it appears unlikely that a junior doctor would retain sufficient amount of information, especially in the theoretical aspects, on their first exposure to this programme. Prior exposure to the information contained in the programme during the interns' undergraduate training might facilitate retention when such information is reinforced in the standard programme introduced during their internship. Earlier studies on formal neonatal resuscitation training programme for undergraduate medical students showed that the students had a substantial learning gain after training. ,
In 2009, we introduced an adapted version of the NRP programme for our final-year students in Monash University Sunway Campus, Clinical School Johor Bahru, Malaysia. The adapted programme, in the form of a three-hour training workshop, aimed at providing students some retainable core knowledge on neonatal resuscitation. We shortened the programme substantially from the standard two-day programme as it was impossible to fit in the two-day workshop within our curriculum. As the skill set covered was complex and required an understanding of related theories, familiarity to algorithms and hands-on practice, it was unclear whether the brief period of training that we offered would confer any lasting benefit to our students. We therefore conducted this study to evaluate the effectiveness of our training workshops in improving the students' knowledge on neonatal resuscitation, using the available data from the written assessments that were incorporated as a standard component in the NRP programme. Besides analysing the total scores, we also assessed student performances in three major domains of theory, infant evaluation and practical action to identify possible areas of deficiency and improve the delivery of our future workshops.
In this study, we set the following research questions:
- Following our neonatal resuscitation training workshop, was there a significant gain in our students' knowledge: (i) Immediately after the workshop and (ii) At the end of their medical training?
- Among the domains of theory, evaluation and practical action, which domain required more emphasis in teaching, as reflected by poor student performance in our assessments?
| Methods|| |
Study design, participants and setting
This was a retrospective study involving a cohort of 56 final-year (year five) medical students of Monash University, Sunway Campus, Jeffrey Cheah School of Medicine, Clinical School Johor Bahru, Malaysia. The school is part of the overseas campus of the Monash University in Australia. Established in 2006, it is one of the largest private universities in Malaysia. Sharing the same educational philosophy and core curriculum as Monash University in Australia, the delivery of teaching-learning activities in Monash University Sunway Campus is nonetheless tailored locally in accordance with the learning environment and cultural considerations, with rigorous efforts in place to ensure equivalence of the curriculum and assessments across campuses. For the School of Medicine in Monash Sunway, the class size ranges from 50 to 120 students per year. The students enter the university directly from high school or equivalent at the age of 18 or 19. The participants of the current study underwent their final-year training from January to December 2009.
The neonatal resuscitation workshop was an adapted version of the AAP NRP training programme. In this workshop, the two-day standard NRP programme is abbreviated to a three-hour session. The session took place once every six weeks, during the first week of the paediatric postings (in January, March, April, May, July, August, September 2009). Each posting group comprised seven to nine students. The workshop included the following components:
- Overview lectures: using the slides from the standard NRP course, we highlighted the major concepts in neonatal resuscitation, including cardiopulmonary adaptations, apneas, important equipments and an overview of the NRP algorithm focusing on the initial steps in resuscitation.
- Video demonstration of key skills such as bag and mask ventilation and cardiac compression, followed by hands-on practice on the manikins.
- Simulation exercise in which selected students worked with the tutor through the mock case scenarios. This was purely a formative exercise with no assessment.
- Interactive video scenarios, taken from the teaching videos of the NRP programme. In this exercise, a computer-generated tutor brought the participants through a series of mock case scenarios in videos. The scenarios would pause at certain intervals during which the tutor would ask the participants questions relating to the next steps in the resuscitation, with options given for each question. The participants' response would lead to different paths in the scenarios, which would reflect the appropriateness of their choices. At the end of the scenario, a summary text would appear, recounting the scenario and highlighting the students' choices along with explanations. The problems illustrated in the scenarios included an infant born through caesarean section, an infant with meconium stained liquor, an infant with heavy maternal blood loss and an infant with congenital anomaly.
- Assessments: in the form of pre- and post-tests, using a test script with 21 multiple-choice questions. The pre-test was administered before the workshop and the post-test immediately after the workshop.
Two authors (NML and CFN), both certified NRP trainers, conducted the workshops. We repeated the workshop for the entire cohort of students in the final week of their medical training in November 2009, when they underwent a recapitulative training programme (the "Back-to-base programme") in preparation for their internship. We repeated the test ("final test") prior to the repeat workshop. We used identical test script for pre-, post- and final-tests. Due to a lack of other studies on medical students that assesses long-term knowledge retention on neonatal resuscitation, we were unable to set an evidence-based target on the amount of knowledge retained at the end of the academic year. We set the following pragmatic target based on the consensus among the authors who were involved in developing and delivering the training programme: We expected our students on average to retain at least half of the knowledge they gained immediately after the workshop by the end of their academic year.
The test script
We used the standard teaching materials, including the test script, from the NRP programme. The multiple-choice assessment script was taken from the "Overview" paper, administered in the standard NRP programme before the workshop commences. In the test script, 17 out of the 21 questions covered practical aspects of neonatal resuscitation (11 covered evaluation and 6 covered practical action) and 4 covered theories (2 on physiology, 1 on device knowledge and 1 on epidemiology). The knowledge assessed in the test scripts were covered directly or indirectly during the workshop. A detailed list of questions according to their classification is shown in [Table 1]. We followed the NRP passing criterion for this paper and considered a score of 18 or above a pass. 
|Table 1: Classifi cation of each question in the neonatal resuscitation test script according to the domain and topic assessed|
Click here to view
Conduct of the study
The students received a briefing on the format of the workshop and our research prior to each session. We informed the students that their test scores would be analysed by the teachers at the end of the year to assess their learning gain and areas of deficiency and to improve the delivery of the future workshops. We also informed the students that our findings might be presented or published later as a research. We asked the students to fill in their student identification numbers in the designated slots on the answer sheets to enable a linkage of the pre-, post- and final-tests for analysis. While all students who participated in the workshop would be required to complete the tests, students who did not wish to have their scores included in the research were asked to indicate so in their answer sheet and leave their student identification numbers blank. We also clarified that the students' decision to participate in the research by having their scores included or otherwise would not in any way affect their university standing.
We linked the test scripts for pre-test, post-test and final test from each student using the student identification numbers, and these were removed from the dataset after all data were transcribed electronically. The anonymised responses were then recoded into correct and wrong answers. The recoded data were used for analysis. We compared students' overall test scores between different test periods using paired t-test. We compared student performances across different subgroups using analysis of variance (ANOVA). We compared the students' pass rates between different periods using McNemar's test. We examined the relationship between the timing of the workshop and the final test scores using linear regression with pre and post test scores as covariates. We compared student performances across different domains (i.e. evaluation, practical actions and theory) using Friedman's test. We performed all statistical analyses using PASW version 18 (Chicago, IL, USA).
This study was approved by the Monash University Human Research and Ethics Committee (Research identification number: CF10/2250-2010001286).
| Results|| |
The entire cohort of students (n = 56) participated in the workshop, sat for the tests, and agreed to have their scores included in the research. All 56 students completed the pre-test, 55 students completed the post-test and 44 students completed the final test; among them, 39 scripts could be linked to the pre-test and 40 scripts to the post-test.
[Figure 1] displays the overall student performances in all three rounds of test. Converting the 21-point scores into percentages, the student scores increased by 21.9% on average from pre-test to post-test (P < 0.001) and decreased by 12.4% from post-test to the final test (P < 0.001), with a net gain of 9.5% on average from pre-test to the final test (P = 0.001). In the pre-test, no student attained the NRP criteria for passing by scoring 18 or above. In the post-test, 17 students (30.4%) passed but in the final test, only three students passed (6.8%) (P = 0.001 comparing the post-test and the final test).
|Figure 1: Student performances in the pre-, post- and the fi nal-tests. The data points represent the mean scores and the error bars represent their corresponding two standard deviations|
Click here to view
The seven subgroups received the training in different periods, ranging from 2 to 10 months before the final test in November 2009. However, there was no significant difference in the final test scores among the subgroups (P = 0.08 for trend). Linear regression revealed that neither the period of training (number of months before the final test) nor the pre- or post-test scores was independently associated with the final test scores ( P ≥ 0.9).
[Figure 2] illustrates student performances in three major domains: theory, evaluation and practical actions across the three periods of assessment. From the performances in the final test, our students scored the lowest in the domain of evaluation compared with practical actions and theory. In the final test, 67.4% (SD 11.0) and 68.5% (SD 9.9) on average answered correctly questions on theory and practical actions respectively, while only 60.6% (SD 26.4) answered correctly the questions under evaluation (P = 0.3 comparing across the three domains). Similar trend in the scores was observed across the three domains in the post-test (P < 0.001), but not the pre-test (P = 0.07) [Figure 2].
|Figure 2: Bar chart showing the mean percentage of correct answers according to the domain assessed|
Click here to view
| Discussion|| |
This study shows that our three-hour simulation-based neonatal resuscitation workshop produced a substantial learning gain immediately after the workshop, and by the end of the year, a modest learning gain remained, regardless of when the workshop was conducted. The overall degree of learning gain was below our expectation. More than half of the knowledge acquired immediately after the workshop was lost by the end of the year, and interestingly, the group that received the training earlier in the year did not appear to lose more knowledge compared with those who received the training later in the year (nearer the final test). In general, the area of weakness for our students appeared to be in the evaluation of infants during resuscitation.
The degree of learning gain from pre- to post-tests in our study was consistent with previous studies on medical students, , although neither of these studies followed up the participants and assessed knowledge retention. The modest amount of knowledge retained by the end of the year, and the small proportions who attained the NRP criterion for passing suggest that our training workshop would best serve as an introduction to neonatal resuscitation skills rather than an opportunity to develop competence expected from the standard NRP training programme. Our findings of a knowledge loss were similar to that of a study on practicing midwives, which showed that on follow-up assessment six months after the neonatal resuscitation training, a substantial amount of the knowledge and skills gained during the initial training were lost.  Such a loss of knowledge, which occurred even for practitioners who cared for newborn infants regularly, indicates a need for repeat training soon after the initial exposure to maintain the trainee's competence in this skill. This in turn supported our rationale in conducting a repeat training session in the final week of the students' medical training.
The abbreviated training was conceived to provide an introduction to neonatal resuscitation for our final-year medical students in a hope to facilitate further knowledge retention when they undergo the standard workshop during their internship. The much-shortened training period of three hours was necessitated mainly by the limited curricular time. Our abbreviated training programme is unsuitable as a replacement of the standard two-day training programme for interns, as the overall much-shortened period of training and a lack of practical skills training, which is crucial to effective neonatal resuscitation, are inadequate to meet the demands of the internship in which there is pressure to perform neonatal resuscitation early into the postings. This is evident by our findings that students performed overall below expectations using the same passing criteria as that of the standard two- day training programme.
Questions on infant evaluation received the lowest scores compared with practical actions and theory. Possible reasons for this included the difficulties in remembering algorithms and specific clinical signs or numbers, such as the critical heart rate and the size and length of the endotracheal tube in relation to the infant's weight, and these areas deserved more emphasis in our future training workshops.
The following are the strengths of this study: First, we assessed knowledge acquisition as well as short-term knowledge retention, using a test script from a well-established training programme, with questions covering three major domains in neonatal resuscitation. The format, educational resources and the delivery of our training workshops followed that of the NRP programme, with certified NRP trainers as tutors. The vast majority of the questions in our test scripts were related to practical aspects of neonatal resuscitation (i.e. evaluation and action), and all items in the assessment were covered in the workshop. Besides, we used identical test scripts for all three rounds of tests, and this minimised the issue of non-equivalence in terms of contents across different assessments.
We also note several limitations in our study. First, we assessed only knowledge and not skills performance in the form of simulated scenarios (such as the "megacodes"). It was therefore unclear to what extent the knowledge demonstrated by the students was translated into actual competence. It has been shown that written test evaluation is a poor predictor of skills performance in resuscitation.  Secondly, we used a single test script with a limited number of questions for all three rounds of assessment. It was unclear how much of the improvement in performances in between the tests were due to an actual gain in knowledge and how much to increased familiarity with the format of the multiple-choice questions and an improved ability to guess a correct response. We used the test script differently compared with the way it is used in the standard two-day training programme, during which the test is administered only once in the beginning to screen the baseline knowledge of the participants. In contrast, we administered the test multiple times to assess the participants' learning gains. It was unclear whether the items contained in the test script were suitable in assessing learning gain. Further, the test script appeared not to have been formally validated in any published report, despite that fact that it was from a well-established programme that is widely used around the world. We were also unable to perform a meaningful sample size or power estimation because there has not been a commonly accepted definition on what constitutes an educationally important learning gain in terms of neonatal resuscitation skills. Next, the students' awareness that they were part of a research project might have influenced their learning and performance. Our small sample size and the retrospective nature of our study represented further limitations.
| Conclusions|| |
This study demonstrates that neonatal resuscitation training workshop conducted at the undergraduate level was feasible and produced a modest short-term gain in knowledge, although it was unclear whether such knowledge gain would translate into a sustained and important gain in the students' future practice. By conducting this study, we have also identified the domain that required more input in our future workshops to improve the quality of our training. Similar research should be conducted by following up medical students into their clinical practice, with outcomes including knowledge retention over a longer term and practical performance.
| Acknowledgement|| |
We are grateful to all our former students who participated in the workshops for agreeing to have their assessment results included as the data for this study.
| References|| |
|1.||Where NRP has been taught. IL: American Academy of Pediatrics, 2009. Available from: http://www.aap.org/nrp/intl/intl_where.html. [Last cited 2009 Aug 21]. |
|2.||Boo NY. Neonatal resuscitation programme in Malaysia: An eight-year experience. Singapore Med J 2009;50:152-9. |
|3.||Duran R, Aladað N, Vatansever U, Süt N, Acunaþ B. The impact of Neonatal Resuscitation Program courses on mortality and morbidity of newborn infants with perinatal asphyxia. Brain Dev 2008;30:43-6. |
|4.||O'Hare BA, Nakakeeto M, Southall DP. A pilot study to determine if nurses trained in basic neonatal resuscitation would impact the outcome of neonates delivered in Kampala, Uganda. J Trop Pediatr 2006;52:376-9. |
|5.||Patel D, Piotrowski ZH, Nelson MR, Sabich R. Effect of a statewide neonatal resuscitation training program on Apgar scores among high-risk neonates in Illinois. Pediatrics 2001;107:648-55. |
|6.||Zhu XY, Fang HQ, Zeng SP, Li YM, Lin HL, Shi SZ. The impact of the neonatal resuscitation program guidelines (NRPG) on the neonatal mortality in a hospital in Zhuhai, China. Singapore Med J 1997;38:485-7. |
|7.||Carlotti AP, Ferlin ML, Martinez FE. Do our newly graduated medical doctors have adequate knowledge about neonatal resuscitation? Sao Paulo Med J 2007;125:180-5. |
|8.||Barrie JR, Greenhalgh DL. Training in neonatal resuscitation: The views of junior paediatricians. J R Coll Physicians Lond 1993;27:151-3. |
|9.||Bismilla Z, Finan E, McNamara PJ, LeBlanc V, Jefferies A, Whyte H. Failure of pediatric and neonatal trainees to meet Canadian Neonatal Resuscitation Program standards for neonatal intubation. J Perinatol 2010;30:182-7. |
|10.||Lee HC, Chitkara R, Halamek LP, Hintz SR. A national survey of pediatric residents and delivery room training experience. J Pediatr 2010;157:158-61. |
|11.||Teale KF. SHOs: The lost tribes. Training inadequate in paediatric resuscitation. BMJ 1994;308:275. |
|12.||Foster K, Craven P, Reid S. Neonatal resuscitation educational experience of staff in New South Wales and Australian Capital Territory hospitals. J Paediatr Child Health 2006;42:16-9. |
|13.||Bhat BV, Biswal N, Bhatia BD, Nalini P. Undergraduate training in neonatal resuscitation - A modified approach. Indian J Matern Child Health 1993;4:87-8. |
|14.||Bhatia BD, Bhat BV, Dey AK, Mohan PV. Training of final year MBBS students in neonatal resuscitation. Indian Pediatr 1993;30:113-5. |
|15.||Rodgers DL, Bhanji F, McKee BR. Written evaluation is not a predictor for skills performance in an Advanced Cardiovascular Life Support course. Resuscitation 2010;81:453-6. |
|16.||Carlo WA, Wright LL, Chomba E, McClure EM, Carlo ME, Bann CM, et al. Educational impact of the neonatal resuscitation program in low-risk delivery centers in a developing country. J Pediatr 2009;154:504-8. |
[Figure 1], [Figure 2]
|This article has been cited by|
||Novel modified Peytonćs approach for knowledge retention on newborn life support training in medical students
| ||Nasenien Nourkami-Tutdibi,Anna-Barbara Hilleke,Michael Zemlin,Gudrun Wagenpfeil,Erol Tutdibi |
| ||Acta Paediatrica. 2020; |
|[Pubmed] | [DOI]|
||Maintaining the knowledge and neonatal resuscitation skills of student midwives 6 months after an educational program
| ||M. Caldelari,L. Floris,C. Marchand,M. Schuler Barazzoni |
| ||Archives de Pédiatrie. 2019; |
|[Pubmed] | [DOI]|
||A meta-analysis for comparing effective teaching in clinical education
| ||Pin-Hsiang Huang,Matthew Haywood,Anthony O’Sullivan,Boaz Shulruf |
| ||Medical Teacher. 2019; : 1 |
|[Pubmed] | [DOI]|
||Care of the mother-infant dyad: a novel approach to conducting and evaluating neonatal resuscitation simulation training in Bihar, India
| ||Brennan Vail,Hilary Spindler,Melissa C. Morgan,Susanna R. Cohen,Amelia Christmas,Pramod Sah,Malay B. Shah,Aritra Das,Dilys M. Walker |
| ||BMC Pregnancy and Childbirth. 2017; 17(1) |
|[Pubmed] | [DOI]|
||Correlations between technical skills and behavioral skills in simulated neonatal resuscitations
| ||T Sawyer,D Leonard,A Sierocka-Castaneda,D Chan,M Thompson |
| ||Journal of Perinatology. 2014; 34(10): 781 |
|[Pubmed] | [DOI]|