Education for Health

INTERVIEW
Year
: 2016  |  Volume : 29  |  Issue : 3  |  Page : 266--270

Creating a longitudinal database in medical education: Perspectives from the pioneers


Rashmi A Kusurkar, Gerda Croiset 
 VU University Medical Center School of Medical Sciences, Institute of Education and Training, Department of Research in Education; LEARN! Research Institute for Learning and Education, Faculty of Psychology and Education, VU University Amsterdam, The Netherlands

Correspondence Address:
Rashmi A Kusurkar
VU University Medical Center Amsterdam, Institute of Education and Training, PK KTC 5.002, Postbus 7057, 1007 MB Amsterdam
The Netherlands

Abstract

The Jefferson Longitudinal Study of Medical Education (JLSME) is the longest running database in medical education and covers the collection and measurement of background, learning, performance, and psychosocial variables before, during, and after medical school. Recently, our research group at VU University Medical Center School of Medical Sciences launched a longitudinal study in medical education, called the “Student Motivation and Success Study.” While setting up this study, we faced many challenges and learning about the JLSME helped us gain a fresh perspective on our work. We interviewed Drs. Joseph Gonnella and Mohammadreza Hojat, the leaders of the JLSME, and present their experiences verbatim in this article and summarize the lessons we learned as tips for others. We conclude that by establishing a longitudinal database, medical educators can test and ensure the quality of the doctors they produce, justify curricular reforms, participate in a continuing inquiry into their educational practices, and produce more generalizable research findings.



How to cite this article:
Kusurkar RA, Croiset G. Creating a longitudinal database in medical education: Perspectives from the pioneers.Educ Health 2016;29:266-270


How to cite this URL:
Kusurkar RA, Croiset G. Creating a longitudinal database in medical education: Perspectives from the pioneers. Educ Health [serial online] 2016 [cited 2021 Oct 22 ];29:266-270
Available from: https://www.educationforhealth.net/text.asp?2016/29/3/266/204214


Full Text

 Background



Our research group at VU University Medical Center School of Medical Sciences recently launched a longitudinal study in medical education, called the “Student Motivation and Success (SMS) study.” The SMS study was initiated because of the dearth of longitudinal studies on students' motivation and outcomes,[1] and to generate a nuanced insight into the model of the influence of motivation on learning and academic performance. The database includes variables reflecting personality characteristics, professional identity, empathy, engagement in the study, burnout, etc. While setting up this study, we faced a number of challenges. One of the challenges was obtaining ethical approval as there was no defined protocol for submission of applications of longitudinal studies in medical education.

During our ethical approval process, we came across the Jefferson Longitudinal Study of Medical Education (JLSME).[2] JLSME helped us take a fresh perspective on our work. We interviewed the pioneers of this study, Drs. Joseph Gonnella and Mohammadreza Hojat, separately through teleconferences to learn of the aims, methodology, and practical problems encountered in setting up longitudinal databases. We present their reported experiences verbatim in this article, as we believe that their comments will be useful for others who want to initiate longitudinal databases in medical education.

The JLSME is the longest running study in medical education and covers the collection and measurement of a wide number of student background, learning, performance, and psychosocial variables before, during, and after medical education.[3],[4] It has resulted in over 194 publications so far and contributed ten instruments to the field of medical education [Table 1].{Table 1}

The JLSME is run from the Center for Research in Medical Education and Health Care, Sidney Kimmel (formerly Jefferson) Medical College at Thomas Jefferson University and was pioneered by Dr. Joseph Gonnella in 1970. Dr. Mohammadreza Hojat has been the director of the JLSME since 1984 and has added the measurement of students' psychosocial attributes and graduates' professional activities.

 Biography of Interviewees



Dr Joseph Gonnella is Distinguished Professor of Medicine, Dean Emeritus of Jefferson Medical College, and founder and emeritus director of the Centre for Research in Medical Education and Health Care at Sidney Kimmel (formerly Jefferson) Medical College, Philadelphia, USA. In 1998, the Association of American Medical Colleges honored him with the Abraham Flexner Award for his extraordinary contributions to the medical education community. Dr. Gonnella attributes this award partly to his setting up of the JLSME. With great humility, he stresses that “Winners are teams, not individuals” [Figure 1].{Figure 1}

Dr. Mohammadreza Hojat is the person of the “Jefferson Scale of Empathy” fame. He is a Research Professor in the Department of Psychiatry and Human Behavior and the Director of the JLSME. He has pioneered ten psychometrically sound instruments to measure medical education and patient outcomes and he is solo author of the book, “Empathy in Patient Care: Antecedents, Development, Measurement, and Outcomes.”[5] An expanded and updated edition of his book was released in 2016.[6] He attributes his success to the leadership of his mentor, Dr. Gonnella. He was given the autonomy and support to follow his passion of investigation of psychosocial attributes in clinical performance without having to chase research grants [Figure 2].{Figure 2}

 Methods Employed to Generate Data



Based on our experiences while setting up the SMS study, we (RAK and GC) independently made a list of questions for the interviewees. The questions were then adapted and finalized through discussion and consensus. At the end of the interviews, the interviewees were asked in an open question for any additional comments. After drafting this paper, the interviewees confirmed the information it presents and gave us permission to publish it.

 Interview Represented in a Question and Answer Format



Why should there be longitudinal databases in medical education?

Dr. Gonnella:Education is an experiment. With any experiment one should collect data before the experiment begins, during, and after. We wanted to document whether the expectations, which we had when we introduced new curricular reforms, were being met. An educational program is like medicine, you need to make a diagnosis first, introduce the treatment, and then make sure the treatment has worked. It creates a different environment if the faculty knows that we are introducing curricular reform not because two or three people have gone to an educational conference and say “this is a good idea,” but because we have the data. In addition, we collected this data as assessment of the quality of our medical graduates. A medical school should be responsible for both the successes and the problems in the performance of their graduates within 1 year after graduation. I consider us as the conscience of our faculty. Other reasons for doing this are documenting the quality of the educational program and the need to introduce new programs and providing the faculty with the opportunity to do educational and health service research.

Dr. Hojat:It is the social responsibility of medical schools to assess their educational outcomes to ensure that the public is safe. Hence if you ask us why we are doing this, it is a social obligation. All medical schools should to this. If they do not do it, there is something wrong. Moreover, the assessment of educational outcomes cannot be based on an anecdotal report on personal observations or experiences, but on empirical data.

What are the ethical considerations in setting up a longitudinal database?

Dr. Gonnella: To me, not conducting this kind of assessment is unethical. Those who do not do it should be meant to feel unethical. Medicine is a service profession, and we should guarantee to the public that our product is safe.

Dr. Hojat: If the study is for publication, you need Institutional Review Board (IRB) approval. If a study is “on-going” like our longitudinal database, IRB has a special code for it. Once it is approved and there is no change in your instruments and data collection, you do not have to replicate the application. If we want to make any change, even a minor modification, we have to get permission. The guidelines are very serious about confidentiality of the data. We do not share individual student's data with any person outside our research team.

Informed consent is important. We have what we call entrance and exit questionnaires, in which we include a statement saying, “The completion of this questionnaire is an indication of my voluntary participation in this project.” Since we also follow the students until the end of their professional life, we need to obtain information from other organizations. Hence, we ask the students for permission to do this. Then, we ask the director of the residency program to give us feedback on behalf of the graduates in the program. Sometimes, these other institutions may need their own IRB approval.

How do you manage to get a response rate as high as 85%?

Dr. Gonnella: As the Dean of Student Affairs, I was able to win the trust of students. While in medical school, we do not need their permission to analyze data collected in the normal course of education for evaluation purposes. After graduation, we do need the cooperation of these ex-students and the directors of postgraduate medical education.

We cannot force our graduates to participate in the study. While the majority of the graduates over time has given us permission, there is still a small number that has not. That small number was more likely to come from the lowest performance quartile, and those are the ones that we should monitor the most.

Dr. Hojat: We have to keep at convincing students. However, students trust us, so we get an 85%–90% response rate. We emphasize that we have the most extensive and comprehensive longitudinal database in all medical schools that this is very valuable and that we appreciate the collaboration of our students and graduates. We ask the students on the last day of their medical school (which is very important for their future placements) to give us permission to ask their future residency program director to complete the evaluation form at the end of the 1st year of residency. About 75% give us such a written permission. When we send the assessment form to the residency program director at the end of the 1st year, we include the original copy of the signature of our students. All these measures together help us to increase the response rate.

And lastly, what would be your advice to medical educators wanting to start this type of a longitudinal database in their own institution?

Dr. Gonnella: Think about and prepare for the costs. You need a director of the program (20%–30% time), a director of the center, someone at PhD level who has expertise in statistical analysis (100% in the beginning, but less later on), a programmer and a secretary. If you think about the damage that one incompetent physician does to the world, the costs will be quickly justified.

Place people from the research group in education implementation positions. For example, the director of such a study should certainly be a member of the Curricular Committee.

Dr. Hojat: Ensure that you have the budget, qualified personnel, and the support of the dean.

Involve faculty in your work. Sometimes, our faculty or students come to us with questions. If there is a merit to the research question and hypothesis, we carry out the analysis and help the faculty and the students in developing their ideas further. We encourage them to read the literature and make a summary, and we help them in the technical part and the methodology. Sometimes, this leads to publications by the faculty and their promotion. Thus, they know that this longitudinal database can help them. Moreover, over time the JSLME has acquired a reputation and the people working in the institution in different departments know of its benefits, so they continue to cooperate with the research group.

Keep a lookout for spin-offs or unpredicted benefits. Sometimes, longitudinal data are useful in predicting at-risk students who can be offered a remedial program.

Ensure cooperation from the different departments and offices in your medical school like the admissions and registrar offices, etc.

 Conclusion



Through these interviews, we conclude that by establishing longitudinal databases, medical educators can test and help ensure the quality of the doctors they produce, justify curricular reforms, participate in a continuing inquiry into their educational practices, and produce more generalizable research findings.[7] While setting up a longitudinal study, streamlined ethical approval procedures, adequate finances, and proactive stakeholder management can be the deciders of success. We have provided a table of tips [Table 2] for medical educators interested in creating longitudinal databases. When we established our longitudinal SMS study, we intended it to be a 6-year long study and were going to collect data solely for research purposes. After learning about the intention behind setting up the JLSME and the outcomes achieved, we are instead considering setting up an ongoing longitudinal database at our institution.{Table 2}

Other longitudinal databases in medical education that we are aware of are the longitudinal study of Norwegian students and doctors,[8] the Longitudinal study of the School of Health Sciences of the University of Minho (ELECSUM)[9],[10] in Portugal, students' and residents' life in health professions study (VERAS) implemented in 22 Brazilian schools,[11] the Medical Schools Outcomes Database and Longitudinal Tracking Project (MSOD) including all medical schools in Australia,[12],[13] and longitudinal cohort studies undertaken by The UK Medical Careers Research Group (UKMCRG).[14],[15]

Acknowledgments

The authors would like to thank Dr. Gonnella and Dr. Hojat, for sparing their valuable time for sharing their views and experience on running a longitudinal database in medical education, Michele van Middelkoop, M.D., for transcribing the interviews, and Anouk Wouters, M. Sc., Cora Visser, M. Sc., Marianne Mak, M.D., and Stephanie van der Burgt, M.A., from VU University Medical Center School of Medical Sciences, for their comments on an earlier version of the paper.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

References

1Kusurkar RA, Ten Cate TJ, van Asperen M, Croiset G. Motivation as an independent and a dependent variable in medical education: A review of the literature. Med Teach 2011;33:e242-62.
2Available from: http://jdc.jefferson.edu/jlsme/. [Last accessed on 2017 Jan 10].
3Gonnella JS, Hojat M, Veloski J. AM last page. The Jefferson Longitudinal Study of Medical Education. Acad Med 2011;86:404.
4Hojat M, Gonnella JS, Veloski JJ, Erdmann JB. Jefferson Medical College Longitudinal Study: A prototype for evaluation of changes. Educ Health 1996;9:99-113.
5Hojat M. Empathy in Patient Care: Antecedents, Development, Measurement, and Outcomes. New York:© 2007 Springer Science+Business Media, LLC; 2007.
6Hojat M. Empathy in Health Professions Education and Patient Care. Switzerland:© 2016 Springer International Publishing; 2016.
7Cook DA, Andriole DA, Durning SJ, Roberts NK, Triola MM. Longitudinal research databases in medical education: Facilitating the study of educational outcomes over time and across institutions. Acad Med 2010;85:1340-6.
8Støen Grotmol K, Gude T, Moum T, Vaglum P, Tyssen R. Risk factors at medical school for later severe depression: A 15-year longitudinal, nationwide study (NORDOC). J Affect Disord 2013;146:106-11.
9Available from: https://www.med.uminho.pt/en/Medical-Degree/Pages/ELECSUM.aspx. [Last accessed on 2017 Jan 10].
10Palha JA, Almeida A, Correia-Pinto J, Costa MJ, Ferreira MA, Sousa N. Longitudinal evaluation, acceptability and long-term retention of knowledge on a horizontally integrated organic and functional systems course. Perspect Med Educ 2015;4:191-5.
11Tempski P, Santos IS, Mayer FB, Enns SC, Perotta B, Paro HB, et al. Relationship among medical student resilience, educational environment and quality of life. PLoS One 2015;10:e0131535.
12Available from: http://www.medicaldeans.org.au/projects-activities/msodproject/. [Last accessed on 2017 Jan 10].
13Wilson I, Griffin B, Lampe L, Eley D, Corrigan G, Kelly B, et al. Variation in personality traits of medical students between schools of medicine. Med Teach 2013;35:944-8.
14Available from: http://www.uhce.ox.ac.uk/ukmcrg/. [Last accessed on 2017 Jan 10].
15Smith F, Lambert TW, Goldacre MJ. Factors influencing junior doctors' choices of future specialty: Trends over time and demographics based on results from UK national surveys. J R Soc Med 2015;108:396-405.