ORIGINAL RESEARCH ARTICLE
Year : 2019 | Volume
: 32 | Issue : 1 | Page : 11--17
Seeking a stable foundation to build on: 1st-Year residents' views of high-value care teaching
Rey Perez1, David Aizenberg2, Trocon Davis3, Kira L Ryskina4,
1 Department of Medicine, University of Pennsylvania, Philadelphia, PA, USA
2 Division of General Internal Medicine, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, USA
3 Department of Family Medicine and Community Health, Mixed Methods Research Laboratory, University of Pennsylvania, Philadelphia, PA, USA
4 Division of General Internal Medicine, Perelman School of Medicine at the University of Pennsylvania; Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA, USA
Kira L Ryskina
12-30 Blockley Hall, 423 Guardian Drive, Philadelphia, PA 19104
Background: United States (US) residency programs have been recently mandated to teach the concept of high-value care (HVC) defined as care that balances the benefits of interventions with their harms and costs. We know that reflective practice is a key to successful learning of HVC; however, little is known about resident perceptions of HVC learning. To better inform HVC teaching in graduate medical education, we asked 1st-year residents to reflect on their HVC learning. Methods: We conducted three focus groups (n = 36) and online forum discussion (n = 13) of 1st-year internal medicine residents. A constructivist grounded theory approach was used to assess transcripts for recurrent themes to identify the perspectives of residents shared about HVC learning. Results: Residents perceived their learning of HVC as limited by cultural and systemic barriers that included limited time, fear of missing a diagnosis, perceived expectations of attending physicians, and poor cost transparency. While the residents reported considerable exposure to the construct of HVC, they desired a more consistent framework that could be applied in different situations. In particular, residents reported frustration with variable incentives, objectives, and definitions pertaining to HVC. Suggestions for improvement in HVC teaching outlined three main needs for: (1) a generalizable framework to systematically approach each case that could be later adapted to independent practice; (2) objective real-time data on costs, benefits, and harms of medical interventions; and (3) standardized approach to assess resident competency in HVC. Discussion: As frontline clinicians and the intended target audience for HVC education, 1st-year residents are in a unique position to provide feedback to improve HVC teaching in residency. Our findings highlight the learners' desire for a more systematic approach to HVC teaching that includes the development of a stable generalizable framework for decision-making, objective data, and standardized assessment. These findings contrast current educational interventions in HVC that aim at reducing the overuse of specific practices.
|How to cite this article:|
Perez R, Aizenberg D, Davis T, Ryskina KL. Seeking a stable foundation to build on: 1st-Year residents' views of high-value care teaching.Educ Health 2019;32:11-17
|How to cite this URL:|
Perez R, Aizenberg D, Davis T, Ryskina KL. Seeking a stable foundation to build on: 1st-Year residents' views of high-value care teaching. Educ Health [serial online] 2019 [cited 2020 May 26 ];32:11-17
Available from: http://www.educationforhealth.net/text.asp?2019/32/1/11/266180
A key objective of residency training is learning to customize recommendations for each patient. Increasingly residents are being asked to optimize their recommendations not only based on patient outcomes but also on prioritizing choices which are “high value.” In the US, rising healthcare expenditures without notable improvements in patient outcomes  heightened concerns of overtreatment and overuse of “low-value” health-care services. To help combat these inefficiencies, increasing emphasis has been placed on the teaching and practice of high-value care (HVC) defined as care that balances the benefits of medical interventions with their harms and costs. In 2012, the Accreditation Council for Graduate Medical Education incorporated cost-effective care into its training competencies for internal medicine residents., Moreover, in 2014, the Institute of Medicine Report on Graduate Medical Education called for a realignment of US graduate medical education financing to produce “a physician workforce better prepared to work in an evolving healthcare delivery system that can provide better individual care, better population health, and lower cost.”
In the past 5 years, a number of strategies have been employed to improve HVC teaching in graduate medical education. The American College of Physicians created a web-based HVC curriculum for use by residency programs. In addition, many residency programs worked to develop formal curricula in this area. In fact, although only 15% of internal medicine residency programs reported having an HVC curriculum in 2012, more than half were in the process of developing and implementing HVC curricula at that time., Despite the increased activity around HVC curriculum design over the past decade, the literature describing educational interventions in HVC teaching in graduate medical education is scant. A 2015 systematic review of HVC educational interventions found that only 6.3% out of 2650 studies reviewed included residents. The review found that, in addition to successful transmission of knowledge and the presence of an environment supportive of HVC, successful learning of HVC was achieved through reflective practice where learners were encouraged to reflect on and adapt their practice to feedback.
Understanding resident perceptions of HVC teaching and practice is necessary to inform educational interventions and to promote learning of HVC. However, few studies have described learners' perceptions of HVC teaching. A 2015 study observed that medical students perceived time constraints, ingrained practices, and defensive medicine as the major barriers to HVC and felt poorly equipped to address these barriers themselves. Studies of residents' views on HVC found that residents generally considered costs of medical interventions when making recommendations to patients although their knowledge of costs was poor , and that they strived to incorporate patients' values and concerns into clinical recommendations. However, while we know residents' attitudes toward HVC, their perceptions of HVC teaching remain underexplored.
A growing body of evidence suggests a long-lasting effect of training on physicians' knowledge, attitudes, and practice of HVC. For example, a survey of practicing US physicians across specialties observed that physicians' attitudes toward cost-consciousness in medical practice were partially explained by the care intensity of the hospital where they trained during residency. Not surprisingly, we also know that residents' practice of HVC is affected by the practice of their attending physicians and cost-conscious culture of institutions where they train., Furthermore, the influence of the practice environment during training also appears to persist in independent practice., Given the importance of HVC teaching during residency, we must understand residents' perceptions of the HVC teaching experience to inform efforts to optimize HVC learning in graduate medical education.
To fill these gaps in the literature, we conducted in-person and online focus groups with the intent of providing 1st-year internal medicine residents a forum to reflect on their practice of HVC and to understand resident experiences with HVC teaching during residency training. The discussion was broadened to explore resident perceptions of barriers and facilitators of HVC teaching and practice during training. In this report, we describe recurrent themes identified in the focus groups to inform future educational interventions for HVC teaching and to improve the practice of HVC during residency.
We used a constructivist ground theory  approach to study 1st-year internal medicine residents' experiences with learning and practicing HVC. We selected 1st-year residents because they are frontline providers of patient care on general medicine teams, typically placing orders and communicating recommendations to patients. First-year residents also have recently transitioned from medical school to residency and have thus experienced training at different institutions that may reflect more diverse perspectives. The authors brought unique experiences and backgrounds to the research and included a graduate medical student who started residency training in internal medicine during the study, an experienced clinician educator, a mixed-methods and qualitative data researcher, and a clinician–investigator and medical education researcher.
Participants and setting
Thirty-six 1st-year residents from the Internal Medicine Residency Program located on the East Coast of the US participated in 3 in-person focus groups and 13 residents participated in the online forum discussion. The focus groups, run by a trained moderator, explored the participants' personal experiences with HVC learning and practice. The online forum discussion was open during the 2-week interval that residents were on their clinic service. Only 1st-year internal medicine residents were able to post comments or respond to other participants' comments. The online forum discussion was moderated by a clinician educator and a medical education researcher. The dual objective of the focus groups and online forum discussion was to provide an opportunity for residents to reflect on their practice of HVC and to elicit feedback about teaching of HVC by soliciting information about residents' experiences with HVC teaching and practice during training. Although most comments focused on participants' experiences during residency, residents were also encouraged to provide comparative comments that reflected on their experiences with HVC teaching during undergraduate medical education.
The focus groups were audio recorded, professionally transcribed verbatim, and coded prior to analysis to preserve participants' confidentiality. The online forum discussion comments were also anonymized and combined with the focus group transcripts. The combined data were entered into NVivo10 database (QSR International, Doncaster, Australia) to facilitate analysis. Using the constant comparison method, three members of the research team read through the data independently to identify recurrent themes and notable deviations. The research team then met and iteratively reviewed first-level codes and was able to identify common themes. These themes were then organized into a codebook in an iterative process of coding, discussio n, and recoding, which was used to produce the final dataset of coded transcripts. All researchers participated in the discussion of final-coded transcripts to achieve a consensus interpretation of main findings and conclusions.
The Institutional Review Board at our university reviewed and approved this study.
Participants' comments were categorized into three broad domains that encompassed the many components of HVC learning that residents discussed: systems and processes, education, and culture. Systems and processes contained any mention of the organizational frameworks in place that guide how physicians interact with patients, influence care delivery, or affect reimbursement. Education captured discussion surrounding how HVC is taught to medical professionals and best practices when it comes to instruction on HVC. Culture refers to the multidimensional concept of norms and expectations by the participants of the US healthcare system (e.g. patients, physicians, and administrators) as well as the local institutional culture of the medical centers or hospitals where residents trained (i.e. medical school and residency hospitals). This domain aligned with previous studies of institutional culture that have identified key aspects of institutional culture within medical centers to be observed institutional values, institutional support, and inclusiveness of employees.
The comments within each of these domains consisted of barriers to HVC learning as well as suggestions for how HVC learning could be improved. The most common of these barriers included limited time, fear of missing a diagnosis, perceived expectations of attending physicians, and poor cost transparency. In addition, residents reported frustration with variable incentives, objectives, and definitions pertaining to HVC, which made learning challenging. While a variety of different suggestions to improve HVC learning across all three domains were identified, they largely expressed one of three major needs identified by the residents. These were (1) the need for a generalizable framework to systematically approach each case that could be later adapted to independent practice; (2) the need for objective real-time data on costs, benefits, and harms of medical interventions; and (3) the need for a standardized approach to assess resident competency in HVC. Examples of resident comments are summarized below by domain.
Systems and processes: Frustration related to varying incentives
Many respondents commented that the incentive structure within the health system inhibited the practice of HVC. They felt that recognition (for the trainees) and compensation (for supervising faculty) was often linked to correctly ordering a test, but that restraint (i.e. not ordering unnecessary or low-value tests) typically went unnoticed. In fact, participants lamented the fact that they occasionally felt penalized for exercising restraint:
“I mean, there are so many pressures for us to do these tests, and that's the legal ones, but also like the financial incentives are there, regardless of what you learn. You have to be very altruistic not to be swayed by that.”
Participants also expressed frustration with uncertainty and variation in costs of medical interventions based on patient and institution as well as changes in what is considered “high value” over time:
“Yeah. It's all so abstract, and it's also a moving target… we could learn one thing is cost effective, but next month (medication name), which used to be really affordable, all of a sudden is astronomically expensive.”
In addition, residents felt uncertainty regarding the trade-offs between time and costs. Depending on perspective, quicker workup may be considered more “high value” but may result in duplicative testing. For example, difficulties associated with obtaining outside medical records in a timely manner may encourage overuse:
“I have a new patient who comes from (another hospital), and they didn't bring their outside patient records. In this case you should take the time and just get their records before you do anything. But it's a lot easier to click a few buttons and get some blood work and just do it all over again.”
Another participant noted that in many instances, doctors are attempting to balance expectations of quick workup and treatment with containing costs, with some believing that a “shotgun” approach to testing might reveal a diagnosis quicker and result in a rapid discharge from the hospital. The residents generally agreed that while a sequential cost-driven approach to diagnostic workup may avoid some unnecessary tests, it would result in longer patient stays and possibly higher overall costs:
“Sometimes doing shotgun labs means that I get the diagnosis quicker and I get the patient out faster, and then (the patient only has) a two-day stay instead of a four-day stay… (sequential) lab ordering is not being supervised right now. So it's no wonder that we're fighting an uphill battle.”
Systems and processes: The need for real-time data on benefits, harms, and costs
As the conversation shifted toward improvements that could be made to these existing systems and processes, many participants spoke about improving cost transparency, making changes to the electronic medical record (EMR) to avail real-time data to inform recommendations, and consultative support services that the hospital could provide to improve resident practice of HVC. Participants offered several mechanisms to increase cost transparency and allow them to make more informed recommendations to patients. Examples of this included listing a dollar amount next to each laboratory test available and seeking pharmacist recommendations to optimize medications based on a patient's specific insurance plan.
Participants discussed possible solutions to improve EMR software to encourage HVC rather than overtesting. These included sharing patient records across institutions and integrating current overtreatment guidelines. Another suggestion related to the EMR included implementing more sophisticated diagnostic and management algorithms that account for costs relative to benefits within the EMR to provide automated decision support to the residents:
“This is where I think the EMR could actually help is if you had somebody who developed an algorithm to figure out the best way to order things. And you said here're the tests I want to order, and then the EMR could process that and say based on the cost that (these tests are) best ordered in this order.”
Finally, residents felt that they could benefit from having HVC experts available for consultation when they had questions about the most value-based course of action. They imagined a consult team of physician HVC experts with whom they could discuss their diagnosis and management strategy for challenging cases.
Culture: Uncertainty regarding norms and expectations
A common sentiment identified by residents was the idea that physicians are strongly driven by a fear of missing a diagnosis or incorrectly diagnosing a patient. This fear incentivizes ordering extra tests “just in case” as opposed to emphasizing the costs of low-yield studies:
“Yeah, because they're only going to get in trouble if they miss it, but… there's going to be nothing wrong if they order like 1,000 (test name). It'll just get tacked onto the patient's bill.”
Patients' concerns and expectations were also identified as a barrier to HVC. One participant noted that many patients request additional tests while not accurately understanding, and frequently overestimating, the benefit of such testing:
(In reference to a patient with a mild abnormality on laboratory values obtained while in hospital). “I get phone calls from her wanting to do more work up, and I think that's more harmful to her than just the cost of this (test) is the anxiety that she's going through.”
Similarly, participants brought up examples when patients overvalued the need to see a specialist that the residents felt was unnecessary or undervalued the need for routine screening and other cost-effective preventive health measures.
Notably, while the other domains contained several suggestions for improvement, the culture domain lacked any ideas that directly address these issues. This could suggest that residents perceived their ability to affect culture of an academic medical center as limited.
Education: The adoption of a standardized approach to assessing resident competency in high-value care
Participants shared a wide range of opinions about the focus of HVC instruction, how the topic should be incorporated into the curriculum, and the best setting for teaching this content. There was a general agreement among the participants that HVC concepts should be introduced as early as possible and need to be reinforced consistently throughout medical school and residency. Participants generally perceived that while they were introduced to the concept in medical school, they received little practical knowledge or were not offered the opportunity to practice HVC in medical school. They often felt that the topic was presented in a few didactic lectures and subsequently left unaddressed. Residents felt that their training so far lacked a systematic approach to assess the value of a given test that could be applied throughout their training and in future independent practice across settings and specialties:
“It has to be experiential. One, why am I ordering this test. Two, what's the likelihood that it's going to be effective or necessary. And then three, how is it going to change my management. Four, are there risks to the patient and what am I doing in this context. And if we can really start thinking about that in a more structured way, and teaching that in a more structured way, I think we would be more prepared to talk about values and outcome-based reimbursements.”
In addition, they wished for consistent opportunities to practice HVC during training in a more controlled setting and desired a standardized approach to the assessment of resident competency in this area of practice. In one comment, a resident suggested the use of standardized patients. A standardized patient is a person (typically an actor) trained to portray a patient with a specific condition in a standardized manner for the purpose of assessment or evaluation of medical trainees:
“If you had a standardized patient come in as one of your patients for upper respiratory tract infection symptoms. And then you work it up somehow and later that afternoon your attending (tells you) this is how much money you cost this patient or the system and you could have evaluated (differently) and spent this much money (instead).”
Residents expressed views that attending physicians should play a larger role in leading their teams to more value-based decisions. They felt that more consistent expectations by the attending physicians would encourage the residents to practice HVC in a more systematic manner:
“If we had all our attendings in a culture where, before we ordered an echo (cardiogram) or CT scan we ask, “Do we really need this?” It does lower the amount that you order things.”
In general, the participants expressed that attending physicians are in a unique position of authority to strongly encourage high-value practices and facilitate practice-based learning of HVC. These comments also highlight a recurrent theme expressed by residents who desired a universal approach to HVC teaching that allows learners to apply “the right method” to patient care. While specific interventions may reduce the use of individual medical tests or treatments, a generalized approach learned in residency could be applied in future independent practice and adapted to different settings.
In this study, we explored 1st-year internal medicine residents' reflections on the practice and teaching of HVC. Reflective practice is instrumental to successful learning and practice improvement during training, and our study adds several insights about resident experiences that may inform future educational interventions in this area. These insights coalesced into three major “needs.”First, residents expressed a need for more constant and generalizable framework for HVC to enable them to systematically approach each case. Such a framework would alleviate concerns regarding unfair allocation of resources and misaligned incentives that cause considerable frustration to physicians., For example, the 2013 Millennium Conference consensus statement (a joint effort of the American College of Physicians and the Association of American Medical Colleges) on teaching HVC recommended that educators use a patient-centered approach which adopts a patient-centric view of value in health care. Second, residents' comments highlighted the need for objective real-time data on costs, benefits, and harms of medical interventions. While efforts to broaden the evidence base necessary to make HVC recommendations are ongoing, existing evidence should be more broadly disseminated via the EMR. Third, residents' comments support the need for a more standardized approach to assess resident competency in HVC and for more consistent expectations from attending physicians regarding HVC practice.
Barriers to HVC learning identified by residents centered around the aspects of the US healthcare system and “the culture of medicine,” such as limited time, the fear of “missing” a diagnosis, financial disincentives, and poor cost transparency. These topics echo the views of barriers to cost-conscious care previously reported by practicing physicians,, medical students, and residents , in earlier studies. These persistent obstacles are concerning given the resources and effort invested in identifying and addressing the barriers to HVC throughout the healthcare system over the past decade. In addition to highlighting the common persistent barriers to HVC, the participants identified unique challenges perceived by 1st-year residents and offered novel solutions that could be adopted to inform the design of interventions to support learning of HVC during early residency training. However, most resident solutions primarily focused on educational interventions, suggesting that residents may feel disempowered to tackle systemic or cultural barriers they observed. This may reflect a lack of agency that residents perceive regarding influencing the culture and changing processes of care in the medical centers where they practice., While studies have demonstrated the efficacy of a number of didactic interventions on cost-conscious behavior,,, durability of their effects is questionable in the face of cultural and systemic barriers to HVC practice identified by the residents. Empowering residents and other frontline providers to tackle cultural and systemic barriers to HVC practice at their institutions may have broad and long-lasting effects not only on the residents' learning and professional development but may also enable more rapid culture change within the institution.
Returning to 2015 systematic review of 27 HVC educational interventions, the three factors most correlated with successful HVC interventions were knowledge transmission, self-reflection, and a supportive environment. The latter two factors are well aligned with the comments under the “culture” and “systems and processes” domains identified in our study. By creating an intervention that prompts self-reflective practice, a physician may be encouraged to challenge a cultural norm. The team-based support structure some of the residents described in their comments may create a more supportive environment for the practice and learning of HVC while simultaneously saving time and addressing patient preference for an expert opinion. As the frontline providers who order the majority of diagnostic tests and other interventions in US hospitals, 1st-year residents' views are uniquely positioned to inform the design of educational interventions on HVC. Keys to the success of these interventions are efforts to go beyond knowledge transmission and actively create a supportive environment that encourages a systematic approach to learning and practice of HVC that could be applied across different cases and settings. These ideas were well aligned with the three main themes that recurred in the participant's comments.
Limitations of this study include its single site and focus group format. In a focus group setting, a single participant may dominate the discussion, drowning out the minority views. Furthermore, time limitations may have prevented certain ideas from being fully discussed. The fact that the focus groups took place in the context of an HVC course, may have prompted participants to recall ideas from the readings or lectures that were part of the course. Nevertheless, the focus groups were moderated by a trained moderator who followed a predetermined script of open-ended questions aiming to sample the range of views within each group. Furthermore, the focus group transcripts were supplemented by online comments from the participants which had more generous time limits. Lastly, participation in the online session was not mandatory, and we may not have captured opinions of those unable to participate.
As medical innovations and other forces continue to drive up healthcare costs, the need to train physicians skilled at providing high-value, cost-effective care has become essential. The US residency programs are adapting to this changing landscape and looking for more effective ways to teach HVC. Understanding resident experiences with HVC teaching and practice during training can provide important insights to inform educational interventions in this area. This study suggests that increasing focus should be placed on teaching HVC skills in a way that not only transmits knowledge but also empowers residents to tackle cultural and systemic barriers to HVC practice. Interventions that facilitate the development of a generalizable framework of HVC decision-making; encourage dissemination of relevant data on benefits, harms, and costs of medical interventions; and adopt standardized approaches to assessing HVC skills are more likely to achieve success in HVC teaching.
Financial support and sponsorship
Dr. Ryskina's work on this study was supported by the Ruth L. Kirschstein National Research Service Award (T32-HP10026) and the NIA Career Development Award (K08AG052572).
Conflicts of interest
There are no conflicts of interest.
|1||Martin AB, Hartman M, Washington B, Catlin A; National Health Expenditure Accounts Team. National health spending: Faster growth in 2015 as coverage expands and utilization increases. Health Aff (Millwood) 2017;36:166-76.|
|2||Berwick DM, Hackbarth AD. Eliminating waste in US health care. JAMA 2012;307:1513-6.|
|3||Owens DK, Qaseem A, Chou R, Shekelle P, Clinical Guidelines Committee of the American College of Physicians. High-value, cost-conscious health care: Concepts for clinicians to evaluate the benefits, harms, and costs of medical interventions. Ann Intern Med 2011;154:174-80.|
|4||Andolsek K, Padmore J, Hauer K, Holmboe ES. Clinical Competency Committees. Guidebook for Programs. Available from: http://www.acgme.org/acgmeweb/Portals/0/ACGMEClinicalCompetencyCommitteeGuidebook.pdf. [Last accessed on 2017 Jul 17].|
|5||Caverzagie KJ, Iobst WF, Aagaard EM, Hood S, Chick DA, Kane GC, et al. The internal medicine reporting milestones and the next accreditation system. Ann Intern Med 2013;158:557-9.|
|6||Eden J, Berwick D, Wilensky G, editors. Committee on the Governance and Financing of Graduate Medical Education; Board on Health Care Services; Institute of Medicine. Graduate Medical Education That Meets the Nation's Health Needs. Washington, DC: National Academies Press (US); 30 September, 2014. Available from: http://www.nationalacademies.org/hmd/Reports/2014/Graduate-Medical-Education-That-Meets-the-Nations-Health-Needs.aspx. Accessed on March 8, 2019.|
|7||Smith CD; Alliance for Academic Internal Medicine – American College of Physicians High Value, Cost-Conscious Care Curriculum Development Committee. Teaching high-value, cost-conscious care to residents: The alliance for academic internal medicine – American College of Physicians curriculum. Ann Intern Med 2012;157:284-6.|
|8||Patel MS, Reed DA, Smith C, Arora VM. Role-modeling cost-conscious care – A national evaluation of perceptions of faculty at teaching hospitals in the United States. J Gen Intern Med 2015;30:1294-8.|
|9||Patel MS, Reed DA, Loertscher L, McDonald FS, Arora VM. Teaching residents to provide cost-conscious care: A national survey of residency program directors. JAMA Intern Med 2014;174:470-2.|
|10||Stammen LA, Stalmeijer RE, Paternotte E, Oudkerk Pool A, Driessen EW, Scheele F, et al. Training physicians to provide high-value, cost-conscious care: A systematic review. JAMA 2015;314:2384-400.|
|11||Tartaglia K, Kman N, Ledford C. Medical student perceptions of cost-conscious care in an internal medicine clerkship: A thematic analysis. J Gen Intern Med 2015;30:1491-6.|
|12||Long T, Silvestri MT, Dashevsky M, Halim A, Fogerty RL. Exit survey of senior residents: Cost conscious but uninformed. J Grad Med Educ 2016;8:248-51.|
|13||Post J, Reed D, Halvorsen AJ, Huddleston J, McDonald F. Teaching high-value, cost-conscious care: Improving residents' knowledge and attitudes. Am J Med 2013;126:838-42.|
|14||Ryskina KL, Smith CD, Weissman A, Post J, Dine CJ, Bollmann K, et al. U.S. Internal medicine residents' knowledge and practice of high-value care: A national survey. Acad Med 2015;90:1373-9.|
|15||Ryskina KL, Halpern SD, Minyanou NS, Goold SD, Tilburt JC. The role of training environment care intensity in US physician cost consciousness. Mayo Clin Proc 2015;90:313-20.|
|16||Ryskina KL, Dine CJ, Kim EJ, Bishop TF, Epstein AJ. Effect of attending practice style on generic medication prescribing by residents in the clinic setting: An observational study. J Gen Intern Med 2015;30:1286-93.|
|17||Ryskina KL, Korenstein D, Weissman A, Masters P, Alguire P, Smith CD, et al. Development of a high-value care subscore on the internal medicine in-training examination. Ann Intern Med 2014;161:733-9.|
|18||Chen C, Petterson S, Phillips R, Bazemore A, Mullan F. Spending patterns in region of residency training and subsequent expenditures for care provided by practicing physicians for medicare beneficiaries. JAMA 2014;312:2385-93.|
|19||Sirovich BE, Lipner RS, Johnston M, Holmboe ES. The association between residency training and internists' ability to practice conservatively. JAMA Intern Med 2014;174:1640-8.|
|20||Charmaz K. Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. London: SAGE; 2006.|
|21||Pololi LH, Krupat E, Civian JT, Ash AS, Brennan RT. Why are a quarter of faculty considering leaving academic medicine? A study of their perceptions of institutional culture and intentions to leave at 26 representative U.S. Medical schools. Acad Med 2012;87:859-69.|
|22||Tilburt JC, Wynia MK, Sheeler RD, Thorsteinsdottir B, James KM, Egginton JS, et al. Views of US physicians about controlling health care costs. JAMA 2013;310:380-8.|
|23||Huang GC, Tibbles CD, Newman LR, Schwartzstein RM. Consensus of the millennium conference on teaching high value care. Teach Learn Med 2016;28:97-104.|
|24||Institute of Medicine. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: National Academies Press; 2012.|
|25||Sedrak MS, Patel MS, Ziemba JB, Murray D, Kim EJ, Dine CJ, et al. Residents' self-report on why they order perceived unnecessary inpatient laboratory tests. J Hosp Med 2016;11:869-72.|
|26||Patel N, Brennan PJ, Metlay J, Bellini L, Shannon RP, Myers JS, et al. Building the pipeline: The creation of a residency training pathway for future physician leaders in health care quality. Acad Med 2015;90:185-90.|
|27||Thakkar RN, Kim D, Knight AM, Riedel S, Vaidya D, Wright SM, et al. Impact of an educational intervention on the frequency of daily blood test orders for hospitalized patients. Am J Clin Pathol 2015;143:393-7.|
|28||Cadogan SL, Browne JP, Bradley CP, Cahill MR. The effectiveness of interventions to improve laboratory requesting patterns among primary care physicians: A systematic review. Implement Sci 2015;10:167.|
|29||Iams WW, Heck J, Kapp M, Leverenz D, Vella M, Szentirmai E, et al. A multidisciplinary housestaff-led initiative to safely reduce daily laboratory testing. Acad Med 2016:91:813-20.|
|30||Braido F, Comaschi M, Valle I, Delgado L, Coccini A, Guerreras P. Knowledge and health care resource allocation: CME/CPD course guidelines-based efficacy. Eur Ann Allergy Clin Immunol 2012;44:193-9.|