|ORIGINAL RESEARCH PAPER
|Year : 2012 | Volume
| Issue : 1 | Page : 24-32
Strengthening Responses to the HIV/AIDS Pandemic: An Internal Evaluation of the Caribbean Health Leadership Institute
K Umble Ph.D., M.P.H. 1, B Bain2, M Ruddock-Small2, E Mahanna1, EL Baker1
1 North Carolina Institute for Public Health, UNC Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
2 Office of the Vice-Chancellor, The University of the West Indies, Kingston, Jamaica
|Date of Submission||02-Apr-2011|
|Date of Revision||21-Apr-2012|
|Date of Acceptance||27-Apr-2012|
|Date of Web Publication||30-Jul-2012|
North Carolina Institute for Public Health, UNC Gillings School of Global Public Health, Campus Box - 8165, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina - 27599
Source of Support: The Caribbean Health Leadership Institute (CHLI) was established through a five-year Cooperative Agreement No. 1-U2GPS00930 between the US Centers for Disease Control and Prevention and the University of the West Indies with funding from the US President's Emergency Fund for AIDS Relief (PEPFAR). Complementary funding has been provided by successive regional grants to the Caribbean from the Global Fund to fight AIDS, Tuberculosis and Malaria (GFATM Rounds 3 and 9). The Pan-American Health Organisation provided support to some scholars, Conflict of Interest: None
Background: Leadership development is a strategy for improving national responses to HIV/AIDS. The University of the West Indies offers the Caribbean Health Leadership Institute (CHLI) to enhance leaders' effectiveness and responses to HIV/AIDS through a cooperative agreement with the Centers for Disease Control and Prevention. CHLI enrolls leaders in annual cohorts numbering 20-40. Objectives: To examine how CHLI influenced graduates' self-understanding, skills, approaches, vision, commitments, courage, confidence, networks, and contributions to program, organizational, policy, and systems improvements. Methods: Web-based surveys and interviews of graduates. Results: CHLI increased graduates' self-understanding and skills and strengthened many graduates' vision, confidence, and commitments to improving systems. It helped graduates improve programs, policies, and systems by: motivating them and giving them ideas for changes to pursue, encouraging them to share their vision, deepening skills in areas such as systems thinking, policy advocacy, and communication, strengthening their inclusion of partners and team members, and influencing how they interacted with others. Training both HIV-focused and general health leaders can help both kinds of leaders foster improvements in HIV services and policies. Discussion: Learners greatly valued self-assessments, highly interactive sessions, and the opportunity to build a network of professional colleagues. Projects provided opportunities to address substantive issues and immediately apply learning to work. Leadership development evaluations in the United States have also emphasized the complementary benefits of assessment and feedback, skills development, and network development. Global leadership programs should find ways to combine these components in both traditional face-to-face and distance-learning contexts.
Keywords: Action learning, Caribbean, continuing education, HIV/AIDS, leadership, leadership development, program evaluation, public administration, public health, training
|How to cite this article:|
Umble K, Bain B, Ruddock-Small M, Mahanna E, Baker E L. Strengthening Responses to the HIV/AIDS Pandemic: An Internal Evaluation of the Caribbean Health Leadership Institute. Educ Health 2012;25:24-32
|How to cite this URL:|
Umble K, Bain B, Ruddock-Small M, Mahanna E, Baker E L. Strengthening Responses to the HIV/AIDS Pandemic: An Internal Evaluation of the Caribbean Health Leadership Institute. Educ Health [serial online] 2012 [cited 2020 May 31];25:24-32. Available from: http://www.educationforhealth.net/text.asp?2012/25/1/24/99203
| Background|| |
Global leaders have identified leadership development as a key strategy for improving national responses to HIV/AIDS, but few evaluations have examined this approach. [1,2] In 2007, the University of the West Indies HIV/AIDS Response Programme (UWI HARP) launched the Caribbean Health Leadership Institute (CHLI) through a cooperative agreement with the Centers for Disease Control and Prevention. CHLI aims to "enhance the skills and effectiveness of Caribbean leaders in the HIV/AIDS arena as well as in the Caribbean health sector as a whole" (http://www.gochli.org), and ultimately to improve the health systems' capacities to respond to the HIV/AIDS epidemic. The program is a partnership with the North Carolina Institute for Public Health. This evaluation describes the program's impact on its graduates and on programs, organizations, policies, and systems.
Scholars and international bodies ,,,, recognize leadership as a vital component of organizational and system capacity and central to the "system strengthening" needed to extend progress on HIV, malaria, tuberculosis, and other initiatives. ,,,,,,,, Strong leadership can improve antiretroviral treatment coverage , and the success of HIV-related non-governmental organizations.  Public health leadership development programs in the United States have strengthened graduates' knowledge, skills, self-awareness, ,, networks, ,, and activities, , while helping graduates foster changes in programs, organizations, policies, and systems. ,, A UNAIDS program in Nigeria improved teamwork,  while a study in Zimbabwe showed how using HIV funds to strengthen an MPH curriculum produced more graduates and additional trained personnel in HIV and other leadership positions. 
| The Caribbean Health Leadership Institute|| |
Since 2008, Caribbean Health Leadership Institute (CHLI) has enrolled both senior and emerging leaders in annual cohorts numbering approximately 20-40. Many work only with HIV/AIDS, while others influence HIV work. The program aims to produce leaders with:
These objectives reflect recent scholarship on the development needs of professionals  and leaders.  The program intends that graduates and their networks  will foster lasting improvements in health systems and organizations, and in HIV/AIDS policies, programs, and services in particular [Figure 1].
- Commitment and vision for providing leadership with others in their organization, nation, and region as agents of system change;
- Vision for how HIV/AIDS programs and systems, and general health systems, should be improved;
- Improved skills, including team-building, negotiation, managing conflict, leading change, systems thinking, communicating with and coaching others, emotional intelligence, and time management; 
- Increased self-awareness: understanding their strengths, liabilities, and customary approaches, and thereby, leading others more reflectively and effectively. 
|Figure 1: Program evaluation logic model: Caribbean Health Leadership Institute|
Click here to view
Program Structure and Learning Methods
The program begins with readings and assessment tools. Direct reports, peers, and supervisors rate participants' competencies in a multirater assessment.  Participants also complete instruments which diagnose their approaches to change and teamwork. At the first 3-day retreat, participants attend seminars [Table 1] and form "action learning" teams of 4-5 members from different countries to work on applicable system improvement projects,  such as "Integrating HIV Services into Primary Health Care," aided by a mentor.
|Table 1: Caribbean Health Leadership Institute On-Site Retreat and Webinar Program Topics|
Click here to view
After the retreat, participants work on their team projects, complete readings, and participate in webinars. After 7 months, they meet for another 3 days for seminars and discussions and project reporting. Participants then complete 2 final months of project work, report results in a webinar, and attend a graduation ceremony by videoconference.
| Methods|| |
The first three cohorts included 75 participants (37% male, 63% female) from 18 countries. The largest numbers were from Barbados (12 leaders), Jamaica (11), and Trinidad and Tobago (14), with the remainder from Anguilla, Antigua, Aruba, Bahamas, Belize, Cayman Islands, Curacao, Dominica, Grenada, Guyana, Montserrat, St. Kitts and Nevis, St. Lucia, St. Vincent, and the Grenadines and Suriname. Most worked for governmental public health units (56%), public (16%), or private (1%) hospitals, community-based or non-governmental organizations (11%), universities (9%), or international organizations (7%).
The current evaluation examined how CHLI influenced graduates' self-understanding, skills, approaches, vision, commitments, courage, confidence, and networks, and their contributions to program, organizational, and systems improvements. The Institutional Review Board at the University of North Carolina at Chapel Hill reviewed and exempted this evaluation.
Two evaluators with training and experience in program evaluation, adult learning, and public health conducted the evaluation.
1-year follow-up survey with cohorts 1 and 2: The evaluators sent an email with a link to a 40-question on-line survey to all Cohorts 1 and 2 graduates 1 year after graduation. Questions examined CHLI's contributions to skills, vision, networking, program organization, policy, and systems changes that graduates had fostered. The response rates were 68% (13/19) for Cohort 1 and 72% (18/25) for Cohort 2, or 70% (31/44) overall. There was no significant difference between the combined Cohorts 1 and 2 1-year follow-up survey respondents and all graduates from cohorts 1-2 with respect to country or type of employing organization, but more graduates who were women responded to the survey than men (P < 0.05).
Open-ended questions asked graduates to explain how CHLI influenced their skills or approach to leadership challenges, and to explain how CHLI had changed their "vision for [their] role as a leader." The survey also asked if graduates could report a program or project change, HIV/TB-related and otherwise, that CHLI had helped them contribute to (Yes or No), and if so, to rate how much CHLI had helped them contribute to the change. Questions were asked about organizational changes, health policy (law) changes, and other health system changes. Respondents then described one such change and explained how CHLI contributed to it.
Graduation surveys for cohorts 2 and 3: Cohorts 2 and 3 graduates completed a 23-question survey shortly after graduation, including questions about CHLI's contribution to their leadership skills, vision, courage, and confidence. Confidence levels predict whether leaders will take on assignments,  initiate changes, , take risks, and how well they will perform under conditions of role ambiguity and constraint. , Graduates also described how CHLI had influenced their commitment to improving health systems and to provide an example. The response rate was 92% (23/25 graduates) for Cohort 2 and 68% (21/31) for Cohort 3, or 79% (44/56) overall. There was no significant difference between the combined Cohorts 2 and 3 graduation survey respondents and all graduates from cohorts 2-3 with respect to country, type of employing organization, or gender.
Interviews with a sample of cohort 1 graduates: Two evaluators interviewed 10 Cohort 1 graduates from diverse countries, professions, and organizations, including at least one member of each team. Interviews lasted 30-60 min and examined CHLI's influence on graduates and the system changes they had contributed to. Interviewees also explained more about their team project reports and responses to open-ended survey questions.
Quantitative and qualitative survey data were analyzed using Excel. An evaluator coded each open-ended survey response,  specifying how each response reflected the program's major objectives and evaluation questions, such as certain skills gained. When they were interrelated, individual respondents' answers to multiple survey questions were treated as narratives,  such as when a respondent explained how growth in understanding and vision led to certain actions and results. The evaluator identified themes in these responses and narratives  to construct answers to the study's research questions.
Each interview was transcribed. An evaluator then coded  the interview data into categories related to the evaluation questions, using the software package Atlas.ti. The evaluators then used the interview data to provide more context and detail to the survey themes and narratives. The evaluators emailed to respondents copies of the narratives in which some evaluator interpretation had been required, asking respondents to verify that their narratives had been properly interpreted.  In all cases, respondents approved the narratives with minimal or no changes.
| Results|| |
Understanding, Skills, and Approaches to Leadership
Nearly all Cohort 2 and 3 respondents reported that CHLI had a moderate (33%), large (48%), or very large influence (14%) on their overall "development as a leader." Majorities indicated that CHLI strengthened to a very large or large extent their understanding of their leadership style, strengths and liabilities, and their abilities to work effectively with other leaders [Table 2]. 24 of 43 respondents from Cohorts 2 and 3 (56%) indicated that CHLI had strengthened to a very large or large extent their ability to lead teams. Fifty-two percent of Cohort 2 and 3 graduates reported that CHLI had strengthened to a very large or large extent their ability to lead major change efforts for systems and organizations, while about one-third reported that CHLI had strengthened this ability to a moderate extent [Table 2].
|Table 2: Caribbean Health Leadership Institute's impact on graduates' self-understanding, abilities, vision, commitments, courage and confidence|
Click here to view
When asked to describe a major program impact on their leadership, over 50% of respondents explained that the program had increased their awareness of their strengths, liabilities, and leadership style, which helped them become more intentional about how they lead teams. A national HIV bureau director stated that as a result of CHLI's assessments:
I have since recognized my strengths and weaknesses and I am very conscious in the way I approach team leadership…. As a visionary I am more confident to promote change and advocate for enhanced services but the approach to doing this is less aggressive and confrontational and more inclusive and approachable.
A graduate who leads HIV training programs explained:
I have become more strategic in my thinking and speaking. There are instances in which I am highly motivated to implement a change and would [previously] proceed to outlining my position and push[ing] for change. Now I will think through very carefully how to roll out the idea in a timely fashion.
Another 30% of respondents described how seminars, readings, and team project work had strengthened their skills and approaches to leading change, including problem-solving, negotiation, stakeholder analysis, partnership and coalition building, communication, and advocacy. These respondents frequently described improving their practice of systems leadership, including identifying partners and developing shared vision. A leader in an HIV-specific regional organization stated:
I have begun approaching significant health issues as manifestations of system weaknesses and increasingly I seek to engage non-health partners in resolving them - [for example] tourism personnel in condom promotion.
Similarly, a project officer in a national HIV/AIDS ministry explained:
Although I had been working with integration of HIV in[to] the general health care setting, it had been in a disorganized manner. The drive was to get it done rather than organize the implementation so as to have a standardized approach that can be replicated nationally. Working with my [project] team members has brought organization and a focused approach to the entire activity. The team approach has expanded the vision to be all inclusive of all relevant stakeholders.
Vision for Leadership
CHLI also was reported as a positive influence on graduates' leadership vision, commitments and confidence. Twenty-seven of 41 Cohort 2 and 3 graduates (65%) reported that CHLI had strengthened to a very large or large extent their vision for their role as a leader. Thirty-five of 44 respondents from Cohorts 2 and 3 (79%) reported that CHLI had strengthened to a very large or large extent their commitment to "continuously growing" in their leadership abilities [Table 2]. Thirty-three of 44 respondents (79%) from Cohorts 2 and 3 perceived that CHLI had strengthened to a very large or large extent their courage to step into leadership roles, while 32 of 44 (73%) said that CHLI had similarly strengthened their confidence in their leadership abilities [Table 2]. Twenty-six of 39 respondents from Cohorts 2 and 3 (66%) said that CHLI had strengthened to a very large or large extent their commitment to improving health systems related to HIV/TB [Table 2]. Twenty-nine of 42 (69%) from Cohorts 2 and 3 reported that CHLI had similarly strengthened their commitment to improving health systems as a general goal of their leadership work. Some stated that they had already been committed before entering CHLI.
In related comments, 15% made statements which indicated that CHLI had strengthened their sense of identity as a leader, which they associated closely with greater vision, commitment or confidence to take initiative in their roles. For example, a county medical officer stated:
0 [CHLI has influenced my vision for my role as a leader by influencing me] to not wait on others … to get things done or moving forward …. Additionally my personal vision has changed from just being a hard working civil servant to someone with leadership aspirations…. I now possess some appreciation of systems thinking and am embarking on improvement of our HIV thrust in my health region.
A national HIV/AIDS program officer described gaining from CHLI a vision of himself as "an invaluable member" of his nation's policy-makers, which he linked to both seeking and being sought for opportunities to lead others:
I now see myself as an invaluable member of the decision making process of national health policy. I have been able to seek new challenges as well as have the opportunities made available for the effective utilization of my skills as a leader. I also see myself as having an influence on the future development of Caribbean leaders not only in [my country] but the wider community.
As for general confidence, one graduate described how CHLI had given her confidence in how to improve her Ministry of Health's community services branch:
[Before CHLI] I felt overwhelmed and I was beginning to doubt that I could make a difference in the trajectory of the organisation. With the various tools that I was exposed to, my self-mastery has improved and I better understand what it takes to get results.
A national HIV/AIDS training program coordinator explained having "a greater confidence in myself and hence feel a greater ability to be a role model and leader" and noted that CHLI's [multi-rater] assessment had "shown me that I have the capacity to deliver way beyond my present scope of work."
About 19% of respondents explained that CHLI had given them a vision for leading teams in a more "inclusive" or participatory manner by identifying stakeholders and involving them in developing vision and goals. For some, this result was linked to CHLI's emphasis on involving the stakeholders who shape the current system. An epidemiologist and planner explained:
My vision as a leader has changed to … be more inclusive in decision making and consensus building. Previously if getting to consensus was proving difficult I would simply deal with the situation on my own; now I put the extra effort needed to reach consensus on the way forward…. I now feel the need to help others understand that where problems exist in service delivery or acceptance of programs that we need to examine the entire system….
A medical officer explained how he had enacted his new vision for inclusive leadership in the learning project by involving both the staff and target population in improving HIV service quality and access, explaining that "involvement" increases a "sense of ownership" and leads to more accessible and acceptable services.
The majority of graduates reported increases in linkages with other leaders with whom they could share ideas and support [Table 2]. Fifty-six percent (14/25) of Cohort 1 and 2 graduates had contacted a fellow graduate or others met through CHLI for input on a leadership matter, while 44% (11/25) had collaborated with another graduate on a leadership project within 1 year after graduation. A small group of graduates is developing an alumni association, but their work is being hampered by lack of time and technical communication problems.
Program and Organizational Improvements
The majority of Cohort 1 and 2 graduates reported program and organizational changes that CHLI had helped them contribute to (both HIV/TB and non-HIV/TB-specific). Two respondents described program changes. One physician, who heads a hospital infectious disease division, explained that since CHLI, with her clinical teams, she had "stressed teamwork, the involvement of relevant partners, peers, and decentralization from a centrally administered vertical program…. [Since] [i]t takes a team with a common vision to work effectively together." Two others described how CHLI had influenced their role in developing new organizations or centers. One started a university-based pharmacy institute to offer continuing education (including on HIV-related topics), conduct research and advocate for policies. She explained:
Attending CHLI sessions seeded the idea of forming an Institute to meet the needs of my profession in this way and provided the impetus and motivation to do it. [This Institute is] fundamentally important for the research base, development and future direction of pharmacy in the Caribbean.
Another respondent, an HIV leader with an international organization, described how CHLI spurred his effort to develop a new training center:
As portrayed by CHLI, the role of leadership is to share one's vision with others and I took the bold step to "sell" the idea of an [regional] HIV-related Training Center to others and it is selling well.
Others described improvements to existing organizations. A physician, who had recently been appointed Senior Medical Officer, gave two examples of how CHLI changed his leadership and hospital. First, as a result of learning proactive and personalized recruitment approaches in CHLI project work, he had recruited "at least ten" senior medical personnel and other junior staff to his hospital. He also firmly advocated keeping the hospital's poorly attended HIV testing and treatment clinics rather than closing them, as his organization had been discussing, and improved them by deploying more of his newly available staff and by reducing patient wait times. As a result, he reported "The patient flow has been excellent."
Another informant explained that after attending CHLI, she "reorganize[ed] the structure of the [her country's national HIV/AIDS] organization to better utilize the staff's skills and energies, in addition to providing incentives and motivation to staff…. The CHLI course … helped me to see the 'bigger picture' and gave me the ability to work at both the 'macro' and 'micro' levels in the organization. The change was important because poor organizational structure was affecting the quality and quantity of outputs." By the time of the interview, this graduate had moved to a new job leading her country's national press office, where she was taking every opportunity to publicize the country's HIV/AIDS work:
[CHLI] got me to see outside of the box, see the wider picture, to start seeing the different links and interconnections… [and] to see my work in a similar way whereby I'm not only looking at what I "have to do" … but actually trying to maximize the impact this organization can have.
Policy and Systems Improvements
Ten of 25 (40%) Cohort 1 and 2 graduates reported an HIV/TB policy change to which CHLI had helped them contribute, while 10 of 24 (42%) who responded reported a non-HIV/TB-specific policy change to which CHLI had helped them contribute. Thirty-eight percent reported health system changes to which CHLI had helped them contribute (both HIV/TB and non-HIV/TB-specific).
An HIV/AIDS Coordinator was working to prepare a National HIV Strategic Plan before joining CHLI. During CHLI, she set out to get this plan passed and implemented. CHLI helped in two ways. First, based on CHLI discussions, she privately presented the plan to each member of the legislature, tailoring her presentations to each member's "agenda" and interests. As a result, the Plan passed "in one sitting." Second, she selected the multisectoral implementation team by assessing who would work well together.
The Sexual and Reproductive Health (SRH) program director in another country explained that due to CHLI's emphasis on visionary leadership, he was "re-inspired" to "be a strategic visionary in … health leadership and management in [his country]." In response, he led efforts to integrate his country's separate STI, HIV, and SRH units:
I felt it was incumbent upon me to do everything within my power … to see how we could at least get some measure of … strategic movement for that integration…. Initial meetings were very favourable and we held a national joint meeting…. This was a first for me approaching joint leadership for agenda setting as well as program implementation to ensure greater synergy among departments.
He also formed an intersectoral work group to develop a national youth health policy:
It was imperative that I utilize the skills that were enhanced during the training to begin multi-sector collaboration, a skill that I had not been very confident to lead previously. The fact that I was able to view myself and [my] leadership style, understanding strengths and weaknesses, allowed me to take the challenge and achieve positive successes.
Another graduate, who is a National AIDS Program Coordinator, explained that CHLI had given her "extra drive" and "impetus" to advocate integrating HIV and STI services in her country, which she believed would increase the sustainability of services. CHLI had helped her pursue this goal with a more effective style that respected the viewpoints of other leaders, rather than reacting angrily to other points of view.
| Discussion|| |
Graduates reported improved knowledge, skills, and practices related to CHLI's major learning goals, including strengthening their self-understanding and reflective leadership, skills, vision and commitment to improving systems, and growing in leadership and confidence. CHLI also helped graduates improve programs, organizations, policies, and systems by motivating them and giving them ideas for changes to pursue, encouraging them to share their vision, deepening skills in areas such as systems thinking, policy advocacy, and communication, strengthening their inclusion of partners and team members, and influencing how they interacted with others. Prior evaluations of leadership programs in other countries have shown similar results. ,,,,
These learning outcomes reflect comprehensive models of leadership  and professional competencies,  including cognitive (e.g., knowledge of trends in integrating HIV and primary care services), functional (e.g., negotiating and communicating), personal or behavioral (e.g., being self-aware and skillfully interacting with teams), and values/ethical competencies (e.g., becoming visionary leaders for system change). Many graduates who worked in HIV/AIDS units explained how they had used their gains to improve them. Others, with wider responsibilities, made changes that benefitted HIV services, such as the hospital director who improved HIV clinics, the graduate who started a pharmacy institute, the communications director who publicized HIV initiatives, and the director who integrated STI, HIV, and SRH units. These findings reinforce earlier reports on how strengthening broader health systems and leadership can improve HIV services. ,,,,,,,,
Learners greatly valued the multirater and leadership style assessments, the highly interactive seminars, and opportunities for discussion at the on-site retreats and the opportunity to build a network of professional colleagues. The action learning projects provided opportunities to address substantive issues of interest to many learners, and to immediately apply learning to work. Leadership development evaluations in the United States have also emphasized the complementary benefits of assessment and feedback, skills development and network development. Global leadership programs should find ways to combine these components in both traditional face-to-face and distance learning contexts.
One limitation is the lack of a comparison group, although questions focused on constructs that CHLI was intended to influence, and asked graduates to explain results "in some detail" (to avoid unverifiable generalities) and to "explain how CHLI contributed." The analysis looked for links to CHLI content and learning methods.  Another limitation is reliance on self-reports. Yet many interviewees described potentially verifiable changes in their activities, and most indicated they were willing to verify and expand on their survey responses by giving their names. Future evaluations should include a pre-post design to more readily document results. More broadly, the authors sought outcomes to which CHLI "contributed," rather than outcomes solely attributable to CHLI. CHLI is only one "force" among many which influence leaders and systems. 
Development programs can strengthen leaders and help them to function more effectively. Training both HIV-focused and general health leaders can help both kinds of leaders foster improvements in HIV services and policies. The CHLI staff will continue to develop leaders and track outcomes over several years to examine the program's long-term influence.
| Acknowledgements|| |
The evaluators thank the CHLI graduates from Cohorts 1-3 for taking their valuable time to complete the surveys and participate in interviews, and the Cohort 1 graduate supervisors who completed an evaluation survey. They are grateful to Dr. Stephen Orton of the North Carolina Institute for Public Health for his support; Mrs. Janey Parris, CDC Consultant, for providing guidance on the direction of the study; and the Pan-American Health Organisation, for providing partial support to CHLI participants from several countries.
| References|| |
|1.||Szekeres G. The next 5 years of global HIV/AIDS policy: Critical gaps and strategies for effective responses. AIDS. 2008; 22(suppl.2):S9-17. |
|2.||Szekeres G, Coates TJ, Ehrhardt AA. Leadership development and AIDS. AIDS. 2008; 22(suppl.2):S19-S26. |
|3.||World Health Organization. Key components of a well functioning health system. Geneva: World Health Organization. Retrieved 2011, from: http://www.who.int/healthsystems/EN_HSSkeycomponents.pdf. |
|4.||Mackay R, Horton D, Dupleich L, Anderson A. Evaluating organizational capacity development. The Canadian Journal of Program Evaluation. 2002; 17(1):121-150. |
|5.||LaFond AK, Brown L, Macintyre K. Mapping capacity in the health sector: A conceptual framework. International Journal of Health Planning Management. 2002; 17(1):3-22. |
|6.||World Health Organization. Everybody business: Strengthening health systems to improve health outcomes: WHO's framework for action. Geneva: World Health Organization; 2007. |
|7.||World Health Organization. Maximizing positive synergies between health systems and Global Health Initiatives. Geneva: World Health Organization; 2008. |
|8.||World Health Organization Maximizing Positive Synergies Collaborative Group, Samb B, Evans T, Dybul M, Atun R, Moatti JP, Nishtar S, Wright A, Celletti F, Hsu J, Kim JY, Brugha R, Russell A, Etienne C. An assessment of interactions between global health initiatives and country health systems. Lancet. 2009; 373(9681):2137-2169. |
|9.||Horton R. Venice statement: Global health initiatives and health systems. Lancet. 2009; 374(9683):10-12. |
|10.||United Nations Program on HIV/AIDS. Joint action for results: UNAIDS outcome framework, 2009-2011. Geneva: World Health Organization; 2010. |
|11.||Chen L, Evans T, Anand S, Boufford JI, Brown H, Chowdhury M, Cueto M, Dare L, Dussault G, Elzinga G, Fee E, Habte D, Hanvoravongchai P, Jacobs M, Kurowski C, Michael S, Pablos-Mendez A, Sewankambo N, Solimano G, Stilwell B, de Waal A, Wibulpolprasert S. Human resources for health: Overcoming the crisis. Lancet. 2004; 364(9449):1984-1990. |
|12.||Hanson C, Van Damme W, Van Dormael M, Van der Roost D, Songane F. Three global health reports: Towards a growing consensus? Tropical Medicine and International Health. 2009; 14(7):710-712. |
|13.||Schneider H, Blaauw D, Gilson L, Chabikuli N, Goudge J. Health systems and access to antiretroviral drugs for HIV in Southern Africa: Service delivery and human resources challenges. Reproductive Health Matters. 2006; 14(27):12-23. |
|14.||Dybul M. Lessons learned from PEPFAR. J Acquired Immune Deficiency Syndrome. 2009; 52 Suppl 1:S12-S13. |
|15.||Schneider H, Coetzee D, Van Rensburg D, Gilson L. Differences in antiretroviral scale up in three South African provinces: The role of implementation management. BMC Health Services Research. 2010; 10 Suppl 1:S4. |
|16.||Kudale A, Solomon S, Rangan S, Kielmann K. Health systems' responses to the roll-out of antiretroviral therapy (ART) in India: A comparison of two HIV high-prevalence settings. AIDS Care. 2010; 22(Suppl 1):85-92. |
|17.||Bechange S. Determinants of project success among HIV/AIDS nongovernmental organizations (NGOs) in Rakai, Uganda. International Journal of Health Planning Management. 2010; 25:215-230. |
|18.||Saleh SS, Williams D, Balougan M. Evaluating the effectiveness of public health leadership training: The NEPHLI Experience. American Journal of Public Health. 2004; 94(7):1245-1249. |
|19.||Umble K, Steffen D, Porter J, Miller D, Hummer-McLaughlin K, Lowman A, et al. The National Public Health Leadership Institute: Evaluation of a team-based approach to developing collaborative public health leaders. American Journal of Public Health. 2005; 95(4):641-644.th |
|20.|| Umble K, Baker EL, Woltring C. An evaluation of the National Public Health Leadership Institute - 1991-2006: part I. Developing individual leaders. Journal of Public Health Management Practice. 2011; 17(3):202-213. |
|21.||Umble K, Baker EL, Diehl SJ, Haws S, Steffen D, Frederick S, Woltring C. An evaluation of the National Public Health Leadership Institute - 1991-2006: part II. Strengthening public health leadership networks, systems, and infrastructure. Journal of Public Health Management Practice. 2011; 17(3):214-224. |
|22.||Umble K, Orton S, Rosen B, Ottoson J. Evaluating the impact of the Management Academy for Public Health: Developing entrepreneurial managers and organizations. Journal of Public Health Management Practice. 2006; 12(5):436-445. |
|23.||United Nations Development Program. Leadership development program: The Nigerian experience. Retrieved 2010, from http://web.ng.undp.org/publications/hiv/LDP.pdf. |
|24.||Jones DS, Tshimanga M, Woelk G, Nsubuga P, Sunderland NL, Hader SL, St. Louis ME. Increasing leadership capacity for HIV/AIDS programs by strengthening public health epidemiology and management training in Zimbabwe. Human Resources Health. 2009; 7:69. |
|25.||Wright K, Rowitz L, Merkle A, Reid WM, Robinson G, Herzog B, Weber D, Carmichael D, Balderson TR, Baker E Competency development in public health leadership. American Journal of Public Health. 2000; 90:1202-1207. |
|26.||Chappelow CT. "360-degree feedback". In: McCauley CD, Moxley RS, Van Velsor E, editors. The Center for creative leadership handbook of leadership development. San Francisco: Jossey-Bass; 1998. p. 29-65. |
|27.||Cheetham G, Chivers G. Professions, competence and informal learning. Cheltenham: Edward Elgar; 2005. |
|28.||Rowitz L. Public health leadership. 2 nd ed. Sudbury, MA: Jones and Bartlett; 2009. |
|29.||Wenger E, McDermott R, Snyder WM. Cultivating communities of practice. Boston: Harvard Business School Press; 2002. |
|30.||Marquardt MJ. Action learning in action: Transforming problems and people for world-class organizational learning. Palo Alto, CA: Davies-Black; 1999. |
|31.||Bandura A. Self-efficacy: The exercise of control. New York: W. H. Freeman; 1997. |
|32.||Paglis LL, Green SG. Leadership self-efficacy and managers' motivation for leading change. Journal of Organizational Behavior. 2002; 23:215-235. |
|33.||Hannah S, Avolio BJ, Luthans F, Harms PD. Leadership efficacy: Review and future directions. The Leadership Quarterly. 2008; 19:669-692. |
|34.||Patton M. Qualitative research and evaluation methods. 2 nd ed. Thousand Oaks: Sage Publications; 1990. |
|35.||Creswell JW. Qualitative inquiry and research design: Choosing among five approaches. 2 nd ed. Thousand Oaks: Sage Publications; 2007. |
|36.||Lincoln YS. Emerging criteria for quality in qualitative and interpretive research. Qualitative Inquiry. 1995; 1:275-289. |
|37.||Eckert A. Situational enhancement of design validity: The case of training evaluation at the World Bank Institute. American Journal of Evaluation. 2000; 21(2):185-194. |
|38.||Grove J, Kibel B, Haas T. EvaluLEAD: An open-systems perspective on evaluating leadership development. In: Hannum K, Martineau J, Reinelt C, editors. Handbook of leadership development evaluation. San Francisco: Jossey-Bass; 2007. p. 71-110. |
[Table 1], [Figure 1]