|Year : 2014 | Volume
| Issue : 2 | Page : 127-131
Peer review for social accountability of health sciences education: A model from South Africa
Desireé Christine Michaels1, Stephen John Reid2, Claudia Susan Naidu3
1 Collaboration for Health Equity through Education and Research, University of Cape Town, Cape Town, South Africa
2 Primary Health Care and Collaboration for Health Equity through Education and Research, University of Cape Town, Cape Town, South Africa
3 University of Cape Town, Cape Town, South Africa
|Date of Web Publication||31-Oct-2014|
Dr. Desireé Christine Michaels
University of Cape Town (UCT) and National Co ordinator for the Collaboration for Health Equity through Education and Research (CHEER), Primary Health Care Directorate, University of Cape Town, E47 Rm68, Old Main Building, Groote Schuur Hospital, Main Road, Observatory, Cape Town 7925
Source of Support: The Cheer peer review process was supported with funding from Atlantic Philanthropies,, Conflict of Interest: None
Background: The Collaboration for Health Equity in Education and Research (CHEER) is unique in the composition of its members who represent all the Faculties of Health Sciences in South Africa. Over the past 10 years, CHEER has conducted 18 peer reviews involving all the institutions. In August 2012, CHEER embarked on its pilot peer review on Social Accountability in Health Sciences in South Africa. This paper shares the lessons learned and insights from the pilot process. Methods: A descriptive study design, using qualitative methods, which focused primarily on semi-structured interviews and focus group discussions, supplemented with supporting documentation, was employed. The protocol was developed by CHEER members and ethics approval was obtained. Results: Arising from our pilot peer review, reviewers identified several key components of the review process that should be noted for future reviews on social accountability. These relate to: (a) The composition of the review team; (b) the review process; (c) data collection and analysis; and (d) the reporting process. Discussion: Peer review is a useful way of building consensus and a common set of values that become more explicit through the process. We found that six criteria, namely, values, reference population, partnerships, student profile, graduate outcomes and impact, provide the basis for establishing standards for reflecting social accountability. The peer review is a process of institutional self-review supported by 'a panel of critical friends' and is useful when considered as part of the process of preparation for the formal accreditation review at Health Sciences educational institutions.
Keywords: Health sciences education, health equity, peer review, social accountability
|How to cite this article:|
Michaels DC, Reid SJ, Naidu CS. Peer review for social accountability of health sciences education: A model from South Africa. Educ Health 2014;27:127-31
| Background|| |
Similar to many other developing countries, South Africa faces enormous healthcare challenges due to the extent of the burden of disease, while the lack of adequate human resources to provide health services to the population exacerbates the situation. South Africa is currently on a drive to improve population health through the transformation of the health system toward equity and social justice. The Human Resources for Health strategy was developed by the Department of Health to address three key issues, namely: (a) The supply of health professionals and equity of access; (b) education, training and research; and (c) the working environment of the health workforce.  There is a stated government commitment to ensuring an "appropriate, trained and sustainable workforce" as a priority for the South African health sector (p8).  Creating the policy environment and establishing the regulatory, financial and organizational structures to support the development, recruitment, retention and equitable deployment of the workforce is recognized as a challenge that must be faced by policy makers in partnership with Higher Education Institutions (HEIs), among others. It is, therefore, incumbent upon HEIs to ensure a workforce "fit for purpose" to meet the health needs of the country by ensuring improved access to health professionals and health care in rural, remote and underserved communities. 
There is currently a disjuncture between health sciences education and health service needs in South Africa with respect to the number of graduates, as well as the curative and specialist orientation of skills and career choices of medical graduates. A fundamental realignment is required including attention to admission processes and criteria, health systems skills, sites of training and accountability of students and graduates, which should shift the graduate outcomes toward meeting the needs of a developing country such as South Africa.
Furthermore, the global call for health sciences education institutions to respond to the health-related needs and challenges in society has sensitized medical schools and health sciences faculties around the world to the concept of social accountability. ,
As far back as 1995, the World Health Organization defined social accountability as it relates to medical schools as the "the obligation to direct their education, research and service activities towards addressing the priority health concerns of the community... they have a mandate to serve" (p3).  The term "social accountability" is often used interchangeably with social responsibility. According to Boelen and Henk,  social responsibility is seen when a health sciences faculty responds to societal needs and acts proactively to fulfill those needs. On the other hand, social accountability is universal, that is, every institution is accountable, whether this is acknowledged or not, and thus social accountability is independent of social responsibility.
The Collaboration for Health Equity through Education and Research Review Process
The Collaboration for Health Equity through Education and Research (CHEER) was established in 2003 as a health equity and human resource research interest group, consisting of representatives from all nine South African Faculty of Health Sciences (FHS).  The seminal research question was focused on the most appropriate educational strategies that would support health science graduates to choose to practice in rural and underserved areas in South Africa once they had qualified. The overall aim of the CHEER collaboration is to promote health equity through appropriate education and research in health science education in South Africa.
One innovative and important component of CHEER has been the process of peer reviews at each of the member universities. To-date, two rounds of peer reviews have been conducted at each of the nine FHS in the country (18 in total). The first round focused on how FHS prepare their graduates for working in rural and underserved areas in South Africa  and the second on the nature of the relationship between FHS and their health service partners.
The CHEER peer review process described in this paper focuses on providing health sciences schools with organizing principles and an opportunity to reflect on the institutional standards that will help them to become more accountable in addressing health inequities and other health system challenges. The aim of further peer review is to determine the nature and extent of social accountability in FHS in South Africa, thereby, building evidence to support effective change toward greater impact and accountability.
| Methods|| |
The third phase of peer reviews has commenced with a focus on the social accountability of health sciences education institutions, and a pilot review was carried out at the University of Limpopo, Faculty of Health Sciences (ULFHS) over a period of two days in August 2012. The purpose of the pilot review was primarily to develop an approach and methods to assess the social accountability of FHS in the South African context.
A descriptive study design, using quantitative and qualitative methods consisting of semi-structured interviews and focus group discussions, supplemented with supporting documentation, was employed. The protocol was developed by CHEER members using an adaptation of an evaluation tool developed by The Training for Health Equity Network (THEnet).
Purposive sampling was done through the identification of partners and key informants within the various FHS departments and the Department of Health (DoH) by the ULFHS. Ethics approval was obtained from the ULFHS Research and Ethics Committee (MREC/M/182/2012: IR) and the University of Cape Town (417/2012).
Measuring Social Accountability
Development of the tool
As a tool intended for sharing and further development, THEnet provided permission for the use and adaptation of the "Evaluation Framework for Socially Accountable Health Professional Education",  and the Executive Director and Co-founder participated as a reviewer in this pilot project.
The framework, which is described as a continually evolving evaluation and quality-improvement tool, is structured into three sections, which address the following questions: (1) How does our school work? (2) What do we do? (3) What difference do we make? These sections explore the various features of health professional schools, giving consideration to components such as: The values that are upheld and demonstrated; reference populations served by the school; internal and external partnership-development; education and training programs; learner and educator profiles; research-focus and service-delivery; peer support; and graduate outcomes and impact.
A draft questionnaire based on the Framework's indicators was developed and reviewed during a workshop with the Undergraduate Education Training Committee of the Health Professionals Council of South Africa (HPCSA). During this workshop, deans and representatives from health sciences disciplines including Pharmacy and Dentistry, working in small groups, were allocated sections of the questionnaire to review. Groups presented their feedback and these suggestions were subsequently used to revise the questionnaire. Major revisions included the translation of Likert-scale-based statements to direct questions, followed by ranking scales for future action with respect to gathering appropriate data and addressing the relevant issue at hand. The final previsit questionnaire consisted of 14 key components, each containing between two and eight question items. Six of the previsit questionnaire's key components were prioritized for the interview schedule used by reviewers in the face-to-face interviews and focus group discussions [Table 1].
The review team
The Director of CHEER, who chaired the review, invited members of the collaboration to volunteer their institution for a pilot review. The Dean of the University of Limpopo invited CHEER to conduct the pilot at their institution. The review team consisted of six academics representing three of the nine South African universities. Two reviewers were experienced in the institutional accreditation review process; one reviewer was completing his doctoral thesis on the topic of social accountability; one reviewer was invited due to her expertise in the dental faculty; two members were from CHEER administration. The Executive Director and Co-Founder of THEnet joined the review team in South Africa.
The host institution
The FHS of the University of Limpopo consists of five schools, namely, School of Health Care Sciences, School of Oral Health Sciences, School of Medicine, School of Pathology and preclinical sciences and the School of Health Sciences. The ULFHS campus has service learning platforms across the boundaries of three Provinces and shares these platforms with two other universities.
The departments involved in the pilot included the School of Health Care Sciences: Occupational Therapy, Physiotherapy, Speech Language Pathology and Audiology, Pharmacy, Radiography, Human Nutrition; Medical clinic departments: Orthopedics, Anesthesiology, Opthalmology, Pediatrics, Family Medicine, Internal Medicine; School of Oral Health Sciences: Operative Dentistry, Periodontology, Community Dentistry, Maxillofacial and Oral Surgery, Orthodontics; School of Pathology and preclinical sciences: Forensic Medicine and Chemical Pathology.
A previsit questionnaire was distributed by the ULFHS to identified participants, prior to the visit. Focus group discussions were also conducted with ULFHS Executive members and faculty staff, stakeholders, including community and DoH representatives, as well as ULFHS students and alumni. Discussions lasted approximately 60 min and were not tape-recorded.
Qualitative analysis was conducted using the recursive abstraction method in which interviewers' notes were summarized according to identified focus areas. After consultation within and across teams, these were further summarized to produce a compact summary that would have been difficult to accurately discern without the preceding steps of distillation. Finally, the quantitative data from the previsit questionnaires were captured in MS Excel and analyzed using SPSS.
| Results|| |
The focus of this paper is on the process of conducting the review and lessons learned. Arising from our pilot peer review, reviewers identified several key components of the review process that should be noted for future reviews on social accountability [Table 2]. These relate to: (a) The composition of the review team; (b) the review process; (c) data collection and analysis; and (d) the reporting process.
|Table 2: Questions to be considered for future reviews on social accountability|
Click here to view
The Review Team
The team should consist of a minimum of six members, balanced in terms of gender, race and profession. The team should also consist of well and less well-resourced schools, and include representatives of the Ministries of Health and Higher Education.
Ideally a peer review should follow an invitation by the Dean of the FHS, thereby, ensuring support for the review process and implementation of recommendations. It should be preceded by an institutional self-evaluation approximately six-eight months prior to the review. This should involve all schools within the FHS in conducting a series of department-wide discussions with students, faculty, community members and members of their health department partners on the topic of social accountability, using tools such as those developed by THEnet to familiarize participants with the concepts and issues.
Adequate preparation time is required to allow for planning, research ethics approval, invitations as well as the completion and submission of the previsit self-assessment activities at the institution. The institution should be provided with guidelines for choosing the key informants to enable appropriate selection of respondents. These key informants should receive an introduction package with one or two reference papers on the concept of social accountability and stating the purpose of the peer review. The inclusion of supporting documents submitted to the review team is recommended to provide a context and background from which to explore key issues during the review visit. This will allow for questions and themes to be targeted according to the selected key informants. Adequate time (at least three full days) should be allocated to the review process to include data collection, reviewer conferencing, interim analysis and preliminary feedback to the institution at the end of the visit.
Specific roles need to be assigned to the reviewers, including nominating a dedicated scribe who will be responsible for drafting the final report. It should be noted that reviewers perform these tasks on a voluntary basis, receiving no remuneration for their time. Therefore, efficient scheduling for the duration of the review is essential.
The peer review team members should have some time allocated between each session to identify the key issues raised by key informants of the preceding session. This would ensure that the reviewers have a shared understanding of the issues raised and will facilitate targeting questions to subsequent interviewees. It will also facilitate the collation and organization of the key points raised during the day's interviews for the summary of the day's proceedings. From our experience, triangulating supporting documentation, both quantitative and qualitative data, is useful in order to build a more valid set of results.
The entire peer review team should be present during interviews with key informants who have the 'big picture' (e.g. senior faculty leadership, DoH officials, student and senate representatives, etc.). This is particularly important when the review involves allied health sciences as well as medical schools.
Preliminary oral feedback at the end of the review process is recommended, followed by a formal oral feedback by a member of the review team after the draft report has been given to the institution, preferably within a few weeks of having conducted the review. This is to prevent or minimize errors of fact for subsequent reporting of results.
| Discussion|| |
We set out to explore the best process for assessing and improving social accountability of FHS in the South African context, and determined it to be a feasible project, requiring a high level of cooperation and mutual understanding between reviewers and those undergoing review. The pilot peer review was successful in highlighting the potential and challenges of the process. Some of the challenges including logistical constraints such as the delay in Ethics Review and approval, which impacted on the prereview questionnaire distribution and resulted in a poor response rate and limited time for previsit analysis of questionnaires. The high volume of interviews planned within a one and one-half day period resulted in the splitting of the review team and very little time between interviews for conference between reviewers.
The peer review process has the potential to align a whole faculty and all stakeholders around a common vision of social accountability. For example, the results of this pilot review has led to the establishment of a position at the University of Limpopo for a 'driver' of social accountability initiatives and monitoring across the health sciences campus.
While we found the evaluation framework developed by THEnet very useful as the basis for the development of an evaluation tool, the results of the review shows that this requires further adaptation to suit the South African context. During this pilot peer review, we found that six criteria provide a basis for establishing standards for reflecting on social accountability and an institutions' progress toward this.
In South Africa, the results of the pilot is likely to influence policy through the undergraduate medical education accreditation body, the HPCSA, which plans to add a dimension of measuring social accountability to its accreditation process. It is important to note the value of a peer review as opposed to the formal accreditation process. The peer review is a process of institutional self-review supported by 'a panel of critical friends' and can be considered as part of the process of preparation for a formal accreditation review. 
The Cheer0 collaboration has entrenched the culture of peer review among all the FHS in the country and, thus, laid the foundation for designing standards to reflect social accountability, which is specific to South Africa. However, it is universally applicable due to the diverse nature of the context of the various FHS in South Africa. The process followed and experience gained in South Africa may therefore prove useful in other countries.
| Acknowledgement|| |
Members of the Collaboration for Health Equith through Education and Research (CHEER) review team: S Reid, V Burch, C Naidu, D Michaels (University of Cape Town), A-J Neusy (Training for Health Equity Network), K Mfenyana (Walter Sisulu University), L Green-Thompson (University of the Witwatersrand), N Veres (University of the Witwatersrand).
| References|| |
Burch V, Reid S. Fit for Purpose? The appropriate education of health professionals in South Africa. S Afr Med J 2011;101:25-6.
Reid SJ. A Cheerful Group: The Collaboration for Health Equity through Education and Research. South African Family Practice 2004;46:3.
Reid SJ, Cakwe M, on behalf of the Collaboration for Health Equity through Education and Research (CHEER). The contribution of South African Curricula to prepare health professionals for working in rural or under-served areas in South Africa: A peer review evaluation. S Afr Med J 2011;101:34-8.
The Training for Health Equity Network THE net′s Social accountability evaluation framework Version 1. Monograph I, 1 st
ed. The Training for Health Equity Network, 2011. Accessed from: http://thenetcommunity.org/files/articles/Monograph
. [Last accessed on 2013 Feb 02].
Maccarrick G, Kelly C, Conroy R. Preparing for an institutional self-review using the WFME standards - An international Medical School case study. Med Teach 2010;32:e227-32.
[Table 1], [Table 2]