Guide to Monitoring Scale-up of Health Practices and Interventions

This guide was developed as a "how-to" resource to provide programme implementers with a low-cost and replicable approach to monitoring the process of scaling up and institutionalising innovations in health. The goal is to assist country stakeholders with identifying if scale-up is happening as intended, assess if the scale-up is adhering to the model or pilot, and determine if the scale-up efforts can be sustained to achieve the desired impact. While the guide focuses on the scale-up of family planning practices and innovations, it can be adapted to any scale-up experience in a given country context.

This guide presents 10 considerations to monitoring scale-up as part of a basic monitoring plan; selected communication-related excerpts and recommendations are included below:

1. Define the innovation and the objectives and scope of the scale-up plan - For instance, in the Ghana Community-based Health Planning and Services (CHPS) Initiative, the goal (or long-term objective) of the scale-up plan was to transition from facility-based health care to high-quality, integrated community-based health services. To achieve that goal, there were several shorter-term objectives, such as: provide outreach to traditional leaders and build community awareness; select a Community Health Committee and train community health officers; mobilise the community to build a simple facility (Community Health Compound); mobilise providers to visit households; procure motorbikes and conduct motorbike rider training; and train and deploy volunteers. After objectives have been identified, there are several questions to be asked, such as: Who are the key stakeholders or policymakers (e.g., religious leaders, parents, specific ministry staff) who must be engaged for the scale-up to succeed?
2. Create a framework - "Table 3 provides a simple yet practical work sheet-style tool intended to be used during the planning phases of scale-up interventions. It can also be used to guide the development of monitoring frameworks for interventions further along in the scale-up process. The purpose is to provide a tangible resource for defining and organizing the key components of a specific scale-up so stakeholders and evaluators have a common vision of scale-up as it relates to their unique perspectives. The work sheet poses general guiding questions for stakeholders to consider when designing a monitoring framework for scale-up. The data collection section provides space for indicator selection..."
3. Identify necessary resources to implement the monitoring plan - "Those developing the monitoring plan, as well as the monitoring framework, should take advantage of readily available technical resources such as indicator guides, M&E [monitoring and evaluation] materials (e.g., manuals, textbooks, free online courses), and communication tools."
4. Select key indicators - "The rationale for selecting these indicators will be based on the conceptual framework and the information needs of decision makers. This process should be an iterative, participatory process with stakeholders....It is helpful to develop an indicator matrix, or indicator reference sheet, summarizing the indicators in the monitoring plan."
5. Establish data sources and reporting systems - "Several tools, systems, and surveys have been developed for the purpose of gathering data to answer specific questions related to the health status or behavior of certain populations. These range from Demographic Health Surveys (DHS) and Reproductive Health Surveys (RHS), to focus group discussions and routine service statistics collected at the health facility level. Although the information obtained from these sources can be highly informative when monitoring a scaled-up practice, it will generally only tell part of the story. For example, health facility data may indicate what percent of postabortion care (PAC) clients received FP counseling, but it will not reveal whether the counseling was correct and consistent, the progress in expanding FP trainings to service providers, the political support behind ensuring FP counseling and commodities to PAC clients, the extent to which FP job aides have been distributed to PAC facilities, or other indicators....In order to make evidence-based decisions, decision makers require information from a variety of sources....Be mindful of the power of qualitative data. The Most Significant Change technique asks the scale-up beneficiaries and implementers to reflect on what they think is the most significant change in their experience as a result of their involvement with a particular practice or innovation. Story telling is another way to poignantly capture and portray (often in the form of success stories) key programmatic lessons and experiences. Anecdotes, quotations, and photographs from project participants and stakeholders can be valuable complements to facts and figures, filling data gaps and providing insight and understanding into statistics."
6. Develop a data use and dissemination plan - Some of the questions the plan should answer are: Who will be responsible for analysing the data? Who will be responsible for interpreting the data? How will the findings be presented (e.g., summary reports, graphs, maps)? How will the findings be disseminated (e.g., meetings, workshops, publications)? Who are the internal (within the scale-up project) and external (stakeholders) audiences for dissemination? Who will be using the data to develop solutions for redirecting or galvanising the aspects of scale-up that have been derailed, delayed, or stalled?
7. Collect data - "The point is to conduct data collection until you have the information you need to answer your priority questions.
8. Analyse data and determine if scale-up is progressing on track - "After specific data sources have been identified and obtained to answer your question of interest, the monitoring data can be transformed into useful information to facilitate decision making and subsequent action. The result of this data analysis can be depicted in visual (e.g., charts, tables, graphs) and narrative formats....To use evidence-based information to improve scale-up, the information received by stakeholders must be available, accessible, relevant, and useful." Amongst the questions to be asked: Has the practice been included in standard operating procedures, job aids, or other health communication materials? Are providers being adequately trained and is there supportive supervision? Do the service delivery points have the necessary commodities, such as information, education, and communication (IEC) materials for the practice to be properly implemented? Is community mobilisation (CM) creating awareness of and support for the practice?
9. Make programme adjustments based on findings and recommendations - "This involves convening key stakeholders to discuss why the expected achievements in the process of scaling up have or have not been met and to make evidence-based program adjustments. It is critical to maintain stakeholder engagement in the process of brainstorming solutions to road blocks in your scale-up to ensure the solutions are actionable and ultimately implemented. A meeting should be convened to use the conclusions identified in your analysis..."
10. Continue the monitoring and evaluation process - "The frequency of monitoring (continued collection, analysis, and interpretation of key indicators) will depend on the nature of the scale-up and practice of interest, the type of the question of interest, and resources - both financial and human."