Monitoring and evaluation are integral parts of any awareness raising initiative. Results should be fed back to interested stakeholders in a timely manner and in a format that can be interpreted and used by the respective parties.
Feedback should encourage improvement and promote learning from experience without fear of negative consequences. Periodic evaluation is important to understand if goals are being accomplished and identify aspects that may need improvement. Evaluation also documents the impact of programs, which is helpful for the scale-up, and replication of interventions. The evaluation should include an analysis of both process and outcome indicators.
Develop a monitoring and evaluation plan
The development of an evaluation plan is helpful in measuring the impact of awareness-raising initiatives. Often process and outcome measures are evaluated quantitatively however the collection of qualitative data can help give a comprehensive view of the extent to which changes are occurring. When developing an evaluation plan, teams should think about the following:
- What to measure?
- How to collect and record data?
- Who will analyze and interpret the data?
- How to feedback results to the different stakeholders?
- Taking action on the results
Measurement and indicators
Process and outcome measures are essential parts of evaluation. The two types of measures give different perspectives on the situation and can be used to get a comprehensive view of the success of awareness raising initiatives. With publically reported measures there is a risk of focusing efforts on one aspect of the program and losing sight of others leading to ‘gaming’ of results. To counteract this, ensure ownership and long-term benefit, indicator development, evaluation, and feedback should be multidisciplinary, involve relevant stakeholders and should be done in a timely manner.
Process measures give a view if improvements have been made in awareness and understanding.
Outcome measures inform the degree to which awareness raising initiatives facilitate behavior change to reduce the spread of antibiotic resistance.
In MEASURE you can find tools and resources for measuring various aspects of antibiotic resistance and use (for example point prevalence survey protocols and knowledge, attitudes, beliefs and practices (KABP) questionnaires).
The data collected during the measurement stage should be analyzed together with the key stakeholders. Involving others from outside can bring additional knowledge and understanding to what is happening in a particular group. By engaging them in the process, it can help them identify problems and take ownership of finding solutions. It is important to have dialogue and discussion in this stage so that everyone can understand the meaning of the results. Where there seems to be a trend towards progress, teams should consider continuing and scaling up interventions. Where there is no indication of change or improvement, teams should discuss possible explanations and decide if changes need to be made to the program.
Communicate results and provide feedback
Feedback on the impact of awareness-raising initiatives should be given in a timely manner to collaborating partners and interested stakeholders, especially management if applicable. A strategy should be developed for communicating the results. Reporting should be transparent to promote confidence in the intervention. Feedback should be in a format that can be interpreted and used by the relevant stakeholders. Communication methods and content can differ between groups and should be tailored to the audience. Positive progress should be acknowledged and public recognition should be made of the contributions and successes of all partners involved in the project. Affirmation of hard work will provide incentive for staff to continually improve and be involved in the process.
|Characteristics and outcomes of public campaigns aimed at improving the use of antibiotics in outpatients in high-income countries||The article identifies and reviews the characteristics and outcomes of 22 campaigns done at a national or regional level in high-income countries between 1990 and 2007.|
|Community Toolbox||The Community Toolbox aims to offer people engaged in local and community work a depository of tools and advice for building healthier communities. Chapter 6, Section 1 gives an overview on how to develop a communication plan and communicating results.|
|Handwashing Promotion Monitoring and Evaluation Module||Toolkit developed by UNICEF useful when implementing a handwashing program. It also includes which steps to take when evaluating the behavior change program.|
|TREND statement and checklist||22 item checklist to guide standardized reporting of nonrandomized evaluations of behavioral and public health interventions, developed by the Transparent Reporting of Evaluations with Nonrandomized Designs (TREND) group.|