Evaluate progress

Monitoring and evaluation provides insight into the impact of the rational use program or interventions. It helps to see if goals are being accomplished and can identify aspects that may need improvement, which is important in the scale-up and replication of interventions.

Outputs from evaluation can be used for evaluating interventions, revising national or local guidelines, or presenting facts for underlying policy decisions. Benchmarking between departments or health care facilities can be used to encourage improvement as no one wants to be seen as the worst. However it is important to encourage improvement and promote learning from experience without fear of negative consequences.

Develop a monitoring and evaluation framework

Often evaluation is done by measuring indicators, however in addition, the collection of descriptive data can be helpful to give a comprehensive view of the extent to which changes are occurring. Unintended consequences of evaluation may include distortion of treatment priorities and promotion of unnecessary antibiotic use. To counteract this, relevant stakeholders should be involved throughout the process including indicator development, analysis and feedback. When developing a monitoring and evaluation framework, teams should consider:

• What will be measured?
• How data will be collected and recorded
• Analysis and interpretation of data
• Feedback of the results to all stakeholders
• Acting on the results

Measurement and indicators

Both process and outcome measures should be used within the monitoring and evaluation framework. The two types of indicators give different perspectives on the success of rational use programs. Measurement through an intermittent audit can help to maintain appropriate antibiotic prescribing/use. 

Process indicators:

Process measures give a view if improvements have been made in appropriate antibiotic use practices. Some examples are:

• % prescriptions adhering to guidelines for appropriateness and timeliness of therapy for a given infection
• % change in the use of an antibiotic or antibiotic class
• % prescribers who accurately applied diagnostic criteria for infections
• % prescribers who obtained cultures and relevant tests prior to treatment and modified antibiotic choices appropriately to microbiological findings
• % prescribers who documented the indication and planned duration of antibiotic therapy
• % prescriptions for restricted antibiotics that are concordant with drug and therapeutics committee approved criteria

Outcome indicators:

Outcome measures inform the degree to which rational use programs or interventions have reduced costs, prevented resistance or other unintended consequences of antibiotic use. Some examples are:

• % change in antibiotic resistance
• % change in drug cost
• % change in mortality
• % adverse drug events
• % patients with complications
• number of patients who develop super-infections, such as C. difficile infection

In MEASURE you can find tools and resources for measuring various aspects of antibiotic resistance and use (for example point prevalence survey protocols and knowledge-attitudes-behaviors-perceptions questionnaires):

• Burden of antibiotic resistance
• Antibiotic resistance
• Consumption of antibiotics
• Appropriateness of use
• Infections
• Quality of antibiotics
• Knowledge, attitudes, beliefs and practices – KABP

Analyze results

Data collected should be analyzed by the committee and additional key stakeholders. Involving people from outside the committee can provide additional understanding of what is happening in a particular setting and helps to build ownership of finding solutions. When reviewing results, dialog and discussion can help to understand the meaning of the results. Interventions showing progress can be considered for scaling up. However when there is no indication of change or improvement, committees should discuss possible explanations and decide if changes need to be made to the program.

Communicate results and provide feedback

Results should be fed back to collaborating partners and interested stakeholders, especially management in a timely manner. Communication methods and content need to be tailored to the audience, and reporting should be transparent to promote confidence in the guidelines and overall program. Positive progress should be acknowledged and public recognition should be made of the contributions and successes of all partners involved in the work. Affirmation of hard work will provide incentive for staff to continually improve and be involved in the process.

Selected Resources