Assessing Monitoring and Evaluation of PharmaceuticalsServices
Session 3: Assessing Monitoring and Evaluation of
PharmaceuticalsServices
Total Session Time: 120 minutes + 2 hours of Assignment
Prerequisites
Learning Tasks
By the end of this session students are expected to be able to:
country pharmaceutical situations
evaluation of country pharmaceutical situations
Resources Needed:
Evaluation CountryPharmaceutical Situation
SESSION OVERVIEW
Activity/
Step Time Content
Method
1 05 minutes Presentation Introduction, Learning Tasks
20 minutes List of Components of WHO Operational
Presentation
2 Package for Assessing Monitoring and
Buzzing
Evaluation of Pharmaceutical
85 minutes Presentation Explanation of Component of WHO
3 Small Group Operational Package for Assessing Monitoring
Discussion and Evaluation of Pharmaceutical
4 05 minutes Presentation Key Points
5 05 minutes Presentation Evaluation
PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide
19
SESSION CONTENTS
STEP 1: Presentation of Session Title and Learning Tasks (5 minutes)
READ or ASK students to read the learning tasks and clarify
ASK students if they have any questions before continuing.
STEP 2: Components of WHO Operational Package for Assessing
Monitoring and Evaluation of Pharmaceutical (20 minutes)
Activity: Buzzing (5 minutes)
ASK students to pair up and buzz on the following question for 2 minutes
evaluation of pharmaceutical situations?
ALLOW few pairs to respond and let other pairs to add on points not mentioned
WRITE their response on the flip chart/board
CLARIFY and SUMMARIZE by using the content below
Pharmaceutical Situations is intended as a useful tool for researchers, policy-makers,
planners and others who need to use standardized measurement tools to gather data and
other information.
approach with three groups of indicators.
Evaluationof CountryPharmaceutical Situations include the following;
o Pharmaceutical indicators for monitoring and assessment
Level I core indicators.
Level II facility-based core indicators
Level III indicators
o Preparing the survey (Level II – facility survey)
o Training data collectors
o The survey
o Data processing, analysis and reporting
PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide
20
STEP 3: Explanation on Each Component of WHO Package for Assessing
Monitoring and Evaluation of Pharmaceutical(85minutes)
Activity: Small group discussion (30 minutes)
ASK students to form small manageable group and discuss the following questions
ALLOW few pairs to respond and let other pairs to add on points not mentioned
WRITE their response on the flip chart/board
CLARIFY and SUMMARIZE by using the content below
o Given the complexity of the pharmaceutical sector, a systematic method of gathering
data is very important for assessing access, quality and rational use of medicines.
o There are multiple, cross-cutting factors that can influence access and rational use of
quality medicines, and a variety of strategies that countries can adopt and implement to
improve their pharmaceutical situations.
o Indicators have been developed for monitoring national medicines policies (NMPs)
that enable systematic assessment, evaluation and monitoring of the formulation and
implementation of pharmaceutical policies and programmes.
o These can be used to:
Assess country capacity, such as available infrastructure, logistics and human
resources to support the pharmaceutical sector and implement NMPs;
monitor the implementation of NMPs;
measure the impact of implementation strategies; and
Evaluate progress towards identified objectives.
o All stakeholders in the pharmaceutical sector can use indicator-based assessment of
the pharmaceutical situation to inform priorities and set targets.
o Indicators provide policy-makers and managers with a clear picture of national and
institutional problems. Policy-makers and managers can refer to study results when
developing strategies to strengthen the pharmaceutical sector
o Information and data can be used to:
Formulate or revise the NMP.
Assess the quality of governance, such as systems and mechanisms that
would safeguard against vulnerability to corruption.
PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide
21
Encourage and facilitate cooperation between the different main players in a
o Level I indicators provide a rapid means of obtaining information on the existing
infrastructure and key processes of each component of the pharmaceutical sector.
These indicators are assessed using a short questionnaire completed at the national
level.
o Level IIindicators
Provide systematic data on access and rational use of quality
medicines through facility-based surveys.
Systematic survey processes to collect data on these indicators are contained in WHO
package.
A population-based survey has been developed as part of the current process and is
discussed in a separate document, Manual for the Household Survey to Measure
Access and Use of Medicines.
o Preparing the survey (Level II – facility survey)
The survey of Level II indicators is a very important part of monitoring the
pharmaceutical sector because these indicators measure the outcome and impact of
pharmaceutical programmes in a country.
Adequate preparation is needed and data collectors must be trained.
o Coordination and survey coordinator
A national coordinator should be selected to take charge of the overall coordination of
the Level II survey, to oversee the survey process, data analysis, reporting and
presentation of results.
The coordinator has to go through the manual carefully, covering concepts of
assessment, monitoring, indicators and how to carry out a systematic survey.
o Selecting data collectors- Based on experience, the most effective data collectors are
those with clinicalexperience such as physicians, nurses, pharmacists or paramedical
staff
o Health ministry/department staff and temporary employees with some health-related
background and experience are possible candidates.
o It can also be useful to select data collectors from different parts of the country to reflect
language differences and enable a more diverse sampling of geographical areas–this
could also resolve any concerns data collectors may have about travelling to unfamiliar
areas.
o Data collection is the most crucial part of the monitoring process. For accuracy and
reliability of information and indicator measurement, the data collectors must be well
trained.
o Training should focus on ensuring data collectors have a common understanding of the
required information and know how to gather the data and complete the survey forms in
a standardized fashion.
PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide
22
PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide
23
o Notifying facilities to be visited- Obtain permission from officials
responsible for public health facilities.
o In some cases verbal permission may be sufficient; in others it may be necessary to send
a formal letter.
o Data collectors must be properly introduced to the facility (both for the field test and
actual survey) through an endorsement letter from the health ministry or department or
the WHO country office.
o What to do with the results.The coordinator should oversee the analysis of the data and
the writing of the report.
o Guidance and a suggested report outline are provided below.
o Data processing should be carefully planned.
o Even before data collection begins, the necessary resources and a person to tabulate the
data and do the computations must be identified.
o Computation of Level II facility indicators
During the survey, the coordinator must check the accuracy of data and
information reported as far as possible in the field
The data entered on the Survey Forms should be calculated following the
instructions and formulas on each of the Survey Forms.
These should be checked for completeness and consistency before making the
final calculations and analysis. Data coding and entry must be double-checked.
o Analysis and interpretation of Level I and Level II facility indicators
Using the information from the Level I questionnaire and data from Level II
facility Summary Forms, the country pharmaceutical situation can be analysed and
discussed.
The information in the Level I questionnaire can be used to discuss the existing
pharmaceutical sector structures, and what activities and strategies have been and
are being implemented in the country.
o Written report
A report should be prepared in order to disseminate the information obtained.
This report is an important source of information and a basis for decisions on
medicine policy and strategies.
It should be persuasive and well prepared.
The results must be discussed in a comprehensive and systematic manner, taking
into account the objectives and strategies of the medicine policy.
Clear recommendations should be included, based on the study findings.
Refer students to Handout3.1: Components of the WHO Operational Package for
Assessing, Monitoring and Evaluation CountryPharmaceutical Situation
PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide
24
STEP 4: Key Points (5 minutes)
o Pharmaceutical indicators for monitoring and assessment
o Preparing the survey (Level II – facility survey)
o Training data collectors
o The survey
o Data processing, analysis and reporting
medicines through facility-based surveys
infrastructure and key processes of each component of the pharmaceutical sector.
STEP 5: Evaluation (5 minutes)
country pharmaceutical situation?
PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide
25
References
Quick, J. D. (1997). Managing drug supply the selection, procurement, distribution and use of
pharmaceuticals. West Hartford, CT: KumarianPress.Bartsch, S. (2007).
The Global Fund to Fight AIDS, Tuberculosis and Malaria. Global Health Governance and the
Fight Against HIV/AIDS,146-171. doi:10.1057/9780230591349
WHO Operational package for assessing, monitoring and evaluating country pharmaceutical
situations. (2015, November 20). Retrieved from
https://www.who.int/medicines/publications/WHO_TCM_2007.2/en/
PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide
26
Handout 3.1: Components of the WHO Operational Package for
Assessing, Monitoring and Evaluation CountryPharmaceutical
Situation
Pharmaceutical indicators for monitoring and assessment
sustainable system of regular monitoring. Resources are not consistently allocated to
this task and there is limited advocacy for a culture of monitoring. Further, many
efforts to develop monitoring tools have been exhaustive, but impractical.
if this was done regularly.
National Drug Policies, a WHO manual, includes approximately 120 indicators covering
structure, process and outcomes of various NMP components. Several countries have
used it to monitor and evaluate their pharmaceutical situations. Another set of
indicators, developed by Management Sciences for Health,8 focuses on rapid
assessment of strengths and weaknesses in the pharmaceutical sector. The WHO
manual ―How to Investigate Drug Use in Health Facilities‖ has been used extensively in
many countries.
Country Pharmaceutical Situations. Guide for Coordinators and Data Collectors, was
developed to provide a practical indicator-based tool that can be implemented
regularly without investing large amounts of human or financial resources. This
package relies on a hierarchical approach to monitoring built around three groups of
core indicators.
data and simple survey techniques. These core
indicators systematically measure the most important information needed to gain a
comprehensive picture of the pharmaceutical situation in a country.
advantages:
pharmaceutical situation:
pharmaceutical policy; and in-depth assessment of specific system components.
PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide
27
components.
among facilities, districts, regions, government and nongovernmental agencies
as well as international organizations.
infrastructure and key processes of each component of the pharmaceutical sector.
These indicators are assessed using a short questionnaire completed at the national
level.
medicines through facility-based surveys. Systematic survey processes to collect data
on these indicators are contained in this package. A population-based survey has been
developed as part of the current process and is discussed in a separate document,
Manual for the Household Survey to Measure Access and Use of Medicines.
The Level I core indicators are used to assess existing structures and processes in a
national pharmaceutical system. They provide a method to rapidly assess the
implementation of NMPs and their components.
information. Most data will be available within the Ministry of Health, although data
on intellectual property rights protection may require consultation with the responsible
ministry.
updated periodically, for example every two years.
Data from Level I indicators can be used to array the achievements and weaknesses of
individual pharmaceutical systems and to illustrate common sectoral strategies and
approaches. Many WHO Member States have submitted data from the Level I
questionnaire to EDM. The WHO MedNet can be consulted to compare results over
time and among countries. Comparisons among countries can be particularly
interesting and convincing for policy-makers. The questionnaire on Level I indicators is
included as Annex 1. The indicators in this questionnaire are summarized below.
o The Level II facility core outcome indicators support the Level I structure and process
indicators by providing specific data about important pharmaceutical outcomes. This
set of indicators requires field surveys. For data to be collected accurately and reliably,
attention must be paid to appropriate survey design, sampling and data gathering
techniques. In selecting the core outcome indicators, consideration was given to the
need to obtain the most relevant information from as limited a data collection process
as possible
PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide
28
o These indicators measure the degree of attainment of strategic pharmaceutical
objectives: improved access, quality and rational use. Access is measured in terms of
the availability and affordability of essential medicines, especially to the poor and in
the public sector. Measuring the actual quality of medicines by testing samples can be
expensive. Instead, the presence of expired medicines on pharmacy shelves and the
adequacy of handling and conservation conditions of medicines are used as indicators
of quality. Finally, rational use is measured by examining prescribing and dispensing
habits and the implementation of key strategies such as standard treatment guidelines
(STGs) and essential medicines lists (EMLs).
o Level II indicators are measured in public health facilities, private drug outlets, and in
warehouses supplying the public sector.
o The survey of Level II indicators is a very important part of monitoring the
pharmaceutical sector because these indicators measure the outcome and impact of
pharmaceutical programmes in a country
the survey of Level II indicators is a very important part of monitoring the
pharmaceutical sector because these indicators measure the outcome and impact of
pharmaceutical programmes in a country. Adequate preparation is needed and data
collectors must be trained.
A national coordinator should be selected to take charge of the overall coordination of
the Level II survey, to oversee the survey process, data analysis, reporting and
presentation of results. The coordinator should be someone who is knowledgeable
about the pharmaceutical sector and experienced in conducting surveys. A thorough
working knowledge of the pharmaceutical sector will help to ensure that important
aspects of the sector are not overlooked in planning for the Level II survey. The results
of the Level I questionnaire will also inform the planning process.
o The roles and functions of the coordinator include:Communicating with government
officials and other local agencies to gain approval for the survey and to request
personnel who will do the survey; Communicating with officials and health facilities to
be visited for the field test and for the actual survey; Selecting geographical areas and
identifying facilities to be surveyed; Allocating budget and, if necessary, requesting
financial (Annex 4,) and technical support. Be sure that main components, such as data
collector training, the survey itself, including transport and per diem details, analysis and
dissemination of results, are covered; Coordinating and identifying sources of
information for the Level I questionnaireon structures and processes of the country
pharmaceutical situation;
basket of medicines, treatment guidelines; Preparing all the necessary materials for training,
the field test, and survey of Level II indicators;
PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide
29
Supervising the actual survey of Level II indicators; Checking that survey forms are
completed correctly and in full;
the report, and presenting/giving feedback of the results to appropriate groups
experience such as physicians, nurses, pharmacists or paramedical staff. Health
ministry/department staff and temporary employees with some health-related
background and experience are possible candidates. It can also be useful to select data
collectors from different parts of the country to reflect language differences and enable
a more diverse sampling of geographical areas–this could also resolve any concerns
data collectors may have about travelling to unfamiliar areas. In all, 10 data collectors
will be needed–one team of two data collectors for each geographical area.
o Notifying facilities to be visited
Obtain permission from officials responsible for public health
facilities. In some cases verbal permission may be sufficient; in
others it may be necessary to send formal letter.
Data collectors must be properly introduced to the facility (both for the
field test and actual survey) through an endorsement letter from the health
ministry/department or the
o The coordinator should oversee the analysis of the data and the writing of the report.
Guidance and a suggested report outline are provided below.
Data processing should be carefully planned. Even before data collection begins, the
necessary resources and a person to encode/tabulate the data and do the computations
must be identified.
PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide
30
Unataka kutumiwa notes hizi kupitia WhatsApp?Kwa notes zilizopangiliwa vizuri kwa kusoma offline au PDF, bonyeza kitufe hapa chini. Ujumbe wenye Level, Semester, Module na Topic utaandaliwa moja kwa moja.TUMIWA NOTES WHATSAPP
