PST06211 Monitoring and Evaluation of Medicine Use

NTA Level 6 • Semester 2 • PST06211 Introduction to Monitoring and Evaluation Monitoring and Evaluation of Medicine Use • Source Session/Topic 1 Full source-text version: all educational wording from the extracted learning source is retained; only presenter/tutor metadata and web-layout noise are removed, while formatting is improved for readability. Session 1: Introduction to Monitoring and Evaluation Total Session Time: 120 minutes + 2 hours of Assignment Prerequisites • None Learning Tasks By the end of this session students are expected to be able to: • Define terms in monitoring and evaluation • Differentiate between monitoring and evaluation • Explain importance of monitoring and evaluation in pharmacy practice Resources Needed: • Flip charts, marker pens, and masking tape • Black/white board and chalk/whiteboard markers • Computer and LCD projector • Handout 1.1: Glossary of Monitoring and Evaluation Terms • Handout 1.2:Difference between monitoring and evaluation SESSION OVERVIEW Activity/ Step Time Content Method 1 05 minutes Presentation Introduction ToLearning Tasks 2 40 minutes Presentation Terms used in Monitoring and Evaluation 25 minutes Presentation and Differences Between Monitoring and 3 Buzzing Evaluation 30 minutes Presentation and Importance of Monitoring and Evaluation in 4 Group Discussion Pharmacy Practice 5 10 minutes Presentation Key Points 6 05 minutes Presentation Evaluation 7 05 minutes Assignment Take home assignment PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 1 SESSION CONTENTS STEP 1: Presentation of Session Title and Learning Tasks (5 minutes) READ or ASK students to read the learning task and clarify ASK students if they have any questions before continuing. STEP 2: Termsused in Monitoring and Evaluation (40 minutes) • Monitoring refers to reviewing on a continuous basis, the degree to which program activities are completed and performance targets or milestones are being met. o Typically monitoring focuses on trucking program inputs such as funding, staff facilities, supplies and training o As such monitoring is part of the operational management of a program o Monitoring also track output such availability of medicines and supplies o It help in identifying potential problems and corrective action to be taken during program implementation • Evaluation refers to analysing progress towards meeting established objectives goals or results. o It provides feedback on the outcomes of activities such as changes in prescribing behaviour and health care seeking behaviour, whether plans have been met and reason for success or failure. o Monitoring can be done through various informal and informal methods: supervisory visits, routine reporting, sentinel reporting system, and special studies. • Monitoring and evaluation plan—a multi-year implementation strategy for the collection, analysis and use of data needed for project management and accountability purposes. • Surveillance—the ongoing, systematic collection, analysis, interpretation, and dissemination of data regarding a health-related event for use in public health action to reduce morbidity and mortality and to improve health. • Target—the objective a program is working towards, expressed as a measurable value or the desired value for an indicator at a particular point in time. • Validity—the extent to which a measurement or test accurately measures what is intended to be measured. • Summative evaluation—a type of evaluation conducted at the end of an intervention to determine the extent to which anticipated outcomes were produced. It is designed to provide information about the merit or worth of the intervention. • Results—the outputs, outcomes, or impacts of an intervention • Quantitative data—data collected using quantitative methods, such as surveys. Qualitative data—data collected using qualitative methods, such as interviews, focus groups, observation, and key informant interviews. PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 2 • Project—an intervention designed to achieve specific objectives within specified resources and implementation schedules, often within the framework of a broader program. • Prevalence—the total number of persons living with a specific disease or condition at a given time. • Performance—the degree to which an intervention or organization operates according to specific criteria or achieves results in accordance with stated goals or plans. • Objective—a statement of a desired program result that meets the criteria of being Specific, Measurable, Achievable, Realistic, and Time-phased (SMART). • Intervention—a specific activity or set of activities intended to bring about change in some aspect(s) of the status of the target population. • Health information system (HIS)—a data system, usually computerized, that routinely collects and reports information about the delivery and cost of health services, and patient demographics and health status. • Goal—a broad statement of a desired, usually longer-term, outcome of a program. • Indicator—a quantitative or qualitative variable that provides a valid and reliable way to measure achievement, assess performance, or reflect changes connected to an intervention • Inputs—the financial, human, and material resources used in a program. • Outputs—the results of program activities; the direct products or deliverables of intervention activities. Refer students to Handout1.1: Glossary of Monitoring and Evaluation Terms STEP 3:Differences between Monitoring and Evaluation (25 minutes) Activity: buzzing (5 minutes) ASK students to pair up and buzz the following question • What is the difference between monitoring and evaluation? ALLOW few pairs to respond and let other pairs to add on points not mentioned WRITE their response on the flip chart/board CLARIFY and SUMMARIZE by using the content below The difference between monitoring and evaluation is summarized in the table 1.1 below; PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 3 Table 1.1; Differences between monitoring and evaluation MONITORING EVALUATION clarify program objectives Analyzes why intended results were or were not achieved Link activities and their resources to Assesses specific causal contribution of objectives activities to results Translate objectives into performance Examine implementation process indicators and set target Routinely collects data on these indicators Explore unintended results compares actual results with target Ensures project accountability Judge the overall merit of the project It is an essential part of good day to day It is an essential activity in a longer term management process Takes place during the implementation phase Mid-term

NTA Level 6 • Semester 2 • PST06211 Pharmacovigillance Methods Monitoring and Evaluation of Medicine Use • Source Session/Topic 9 Full source-text version: all educational wording from the extracted learning source is retained; only presenter/tutor metadata and web-layout noise are removed, while formatting is improved for readability. Session 9: Pharmacovigillance Methods Total Session Time: 120 minutes Prerequisites • None Learning Tasks By the end of this session students are expected to be able to: • Define pharmacovigillance methods • List pharmacovigillance methods • Explain each pharmacovigillance methods • Compare reporting methods in pharmacovigillance Resources Needed: • Flip charts, marker pens, and masking tape • Black/white board and chalk/whiteboard markers • Computer and LCD projector • Handout 9.1 pharmacovigillance methods SESSION OVERVIEW Activity/ Step Time Content Method 1 05 minutes Presentation Introduction, Learning Tasks 2 05 minutes Presentation Definition of PharmacovigilanceMethods 45 minutes Presentation 3 Small group Pharmacovigillance Methods discussion 4 20 minutes Presentation Explanations on Pharmacovigillance Methods 25 minutes Presentation Comparing the Reporting Methods of 5 Buzzing Pharmacovigillance 6 10 minutes Presentation Key Points 7 10 minutes Presentation Evaluation PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 69 SESSION CONTENTS STEP 1: Presentation of Session Title and Learning Tasks (5 minutes) READ or ASK students to read the learning tasks and clarify ASK students if they have any questions before continuing. STEP 2: Definition of Pharmacovigillance Methods (5 minutes) • Pharmacovigillance methods are those methods used in pharmacovigillance in conducting surveillance of adverse events or any other drug related problem. It can be either active surveillance or passive surveillance • It‘s a way to monitor drug safety. • Are methods for gathering information STEP 3: Pharmacovigillance Methods (45 minutes) Activity: Small Group Discussion (20 minutes) DIVIDE students into small manageable groups ASK students to discuss on the following questions • What are the pharmacovigillance methods? ALLOW students to discuss for 15 minutes ALLOW few groups to present and the rest to add points not mentioned CLARIFY and SUMMARIZE by using the contents below • The following are pharmacovigillance methods used in monitoring drug safety; o Spontaneous reporting o Intensified ADR reporting o Targeted reporting o Cohort Event Monitoring o Electronic health record(EHR) mining PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 70 • Also pharmacovigillance methods can be categorized into; o Passive surveillance methods include; Spontaneous reports, Case series, Stimulated reporting o Active surveillance methods include; Sentinel sites, Medicine event monitoring, Cross- sectional study (survey) ,Case-control study Refer students to Handout9.1: pharmacovigillance methods STEP 4:Explanations on PharmacovigillanceMethods(20 minutes) • Spontaneous reporting o A system to monitor the safety of all medicines on the market o Voluntary submission of ICSRs by health professionals, pharmaceutical manufacturers and patients to the nationalpharmacovigillance centre o Requires two initial steps:  A reporter  Suspects that an undesirable medical event may have been caused by exposure to a medicine o Reports the suspicion to the national pharmacovigillance centre o Reports may include;  Serious ADRs  Severe ADRs  New drug  Unknown (unlabelled reactions)  ADRs in vulnerable groups (children, pregnant women, elderly o Advantages of this method  Most commonly used method  Easiest method to establish  least labour intensive  relatively inexpensive o Disadvantages of this method  Inherent under‐reporting  Captures only suspected ADRs  Possibility ofReporting bias • Intensified ADRs reporting o To enhance ADR reporting of specific medicines in early post‐marketing phase o Extension of Spontaneous Reporting Programme o Medicines under additional monitoring include, medicines contain new active substance, biologicalmedicines that require additional studies e.g. more data on long term use or on rare side effects in clinical trials PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 71 • Targeted reporting o intensified ADR Reporting within a defined cohort o it target specific medicines, population, ADR, and clinics • Cohort event monitoring o To gather more information on the safety profile of a new chemical entity in early post‐ marketing phase o Observational cohort study design o Patients enrolled into cohort and actively followed‐up during treatment to record all adverse events (not just suspected ADRs)Characterise known reactions o Detect signals of unrecognised reactions o Identify interactions with other medicines o Detect inefficacy of medicine o Assess safety in pregnancy and lactation • Electronic health record(EHR) mining o Make use of existing health records to supplement pharmacovigillance activities o Potentially rich source of ADR data o Mining of the THIN data base is currently being evaluated • Which method and when?Routine pharmacovigillance (spontaneous reporting) is recommended for all products focused pharmacovigillance is better out under specified conditions when safety issues or potential safety issues need to be addressed .Active studies are undertaken when data is needed quickly STEP 5: Comparing the Reporting Methods (25 minutes) Activity: Buzzing (10minutes) ASK students to pair up and buzz on the following question • Which methods report all types of ARDs? ALLOW few pairs to respond and let other pairs to add on points not mentioned WRITE their response on the flip chart/board CLARIFY and SUMMARIZE by using the content below PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 72 METHOD MEDICINES POPULATION REPORTS Spontaneous Reporting All medicines life, All exposure All ADRs cycle of products individuals but denominator unknown Intensified ADR Reporting Specific medicines All exposure All ADRs individuals but denominator unknown Targeted Reporting Specific medicines Defined cohort Specific ADRS All ADRs Cohort Event Monitoring Specific medicines Defined cohort All events Electronic health record All medicines Defined cohort All events mining STEP 6: Key Points (10 minutes) • Pharmacovigillance methods are ways to monitor drug safety. • Pharmacovigillance method spectrum include;Spontaneous Reporting, Intensified ADR Reporting, Targeted Reporting and Cohort Event Monitoring • Pharmacovigillance methods can be categorized into passive and active surveillance • Spontaneous reporting involves a system to monitor all medicines • Intensified ADR reporting involves reporting of specific medicines at early

NTA Level 6 • Semester 2 • PST06211 The Concept of Pharmacovigillance Monitoring and Evaluation of Medicine Use • Source Session/Topic 8 Full source-text version: all educational wording from the extracted learning source is retained; only presenter/tutor metadata and web-layout noise are removed, while formatting is improved for readability. Session 8: The Concept of Pharmacovigillance: Total Session Time: 120 minutes Prerequisites • None Learning Tasks By the end of this session students are expected to be able to: • Explain pharmacovigillance • Define terminologies used in pharmacovigillance • List the aims of pharmacovigillance • List key partners in pharmacovigillance • Explain the importance of pharmacovigillance Resources Needed: • Flip charts, marker pens, and masking tape • Black/white board and chalk/whiteboard markers • Computer and LCD projector SESSION OVERVIEW Activity/ Step Time Content Method 1 05 minutes Presentation Introduction, Learning Tasks 2 05 minutes Presentation Explanation of Pharmacovigillance 3 15 minutes Presentation Definition of Terms in Pharmacovigillance 15 minutes Presentation and 4 Aims of Pharmacovigillance Buzzing 5 10 minutes Presentation Key Partners in Pharmacovigillance 50 minutes Presentation and 6 Importance of Pharmacovigillance Group discussion 7 10 minutes Presentation Key Points 8 10 minutes Presentation Evaluation PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 62 SESSION CONTENTS STEP 1: Presentation of Session Title and Learning Tasks (5 minutes) READ or ASK students to read the learning tasks and clarify ASK students if they have any questions before continuing. STEP 2: Pharmacovigillance(5 minutes) • The etymological roots for the word "pharmacovigillance" are: pharmakon (Greek for drug) and vigilare(Latin for to keep watch). As such, pharmacovigillance heavily focuses on adverse drug reactions. • Pharmacovigillance (PV) is defined as the science and activities relating to the detection, assessment, understanding and prevention of adverse effects or any other drug-related problem(According to WHO definition) • Why Pharmacovigillance is needed? The processes involved in the clinical development of medicines once put onto the market; a medicine leaves the secure and protected scientific environment of clinical trials and is legally set free for consumption by the general population. At this point, most medicines will only have been tested for short-term safety and efficacy on a limited number of carefully selected individuals. STEP 3: Definition of Terms in Pharmacovigillance (15 minutes) The following are some of pharmacovigillance terminologies as adopted in TFDA guideline: • Adverse Drug Reactions (ADRs) a response to a medicinal product that is noxious or potentially harmful and unintended and which occurs at doses normally used in human for prophylaxis, diagnosis or therapy of a disease or for the modification of physiological function in which individual factors may play an important role. • Adverse eventAny unfavourable medical occurrence that in coincidence may present during treatment with a pharmaceutical product, but which does not necessarily have a causal relationship with the treatment. • Benefit/risk analysis Examination of the favourable and unfavourable results of undertaking a specific course of action. • Life-threatening reaction isa reaction in which the patient was at risk of death at the time of the event and does not refer to an event, which hypothetically might have caused death if it was more severe. • Serious adverse drug reaction A noxious and unintended response to a drug that at any dose, may result in death, is life threatening (such as Stevens-Johnson Syndrome), requires patient hospitalization or prolongation of existing hospitalization, causes a congenital PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 63 anomaly or birth defect, results in persistent or significantly is ability or incapacity, or require intervention to prevent permanent impairment or damage. • Side effect any unintended effect of a pharmaceutical product occurring at doses used in man for medical treatment which is related to the pharmacological properties of the product and in which there is no deliberate overdose. • Unexpected adverse drug reaction An adverse reaction, the nature or severity of which is not mentioned in the summary of product characteristic or market authorization, or expected from characteristics • Biological products Medical products prepared from biological material of human, animal or microbiologic origin (such as blood products, vaccines, insulin). • Clinical trial A systematic study on pharmaceutical products in human subjects (including patients and other volunteers) in order to discover or verify the effects of and/or identify any adverse reaction to investigational products, and/or to study the absorption, distribution, metabolism and excretion of the products with the objective of ascertaining their efficacy and safety. • Efficacy The ability of a drug to produce the intended effect as determined by scientific methods, for example in pre-clinical research conditions (opposite of hazard). • Potency is a measure of drug activity expressed in terms of the amount required to produce an effect of given intensity. • Individual case safety reports (ICSR)A report that contains ‗information describing a suspected adverse drug reaction related to the administration of one or more medicinal products to an individual patient. • Post-marketing The stage when a drug is generally available on the market. • Pre-marketing The stage before a drug is available for prescription or sale to the public. • Poly-pharmacy The concomitant use of more than one drug, sometimes prescribed by different practitioners. • Placebo An inactive substance (often called a sugar pill) given to a group being studied to compare results with the effects of the active drug. • Cohort study – A study that identifies defined populations and follows them forward in time, examining their rates of disease. • National pharmacovigillance centre – A single, governmentally recognized centre within a country with the clinical and scientific expertise to collect, collate, analyse and give advice on all information related to drug safety. PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 64 STEP 4: Aims of Pharmacovigillance (15 minutes) Activity: Buzzing (5minutes) ASK students to pair up and buzz on the following questions for 5 minutes • What are the aims of pharmacovigillance? ALLOW few pairs to

NTA Level 6 • Semester 2 • PST06211 Tools for Monitoring and Evaluation of Medicine Monitoring and Evaluation of Medicine Use • Source Session/Topic 7 Full source-text version: all educational wording from the extracted learning source is retained; only presenter/tutor metadata and web-layout noise are removed, while formatting is improved for readability. Session 7: Tools for Monitoring and Evaluation of Medicine Use Total Session Time: 120 minutes Prerequisites • None Learning Tasks By the end of this session students are expected to be able to: • List tools used in monitoring and evaluation of medicine use • Describe tools used in monitoring and evaluation of medicines use Resources Needed: • Flip charts, marker pens, and masking tape • Black/white board and chalk/whiteboard markers • Computer and LCD projector • Handout 7.1. Description of tools used in monitoring and evaluation of medicines use SESSION OVERVIEW Activity/ Step Time Content Method 1 05 minutes Presentation Introduction, Learning Tasks 25 minutes Presentation Tools used in Monitoring and Evaluation of 2 Buzzing Medicine use 80 minutes Presentation Description of Tools used in Monitoring and 3 Small Group Evaluation of Medicines use Discussion 4 05 minutes Presentation Key Points 5 05 minutes Presentation Evaluation PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 54 SESSION CONTENTS STEP 1: Presentation of Session Title and Learning Tasks (5 minutes) READ or ASK students to read the learning task and clarify ASK students if they have any questions before continuing. STEP 2: Tools used in Monitoring and Evaluation of Medicine use (25 minutes) Activity: Buzzing (5 minutes) ASK students to pair up and buzz on the following question for 2 minutes • What are tools used in monitoring and evaluation of medicine use? ALLOW few pairs to respond and let other pairs to add on points not mentioned WRITE their response on the flip chart/board CLARIFY and SUMMARIZE by using the content below • The WHO Operational Package for Assessing, Monitoring and Evaluating Country Pharmaceutical Situations is intended as a useful tool for researchers, policy-makers, planners and others who need to use standardized measurement tools to gather data and other information. • The tools presented here have already been used for several years at global and country levels. • The listed below are tools used in monitoring and evaluation of medicine use as per WHO operational package for assessing and monitoring; o Level I questionnaire.The Level I questionnaire, which is sent to countries once every four years to update global pharmaceutical data, is included in the annexes. It can also serve as a rapid assessment and checklist for countries to check current the pharmaceutical structure and processes of their national pharmaceutical systems. o Level II survey forms. The annexes contain the technical descriptions of Level II facility indicators and the sampling process. Survey forms are included, and graphs and tables can be from the analysis template. o Medicine use indicators-prescribing indicator form o A Guide for Coordinators and Data Collectors of the Level II Facility Survey Checklist for data collectors PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 55 o Coordinator checklist o Copies of the National Medicines List. o Patient records and stock cards. • The above are necessary tools use to gather information that are brought together with WHO operational package. • A diskette that contains Level II facility survey forms, summary forms and training slides. STEP 3: Description of Tools used in Monitoring and Evaluation of Medicines use (80 minutes) Activity: Small Group Discussion ( 30 minutes) DIVIDE students into small manageable groups ASK students to discuss on the following question • What are the Tools used in Monitoring and Evaluation of Medicines use? ALLOW students to discuss for 20 minutes ALLOW few groups to present and the rest to add points not mentioned CLARIFY and SUMMARIZE by using the contents below • Level I questionnaire o The Level I questionnaire, which is sent to countries once every four years to update global pharmaceutical data, is included in the annexes. It can also serve as a rapid assessment and checklist for countries to check current the pharmaceutical structure and processes of their national pharmaceutical systems o Level I questionnaire is a questionnaire on theexisting infrastructures and key processes of each component of the pharmaceutical sector. The completion of the questionnaire can be accomplished in a relatively short time after identifying sourcesof accurate information. o The survey of Level II indicators is a very important part of monitoring the pharmaceutical sector because these indicators measure the outcome and impact of pharmaceutical programmes in a country. • Level II Survey Forms numbered 1—17; these are various forms used for gathering data in monitoring and evaluation of medicine use. o Adequate preparation is needed and data collectors must be trained. PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 56 o The forms are given below together with their indicators use to monitor and assess medicine use • Survey Forms 1-6 o in Public health facility pharmacies/dispensaries • Survey form 1 o percentage key medicines available o percentage medicines expired • Survey form 2 o Price of key medicine o Price of paediatric medicines • Survey form 3 o Affordability of treatment for adults and children under 5 years of age • Survey form 4 o Average stock out duration o Adequate record keeping • Survey form 5 o Adequate conservation conditions and handling of medicines in the storeroom and dispensing area • Survey form 6 o Average number of medicines per prescription o percentage medicines dispensed or administered o percentage medicines adequately labelled o percentage patients knowing how to take medicines o Average cost of medicines o Geographical accessibility of dispensing facilities • Survey Forms 7–9Gather information regardingRational medicine use – Prescribing indicator form • Indicators: o prescribed medicines on EML o percentage patients prescribed antibiotics/injections o percentage medicines prescribed by generic name o public health facilities

NTA Level 6 • Semester 2 • PST06211 Indicators in Monitoring and Evaluation Monitoring and Evaluation of Medicine Use • Source Session/Topic 6 Full source-text version: all educational wording from the extracted learning source is retained; only presenter/tutor metadata and web-layout noise are removed, while formatting is improved for readability. Session 6: Indicators in Monitoring and Evaluation Medicines Use Total Session Time: 120 minutes + 2 hours of Assignment Prerequisites • None Learning Tasks By the end of this session students are expected to be able to: • List indicators used in monitoring and evaluation of medicines use • Explain application of each indicator in monitoring and evaluation of medicines use Resources Needed: • Flip charts, marker pens, and masking tape • Black/white board and chalk/whiteboard markers • Computer and LCD Projector SESSION OVERVIEW Activity/ Step Time Content Method 1 05 minutes Presentation Introduction, Learning Tasks 35 minutes Indicators used in Monitoring and Evaluation Presentation 2 of Medicines use Buzzing 65 minutes Presentation Application of Each Indicator in Monitoring 3 Small Group and Evaluation of Medicines use Discussion 4 05 minutes Presentation Key Points 5 05 minutes Presentation Evaluation PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 47 SESSION CONTENTS STEP 1: Presentation of Session Title and Learning Tasks (5 minutes) READ or ASK students to read the learning tasks and clarify ASK students if they have any questions before continuing. STEP 2: Indicators used in Monitoring and Evaluation of Medicines use (35 minutes) Activity: Buzzing (10 minutes) ASK students to pair up and buzz on the following question for 5 minutes • What are indicators used in monitoring and evaluation of medicine use? ALLOW few pairs to respond and let other pairs add on points not mentioned WRITE their response on the flip chart/board CLARIFY and SUMMARIZE by using the content below • Medicine use indicators were developed to be used as measures of performances in three general areas related to the rational use of medicine in primary care o Pharmaceutical prescribing practices by health providers o Key elements of patient care covering both clinical consultation and pharmaceutical dispensing o Availability of facility specific factors which support rational use such as key essential drugs and minimum pharmaceutical information • The medicine use indicators would typically be measured within a defined geographic or administrative area either to describe medicine use at a given point in time or to monitor changes over time • Usually they tend to measure the rational use of medicine measured by facilities • The level II indicators are used to monitor and evaluate medicine uses and they can be group into the following aspects as from the WHO operational package ; o Prescribing Indicators  Average number of medicines per encounter  Percentage of medicines prescribed by generic name  Percentage of encounters with an antibiotic prescribed  Percentage of encounters with an injection prescribed  Percentage of medicines prescribed which are from the essential medicines list or formulary list PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 48 o Patient Care Indicators  Average consultation time  Average dispensing times  Percentage of medicines actually dispensed  Percentage of medicines that are adequately labeled  Percentage of patients who know how to take their medicines o Health Facility Indicators  Availability of essential medicine list or formulary  Availability of key set of indicator medicines  Availability of standard treatment guideline (STG)  The above indicators are the minimum set of measures to be calculated during a single medicine use indicators survey o Complementary Indicators (this have less standardization and less experience in actual use  Percentage of patients treated without medicines  Average medicine costs per encounter  Percentage of medicine cost spent on antibiotics  Percentage of medicine cost spent on injections  Percentage of prescriptions in accordance with STG  Percentage of patients satisfied with care provided  Percentage of facilities with access to impartial information STEP 3: Application of Each Indicator in Monitoring and Evaluation of Medicines Use (65 minutes) Activity: Small Group Discussion ( 30 minutes) DIVIDE students into small manageable groups ASK students to discuss on the following question • What are applications of medicine use indicators? ALLOW students to discuss for 15 minutes ALLOW few groups to present and the rest to add points not mentioned CLARIFY and SUMMARIZE by using the contents below PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 49 Below is the description of application of each indicator for motoring and evaluation of medicine use; • Prescribing Indicators o The indicators of prescribing practices measure the performance of healthcare providers in several key dimensions related to appropriate use of medicines  Average number of medicines per encounter  Purpose-To determine the prevalence of polypharmacy, which is one measure of unnecessary prescribing. o Percentage of medicines prescribed by generic name  Purpose -To measure the degree to which prescribing practice conforms to the principles of generic prescribing. o Percentage of encounters with an antibiotic prescribed  Purpose -To determine the prevalence of antibiotic prescribing, since over- prescribing of antibiotics is one common type of inappropriate medicine use. o Percentage of encounters with an injection prescribed  Purpose -To determine the prevalence of injection use, since over-prescribing of injections is one common type of inappropriate medicine use. o Percentage of medicines prescribed which are from the essential medicines list or formulary list  Purpose -To measure the degree to which prescribing practice conforms to the national essential medicines list (EML). The essential medicines concept is one of the main strategies being promoted in medicines policy. More and more countries are formulating national EMLs. For most countries, this should be the basis for all public medicines procurement and prescribing. • Patient Care Indicators o Patient care indicators address key aspects of what patients experience at health facilities and how well they have been prepared to

NTA Level 6 • Semester 2 • PST06211 Factors Hindering Monitoring and Evaluation of Medicines Use Monitoring and Evaluation of Medicine Use • Source Session/Topic 5 Full source-text version: all educational wording from the extracted learning source is retained; only presenter/tutor metadata and web-layout noise are removed, while formatting is improved for readability. Session 5: Factors Hindering Monitoring and Evaluation of Medicines Use Total Session Time: 60 minutes Prerequisites • None Learning Tasks By the end of this session students are expected to be able to: • List factors hindering monitoring and evaluation of medicines use • Explain factors hindering monitoring and evaluation of medicines use • Identify measures to improve monitoring and evaluation of medicines uses Resources Needed: • Flip charts, marker pens, and masking tape • Black/white board and chalk/whiteboard markers • Computer and LCD projector SESSION OVERVIEW Activity/ Step Time Content Method 1 05 minutes Presentation Introduction, Learning Tasks 15 minutes Factors Hindering Monitoring and Evaluation Presentation 2 of Medicines Use Buzzing 15 minutes Explanation on Factors Hindering Monitoring 3 Presentation and Evaluation of Medicines Use 15 minutes Presentation Measures to Improve Monitoring and 4 Brainstorming Evaluation of Medicines Uses 5 05 minutes Presentation Key Points 6 05 minutes Presentation Evaluation PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 42 SESSION CONTENTS STEP 1: Presentation of Session Title and Learning Tasks (5 minutes) READ or ASK students to read the learning tasks and clarify ASK students if they have any questions before continuing. STEP 2: Factors Hindering Monitoring and Evaluation of Medicines Use (15 minutes) Activity: Buzzing (5minutes) ASK students to pair up and buzz on the following question for 5 minutes • What are the factors hinder monitoring and evaluation of medicine uses? ALLOW few pairs to respond and let other pairs add on points not mentioned WRITE their response on the flip chart/board CLARIFY and SUMMARIZE by using the content below • Poor record keeping, availability and accessibility of medical records • Unsustainable system of regular monitoring and evaluation of medicine use • There is limited advocacy for culture of monitoring and evaluation of medicine use • Economic factors • Patient factors such as a delay of patient to be included in the sampling frame • Inadequate number of human resource • Lack of expertise • Most monitoring tools include indicators that are difficult to collect data especially if done regularly • Limited resources that are not consistently allocated STEP 3: Factors hindering Monitoring and Evaluation of Medicines Use (15 minutes) • Poor record keeping, availability and accessibility of medical records. Difficulties in assessing the patient‘s medical files it becomes a challenge in obtaining data during monitoring and evaluation of medicine use although others might be available but accessibility becomes also a challenge. • Unsustainable system of regular monitoring and evaluation of medicine use. Many established systems for monitoring and evaluation of medicine use they do not sustain for long time therefore it becomesdifficult to establish it again. PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 43 • There is limited advocacy for culture of monitoring and evaluation of medicine use. Many facilities have no tendency or behaviour of carrying out monitoring and evaluation of medicine use this lack support from facilities, pharmacies and the government itself have no tendency of promoting monitoring and evaluation of medicine use • Economic factor. Economy stability to support the monitoring and evaluation of medicine use also have become unreliable due to inadequate funds to take people to field for data collection • Patient factors. Delay of patient to be included in the sampling frame. Delay of patients to participate in interview during data collection on the field is ain obtaining information from some medicine use indicators in monitoring and evaluation of medicine use. • Inadequate number of human resource. Lack of expertise and adequate number of personnel to perform monitoring and evaluation of medicine use • Most monitoring tools include indicators that are difficult to collect data especially if done regularly. • Limited resources that are not consistently allocated. The allocation of resource for conducting monitoring and evaluation of medicine use are not consistently allocated which becomes difficult in conducting the assessments and monitoring of medicine uses. STEP 4: Measures to Improve Monitoring and Evaluation of Medicines Uses (15 minutes) Activity: Brainstorming (5 minutes) Ask students to brainstorm on the following question: • What are the measures to improve monitoring and evaluation of medicine uses? ALLOW few students to respond? WRITE their responses on the flip chart/ board CLARIFY and SUMMARISE by using the content below • Establishment of sustainable system of monitoring and evaluation • There should be a clear communication of plans and targets • To promote advocacy on culture of monitoring and evaluation of medicine use • Training personnel on importance of monitoring and evaluation of medicine use well as how to carry it • To put a clear availability and accessibility of medical record this will enable expertise to carry the task easily. PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 44 • To allocate resources consistently • To increase more number of human resource to coordinate the activity. STEP 6: Key Points(5 minutes) • Factor hindering monitoring and evaluation of medicine uses includes; lackof expertise, economic factors, patient factors and insufficient human resource. • Measures to improve monitoring evaluation of medicine uses is by allocation of more resources consistently, making availability and accessibility of medical records STEP 7: Evaluation (5 minutes) • What are factors hindering monitoring and evaluation of medicine uses? • What are measures to improve monitoring and evaluation of medicine uses? PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 45 References Quick, J. D. (1997). Managing drug supply the selection, procurement, distribution and use of pharmaceuticals. West Hartford, CT: KumarianPress.Bartsch, S. (2007). The Global Fund to Fight AIDS, Tuberculosis and Malaria.

NTA Level 6 • Semester 2 • PST06211 Performance Indicators for Monitoring and Evaluation Monitoring and Evaluation of Medicine Use • Source Session/Topic 4 Full source-text version: all educational wording from the extracted learning source is retained; only presenter/tutor metadata and web-layout noise are removed, while formatting is improved for readability. Session 4: Performance Indicators for Monitoring and Evaluation Total Session Time: 120 minutes + 2 hours Assignment Prerequisites • None Learning Tasks By the end of this session students are expected to be able to: • List performance indicators in monitoring and evaluation of different pharmaceutical situations • Explain each performance indicators in monitoring and evaluation of different pharmaceutical situations Resources Needed: • Flip charts, marker pens, and masking tape • Black/white board and chalk/whiteboard markers • Pointer • Computer and LCD projector • Handout 4.1:Summary list of performance indicators SESSION OVERVIEW Activity/ Step Time Content Method 1 05 minutes Presentation Introduction, Learning Tasks 20 minutes Performance Indicators in Monitoring and Presentation 2 Evaluation of Pharmaceutical Situations Buzzing 80 minutes Explanation on Performance Indicators in Presentation Small 3 Monitoring and Evaluation of Pharmaceutical group discussion Situations 4 10 minutes Presentation Key Points 5 05 minutes Presentation Evaluation PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 31 SESSION CONTENTS STEP 1: Presentation of Session Title and Learning Tasks (5 minutes) READ or ASK students to read the learning tasks and clarify ASK students if they have any questions before continuing. STEP 2: Performance Indicators in Monitoring and Evaluation of PharmaceuticalSituations (20 minutes) Activity: Buzzing (5 minutes) ASK students to pair up and buzz on the following question for 2 minutes • What are performance indicators in monitoring and evaluation of different pharmaceutical situations? ALLOW few pairs to respond and let other pairs to add on points not mentioned WRITE their response on the flip chart/board CLARIFY and SUMMARIZE by using the content below • The WHO process for pharmaceutical monitoring and assessment uses a hierarchical approach with three groups of indicators: Level I, Level II and Level III. This provides a standard methodology to follow progress over time and to compare situations in different facilities, districts and countries • Level of core indicators are shown on diagram below PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 32 Level II indicators (outcome) Level III • WHO NDP indicators • Indicators for specific pharm • How to investigate drug • Assessing regulatory ca Level I • TRIPS, Drug pricing, Tra indicators (structure& process) • Level I indicators provide a rapid means of obtaining information on the existing infrastructure and key processes of each component of the pharmaceutical sector. • Level II health facility indicators provide systematic data to measure outcomes on access (affordability and availability of key medicines and geographical accessibility of dispensing facilities) and rational use of quality medicines, including some indication of the quality of medicines at health facilities and pharmacies. • Level III indicators are a more detailed and expanded list of indicators covering key components and areas such as those elaborated in several indicator documents in medicine pricing, medicine supply management, HIV/AIDS, TRIPS, rational drug use (RDU) and regulatory capacity assessment. STEP 3: Performance Indicators in Monitoring and Evaluation of Pharmaceutical Situations (80 minutes) PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 33 Activity: Small Group Discussion ( 30 minutes) DIVIDE students into small manageable groups ASK students to discuss on the following question • What components are in level I indicators? ALLOW students to discuss for 15 minutes ALLOW few groups to present and the rest to add points not mentioned CLARIFY and SUMMARIZE by using the contents below • Level I core indicators o The Level I core indicators are used to assess existing structures and processes of national pharmaceutical system. They provide a method to rapidly assess the implementation of NMPs and their components. o Indicators, a knowledgeable informant can coordinate the gathering of information. Most data will be available within the Ministry of Health, although data on intellectual property rights protection may require consultation with the responsible ministry. Data collection does not require field surveys and information can easily be updated periodically, for example every two years. o The questionnaire on Level I indicators is included as Annex of the WHO operational package for assessing and monitoring pharmaceutical situations. The indicators in this questionnaire are summarized below. • Pharmaceutical components in Level I indicators o National Medicines Policy (NMP)- An NMP document that covers the public and private sectors, a written implementation plan and the integration of medicine and health policies provide a basic framework to organize and improve the pharmaceutical system.  They also assist in coordinating the functions and strategies of each component as they are being implemented. Regular monitoring helps to inform the NMP and its implementation. o Regulatory system-Regulations on medicine manufacturing, promotion and advertising, sales, distribution, dispensing and prescribing must be in place. Legislation directed at generic prescribing, dispensing and substitution can help increase access to essential medicines in both the private and public sectors o Medicines supply system- Access and availability of essential medicines, especially at public sector facilities, are affected by how medicines are purchased and distributed and how medicines are managed in the health system. PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 34 o Medicines financing- Access and availability are also affected by how much money the government can allocate to medicines, pricing policies, financing schemes (such as insurance programmes and user fees) and medicine donations. o Production and trade – Activities ranging from repackaging to formulation of products to developing new medicines are important in assessing the pharmaceutical sector. Implementing TRIPS flexibilities in public health can increase access to medicines. o Rational use of medicines – Medicines policies can often have greater impact with effective use of strategies to improve the prescribing and dispensing practices

NTA Level 6 • Semester 2 • PST06211 Assessing Monitoring and Evaluation of PharmaceuticalsServices Monitoring and Evaluation of Medicine Use • Source Session/Topic 3 Full source-text version: all educational wording from the extracted learning source is retained; only presenter/tutor metadata and web-layout noise are removed, while formatting is improved for readability. Session 3: Assessing Monitoring and Evaluation of PharmaceuticalsServices Total Session Time: 120 minutes + 2 hours of Assignment Prerequisites • None Learning Tasks By the end of this session students are expected to be able to: • List components of WHO operational package for assessing monitoring and evaluation of country pharmaceutical situations • Explain each component of WHO operational package for assessing monitoring and evaluation of country pharmaceutical situations Resources Needed: • Flip charts, marker pens, and masking tape • Black/white board and chalk/whiteboard markers • Computer and LCD projector • Handout 3.1 :Components of the WHO Operational Package for Assessing, Monitoring and Evaluation CountryPharmaceutical Situation SESSION OVERVIEW Activity/ Step Time Content Method 1 05 minutes Presentation Introduction, Learning Tasks 20 minutes List of Components of WHO Operational Presentation 2 Package for Assessing Monitoring and Buzzing Evaluation of Pharmaceutical 85 minutes Presentation Explanation of Component of WHO 3 Small Group Operational Package for Assessing Monitoring Discussion and Evaluation of Pharmaceutical 4 05 minutes Presentation Key Points 5 05 minutes Presentation Evaluation PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 19 SESSION CONTENTS STEP 1: Presentation of Session Title and Learning Tasks (5 minutes) READ or ASK students to read the learning tasks and clarify ASK students if they have any questions before continuing. STEP 2: Components of WHO Operational Package for Assessing Monitoring and Evaluation of Pharmaceutical (20 minutes) Activity: Buzzing (5 minutes) ASK students to pair up and buzz on the following question for 2 minutes • What are the components of WHO operational package for assessing monitoring and evaluation of pharmaceutical situations? ALLOW few pairs to respond and let other pairs to add on points not mentioned WRITE their response on the flip chart/board CLARIFY and SUMMARIZE by using the content below • The WHO Operational Package for Assessing, Monitoring and Evaluating Country Pharmaceutical Situations is intended as a useful tool for researchers, policy-makers, planners and others who need to use standardized measurement tools to gather data and other information. • The WHO process for pharmaceutical monitoring and assessment uses a hierarchical approach with three groups of indicators. • The components of the WHO Operational Package for Assessing, Monitoring and Evaluationof CountryPharmaceutical Situations include the following; o Pharmaceutical indicators for monitoring and assessment  Level I core indicators.  Level II facility-based core indicators  Level III indicators o Preparing the survey (Level II – facility survey) o Training data collectors o The survey o Data processing, analysis and reporting PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 20 STEP 3: Explanation on Each Component of WHO Package for Assessing Monitoring and Evaluation of Pharmaceutical(85minutes) Activity: Small group discussion (30 minutes) ASK students to form small manageable group and discuss the following questions • What are pharmaceutical indicator for monitoring and assessment? ALLOW few pairs to respond and let other pairs to add on points not mentioned WRITE their response on the flip chart/board CLARIFY and SUMMARIZE by using the content below • Pharmaceutical indicators for monitoring and assessment o Given the complexity of the pharmaceutical sector, a systematic method of gathering data is very important for assessing access, quality and rational use of medicines. o There are multiple, cross-cutting factors that can influence access and rational use of quality medicines, and a variety of strategies that countries can adopt and implement to improve their pharmaceutical situations. o Indicators have been developed for monitoring national medicines policies (NMPs) that enable systematic assessment, evaluation and monitoring of the formulation and implementation of pharmaceutical policies and programmes. o These can be used to:  Assess country capacity, such as available infrastructure, logistics and human resources to support the pharmaceutical sector and implement NMPs;  monitor the implementation of NMPs;  measure the impact of implementation strategies; and  Evaluate progress towards identified objectives. o All stakeholders in the pharmaceutical sector can use indicator-based assessment of the pharmaceutical situation to inform priorities and set targets. o Indicators provide policy-makers and managers with a clear picture of national and institutional problems. Policy-makers and managers can refer to study results when developing strategies to strengthen the pharmaceutical sector o Information and data can be used to:  Formulate or revise the NMP.  Assess the quality of governance, such as systems and mechanisms that would safeguard against vulnerability to corruption. PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 21  Encourage and facilitate cooperation between the different main players in a o Level I indicators provide a rapid means of obtaining information on the existing infrastructure and key processes of each component of the pharmaceutical sector.  These indicators are assessed using a short questionnaire completed at the national level. o Level IIindicators  Provide systematic data on access and rational use of quality medicines through facility-based surveys.  Systematic survey processes to collect data on these indicators are contained in WHO package.  A population-based survey has been developed as part of the current process and is discussed in a separate document, Manual for the Household Survey to Measure Access and Use of Medicines. o Preparing the survey (Level II – facility survey)  The survey of Level II indicators is a very important part of monitoring the pharmaceutical sector because these indicators measure the outcome and impact of pharmaceutical programmes in a country.  Adequate preparation is needed and data collectors must be trained. o Coordination and survey coordinator  A national coordinator should be selected to take charge of the overall coordination of the Level II survey, to oversee the survey process, data analysis, reporting and presentation

NTA Level 6 • Semester 2 • PST06211 Introduction to Pharmaceutical Assessment and Monitoring Monitoring and Evaluation of Medicine Use • Source Session/Topic 2 Full source-text version: all educational wording from the extracted learning source is retained; only presenter/tutor metadata and web-layout noise are removed, while formatting is improved for readability. Session 2: Introduction to Pharmaceutical Assessment and Monitoring Total Session Time: 60 minutes Prerequisites • None Learning Tasks By the end of this session students are expected to be able to: • List the aims of pharmaceutical assessment and monitoring • Give reasons for monitoring and assessment of pharmaceutical services • Explain the use of assessment and monitoring results • List importance of using indicators in monitoring and evaluation of pharmaceuticals Resources Needed: • Flip charts, marker pens, and masking tape • Black/white board and chalk/whiteboard markers • Computer and LCD Projector • Handout 2.1:Pharmaceutical assessment and monitoring SESSION OVERVIEW Activity/ Step Time Content Method 1 05 minutes Presentation Introduction, Learning Tasks 10 minutes Presentation Aims of Pharmaceutical Assessment and 2 Buzzing Monitoring 10 minutes Presentation Reasons for Monitoring and Assessment of 3 Brainstorming PharmaceuticalServices 15 minutes Presentation 4 Usage of Assessment and Monitoring Results Buzzing 10 minutes Presentation Importance of Indicators in Monitoring and 5 Evaluation of Pharmaceuticals services 6 05 minutes Presentation Key Points 7 05 minutes Presentation Evaluation PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 11 SESSION CONTENTS STEP 1: Presentation of Session Title and Learning Tasks (5 minutes) READ or ASK students to read the learning tasks and clarify ASK students if they have any questions before continuing. STEP 2: Aims of Pharmaceutical assessment and Monitoring (10 minutes) Activity: Buzzing (5 minutes) ASK students to pair up and buzz on the following question for 2 minutes • What are aims of pharmaceutical assessment and monitoring? ALLOW few pairs to respond and let other pairs to add on points not mentioned WRITE their response on the flip chart/board CLARIFY and SUMMARIZE by using the content below • The following are aims of pharmaceutical assessment and monitoring; o Monitoring and assessing the pharmaceutical sector are vital to determine if the key pharmaceutical objectives are met. o Evaluate accessibility of essential medicines to people. o Help to assess the safety, efficacy and quality of medicines o Monitoring of rational use of medicines. STEP 3: Reasons for Monitoring and Assessment of Pharmaceuticals Services (10 minutes) Activity: Brainstorming (5 minutes) Ask students to brainstorm on the following question: • What Reasons for Monitoring and Assessment of Pharmaceuticals Services? ALLOW few students to respond? WRITE their responses on the flip chart/ board CLARIFY and SUMMARISE by using the content below PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 12 • There are various reasons why we monitor and assess pharmaceutical situations these includes; o To assess country capacity in infrastructures, resources and human resources to support the pharmaceutical sector and implement NMPs; o To review implementation strategies so adjustments can be made. Such as implementation of national medicine policy o To measure outcome of pharmaceutical objectives in terms of access and rational use of quality medicines. o To evaluate progress towards identified objectives STEP 4: Usage of Assessment and Monitoring Results (15 minutes) Activity: Buzzing (5minutes) ASK students to pair up and buzz on the following question for 5 minutes • What are usage of assessment and monitoring pharmaceuticals results? ALLOW few pairs to respond and let other pairs add on points not mentioned WRITE their response on the flip chart/board CLARIFY and SUMMARIZE by using the content below • Various results from monitoring and assessment of pharmaceuticals situations can be used by different stakeholders ; • Countries o To focus action, prioritize, measure achievement • National policy-makers o To synchronize health and economic policies o To get a clear picture of national problems and identify and prioritize strategies o To present the performance of the pharmaceutical sector to donors and other governmental agencies • Health facilities to be aware of the institutional problems and identify strategies to improve the situation • International agencies o To assess the structure and capability of countries when developing new projects PST 06211 Monitoring and Evaluation of Medicines Use NTA Level 6 Semester 2 Facilitator Guide 13 o To assess the progress and accomplishment of current projects o To assess the impact of aid • Professional groups, NGOs and academia o To focus their activities on advocacy activities and information campaigns Refer students to Handout 2.1: Usage of Assessment and Monitoring Results STEP 5:Importance of Indicators in Monitoring and Evaluation of Pharmaceuticals Services (10 minutes) • The following are importance‘s of using indicators in monitoring and evaluation of pharmaceutical services o It helps in comparing the performance of facilities, districts, regions and the overall situation in the country. By using indicators we can compare situations such the availability and accessibility of essential medicines between one country and another, one region and another and districts as well. o It helps in seeing trends over time. Helps to identify the rise and fall in availability of essential medicine in particular time and specifically the demands of the health facilities o It helps in setting targets. It helps set goals and guiding policy-makers in developing and planning national strategy, including budget planning, reallocation of funding and human resource management. Which are key cross-cutting factors that can influence access to medicines STEP 6: Key Points (5 minutes) • Various results from monitoring and assessment of pharmaceuticals situations can be use by; countries, international agencies,national policy makers, health facilities, Professional groups, NGOs and academia • Monitoring and evaluation of pharmaceuticals by using indicators facilitates; seeing trends overtime, setting targets and comparing performance of facilities. • Key objectives in monitoring and assessment of pharmaceuticals are to ensure people have access to essential medicines, quality medicines, and use of medicines rationally. STEP 7: Evaluation (5 minutes) • Who can use the results from assessment and monitoring