Public Policy Monitoring and Evaluation Training (27th – 30th October 2015)

Conducted by: Mata’afa Dr. Desmond Amosa, Regional Advisor, Pacific Island Center of Public Administration (PICPA)

Policy Monitoring and Evaluation in the main, is about accountability and learning. The workshop in essence was a step by step guide on how to evaluate a policy within a government ministry for accountability and learning purposes.

Some common problems with evaluation processes across the Region were identified at the very beginning of the workshop, which are:

  1. Rarely any practice of policy evaluation with respect to policy outcomes (effects/benefits)
  2. Most focus of the evaluation process is on activities and outputs
  3. Where it is done – it is poorly conducted
  • Emphasis was made on the failure to evaluate outcomes of each policy (its effects/benefits to society)
  • Distinction between output and outcome:

      Output: represents the amount of work done, are the immediate results of activities, are products and services, and reflect a change in the supply of these products and services e.g. number of schools built, number of new curricula adopted…

      Outcome: represents the use or application of outputs. Outcomes reflect a program’s benefits e.g. improved learning among students….

The 4 days period training was a step by step guidance of the evaluation processes of a Policy which are as follow:

Policy Evaluation Steps

  1. Construct a Theory of Change
  • Theory of Change is a knowledge – based diagram of how an intervention intends to achieve results e.g. If we do X, we will get Y
  1. Transforming the theory of Change to a Logic Model
  • A Logic Model links planned work (inputs and activities) with intended results (outputs, outcomes and impacts), hence a Logic Model illustrates a policy’s design and specifies a theory of change
  • Constructing Logic Models

Impacts         Outcomes         Outputs          Activities         Inputs

      Start with the Desired Impacts and work back to define process through which they will be achieved

  1. Formulating Target Indicators
  • Logic Models identify the results we want to achieve and the means for doing so, Indicators, in contrast, tell us what to measure to determine whether we are achieving these results
  • Indicators of Policy Outcomes should answer 2 main questions
  1. How would we know success or achievement when we see it?
  2. Are we moving towards achieving our desired outcome?
  1. Formulating Evaluation Questions
  • When we do evaluation, we ask questions on what we want to know – hence the need for evaluation questions e.g. what is the total cost of resources needed for activities? What are the benefits that will be derived from the output?
  1. Decide on the Evaluation Design
  • Evaluation Design represents the overall strategy/plan of how to collect and analyze data to answer evaluation questions
  1. Select Information Collection Tools
  • Choice of which Data Collection Approach to use depends on the situation. Each technique is more appropriate in some situations than others, however, all techniques are subject to bias e.g. surveys and interviews
  1. Conduct Data Analysis
    1. Qualitative Analysis

      Best used for in – depth understanding of the intervention

  1. Quantitative Analysis

      Are numerical and analyzed with statistics

  1. Presenting the Findings
  • Analyzing and Reporting Performance findings are a critical step. It determines what is reported, when it is reported and to whom it is reported
  • Findings are mainly used for the purpose of: improving performance, accountability and learning

 “One of the great mistakes is to judge policies and programs by their intentions rather than their results” – Milton Friedman

workshop attended : Ms Maile Tu'itavuki and Mr Pesalili Latu 

Go to top