Overview of webinar topics : Overview of webinar topics Need for underlying logic in project planning
Characteristics of indicators
Choosing and improving indicators
Underlying project logic : Underlying project logic The ‘results chain’ demonstrates a clear relationship between the project goal, strategies (for anticipated outcomes) and supporting activities
We have a clear idea about the change process(es) anticipated
Activities need to be sufficient to bring about the outcomes (but may not be the only influence on the outcomes we are seeking)
Potential problems with project logic : Potential problems with project logic Anticipated results not logically linked to activities
Results not sufficiently specific
Results are composites of several results
Outcomes don’t express change
Results statements are too wordy
Confusion between levels of results (outputs, short- and medium-term outcomes)
Indicators
Not logically linked to the result
Not measurable
Are new results
Refining results… : Refining results… To strengthen the capacity of civil servants to do X by undertaking Y,… The capacity of civil servants is strengthened to do X by undertaking Y,… The capacity of civil servants in the
4 poorest districts is strengthened to
do X by undertaking Y,… The capacity of civil servants in the 4 poorest districts is strengthened to do X
by undertaking Y,… Civil servants in the 4 poorest districts are better able to X Let’s use results language to emphasis the future condition we want to achieve. All civil servants, everywhere? Can you be more specific? Are there particularly weak or under-resourced civil servants we should emphasise? We can take out information that relates to either strategy or activities. Now, let’s try bringing the subject of change to the front, and shifting from passive to active language.
What is an indicator? : What is an indicator? A tool to measure evidence of results: accomplishments, changes made, or progress achieved
Slide 6 : Indicators
Indicators describe how the intended results will be measured - accountability
Objectively verifiable, repeatable measures of a particular condition
They force clarification of what is meant by the result …….the fine print!
Often accompanied by baselines and targets
Slide 7 : Examples: Quantitative indicators
1. Quantity of rice produced
2. Comparative (women and men) rate of employment in the public sector
3. Proportion of children aged five or under who are fully immunized
Slide 8 : 1. Level of satisfaction of the beneficiaries
2. Quality of the service provided
3. Perception of men on women’s participation in the local Committee
4. Attitudes towards condom use among sexually active adolescents
Often these are quantifiable. Examples: Qualitative indicators
Considerations : Considerations Indicators do not exist in a vacuum – they should always be related to results
Need a balance of quantitative and qualitative
Some results are more suitable for indicators than others
It takes time to get indicators right
Performance Indicator Selection Criteria : Performance Indicator Selection Criteria Validity - Does it measure the result?
Reliability - Is it a consistent measure over time and, if supplied externally, will it continue to be available?
Sensitivity - When a change occurs will it be sensitive to those changes?
Simplicity - Will it be easy to collect and analyze the information?
Utility - Will the information be useful for decision-making and learning?
Affordable – Do we have the resources to collect the information?
Baseline, Target and Achievement : Baseline, Target and Achievement Base-
line Commitment Current
Level of
Achievement Actual
Achieve
ment Performance Achievement
At end of period Target Planned
Level of
Achievement
Outputs: Definition & key features : Outputs are deliverables
Operational changes: new skills or abilities, the availability of new products and services
Must be achieved within the programme period
Managers have a high degree of control
If the result is mostly beyond the control or influence of the programme or project, or carried out by others, it cannot be an output
Failure to deliver is failure of the programme or project Outputs: Definition & key features
Slide 13 : Measuring Outputs
Easier to measure than outcomes (tangible!)
Indicators or MOV usually coming from existing service data or from routine progress reports
Indicators will often be ‘yes-no’
1 to 4 indicators enough - the fewer the better to reduce costs
Promises!
(…and a critical insurance policy with donors)
Output Indicators, Baseline, Target and Sources of Data : Output Indicators, Baseline, Target and Sources of Data The Coordination Body for Service Delivery is able to design and implement environmental management initiatives Indicator:
- National SD strategy working papers (y/n)
- Compliance monitoring reports (y/n) Source of Data:
Draft papers
Monitoring reports Communities participate in the development of Environmental Action Plans (EAPs) Indicator:
- # Community consultations
- # Community EAPs
Baseline: 0
Target: 20 per year Source of Data:
Community-based service delivery (CBSD) Consultation reports
Community EAPs
Outcomes: Definition & key features : Represent changes in institutional performance or behaviours
3 to 5 suggested, >5 Ok, based on country context and
High-level outcomes produced by the combined effects of the lower level outcomes Outcomes: Definition & key features
Examples : Examples RBM Technical Briefs, 2007.
Outcome Indicators, Baseline, Target and Sources of Data : Outcome Indicators, Baseline, Target and Sources of Data Output:
800 teachers in Belem Province can deliver the new curriculum effectively Outcome:
By 2007, more girls in Belem Province enjoy a quality, basic education Indicator:
Net enrolment ratio
(M; F)
Baseline: F:45%
Target: F: 75%
- Improvement in school test scores Source of Data
School enrollment data
Annual school test scores report Indicator:
# Teachers with new certification
Baseline: 0
Target: 800
Teacher proficiency reports
Improvement of school satisfaction ratings Source of Data:
Teacher certification data (MoE)
Teacher evaluations reports (school-based)
School satisfaction surveys
Outcome Indicators, Baseline, Target and Source of Data : Outcome Indicators, Baseline, Target and Source of Data Outcome:
By 2007, a national youth policy adopted Indicator:
Policy adopted (yes/no)
Baseline: na
Target: na Source of Data
- Parliamentary reports
Typical pitfalls of indicator construction : Typical pitfalls of indicator construction Wordiness
Unclear logic and confusing
Overly ambitious
Passive voice and wishy-washy wording (i.e., Support provided to improve, including but not limited to,...)
Typical pitfalls : Typical pitfalls Wordiness
“To promote equitable economic development and democratic governance in accordance with international norms by strengthening national capacities at all levels and empowering citizens and increasing their participation in decision-making processes”
Too ambitious
“Strengthened rule of law, equal access to justice and the promotion of rights”
Containing multiple results
“The state improves its delivery of services and its protection of rights—with the involvement of civil society and in compliance with its international commitments “
Typical pitfalls : Typical pitfalls Wishy-washy
Support to institutional capacity building for improved governance
So general, they could mean anything
“To promote sustainable development and increase capacity at municipal level”
Confusing means and ends
“Strengthen the protection of natural resources through the creation of an enabling environment that promotes sound resources management”
Wordiness : Wordiness
Unclear logic and confused indicators : Unclear logic and confused indicators
Wording (subject hidden & passive voice) and mixed indicators : Wording (subject hidden & passive voice) and mixed indicators