Commissioning Guide 7
How to commission EDI evaluation and impact measurement
Evaluation assesses whether an EDI intervention, programme or strategy is achieving its intended outcomes.
1. What is this service?
Impact measurement goes further by examining whether meaningful change is occurring and what contribution the intervention may have made.
2. When might you commission it?
- evaluating an EDI strategy
- evaluating training
- assessing a new programme
- measuring organisational change
- reporting to funders or boards
- deciding whether to continue an intervention
- understanding what is working.
3. Before you commission
Ask:
- What are we trying to change?
- What is the baseline?
- What can realistically change?
- What evidence already exists?
- What decisions will the evaluation inform?
- When should change be measured?
- What cannot realistically be attributed to the intervention?
4. Define your brief
Specify:
- intervention
- theory of change, if available
- outcomes
- indicators
- baseline
- population
- timeframe
- data sources
- evaluation questions
- reporting requirements.
5. What to ask providers
- What methodology would you recommend?
- How will you establish a baseline?
- How will you measure outcomes rather than activities?
- How will you deal with attribution?
- What data will you need?
- How will you assess qualitative change?
- How will you communicate uncertainty?
- How will findings inform future decisions?
6. How to compare proposals
Assess:
- evaluation design
- methodological credibility
- appropriateness of measures
- data capability
- qualitative capability
- understanding of EDI context
- reporting
- practicality.
7. What good looks like
- Activity: What did we do?
- Output: What did we produce?
- Outcome: What changed?
- Impact: What meaningful difference resulted?
8. Things to watch out for
- measuring attendance as impact
- arbitrary targets
- no baseline
- no comparison over time
- claiming causation without evidence
- ignoring negative or unexpected findings
- evaluation designed after the intervention has ended.
9. What should you receive?
- evaluation framework
- baseline
- methodology
- data collection tools
- findings
- outcome analysis
- limitations
- recommendations
- impact report.
10. After commissioning
Agree:
- who monitors indicators
- when data is reviewed
- what triggers action
- when the evaluation is repeated.