The notion of utilization-focused evaluation is that we do not wait to the end of an evaluation to think about how we will use it, rather we design and manage the whole evaluation process with its intended use by intended users in mind. Use can mean different things, however. The obvious kind of use is to improve the programme or project, but, an evaluation can have other kinds of uses.
For example, it might change ideas and understanding about the issue, change relationships among stakeholders, empower communities, change processes of decision making, and provide justification for political action (or inaction). Some of these uses the evaluation manager can, and should try to influence, but one also needs to recognize that once the report is made public, it may take on a life of its own, and be used by others in ways not anticipated.
The evaluation manager needs to facilitate use and impact. This can be accomplished by revisiting initial evaluation materials, the stakeholder analysis, the ToR and reference group discussions to reassess the use and users that were initially developed: are they still valid? what is needed to take action on the evaluation? what are the constraints to using this evaluation?
Use should be considered in the context of the programme being evaluated (addressing recommendations) and in the context of broader learning (lessons learned) for the organisation and others undertaking GE & HR based work/intervention and evaluations. This means considering how evaluations can inform the design of new programmes and future operational and strategic planning.
Some common risks to utilization and mitigation approaches are:
- A lack of consensus on the recommendations and required action – share the results and engage stakeholders early, and make sure that the consultation process is not rushed.
- The report is “deniable” or those who don’t like the results attack the process – ensure the methodology was rigorous and defensible and report recommendations are well substantiated.
- The report is disowned by the client or other stakeholders – try to get a good understanding of why this is happening. Engage the reference and management groups in committing to action.
- Dissemination is minimal: there are no funds, interest or time for dissemination – plan and budget for communicating results at the start of the evaluation process; be creative in your dissemination strategy, stakeholders may be able to assist with communication at minimal cost.
- There is no follow-up process – in developing the management response to the evaluation consider how and when commitments on action can be fulfilled and by whom. Build follow up on these commitments into ongoing programme monitoring and business planning processes.
A particular challenge in GE/HR programmes is that when people do not see immediate impact, they assume the programme is not working. Given the many structural, economic, social and cultural factors involved in GE/HR programming, modification of people’s behavior and practices may take a long time to occur or to become evident. It is therefore difficult to know if the evaluation results were not being used and have had no effect, or if they have been used, but change simply takes a long time. Change may be incremental, evaluation is one step!
Tip: Good use of evaluation results is more than action by the manager to respond to recommendations. It is aboout engaging with stakeholders to implement change.
