A Manager's Guide to Gender Equality and Human Rights Responsive Evaluation

When to consider evaluation?

Evaluation needs to be considered as part of regular strategic and operational planning and at the very beginning – when designing or formulating a new programme. Managers (of organizational units) need to consider what, across the spectrum of all programming, will be evaluated within a given period and to allocate the resources and time required. Further guidance on devising an Evaluation Plan is found in Evaluation Planning at UN Women . Managers also need to consider evaluation during the design of new programmes as:

• the design of the program affects how it can be evaluated
• specifying programme results and indicators are foundational to evaluation
• monitoring results throughout programme implementation is critical to having valid information available for an evaluation, and
• there is a need to allow time for the evaluation in work plans and allocate funding in the budget (for more information, see the Evaluation Preparation chapter).

How to consider evaluation?

Evaluation is closely related to results based management (RBM). All of the steps in RBM have evaluation implications, and evaluation influences the work undertaken in each step as well, not only in step 4.  RBM is a manager’s responsibility and part of managing for results is be clear on what the programme is designed to achieve, measure progress towards and attainment of results, as well as learn from programme experiences. Evaluation is an important tool for managers in their RBM responsibilities.

RBM step 1, the context / situation analysis, informs the design of a programme by knowing more about what exists at the outset, the nature of the ‘problem’, including regard for GE and HR issues. This type of information is also found in baseline reports. Baseline studies examine the situation related to all of the programme’s desired results, as they exist(ed) at the beginning of the programme. Managers frequently draw from the situational analysis for baseline information. Baseline studies or baseline stories are essential to evaluations as they provide a point of comparison. An evaluation will examine changes in results and compare these to baseline data and information. Where baseline is not available to the evaluation, there is a need to consider how the evaluation may help re-create it.

Developing the results chain (RBM step 2) involves anticipating what the situation will or should look like once the programme has been implemented successfully. This is a strategic exercise in the formulation of a programme, making good use of best practices (from the literature and from past evaluations), the context/situation analysis, the participation of stakeholders and the manager’s own expertise in development interventions.

Results include outputs, outcomes and impacts and these are all articulations of what is hoped to be achieved in furthering human rights and gender equality through this programme. Outputs are the products, services, and capacities that result from the completion of activities within a development intervention. Outcomes are the intended or achieved short-term and medium-term effects of an intervention’s outputs, usually requiring the collective effort of partners. Outcomes represent changes in development conditions which occur between the completion of outputs and the achievement of impact. Impacts are the long term effects or change to which the programme, through collective effort with partners, will contribute.

Results should be SMART:

• Specific
• Measurable/observable
• Attainable within scheduled time, budget, and conditions
• Results-oriented
• Targeted to the identified need, desired impact on or benefit to

Formulating a new programme also includes defining what activities will be undertaken in order to achieve these results. Again, this is importantly a collaborative exercise as it involves deciding with programme partners, stakeholders and beneficiaries what actions are most likely to achieve the desired results. This is the theory of change or programme theory, drawing relationships between what is planned in action or activities in order to achieve certain outcomes.  Or, in reverse, if we want to achieve a certain result, we can identify what actions are most likely to have this effect. A logic model is a diagram or graphic display of these elements – goals, activities and results chain (outputs, outcomes, and impacts) – in a way that demonstrates the links or causal relationships among them. A logic model says, IF I do these activities, THEN I expect these outputs, outcomes and finally impact. This Guide provides a tool for developing a logic model. A GE/HR responsive evaluation will examine the different results or programme achievements and how (and if) programme activities contributed to these, paying attention to GE and HR as part of the intervention and throughout the results chain and being conducted in a manner that is inclusive, participatory, transparent, and makes use of mixed methods.

Indicators (RBM step 3) provide the means or way to measure progress towards and achievement of the results defined for the programme. Indicators are developed for all results (outputs, outcomes and impacts) and can be qualitative or quantitative. Development of indicators, like identification of desired results is best done collaboratively with stakeholders and partners to:

• get different views of what is important
• gain consensus on what constitutes ‘proof’ that a result has been achieved, and
• identify and agree upon what information will be collected along the way

While collaboration can involve additional time and cost, the process of developing results and indicators is as beneficial as having them at all! Better results and indicators will result from broader participation in the process, including greater programme buy-in and understanding.

Indicators should be:

• Simple – easy to use, simple to understand and easy to operationalize (technically feasible)
• Valid – significant, relevant, and directly linked to programme objectives
• Reliable – robust, accurate, replicable, verifiable and objective
• Sensitive – capable of demonstrating and capturing change in the outcome of interest (responsive to changes)
• Direct – measure the result they are intended to measure as closely as possible. Proxy indicators are used as alternative measures if a direct measure is not possible
• Quantitative and Qualitative – based on precise and replicable measurements which are comparable. Qualitative indicators should supplement and complement quantitative indicators for richer information
• Disaggregated – disaggregated by sex, age, education, location or other dimensions to give a better picture of programme results

Gender sensitive and rights based indicators are critical to GE/HR responsive evaluation work, as they set the stage for what will be measured. Some examples include:  proportion of target group by sex aware of the benefits of birth registration, employment to population ratio by age and sex, gross primary graduation ratio by sex, percentage of women in parliament. This Guide provides a reference for GE /HR sensitive indicator types and sources and links to guides for further direction and information on this subject.

Planning for monitoring and GE/HR responsive evaluation (RBM step 4) is often done through the development of a logframe or performance measurement framework, which are plans for measuring the programme. Example templates are provided. It is important to think through and document what will be measured, who will do it, when and how it will be done. Monitoring is done by those who manage and deliver the programme – the programme manager and programme partners. Ongoing data and information about how the programme is progressing is crucial to good evaluation. A lack of monitoring information is often cited in evaluations as the reason programmes cannot be fully and accurately assessed. Planning for GE/HR responsive evaluation is explained in the subsequent section Evaluation Planning at and UN Women in the Evaluation Preparation chapter.

Reporting on results (RBM step 5) includes result-oriented progress report writing done throughout the life of the programme and evaluation(s) conducted on the programme. Reporting on results is important to share lessons learned from the programme and for accountability – to and amongst partners, to stakeholders and beneficiaries and to donors. Monitoring and evaluation reports are used to make adjustments in the programme, during implementation in order to improve programme delivery and the likelihood of achieving results. Lessons learned through evaluations are instrumental in designing new programmes and enhancing knowledge on what works to further gender equality and human rights in development programming. Evaluation reporting and reports is discussed in detail later in this Guide.

Tips: Revisit the logic model and measurement plan throughout the life of the programme to ensure the vision for the programme is aligned with the reality of implementation. Be discriminating in identifying indicators. Keep the number of indicators small and meaningful. Create a system (or a common system in programmes involving partners) for organizing ongoing monitoring information during programme implementation, so that it may be easily retrieved during evaluations.