Evaluation of Climate Adaptation Projects

\label{evaluation-of-climate-change-adaptation-projects}The concept of climate change adaptation cannot be easily and unequivocally defined. Measures as diverse as land use planning, infrastructure design, community development and farming practices might all include considerations related to climate change, but it remains difficult to isolate and evaluate the individual adaptation elements. Furthermore, activities which have a clear adaption focus require comparison against a counter-factual baseline which is often difficult to establish. It should also be considered that adaptation strategies considered as successful within a short time frame might actually exacerbate longer-term vulnerability. By way of illustration, poorly designed coastal defenses may in the short-term lower the level of vulnerability, thereby encouraging among others population growth and development. In the long-term however vulnerability may increase if extreme weather events exceed the design threshold of the defenses. These complexities need to be considered when designing and implementing adaptation activities, and also when evaluating them, as evaluation is a key stage of any process of public intervention (OECD 2011).
Evaluation activities can be categorised by their timing: a) ex-ante evaluations take place prior to the implementation of an initiative and are forward-looking assessments of the likely future effects (this may include vulnerability or climate change impact assessments), b) mid-term evaluations are conducted during the implementation phase of projects and intend to improve performance, and c) ex-post evaluations normally serve the purpose of a summative evaluation since they are undertaken at the end of the implementation phase of projects, and are directed at investigating how well the initiative served its aims, at assessing the sustainability of results and impacts and at drawing conclusions that may be relevant to similar initiatives in the future (UNDP 2007, GEF 2008, UNFCCC 2010).
There are mainly three ways to carry out ex-ante evaluations of climate change adaptation. One is to apply a Cost-Effectiveness Analysis (CEA) where the societal costs and benefits of adaptation projects are determined, and thus to arrive at a prioritized ranking of the most cost-effective adaptation measures (ECAWG 2009). This results in a ‘cost curve’ showing measures that are cost-negative (i.e. create savings), measures with a cost-benefit ratio below 1 (i.e. benefits outweigh costs), and measures with a cost-benefit ratio above 1 (i.e. cost inefficient). Cost-Benefit Analysis (CBA) is often used to assess adaptation options when efficiency is the only decision making criteria. A CBA involves calculating and comparing all of the costs and benefits, which are expressed in monetary terms, thus providing a basis for prioritizing possible adaptation measures. Multi-Criteria Analysis, the third evaluation approach, allows for different adaptation options to be assessed against a number of criteria, where each criteria is given a weight and an overall score for each adaptation option is obtained. A recent review of approaches for assessing costs and benefits of adaptation options (UNFCCC 2010) shows that most approaches focus on either adaptation costs or vulnerability/risk management. Only two studies actually compare costs and impacts (Moench et al., 2009, ECAWG 2009). Another study incorporates in the analysis costs and multiple aspects and benefits for the prioritization of adaptation measures for disaster risk and vulnerability reduction (Haque et al., 2012).
In terms of ex-post evaluation, such economic-based analyses (i.e. CBA, CEA and MCA) are often contested and characterised by thorny methodological issues, particularly in relation to uncertainty, valuation and equity (Grafakos and Olivotto 2012). Uncertainty surrounding future climate change dynamics and impacts and future socio-economic development constrains the identification of optimal adaptation options. The valuation of costs and benefits, both from a financial and economic point of view, inevitably includes elements of subjectivity. Last but not least, it is important to consider not only the level of net benefits of adaption project but also their societal distribution, for example by giving weights to different costs and benefits according to who receives the benefits and who bears the costs (UNFCCC 2010).
The focus of the chapter is on the methodological challenges of ex-post evaluations of climate change adaptation interventions. In relation to this, it should be pointed out that adaption initiatives may be implemented at different levels. This realization leads to the necessity to measure the results of adaptation interventions at different scales and therefore to develop methodologies able to assess national policy level, community level and project level related adaption measures. The framework of choice depends on the type of adaptation effort undertaken (e.g. reduction of community vulnerability, implementation of adaptation investment portfolio of actions, mainstreaming adaptation in national processes) and by whom (e.g. NGO, donor or bilateral agencies, national governments). As project-related evaluation systems are widely used among international donor agencies such as DFID, USAID and the World Bank but also among international climate funds like Global Environment Facility (GEF), the Adaptation Fund, the Green Climate Fund (GCF) and the Climate Investment Funds (CIF/PPCR), this chapter focuses on this particular level of adaptation interventions. In need to justify spending by assessing results locally and aggregating information to the portfolio level, the evaluation systems at this level are usually built around evaluation criteria, expressed through logical framework models (input, output, process, outcome), result-based frameworks and in some cases supported by a theory of change.