Assessing Skill for Impacts in Seasonal to Decadal Climate Forecasts
Journal of Geology & Geophysics

Journal of Geology & Geophysics
Open Access

ISSN: 2381-8719

+44 20 3868 9735

Editorial - (2013) Volume 2, Issue 3

Assessing Skill for Impacts in Seasonal to Decadal Climate Forecasts

Pete Falloon1*, David Fereday1, Nicky Stringer1, Karina Williams1, Jemma Gornall1, Emily Wallace1, Rosie Eade1, Anca Brookshaw1, Joanne Camp1, Richard Betts1, Rutger Dankers1, Kathryn Nicklin2, Michael Vellinga1, Richard Graham1, Alberto Arribas1 and Craig MacLachlan1
1Met Office Hadley Centre, Fitzroy Road, Exeter, Devon, EX1 3PB, UK
2School of Earth and Environment, The University of Leeds, Leeds, LS2 9JT, UK
*Corresponding Author: Pete Falloon, Met Office Hadley Centre, Fitzroy Road, Exeter, Devon, EX1 3PB, UK, Tel: +44(0)1392 886336, Fax: +44(0)1392 885681 Email:

Despite the recent effort to develop underpinning climate prediction science for seasonal to decadal (S2D) climate predictions, there has been relatively little uptake and use of S2D climate forecasts by users for decision making in Europe [1]. On the other hand, there is a much longer tradition in applying seasonal climate forecast information for user applications in other parts of the World, notably in Africa, the USA and Australia [1,2] one notable exception is the use of precipitation forecasts for hydropower generation management by EDF [3,4]. In part, this is related to the relatively limited skill of S2D climate forecasts in Europe; in contrast predictability in decadal hindcasts (forecasts of the past) is greatest in the Tropics [5]. This illustrates the importance of understanding skill in user uptake of such products [6-8]. However, accuracy, lead time, and appropriate spatial and temporal scale of S2D climate forecast information may not be the main (or only) factors influencing user uptake; potential economic and environmental benefits may be of greater importance [9]. In addition, probabilistic (ensemble) prediction systems are more commonly used in mediumrange applications, bringing additional challenges in communicating forecast information to end-users.

The use of basic S2D climate forecast outputs (e.g. temperature, precipitation etc) has significant potential to support both shorterterm decision making (thus helping avoid potential risks and losses, and optimize profits), and longer-term climate adaptation plans in numerous sectors (e.g. agriculture, water, health and energy [10]) as noted above these products are already widely used in some regions of the World. Further benefit could also be realized by providing information more directly relevant to potential users, such as changes in extreme rainfall events, heat-waves, crop yields and river flows, which we refer to here as “impacts”. In addition, further processing of direct S2D forecast outputs and the use of impact models may improve the usability of S2D forecasts with weak skill [3,4]. However, as noted above, the skill of S2D forecasts for impacts (as opposed to generic assessments of weather and climate skill) may limit the usability of S2D impacts products.

S2D predictions of weather and climate can be derived both from statistical (or empirical) and dynamic models [7]. The former approach is usually based on regional historic relationships between climate variables; most recent dynamic approaches use fully coupled ocean-atmosphere general circulation models (CGCMs). Some S2D forecasting systems, particularly the CGCM approaches, may include impact-relevant outputs directly (“online” approaches), for example via river flow models, soil moisture calculations, or estimates of vegetation productivity. Validation and skill assessment in these systems may also provide valuable information on the overall performance of the seasonal prediction system itself. For example, rivers integrate land hydrology over large geographic areas and are important sources of freshwater input to the oceans [11]. However, drivers for, and focus of skill assessments for CGCM development versus impact (and user) application may differ, and obtaining reliable observations for impacts may be challenging.

For user applications, the focus is on the reliability of the outcome for societal purposes (e.g. purely the crop yield or river flow volume). The CGCM development perspective tends to focus on better representing and understanding earth system processes, in turn leading to more holistic assessments targeted at understanding the role of individual processes or components and their connections (e.g. for river flow: precipitation, evaporation, soil moisture and runoff generation); or broader understanding of the role of wider climate processes (e.g. variability, climate modes). However, the user application focus would also clearly benefit from understanding gained in ‘climate’ focused studies since the ‘right’ impact model outcomes may result for wrong model reasons (e.g. from a combination of two incorrectly simulated processes or variables [11,12]). More in-depth assessments would also aid confidence in S2D impact predictions in general.

Skill for impacts in S2D forecasts varies considerably, depending on the variable or impact in question, the region and the lead time [13]. There have been relatively few co-ordinated impact model intercomparisons for S2D skill, either within a sector (agriculture) or across sectors. However, a co-ordinated assessment of impact skill for S2D forecasts could further our understanding of the robustness of S2D impact forecasts, across impacts, regions and lead times. At the top level, the greatest skill for S2D impacts might be expected where S2D forecast skill is greatest (both spatially and temporally). However, this is likely to be an oversimplification for several reasons. Firstly, many impacts depend on a number of climate and weather variables (and the interactions between then) and so climate skill may not translate directly to impact skill. Secondly, impact models themselves may have regional biases even when driven by observations [11], which may distort the effect of climate skill, particularly if using raw S2D model outputs. Thirdly, various parts of the impact modelling “post-processing chain” (for example, downscaling and bias-correction) may actually improve skill for impacts relative to the original skill for S2D climate variables. However, clearly any improvement in climate skill in S2D forecasts has the potential to increase skill for impacts [14]. There is also a need to recognize that S2D impact forecasts may only be one part of the user’s decision making process, which suggests a further need to understand the role that S2D forecasts play in the wider decision-making context.

Greater use should be made of existing techniques for assessing impact (and climate) model skill from model inter-comparison projects (e.g. the Coupled Model Intercomparison Project (CMIP5 [15]); Inter-sectoral Impact Model Intercomparison Project (ISI-MIP [16]), Water Model Intercomparison Project (WaterMIP [17]), Protocol for the Analysis of Land Surface models (PALS), and the Agricultural Model Intercomparison and Improvement Project (AgMIP [18]), which are currently largely focused on longer-term climate change, or climatological timescales. Therefore, a co-ordinate, in-depth assessment of S2D impacts would also be beneficial, and would enable assessment of the robustness of S2D impacts and guidance on the form for their dissemination (e.g. as text, probabilities, terciles, absolute values or delta changes), and the platform (e.g. internet, print, email, tools etc.). This may also require a more co-ordinate approach for using assessment metrics, although metrics chosen should be appropriate to the application. The WMO set up a standardized verification system for long-range forecasts (SVS-LRF) in 2005-while it may be possible to build on, or adapt this for impact purposes, it is technical in nature and not well suited for end user applications [7]. The core focus of model inter-comparison projects is often a top-level comparison of model results rather than on process understanding, which may limit model development and improvement activities. Therefore, there is also a need to better link the understanding of S2D impact skill with both climate skill understanding, and that of broader climate processes and driving factors (e.g. El Nino Southern Oscillation (ENSO) and the North Atlantic Oscillation (NAO)) in S2D forecasts.

The modelling protocols in such an inter-comparison project may depend on the overall aim - whether this is on model comparison and improvement (more rigid, stricter experimental design), or producing the best outcomes for users (more flexible experimental design, orientated to producing greatest skill). Furthermore, there is a need to better tailor S2D skill assessments to meet needs of users [7], which includes skill for impacts; in turn this requires an improved understanding of their needs for S2D forecast information [1,19]. The EU project European Provision of Regional Impacts Assessments on Seasonal and Decadal Timescales (EUPORIAS [20]) is surveying a range of users in this respect. In addition, it would be beneficial to present S2D impact forecasts alongside estimates of skill to aid understanding of their strengths and weaknesses.

Some specific aspects of S2D impact skill assessment requiring further work include:

• Better understanding of the role of initial conditions (the EU project EUPORIAS is investigating this issue for a range of impacts). Different spin-up approaches may be needed for different impacts [21].

• Development and comparison of techniques for evaluating probabilistic impact forecasts [22].

• Understanding the role of downscaling of seasonal forecasts to capture local influences (topography, land use etc. [22]) in S2D impacts skill.

• Improvements in the use of observational data (for example, enhanced use of remote sensing and spatial data [22]; and the use of longer time series of data to assess model performance [14]).

• Further studies on the impact of extreme events in seasonal forecasts [14,23-25].

• Linking impact and climate models (‘online approaches’), which will enable impact results to be generated more rapidly and will allow feedbacks between impacts and climate to be captured (e.g. the effects of seasonal soil cover in crop models [22]).

• Understanding the effect of bias correction on S2D impact forecast results, which recent work on climate timescales has shown may be significant depending on variable, spatial and temporal scale [26,27].


This article was supported the Joint DECC/Defra Met Office Hadley Centre Climate Programme (GA01101) and the EUPORIAS project, funded by the European Commission 7th Framework Programme for Research, grant agreement 308291.


  1. Dessai S, Soares MB (2013) EUPORIAS (Grant agreement 308291) Deliverable 12.1, Systematic literature review on the use of seasonal to decadal climate and climate impacts predictions across European sectors, University of Leeds, Leeds, UK.
  2. Hansen JW, Mason SJ, Sun L, Tall A (2011) Review of seasonal climate forecasting for agriculture in Sub-Saharan Africa. Experimental Agriculture 47: 205-240.
  3. Dubus L (2012) Monthly and seasonal forecasts in the French power system. ECMWF Seminar, Personal presentation.
  4. Dubus L (2013) Weather & climate and the power sector: Needs, recent developments and challenges. Weather matters for energy, Springer.
  5. Macleod D, Caminade C, Morse A (2012) Useful decadal climate prediction at regional scales? A look at the ENSEMBLES stream 2 decadal hindcasts. Environmental Research Letters 7.
  6. Meinke H, Nelson R, Kokic P, Stone R, Selvaraju R, et al. (2006) Actionable climate knowledge - from analysis to synthesis. Climate Research 33: 101-110.
  7. Davey M, Brookshaw A (2011) Long-range meteorological forecasting and links to agricultural applications. Food Policy 36: S88-S93.
  8. Demeritt D, Nobert S, Cloke HL, Pappenberger F (2013) The European Flood Alert System and the communication, perception, and use of ensemble predictions for operational flood risk management. Hydrological Processes 27: 147-157.
  9. Marshall NA, Gordon IJ, Ash AJ (2011) The reluctance of resource-users to adopt seasonal climate forecasts to enhance resilience to climate variability on the rangelands. Climatic Change, 107: 511-529.
  10. Van der Linden P, Mitchell JFB (2009) ENSEMBLES: Climate Change and its Impacts: Summary of research and results from the ENSEMBLES project, Met Office Hadley Centre, Exeter EX1 3PB, UK.
  11. Falloon P, Betts R, Wiltshire A, Dankers R, Mathison C et al. (2011) Validation of river flows in HadGEM1 and HadCM3 with the TRIP river flow model. Journal of Hydrometeorology 12: 1157-1180.
  12. Beven KJ, Freer J (2001) Equifinality, data assimilation, and uncertainty estimation in mechanistic modelling of complex environmental systems. Journal of Hydrology 249: 11-29.
  13. Falloon P, Fereday D, Stringer N, Williams K, Gornall J et al. (2013) Policy Deliverable E3: Assessment of potential skill for impacts quantities in seasonal, interannual and decadal forecasts. For Joint DECC and Defra Met Office Hadley Centre Climate Programme, Reference: DECC/Defra GA01101, Met Office Hadley Centre, Exeter.
  14. Challinor AJ, Slingo JM, Wheeler TR, Doblas-Reyes FJ (2005) Probabilistic simulations of crop yield over western India using the DEMETER seasonal hindcast ensembles. Tellus A 57: 498-512.
  15. Taylor KE, Stouffer RJ, Meehl GA (2012) An Overview of CMIP5 and the Experiment Design.
  16. Warszawski L, Frieler K, Piontek F, Schewe J, Serdeczny O (2013) Research Design of the Intersectoral Impact Model Intercomparison Project (ISI-MIP), P Natl Acad Sci.
  17. Haddeland I, Clark DB, Franssen W, Ludwig F, Voß F, et al. (2011) Multimodel Estimate of the Global Terrestrial Water Balance: Setup and First Results. J Hydrometeorol 12: 869-88.
  18. Rosenzweig C, Jones JW, Hatfield JL, Ruane AC, Boote KJ, et al. (2013) The Agricultural Model Intercomparison and Improvement Project (AgMIP): Protocols and pilot studies. Forest Agr Meteorol 170: 166-182.
  19. Coelho CAS, Costa SM (2010) Challenges for integrating seasonal climate forecasts in user applications. Current Opinion in Environmental Sustainability 2: 317-325.
  20. Hewitt CD, Buontempo C, Newton PC (2013) Using climate predictions to better serve society's needs. Eos 94: 105-107.
  21. Cosgrove BA, Lohmann D, Mitchell KE, Houser PR, Wood EF, et al. (2003) Land surface model spin-up behavior in the North American Land Data Assimilation System (NLDAS). J Geophys Res 108: 8845.
  22. Hansen JW, Challinor A, Ines AVM, Wheeler T, Moron V (2006) Translating climate forecasts into agricultural terms: advances and challenges. Climate Research 33: 27-41.
  23. Challinor AJ, Wheeler TR, Slingo JM, Osborne TM (2006) Combining weather and crop yield forecasting for seasonal to multi-decadal prediction, Inside Agriculture 1: 40-45.
  24. Eade R, Hamilton E, Smith DM, Graham RJ, et al. (2012) Forecasting the number of extreme daily events out to a decade ahead. J Geophys Res 117.
  25. Hamilton E, Eade R, Graham RJ, Scaife A, Smith DM, et al. (2012) Forecasting the number of extreme daily events on seasonal timescales, Journal of Geophysical Research-Atmospheres 117.
  26. Ehret U, Zehe E, Wulfmeyer V, Warrach-Sagi K, Liebert J (2012) HESS Opinions “Should we apply bias correction to global and regional climate model data?” Hydrol Earth Syst Sci: 16: 3391-3404.
  27. Hagemann S, Chen C, Haerter JO, Heinke J, Gerten D, et al. (2011) Impact of a Statistical Bias Correction on the Projected Hydrological Changes Obtained from three GCMs and Two Hydrology Models. J. Hydrometeorol 12: 556-578.
Citation: Falloon P, Fereday D, Stringer N, Williams K, Gornall J, et al. (2013) Assessing Skill for Impacts in Seasonal to Decadal Climate Forecasts. J Geol Geosci 2:e111.

Copyright: © 2013 Falloon P, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.