Deriving user-informed climate information from climate model ensemble results

Communication between providers and users of climate model simulation results still needs to be improved. In the German regional climate modeling project ReKliEs-De a midterm user workshop was conducted to allow the intended users of the project results to assess the preliminary results and to streamline the final project results to their needs. The user feedback highlighted, in particular, the still considerable gap between climate research output and user-tailored input for climate impact research. Two major requests from the user community addressed the selection of sub-ensembles and some condensed, easy to understand information on the strengths and weaknesses of the climate models involved in the project.


Introduction
In the context of the fifth IPCC report (2013/2014), a new generation of scenarios for greenhouse gas (GHG) concentrations were developed, called representative concentration pathways (RCPs; Moss et al., 2010;van Vuuren et al., 2011;Stouffer et al., 2010).For impact research and policy advocacy, high-resolution downscaled information for the respective global ensemble simulation results are required.The Coordinated Downscaling Experiment for Europe (EURO-CORDEX; Jacob et al., 2013) provides several such simulations using dynamical downscaling methods.
Germany has a variety of institutions and projects concerned with making climate model results accessible and usable for users outside the direct climate research community (i.e., climate services).This includes the German Weather Service (DWD), the Climate Service Center Germany (GERICS), the Potsdam Institute for Climate Impact Research (PIK), several regional climate service providers by the Helmholtz group, and further institutions at universities, research institutions, and projects.
As a German particularity, there are two informal groups of federal and state environmental agency staff members responsible for providing policy advisory on climate-related topics: a group on "Climate change impacts and adaptation" (formed in 2003) and another group on "Interpretation of regional climate model data" (formed in 2006).Both groups meet semiannually.The working group "Interpretation of regional climate model data" has discussed and published recommendations for the use and presentation of climate model ensemble results (Kreienkamp et al., 2013).Both groups together initiated further contributions to the EURO-CORDEX high-resolution (12 km) downscaling ensemble, augmenting it by statistical downscaling methods.The resulting project ReKliEs-De (german: Regionale Klimaszenarien Ensemble H. Huebener et al.: Deriving user-informed climate information für Deutschland) is funded by the German Federal Ministry of Education and Research (BMBF) and helps to improve policy consulting based on the most recent climate model scenarios.The two environmental agencies' member groups monitor the project's progress and provide input from the user perspective within a project advisory board.
Although there are multiple projects and initiatives producing and providing climate research results to all kinds of stakeholders, there is still a communication gap between the climate research community and many of the users of climate change information (see McNie et al., 2007;Hewitt et al., 2013;Black, 2015).Presently, a growing number of institutions, projects, and initiatives address this gap (e.g., Groot et al., 2014;Harold et al., 2016;Roessler et al., 2017).Additionally, the Global Framework for Climate Services (GFCS, www.wmo.int/gfcs/) has started a discussion towards an ethical framework for climate services, which calls for integrity, transparency, humility, and collaboration when providing climate services.Consequently, an important part of the ReKliEs-De project, besides conducting and providing the simulations, is the systematic analysis and processing of the resulting output to improve its usability in impact research and policy consultancy.
To ensure the adequacy of the resulting information for the intended purpose and users, a midterm user workshop was conducted on 14 and 15 June 2016 in Potsdam to allow the envisioned users to interact with the project and streamline the results to their needs.Overall, 43 users attended the workshop, coming from a variety of impact research and application areas.Those encompassed meteorological service providers, hydrological modelers, forest researchers, viticultural and agricultural researchers, political advisors at federal states (federal and state environmental agencies, federal forest agencies, geological services, etc.), and city planners for water infrastructure.

The workshop format
The user workshop was dedicated to an interactive format to facilitate the users' input into the project.Thus, no oral presentations -apart from a welcome address, an introduction to the workshop concept, and the final discussion of the workshop results -were given.All (interim) project results were presented as posters, exclusively.Poster viewing sessions took place on both workshop days.On the second day, users were asked to write thoughts, questions and remarks regarding, recommendations for, or criticism of the posters on sticky notes and stick them on the respective poster.All project partners were in attendance at the posters in both poster sessions to answer questions, discuss and take in the oral user feedback.The other component of the workshop encompassed topical working groups, regarding "use of ensembles of climate change simulations", "analysis of extreme events from climate change simulations" and "data handling".Two of the working groups -on "Ensembles" and "Extremes" -were conducted twice (once on each workshop day); the "Data" working group only took place once, since all interested participants could be accommodated in one session.Consequently, all participants had the opportunity to participate in two different working groups.A further contribution to the interactive concept was an "ice-breaker" event at the evening of the first day, set up to foster informal communication about the workshop topics.
To our knowledge, it was the first time that an entire workshop was organized consequently in such a dedicated interactive format.Further information about the workshop (e.g., all posters) can be accessed via: http://reklies.hlnug.de/nutzerworkshop.html(in German).
The intensive discussion with the participants of the workshop provided a lot of helpful information on the demands of the different users, which are summarized in the following chapter together with some selected examples.

User needs
The discussions at the workshop covered several topics that were of particular interest to the users.These included data problems (accessing, processing and analyzing large data sets), biased climate model results, how to work with large model ensembles, how to communicate and use the different scenarios (for example, calculation of flood protection measures), how to treat results obtained with older scenarios or model versions, features specific to general circulation models (GCMs) (e.g., climate sensitivity, performance in simulating spatial and temporal patterns), and model (GCMs, regional climate models, empirical statistical downscaling) performance in simulating extreme precipitation events, stationary weather patterns (blockings, stationary troughs), interannual variability, clustering of events (e.g., storms) and combined events (e.g., heavy rainfall after a dry period).
A recurring request in several discussions was for more information on the climate models' strengths and weaknesses, preferably as a short "climate model fact sheet".

Information on climate models
Climate modelers are familiar with the inherent strengths and weaknesses of global and regional climate models in general and of some model features in specific models or model groups.However, the climate impact research community and the climate advisory staff in administration, economy and society present at the workshop were mostly unfamiliar with these facts in detail.They (have to) rely on the information provided with the climate model results.The user feedback at the ReKliEs-De project workshop showed a strong request for predigested climate model information (like climate sensitivity, expressed as the ratio T / GHG, or hydrological sensitivity, expressed as the ratio P / T , performance in simulating blockings or storms, or other features of the models comprising the ensemble).While this information is mostly available in the scientific literature (e.g., IPCC reports), it is not readily accessible for people from climate impact disciplines or from policy advisory.The problems in accessing the scientific information from the available literature sometimes begin with language problems (often users prefer information in their native language).They include accessibility of scientific journals and culminate in the scientific language used in scientific publications that is often quite discipline-oriented and difficult to understand for nonexperts.However, the climate research community -and specifically the climate modelers from the ReKliEs-De project -are requested to provide basic information on the models and their strengths and weaknesses in a manner that is short, precise and easy to understand.It is simply not sufficient to state general rules of caution like "Always use all available model results" or "Always average over at least 3 × 3 grid boxes" or issue similar statements.While there are good arguments for rules like that, they are sometimes considered not applicable for answering local impact questions by the users.Rather, which model has which weakness where and when should be explained.This could lead to a statement like "The Rhine river valley is not resolved in all model runs of resolution 12 km or larger; thus, the simulated properties like temperature or rainfall are not representative for the Rhine river valley but represent an average for a much broader area" or "The winter rainfall simulated with the old model version is probably not realistic, because it contains an extremely high number of warm-rainy winter days in the future.This is mostly remedied in the new model version".
With more information about model strengths and weaknesses, the users argued that they would be better equipped to decide which input might serve their needs best.

Information on ensemble selection
In climate modeling, most multi-model ensembles are ensembles of opportunity (e.g., CMIP5, Taylor et al., 2012).They might comprise a number of models that share some "genealogy" (Knutti et al., 2013), common sub-models, common characteristics (e.g., aliasing at strong gradients in spectral models) or weaknesses (e.g., reduced winter blocking frequency over the north Atlantic, Anstey et al., 2013).Another approach is to use a single-model perturbed physics ensemble (e.g., the climateprediction.netproject by the Met Office Hadley Centre; Murphy et al., 2009;Frame et al., 2009), which covers a wide range of possible simulation results, albeit all of them derived from the same GCM.To provide regional climate change information, usually a sub-ensemble of the available GCM ensemble is selected for downscaling (e.g., McSweeney et al., 2012;Dalelane et al., 2017).The project ReKliEs-De, like most other downscaling efforts, uses a sub-ensemble of the available GCM ensemble too.The selection of a sub-ensemble (and the possible exclusion of certain model runs from an ensemble, see McSweeney et al., 2015) is usually the result of several selection criteria.First and foremost, scientific aspects are used to select a subensemble which should cover the spread of the whole ensemble as well as possible for the research subject and region of interest (e.g., Asian summer monsoon by McSweeney et al., 2012;Arctic by Overland et al., 2011).Further selection criteria often include aspects like avoiding models too similar in their genealogy.Additional criteria are often derived from expert knowledge on, for example, regional performance of the models, specific strengths or specific sub-models, or parameterizations necessary for the respective analysis (e.g., interactive vegetation for analysis of a possible Amazonian forest dieback).Additionally, technical aspects might impede the use of one or another model, like insufficient resolution, missing variables, and missing data availability.However, all these considerations are usually not communicated when the downscaling results derived from such a sub-ensemble are presented to the users.Consequently, the users often deal with a "black box" when they analyze the ensemble results or when their task is the selection of a certain sub-ensemble.
At the workshop, the ensemble selection criteria applied in the ReKliEs-De project were briefly presented.Several users formulated that understanding the selection criteria for a certain sub-ensemble would enable them to better understand the specific strengths and weaknesses -and perhaps limitations -of a certain sub-ensemble.They asked for understandable quality criteria for inclusion or exclusion of certain models in a sub-ensemble.Thus, we conclude that to improve the understanding of the resulting data it would be beneficial to communicate the decision criteria used for selecting sub-ensembles (particularly concerning the "expert judgment" aspects) to the users.

Support interpretation of model results
The workshop participants also requested more support with interpreting the model results.This encompasses assessing the model performance for different meteorological variables (temperature, precipitation, wind, etc.) or weather features (blockings, storm tracks, stationary troughs, etc.).
With respect to the above mentioned expert knowledge of climate modelers on selected model results, a communication strategy should also include a basic interpretation of important features of the ensemble results.Climate modelers' confidence in some results is much higher than in other results, and it seems necessary to explain these experiences and their reasons to the users.For example, there is high confidence in projected temperature changes (in a certain range) but much less in projected rainfall changes.This is due to the fact that temperature is calculated at the grid scale in the models and is thus relatively well simulated.On the other hand only the large scale part of precipitation is calculated at the grid scale, while convective precipitation and its precursor processes like convective cloud formation, cloud properties, and so forth are parameterized.Climate modelers are relatively sure about some general changes in the global circulation like poleward displacement of the subtropical and polar fronts (e.g., IPCC, 2007) but much less sure on the frequencies of wave activities particularly in the polar front (e.g., IPCC, 2013, chap.14.6.2).The global models still have trouble simulating a realistic North Atlantic storm track (see IPCC, 2013, chap. 9.4.1.4.3) including the number and persistence of midlatitude blockings (IPCC, 2013, chap.9.5.2.2).Thus, phenomena driven by the wave activity of the polar front like persistent weather events at the leading edge of the trough (e.g., consecutively occurring heavy rainfall events as seen in early summer 2016 in Germany) cannot be expected to be simulated realistically (see IPCC, 2013, chap.14, Box 14.2).
There is evidence that higher-resolution modeling with a regional climate model improves processes that are steered by orography or by soil and land use properties (e.g., Christensen and Christensen, 2007;van der Linden and Mitchell, 2009;Rummukainen, 2010).Precipitation simulation is particularly improved when resolutions reach 2 to 4 km, so convection does not have to be parameterized anymore (convection permitting simulations, see Bauer et al., 2011;Warrach-Sagi et al., 2013;Prein et al., 2014).
However, some problems cannot be solved by downscaling.If the driving global model, for instance, does not simulate a blocking high-pressure system or a stationary trough, the regional model cannot fully generate these features in its inner simulation domain as it is limited by the boundary conditions imported from the global model (e.g., Maraun et al., 2010, file those effects under "errors inherited from the driving global climate model").It is only possible to carefully interpret an increase or reduction in the number or duration of such events simulated for current and future climate.Another example is that even though storms cannot be simulated by most global models realistically (because the spatial resolution is too coarse) we might still gain useful insights using downscaling methods (e.g., Pardowitz et al., 2016).An interesting case is the future trend for increased heavy precipitation.This was first postulated from theory (according to Clausius-Clapeyron), was then simulated by climate models (e.g., Fowler and Hennessy, 1995) and only recently emerged out of the highly variable rainfall data as an observable trend (Fischer and Knutti, 2016).This example shows that understanding of first principles and the modeling exercises can give useful and useable hints for the interpretation and for the subsequent use of the information in impact research and advisory.
In several cases specific model or sub-ensemble results can be attributed to joint characteristics of the models contributing to the sub-ensemble.For example, results from statistical downscaling methods might systematically display characteristics that are not evident in dynamical downscaling results.
This or other effects might be interpretable in the knowledge of the general differences between the models or methods.However, users from the climate impact research community or from administration or policy advice might not have the necessary climate modeling knowledge to draw these conclusions themselves.It is therefore necessary to support the users with this kind of interpretation of ensemble results.Munaretto and Huitema (2015) propose so-called boundary organizations at the science-policy interface to facilitate this crucial exchange of information between the different groups of scientists or users.According to Mc-Nie (2007) this boundary work needs to be "credible, salient and legitimate".

Support users in ensemble analysis
From the interpretation of specific model results there is a close connection to the robustness or spread (scientifically termed uncertainty) of the results of an ensemble of climate change simulations.The users asked for more explanations on how to interpret results with a spread.
The composition of climate model ensembles continually changes due to several reasons.Sometimes new global and/or regional models or model versions become available, simulations are repeated with increased resolution, or modified scenarios are formulated and call for new simulations.Particularly, climate impact modelers are sometimes in a dilemma: they would prefer a fixed ensemble which does not change in the near-future so they could analyze the impacts and provide guidance for decision making.However, this is not how science works; climate models are always imperfect and climate modelers will always strive to improve the models and provide more and improved simulations for an improved assessment of future changes.Thus, at least from within the German impact research community, the definition of a fixed standard climate model ensemble was requested (at the ReKliEs-De midterm workshop, but also at other opportunities).Since this is not practicable (i.e., an optimal ensemble might consist of different simulations for different impact research questions) another approach is needed.In the ReKliEs-De project an analysis of ensemble stability is conducted.The aim of this analysis is to identify the necessary number of climate model simulations which represent a stable spread for a specific climate change signal, i.e., a spread which is largely insensitive (within predefined thresholds) to a further expansion of the simulation ensemble.The resulting minimum ensemble size will probably differ for different variables, since multi-model spread is much larger for some parameters (e.g., precipitation) than for others (e.g., temperature).This would at least give the users a number they could use to determine whether their results cover the whole probability range or not.
With respect to displaying climate model ensemble results, several users at the project workshop preferred displays of the multi-model ensemble as discrete model results instead of a shaded band of results.An example graphic (http://reklies.hlnug.de/fileadmin/tmpl/reklies/dokumente/workshop/Juni_2016/Poster_3.2.pdf, Fig. 3) showing percentage precipitation change in the form of a histogram of positive and negative changes, color-coded for direction of change (blue for increase, red for reduction) and significance or absence of it (dark colored for significant changes, light colored for nonsignificant changes), received quite positive evaluations.This was despite the fact that the single models contributing to the histogram were not named.Such a diagram enables a straightforward visual interpretation.It allows assessing the significance of the simulations contributing to the ensemble, and clustering of model results or identification of outliers by the users while retaining the decision on what part of the information to use or what to discard.They could take into account only the simulations within a certain range, e.g., between the 15th and 85th percentile of all climate change signals, or have the option to include or exclude extremes/outliers.

Looking at the problem from the opposite direction
There is much to be gained from improved communication and dissemination of climate model ensemble results and their interpretation.However, there is a gap between the climate model output and the impact model input or the information the advisors need which cannot be bridged from one side only.Some discussions at the user workshop considered options for using climate model results in impact research or decision-making support despite the existing limitations of the available climate change information.
The responsibility of the impact researcher or policy advisor is to identify the critical thresholds for the respective impact assessment or decision.Methods to do this particularly include sensitivity studies.These might be using past extremes (e.g., Rammig et al., 2015;Bastos et al., 2014;Wagner et al., 2014), simulation of successive recurrence of past extremes (e.g., LfU, 2017), identifying climate response surfaces for the current climate (Fronzek et al., 2011;Weiß, 2011, in some cases termed climate envelopes, Kölling, 2007;Ferrise et al., 2011) or simulations using synthetic driving data either for the climatic component (e.g., Seneviratne et al., 2002) or for the impact system component (e.g., Heck et al., 2001 for actual versus potential vegetation).However, extrapolation of observed trends does not allow a reliable assessment of the state of the system to future conditions, at least for systems with nonlinear interactions (e.g., Vicca et al., 2014, for soil-CO 2 efflux).
The identification and quantification of critical threshold values of the impact systems are prerequisites for developing new (and hopefully better-constrained) climate change indices from the climate simulations.If, for instance, what constitutes a "dry summer" or an "extreme rainfall event" can be quantified, it might be possible to better serve the impact modelers' needs with the necessary user-tailored information.
In many cases it might not be possible to obtain an unequivocal answer whether the threshold will be exceeded or not from an ensemble of climate model results, independent of their spatial resolution, applied bias corrections, or other terms.One should bear in mind that in all cases an ensemble will only provide a likelihood of exceeding or not exceeding a certain threshold.It remains the responsibility and expert judgment of the impact researcher or advisor to determine the relevant threshold and to decide which risk in terms of likelihood of threshold exceedance the system could or should take and which to avoid.

Conclusions
Several climate service institutions and research project consortia have held user workshops on using regional climate change data.Within the German community, in particular, DWD and GERICS have conducted such workshops.The results from the workshops (not published) already formulated a number of questions and problems that were again brought to the ReKliEs-De user workshop.These questions and problems encompassed among others the strong need for support in understanding the strengths and weaknesses of climate models and of their results, the request for reducing the ensemble size or for a "standard ensemble of climate simulations", and the request for support in interpreting ensemble results.
In all those workshops, a recurring theme was the improvement of the communication between scientists and practitioners to find or define a "common language" and to explain complex facts to "time-poor generalists" (Black, 2015).This also corresponds to experiences gathered in other countries (Formayer et al., 2011;MeteoSwiss, 2016) and projects (e.g., Swart and Avelar, 2011;Hewitt et al., 2013;Groot et al., 2014;Roessler et al., 2017).Thus, even though we focus on the ReKliEs-De project workshop, the questions and problems that were formulated there are shared by other users -albeit perhaps to different degrees.
H. Huebener et al.: Deriving user-informed climate information

Conclusions for climate modelers
As stated clearly by many users at the workshop, climate model data users need a basic understanding of the strengths and weaknesses of climate models in general and of certain models or methods (e.g., statistical versus dynamical downscaling methods) with the data themselves.Thus, climate data providers (in the definition of Roessler et al., 2017) or climate service providers (in the definition of the GFCS, www.wmo.int/gfcs/)need to deliver easily understandable and condensed climate model information (one user at the workshop suggested the concept of "climate model fact sheets").This information should refrain from using the discipline-specific climate modeling "lingo" and should instead try to be as simple and as short as possible.The climate data providers should explain clearly the model strengths and weaknesses (like different biases in temperature or rainfall), assisting the users from other disciplines or from outside sciences in drawing their own conclusions.It will remain challenging to present the requested information in a short and easily understandable form.
Climate modelers have developed objective methods for sub-ensemble selection or exclusion of models from a subensemble (e.g., Overland et al., 2011;McSweeney et al., 2015;Dalelane et al., 2017).A lot of expert knowledge is applied to decide for or against using a certain model in an analysis.It is essential to make this expert knowledge transparent to the users.
Climate modelers should communicate their level of trust in their own research results (e.g., using the IPCC language of "virtually certain" and so forth).It is basically true that there might still be unresolved questions or mechanisms which cannot be modeled, which are not yet understood or are not even suspected.Nevertheless, climate change science still confirms -with increasing complexity and confidence -central climate change research results over and over again.With respect to mechanisms that are known but not yet included in (all) climate models (like possible feedbacks between melting of permafrost and GHG concentrations or between climate and land use changes), it is possible to formulate them and give at least a direction of the expected (or even suspected) deviation from the existing research results.It is even possible to formulate hypotheses (or lack thereof) regarding processes which are not yet fully understood.However, the climate modeling community should formulate the remaining scientific doubts carefully to avoid the misinterpretation that all results derived from decades of climate change research are in question.There are cases where simulated climate change signals may have opposite directions (like precipitation in some regions).These are results that can and must be interpreted as still open questions.In a case like this, impact research and policy advice have to take both possible change directions into account.However, a statement like "it could also be completely different" should be avoided, since it might be misunderstood in the general perception as if the complete climate change research results were obsolete.

Conclusions for impact researchers
In some impact research subjects, knowledge about the relative importance of different drivers of the system (like temperatures, soil moisture, or rainfall in different phases of a biological cycle) are not clear from previous research and might not be easy to determine.Obviously, many natural systems are exceedingly complex.Past data may allow only very specific conclusions to be drawn with respect to the survival success or failure of the system.Apparently, there is a need for more in-depth analysis of the respective impact systems to identify the relevant parameters and the critical thresholds that lead a system into a new and possibly less favorable state.Sensitivity studies (particularly modeling or laboratory studies) can help disentangle the complex interactions in a system.The identification and quantification of critical threshold values for the impact systems should be a research priority.It would constitute an important step towards bridging the gap between the climate models' output (and the associated spread and other aspects that are often considered limitations by the impact researchers) and the necessary input for impact research and policy advice.
The climate impact research community is increasingly using ensembles of climate model simulation results.However, there are still a few cases where only one climate model projection is used to drive an impact model.In these cases, overconfidence in the results should be avoided.If it is not possible to use a large ensemble of driving climate model simulations, at least the performance of the model simulation used as input for the impact research should be assessed with respect to a large ensemble.Here are some key questions: is the simulation used as a driving condition at the warm or cool end of the ensemble's spectrum?Is it rather moist or dry compared to other models?With this information it is possible to arrive at an informed guess of the position of the impact results in a range of possible results.It is the basis for assessing and communicating the probabilities of the impact systems' resilience or lack thereof regarding the projected future climate change.

Conclusions for advisors
Advising policy, economy and society on how to deal with climate change and its expected impacts requires an understanding of the scientific background of the climate information.This profound understanding should and could be requested from the existing climate service institutions.If necessary, new "border institutions" (Munaretto and Huitema, 2015) should be developed.If possible, scientific experts in climate change modeling or analysis should be incorporated into decision making or advising groups to provide the sci-entific understanding of the climate research results directly to the user groups.
However, advising policy, economy or society on expected climate change impacts does not require the perfect "precise amount of change" knowledge.As in many other sectors of decision making, the stakeholders need to decide under uncertainty.In our everyday decisions with uncertainty, we think about risks before thinking about numbers.What were the potential impacts of a decision which would be either too risky or too risk averse?If the risk is getting soaked because you left your umbrella at home, then it might be bearable.If, however, the risk is something threatening the survival of numerous people, then the decision should be much more risk averse.Decision support with respect to climate change should be treated like decision support with respect to other information with a probability distribution like demography, economy or others.The established decision-making processes in these areas can be taken as a first approximation for the decision-making process with respect to climate change.