The FP7 project EUPORIAS was a great opportunity for the climate community to co-design with stakeholders some original and innovative climate services at seasonal time scales. In this framework, Météo-France proposed a prototype that aimed to provide to water resource managers some tailored information to better anticipate the coming season. It is based on a forecasting system, built on a refined hydrological suite, forced by a coupled seasonal forecast model. It particularly delivers probabilistic river flow prediction on river basins all over the French territory.
This paper presents the work we have done with “EPTB Seine Grands Lacs” (EPTB SGL), an institutional stakeholder in charge of the management of 4 great reservoirs on the upper Seine Basin. First, we present the co-design phase, which means the translation of classical climate outputs into several indices, relevant to influence the stakeholder's decision making process (DMP). And second, we detail the evaluation of the impact of the forecast on the DMP. This evaluation is based on an experiment realised in collaboration with the stakeholder. Concretely EPTB SGL has replayed some past decisions, in three different contexts: without any forecast, with a forecast A and with a forecast B. One of forecast A and B really contained seasonal forecast, the other only contained random forecasts taken from past climate. This placebo experiment, realised in a blind test, allowed us to calculate promising skill scores of the DMP based on seasonal forecast in comparison to a classical approach based on climatology, and to EPTG SGL current practice.
Past climate information is very often used by stakeholders who are aware of their climate-sensitivity. It generally consists in using past climate events to force their impact model, to anticipate potential crises in the next months. The logic behind is a “stationary climate” hypothesis, which could be misleading considering the current rapid rhythm of climate change.
Seasonal forecasts are an obvious alternative to take into account the climate evolution. It has effectively partly replaced the former strategy in different economic sectors in some parts of the world where seasonal predictability is reasonably good: for example in Australia, in Brazil and in Africa in the agricultural sector (Hammer et al., 2001; Lemos et al., 2002) and for water resource management (Bader et al., 2006). Over Europe, such practical applications of seasonal forecasts are very rare, mainly due to the low skill of such forecasts in this region (Doblas-Reyes, 2012).
The FP7 project EUPORIAS (
Water resource management generally requires long range forecasts, especially when it consists in optimizing a dam's water stock. This kind of programming is exactly one of the role of “Etablissement Public Territorial de Bassin Seine – Grands Lacs” (EPTB SGL) in France.
In accordance with EPTB SGL, we have chosen to deal with the Marne basin and
its dam-reservoir (Fig. 1). Built in 1974, it could stock 350 Mm
Area managed by EPTB Seine-Grands Lacs and location of the Marne lake-reservoir (grey arrow). Source: EPTB Seine-Grands Lacs.
Each year in May, EPTB SGL needs to program the dam water release for the entire dry period, i.e. up to November. This decision concretely consists in drawing a water release curve that would be approved in a decision committee and eventually applied by the dam operators. Of course this projection could be adjusted if needed, but the more this program is known in advance by the actors of the basin, the best it is.
To achieve this, EPTB SGL has defined a method based on the knowledge of the initial conditions (actual filling of the dam, previous month's upstream river flow), simple tools to extrapolate river flow evolution based on past years, and a great dose of expertise of its team that has now a long experience in such an exercise.
Their expectation is very clear: to dispose of a service that could strengthen their current practice.
Our prototype is based on a seasonal hydrological forecasting suite. In particular, it is able to deliver river flow forecasts on more than 880 river gauging stations in France.
The hydrological suite is composed by a SVAT (soil vegetation atmosphere transfer model) called ISBA, coupled to a water routing model called MODCOU (Habets et al., 2008). This suit, forced by a daily atmospheric analysis, has been used operationally at Météo-France since 2004 for real time climate monitoring of soil wetness indices or snow cover for example. Moreover, a reanalysis from 1958 has been calculated, allowing daily outputs to be compared to climate references.
These models have also been used in a forecast mode, forced by ECMWF medium range forecasts (Rousset-Regimbeau, 2007). An objective evaluation (through classical scores) has been performed for high and low flows, it has concluded that the system is efficient for early flood warning, especially for large river basins such as the Seine.
For the EUPORIAS prototype, we have forced ISBA-MODCOU by seasonal forecasts coming from ARPEGE-System 3 model (see Fig 2), to deliver river flow ensemble seasonal forecasts. This hydrological suite had been run over the long hindcast period of ARPEGE-System 3 (1979–2007), composed of 11 members and delivering 7 months' forecasts. Objective river flow scores (Singla et al., 2012) show that this suit is able to provide significant skilful forecasts up to several months, especially in river basins influenced by snow melting or groundwater.
The seasonal hydrological forecast suite used to produce riverflow forecasts.
The construction of the RIFF prototype (river flow forecast over France,
Figure 3 shows two examples of these tailored probabilistic products, presenting what would be the situation without any river flow sustain from the dam. The left one shows the forecast of monthly river flow downstream from the dam, compared to climatology. The right one is a time-integrated product that represents the forecast of the number of days below a specific river flow threshold, called “vigilance”. This threshold is specified by EPTB SGL, it corresponds to a level below which use restrictions could be taken by authorities.
These products were assessed by classical scores (Brier scores, ROC scores, correlations) firstly to evaluate their intrinsic value and secondly to provide some recommendation of using. For example, for the May initial condition forecast, it was specified to the stakeholder that the mean monthly skill downstream from the dam does not extent more than August.
The assessment of the usefulness of the climate information was a major ambition of EUPORIAS. To achieve this goal, Météo-France has proposed a specific protocol, so called Placebo Protocol, as a reference to the methodology used to test new medication.
The principle of this protocol is to provide two sets of impact forecasts, one issued with “real” seasonal forecasts (i.e. N scenarios coming from the hydrological suite forced by a climate forecast model composed of N members); the other one called “climatological forecasts” is issued from the same hydrological suite but forced by random atmospheric forcing (i.e. a selection of N years coming from the atmospheric reanalysis). These two sets are to be used in a blind test, over several past situations, to make decisions.
The main advantage of this protocol is that it ensures a perfect objectivity when comparing the decisions made with the Placebo and with Seasonal Forecasts. But of course, it could not take into account all the context in which the decision is made (specific environmental context, or pressure from the decision committee for instance).
For a specific situation, the stakeholder has to make its decision with each set of forecast. Precisely, for each situation, in addition to seasonal forecast, it had access to the filling of the dam (for certain years the filling could be incomplete) and to the last months upstream river flow (to have an insight into the recent watershed condition). Its decision consists in drawing the most appropriate release curve considering the context and the forecast.
Examples of tailored products. On the left, monthly mean river flow forecast (grey boxplot) compared to climatology (yellow boxplot) for the next seven months. On the right, forecast of the number of days of low flow (grey boxplot) compared to climatology (yellow boxplot), integrated over the whole dry season.
The quality of each decision is evaluated using a specific metric, defined in agreement with EPTB SGL. It is based on the number of days below its “vigilance” threshold. It means that for a specific decision, we have calculated the downstream river flow as the sum of “natural” flow plus the flow coming from the water release curve. We could then calculate the resulting number of days below the “vigilance” threshold.
Redoing this on a set of situation the added-value of seasonal forecasts compared to climatological forecasts could be assessed.
This protocol was applied by EPTB SGL for its May programming, for 29 situations corresponding to the 1979–2007 hindcast (also called “re-forecast”) period of the seasonal forecast model.
To complement this experiment, EPTB SGL has accepted to make decisions without any forecast information (neither seasonal forecast, nor Placebo), which corresponds more or less to its current practice.
Evaluation of the quality of the May initial conditions forecasts. The metric is applied for four periods of integration, from 1 month (May) to 4 months (May to August). The best decisions would lead to “zero day below the threshold”
In Fig. 4, we can see that for this station, the added-value of seasonal forecast compared to Placebo is anything but obvious: looking carefully at the different situations, it appears that the little advantage of seasonal forecast is very fragile. We could not reasonably conclude on this only result, based on one station.
These good results of Placebo, which may at first sight seem surprising, could be explained by the presence of aquifers, main source of predictability of this basin. Indeed, during summer, the percentage of groundwater contribution to river flow reaches 80 % (Singla, 2012) and Placebo benefits from MODCOU's good representation of the Seine aquifer, as much as seasonal forecasts.
However, the “no forecast” experiment obtains the worse scores than Seasonal Forecast and Placebo. It means that our hydrological system brings relevant information to the stakeholder, comparing to its current practice. Moreover, the difference between “no forecast” in the one hand, and Placebo and Seasonal Forecast in the other hand, is smaller for May only than for longer integrations: this reinforces stakeholder's interest in using our climate information.
This work in the framework of EUPORIAS presents a promising example of application of seasonal forecast over Europe. Our interaction with EPTB SGL, and other similar French institutions, shows the great expectation they have in seasonal forecast. It also shows the gap between the classical way of presenting seasonal forecasts and the tailoring needs, to be able to connect this climate information to their tools and decision making processes.
Once the interaction between climate experts and potential users has been set up, a crucial phase is to assess its real usefulness. “Low skill” doesn't mean “no skill”, and the little added-value (from the climate expert point of view) could sometimes correspond to a significant interest (from the stakeholder point of view). The Placebo protocol we have proposed and experimented in this project intends to objectively measure the impact of climate information on decision made. It implies a strong involvement from the stakeholder, who has to remake past decisions in a kind of theoretical framework. But at the end it allows to have a measure of the added-value (or loss) of seasonal forecast, with a metric adapted to the stakeholder. In the example presented in this paper, we have shown that our forecast system was able to improve dam management at seasonal timescale, compared to current practice.
Our technical perspectives are to consolidate our experimental system by experimenting it in a real-time mode, to update model versions (seasonal forecasts and hydrological suite) and to develop a delivery web platform.
Concerning applications, we have already enlarged our panel of potential users to be able to test other products and other potentialities of our suite: such as soil wetness or snow cover forecast.
We acknowledge Stéphane Dermerliac and Claudine Jost from EPTB Seine-Grands Lacs for their strong involvement in this experiment. This work was supported by the EUPORIAS project, funded by the European Commission 7th Framework Programme for Research, grant agreement 308291. Edited by: M. Zappa Reviewed by: L. Arnal and one anonymous referee