The Weather Research and Forecasting model (WRF), like other numerical models, can make use of several parameterization schemes. The purpose of this study is to determine how available cumulus parameterization (CP) and microphysics (MP) schemes in the WRF model simulate extreme precipitation events in the Basque Country. Possible combinations among two CP schemes (Kain–Fritsch and Betts–Miller–Janjic) and five MP (WSM3, Lin, WSM6, new Thompson and WDM6) schemes were tested. A set of simulations, corresponding to 21st century extreme precipitation events that have caused significant flood episodes have been compared with point observational data coming from the Basque Country Automatic Weather Station Mesonetwork.
Configurations with Kain–Fritsch CP scheme produce better quantity of precipitation forecast (QPF) than BMJ scheme configurations. Depending on the severity level and the river basin analysed different MP schemes show the best behaviours, demonstrating that there is not a unique configuration that solve exactly all the studied events.
In the last years, several events of heavy precipitation over the Basque Country have caused different flood episodes. The complex orography and rivers characteristics, among other factors, favour the occurrence of these episodes. Figure 1 shows the location of the studied river basins. In order to understand the occurrence and dangerousness of these flood episodes, a full study was made including synoptic, mesoscale information and local meteorological characteristics.
In the Basque Country Agency (Euskalmet), different Numerical Weather Prediction (NWP) models run operationally (Egaña et al., 2008; Gaztelumendi et al., 2007, 2009; Gelpi et al., 2007, 2013). One of them is the Weather Research and Forecasting model (WRF) (Skamarock et al., 2005). In this paper, we present a preliminary comparison of different microphysics and cumulus parameterization schemes in the WRF quantity precipitation forecast (QPF) for 21st century extreme precipitation events in the Basque Country. We focus on particular river basins in the Basque Country for a selection of severe episodes. Analysis and validation are based mainly on rain data from the Basque Country Automatic Weather Station (AWS) Mesonetwork (Gaztelumendi et al., 2003).
The purpose of this study is to know skill and reliability of different WRF parameterization configurations on extreme precipitation forecast, allowing us to improve the forecast tasks.
The rest of the paper is organised as follows. Section 2 describes the experimental framework used to evaluate performance of parameterization configurations. Section 3 describes the validation process and observational data. Section 4 describes the results. Finally, Sect. 5 presents some conclusions and recommendations for future work.
Basque Country, River basins and rain-gauge location (red and blue points).
The performance of several cumulus and microphysics parameterizations is studied for extreme rainfall events. Combinations among two Cumulus Parameterization (CP) and five Microphysics Parameterization (MP) schemes result in ten different physics configurations for WRF-system.
Planetary boundary layer YSU-PBL, RRTM/Dudhia radiation scheme and Noah land surface model parameterizations remain unaltered for the whole set of experiments. These parameterizations are used in WRF-Euskalmet since its operational installation.
The main characteristics of WRF-Euskalmet, based on WRF-ARW 3.2.1, are
Lambert projection and “two-way” nesting technique with a 3 ratio for the
four nested domains. Grid resolutions are 81 (55
Combinations used on experiments (E1–E10), showed in Table 1, were chosen based on literature review (Cossu and Hocke, 2014; Gallus Jr. and Pfeifer, 2008; Gilliland and Rowe, 2007; Hong et al., 2010; Jankov et al., 2005; Otkin et al., 2006). E0 experiment corresponds to WRF-Euskalmet operational configuration. The cumulus parameterization scheme used is a combination of Kain–Fritsch for the coarser domains (81 and 27 km) and Grell–Deyeni for the finer domains (9 and 3 km), with WSM3 scheme for microphysics parameterization.
The initial and boundary conditions for the coarser grid are obtained from the Global Forecast System (GFS), run by NCEP (National Center for Environmental Prediction), 1-degree analysis data.
MP schemes selected are: WRF Single-Moment 3-class (WSM3), Purdue Lin (LIN), WRF Single-Moment 6-class (WSM6), New Thompson (NT) and WRF Double-Moment 6-class scheme (WDM6).
The WSM3 (Hong et al., 2004) categories are vapor, cloud water/ice, and rain/snow. The cloud ice and cloud water are counted as the same category, and they are distinguished by temperature. The WSM6 (Hong and Lim, 2006), LIN (Chen and Sun, 2002), and NT (Thompson et al., 2008) schemes contain prognostic equations for cloud water, rain water, ice, snow, and graupel mixing ratios. NT scheme also predicts the total concentration of ice. The WDM6 scheme (Lim and Hong, 2010) is the extended version of the WSM6 adding the prognostic of cloud and rainwater together with the cloud condensation nuclei (CCN) concentration. The inclusion of prognostic equations for the total concentration of each species is computationally demanding but it allows for a more realistic treatment of many microphysical processes.
CP schemes are: Kain–Fritsch (KF), Betts–Miller–Janjic (BMJ) and Grell 3-D schemes.
KF scheme is a shallow sub-grid scheme that uses a mass flux approach with downdrafts and CAPE removal timescale closure, includes condensed and gaseous water detrainment, and the clouds persist over the convective time scale (Kain, 2004; Kain and Fritsch, 1990). BMJ scheme is an adjustment type scheme that generates deep and shallow convection. Relaxing is applied towards variable temperature and humidity profiles determined from thermodynamic considerations (Janjic, 1994). GD, Grell 3-D is an improved version of the GD (Grell–Devenyi) scheme.
Configurations of parameterizations for tested experiments.
A set of 35 severe weather episodes of heavy/persistent precipitation in the Basque Country for the 21st century have been selected. The selection criterion used is related to precipitation episodes that have caused flooding and/or damages in river basins that include highly populated areas, as Bilbao and San Sebastian surroundings. An episode of precipitation may correspond to one or more days (Table 2).
Date (YYYYMMDD) and number of days for heavy/persistent precipitation episodes selected for the study.
For validation purposes, objective point to point comparisons are made between simulated daily precipitation amount values versus observed at some selected stations from the AWS Basque Mesonetwork (Gaztelumendi et al., 2003). The stations were selected to ensure that the observations are representative for Kadagua and Urumea river basins (see Fig. 1).
To carry out the validation and analysis process, several scatter plots and graphs are prepared, including statistical continuous parameters, to know the behaviour in quantity precipitation forecast as MAE (Mean Absolut Error) of the absolute values of the individual forecast errors. RMSE (Root Mean Square Error) is more sensitive to large forecast errors than MAE. NRMSE (Normalized RMSE) facilitates the comparison between datasets or models with different scales; the approach taken is to normalise by the mean value of the observations.
More interesting to us than errors in quantity, is proper detection of severe weather forecast events operationally. A useful summary of the forecast of observed weather events can be presented in a contingency table, which does not constitute a verification method by itself, but provides the basis from which useful scores can be obtained.
Contingency tables are useful to understand dichotomous (yes/no) forecasts, yes (event will happen), no (event will not happen), rain is a common example of this type. The four combinations of forecasts (yes or no) and observations (yes or no) are: hits (event was forecast to occur, and did occur), false alarms (event was forecast to occur, but did not occur), misses (event was forecast not to occur, but did occur) correct non-events (event was forecast not to occur, and did not occur).
A large variety of categorical statistics are computed from the elements in
the contingency tables to describe particular aspects of forecast
performance. We have worked with Proportion Correct score (PC), Probability
of Detection (POD), False Alarm Rate (FAR), Critical Success Index (CSI) and
Heidke Skill Score (HSS). Proportion Correct score indicates what fraction of
the forecasts was correct. It is simple and intuitive, and heavily influenced
by the most common category (possible values Perfect
The Basque Meteorology Agency (Euskalmet) is responsible for issuing severe weather warnings in the Basque Country area. To assess the ability of the different configurations in defining the level of risk of adverse events rainfall, 4-category contingency tables were created, based on Euskalmet warning system thresholds (Gaztelumendi et al., 2012) and also 11-category with regular 20 mm intervals, summarized in Table 3. PC, FAR, POD, HSS and CSI indexes related to contingency tables were calculated (see Figs. 4–6).
Scatterplots of precipitation episodes, forecast versus observed, for the whole set of experiments.
NRMSE of precipitation (%) for western river basin (top panel) and eastern river basin (bottom panel) automatic weather stations.
PC and HSS (top to bottom panels) for 4-category contingency tables for each experiment in western river basin (blue bars) and in eastern river basin (pink bars).
POD
POD
The analysis of the results show that the 9 km resolution domain, for most of the events and experiments, works better for QPF (Quantitative Precipitation Forecast) than other domains, including 3 km resolution. This behaviour can be explained by the validation methodology, based on rain-gauge comparison that penalizes high resolution precipitation patterns with good shape but with poor accuracy, and also by the characteristics of microphysics and cumulus parameterization for resolutions smaller than 10 km.
Scatter-plots give information about the correspondence between forecasts and observations and offer the advantage of presenting in a synthetic way all the statistical information in the data set. An accurate forecast will have points on or near the diagonal.
Figure 2 shows the scatter plots for the whole set of experiments (E0–E10),
using all the events and stations of both river basins. If we focus on
highest values of the
Most of the BMJ cumulus parameterizations (E1–E5) produce underestimation of
precipitation for the majority of the events in the eastern river basin
(Urumea), especially using NTH (E4) and WDM6 (E5) microphysics
parameterization schemes. The exception is the WSM3
In the western river basin (Kadagua), for events exceeding the 100 mm (not
observed events exceeding 200 mm), the model configurations with BMJ cumulus
parameterization underestimate precipitation forecast, more noticeable with
WSM3 (E1), LIN (E2) and NTH (E4) microphysics schemes. Similar behaviour is
observed in the NTH
The configurations with WSM3 microphysics scheme (E0, E1 and E6) seem to
works properly, regardless of the cumulus parameterization used (see
Figs. 2–5 and Table 4), in both river basins. The NRMSE values in the
western river basin are lower for WSM3
Analysing indexes coming from contingency table, Proportion Correct (PC) index values are higher in the western river basin than the eastern. All configurations show similar values for each of the river basins. Most of the HSS indexes are higher for KF cumulus parameterization configurations and higher for the western river basin than for the eastern one (see Fig. 4).
For 11-category daily precipitation contingency tables (see Fig. 6),
increasing the event severity causes skill indexes to worsen. Probability of
detection (POD) of yellow-orange-red cases is higher in KF cumulus
parameterization (E0, E6–E10) than in BMJ cumulus parameterization
configurations (E1–E5), pointing out that the precipitation events in the
0–20 mm day
A preliminary subjective validation was made by comparison of simulated precipitation patterns against observed precipitation fields. Observed maps have been generated using geostatistical techniques (Hernandez et al., 2003). In Fig. 7, some examples of forecasted precipitation patterns vs. observed are showed. Maximum values in eastern Basque Country are correctly located but underestimated, while quantity differences between north and south precipitation patterns (top panels) are properly simulated. Maximums of rainfalls are acceptably simulated in location and quantity, as well as other secondary patterns (medium panels). For maximums located in the east and centre of the Basque Country, the amount of precipitations and its location is correctly forecasted (bottom panels).
Examples of daily precipitation distribution maps (mm), observed (left panels) and simulated (right panels) for E0 7 November 2001, E9 31 January 2015 and E6 26 February 2015 (from top to bottom panels).
Daily precipitation thresholds for contingency tables and number of observed data for each category. For the 4-category contingency table, based on Euskalmet colour coded warning system, and for the 11-category one each 20 mm.
Is this work, the performance of different configurations in predicting adverse precipitation episodes on daily operational numerical weather prediction has been studied.
Configurations with KF scheme for cumulus parameterization forecast better the quantity of precipitation than BMJ scheme configurations. The configurations present better performance in the western river basin than in the eastern one. Depending on the severity level and the river basin analysed various microphysical schemes show the best behaviour, implying that a single configuration is not accurate enough to simulate all analysed events. For extreme precipitation events, neither acceptable nor overall results were found.
Error index values for accumulated precipitation forecast for the full set of experiments, considering all the AWS located at river basins.
With these results an optimal setup is not possible for operational usage in the Basque country, although poor results of BMJ cumulus parameterization configurations advise against its use.
We have just begun to analyse simulation results and to extract some preliminary conclusions. To obtain full concussions for parameterizations performance in operational models, further research is planned including different aspects as commented in following paragraph,.
The number of events and the stratification should be increased, including more severe events and other categorization groups (synoptic forcing characteristics, seasonal, weather types, etc.). The subsets should contain enough cases to produce reliable verification results. If not possible, as usual for rare events, we need to include quantitative uncertainty estimations of the verification results. This will allow us to judge whether it is likely that differences in model performance are real or just an artificial outcome of sampling variability.
The number of experiments should be increased, to also test the influence of different planetary boundary layer schemes in the precipitation forecast of extreme events around the selected areas.
Some skill scores related to persistence should be used to put verification results in perspective and to show the usefulness of the analysed options for operational purposes
NWP data used in this study are not publicly available, but they are archived
in the Basque Meteorology Agency (Euskalmet). Observed data is available at
The authors would like to thank the Emergencies and Meteorology Directorate – Security Department – Basque Government for public provision of data and operational service financial support. We also would like to thank all our colleagues from Euskalmet for their daily effort in promoting valuable services for the Basque community. Edited by: P. Nurmi Reviewed by: S. Tijm and one anonymous referee