Study of NWP parameterizations on extreme precipitation events over Basque Country

The Weather Research and Forecasting model (WRF), like other numerical models, can make use of several parameterization schemes. The purpose of this study is to determine how available cumulus parameterization (CP) and microphysics (MP) schemes in the WRF model simulate extreme precipitation events in the Basque Country. Possible combinations among two CP schemes (Kain–Fritsch and Betts–Miller–Janjic) and five MP (WSM3, Lin, WSM6, new Thompson and WDM6) schemes were tested. A set of simulations, corresponding to 21st century extreme precipitation events that have caused significant flood episodes have been compared with point observational data coming from the Basque Country Automatic Weather Station Mesonetwork. Configurations with Kain–Fritsch CP scheme produce better quantity of precipitation forecast (QPF) than BMJ scheme configurations. Depending on the severity level and the river basin analysed different MP schemes show the best behaviours, demonstrating that there is not a unique configuration that solve exactly all the studied events.


Introduction
In the last years, several events of heavy precipitation over the Basque Country have caused different flood episodes.The complex orography and rivers characteristics, among other factors, favour the occurrence of these episodes.Figure 1 shows the location of the studied river basins.In order to understand the occurrence and dangerousness of these flood episodes, a full study was made including synoptic, mesoscale information and local meteorological characteristics.
In the Basque Country Agency (Euskalmet), different Numerical Weather Prediction (NWP) models run operationally (Egaña et al., 2008;Gaztelumendi et al., 2007Gaztelumendi et al., , 2009;;Gelpi et al., 2007Gelpi et al., , 2013)).One of them is the Weather Research and Forecasting model (WRF) (Skamarock et al., 2005).In this paper, we present a preliminary comparison of different microphysics and cumulus parameterization schemes in the WRF quantity precipitation forecast (QPF) for 21st cen-tury extreme precipitation events in the Basque Country.We focus on particular river basins in the Basque Country for a selection of severe episodes.Analysis and validation are based mainly on rain data from the Basque Country Automatic Weather Station (AWS) Mesonetwork (Gaztelumendi et al., 2003).
The purpose of this study is to know skill and reliability of different WRF parameterization configurations on extreme precipitation forecast, allowing us to improve the forecast tasks.
The rest of the paper is organised as follows.Section 2 describes the experimental framework used to evaluate performance of parameterization configurations.Section 3 describes the validation process and observational data.Section 4 describes the results.Finally, Sect. 5 presents some conclusions and recommendations for future work.
Published by Copernicus Publications.

System and experiment description
The performance of several cumulus and microphysics parameterizations is studied for extreme rainfall events.Combinations among two Cumulus Parameterization (CP) and five Microphysics Parameterization (MP) schemes result in ten different physics configurations for WRF-system.
Planetary boundary layer YSU-PBL, RRTM/Dudhia radiation scheme and Noah land surface model parameterizations remain unaltered for the whole set of experiments.These parameterizations are used in WRF-Euskalmet since its operational installation.
The initial and boundary conditions for the coarser grid are obtained from the Global Forecast System (GFS), run by NCEP (National Center for Environmental Prediction), 1degree analysis data.
MP schemes selected are: WRF Single-Moment 3class (WSM3), Purdue Lin (LIN), WRF Single-Moment 6class (WSM6), New Thompson (NT) and WRF Double-Moment 6-class scheme (WDM6).The WSM3 (Hong et al., 2004) categories are vapor, cloud water/ice, and rain/snow.The cloud ice and cloud water are counted as the same category, and they are distinguished by temperature.The WSM6 (Hong and Lim, 2006), LIN (Chen and Sun, 2002), and NT (Thompson et al., 2008) schemes contain prognostic equations for cloud water, rain water, ice, snow, and graupel mixing ratios.NT scheme also predicts the total concentration of ice.The WDM6 scheme (Lim and Hong, 2010) is the extended version of the WSM6 adding the prognostic of cloud and rainwater together with the cloud condensation nuclei (CCN) concentration.The inclusion of prognostic equations for the total concentration of each species is computationally demanding but it allows for a more realistic treatment of many microphysical processes.
KF scheme is a shallow sub-grid scheme that uses a mass flux approach with downdrafts and CAPE removal timescale closure, includes condensed and gaseous water detrainment, and the clouds persist over the convective time scale (Kain, 2004;Kain and Fritsch, 1990).BMJ scheme is an adjustment type scheme that generates deep and shallow convection.Relaxing is applied towards variable temperature and humidity profiles determined from thermodynamic considerations (Janjic, 1994).GD, Grell 3-D is an improved version of the GD (Grell-Devenyi) scheme.

Validation and observational data
A set of 35 severe weather episodes of heavy/persistent precipitation in the Basque Country for the 21st century have been selected.The selection criterion used is related to precipitation episodes that have caused flooding and/or damages in river basins that include highly populated areas, as Bilbao and San Sebastian surroundings.An episode of precipitation may correspond to one or more days (Table 2).
For validation purposes, objective point to point comparisons are made between simulated daily precipitation amount values versus observed at some selected stations from the AWS Basque Mesonetwork (Gaztelumendi et al., 2003).The stations were selected to ensure that the observations are representative for Kadagua and Urumea river basins (see Fig. 1).
To carry out the validation and analysis process, several scatter plots and graphs are prepared, including statistical continuous parameters, to know the behaviour in quantity precipitation forecast as MAE (Mean Absolut Error) of the absolute values of the individual forecast errors.RMSE (Root Mean Square Error) is more sensitive to large forecast errors than MAE.NRMSE (Normalized RMSE) facilitates the comparison between datasets or models with different scales; the approach taken is to normalise by the mean value of the observations.More interesting to us than errors in quantity, is proper detection of severe weather forecast events operationally.A useful summary of the forecast of observed weather events can be presented in a contingency table, which does not constitute a verification method by itself, but provides the basis from which useful scores can be obtained.
Contingency tables are useful to understand dichotomous (yes/no) forecasts, yes (event will happen), no (event will not happen), rain is a common example of this type.The four combinations of forecasts (yes or no) and observations (yes or no) are: hits (event was forecast to occur, and did occur), false alarms (event was forecast to occur, but did not occur), misses (event was forecast not to occur, but did occur) correct non-events (event was forecast not to occur, and did not occur).
A large variety of categorical statistics are computed from the elements in the contingency tables to describe particular aspects of forecast performance.We have worked with Proportion Correct score (PC), Probability of Detection (POD), False Alarm Rate (FAR), Critical Success Index (CSI) and Heidke Skill Score (HSS).Proportion Correct score indicates what fraction of the forecasts was correct.It is simple and intuitive, and heavily influenced by the most common category (possible values Perfect = 0, No skill = 1).Probability of Detection indicates what fraction of the observed yes events was correctly forecast (Perfect = 1, No skill = 0).POD is sensitive to hits but ignore false alarms.It is good for rare events and should be used in conjunction with the False Alarm Ratio index.False Alarm Ratio indicates what fraction of the predicted yes events actually did not occur, i.e. the fraction of non-events which were forecast as false alarms.It is sensitive to false alarms but ignore miss values (Perfect = 0, No skill = 1).Critical Success Index indicates how well the forecast yes events corresponded to the observed yes events.It measures the fraction of observed and forecast events that were correctly predicted.It is quite sensitive to hits and penalizes both misses and false alarm (Perfect = 1, No skill = 0).Heidke Skill Score indicates the accuracy of the forecast in predicting the correct category, relative to that of random choice.It measures the fraction of correct forecasts after removing those forecasts that would be correct due to purely random chance.(Perfect = 1, No skill = 0).
The Basque Meteorology Agency (Euskalmet) is responsible for issuing severe weather warnings in the Basque Country area.To assess the ability of the different configurations in defining the level of risk of adverse events rainfall, 4category contingency tables were created, based on Euskalmet warning system thresholds (Gaztelumendi et al., 2012) and also 11-category with regular 20 mm intervals, summarized in Table 3. PC, FAR, POD, HSS and CSI indexes related to contingency tables were calculated (see Figs. 4-6). www.adv-sci-res.net/13/137/2016/

Results
The analysis of the results show that the 9 km resolution domain, for most of the events and experiments, works better for QPF (Quantitative Precipitation Forecast) than other domains, including 3 km resolution.This behaviour can be explained by the validation methodology, based on rain-gauge comparison that penalizes high resolution precipitation patterns with good shape but with poor accuracy, and also by the characteristics of microphysics and cumulus parameterization for resolutions smaller than 10 km.
Scatter-plots give information about the correspondence between forecasts and observations and offer the advantage of presenting in a synthetic way all the statistical information in the data set.An accurate forecast will have points on or near the diagonal.
Figure 2 shows the scatter plots for the whole set of experiments (E0-E10), using all the events and stations of both river basins.If we focus on highest values of the y axis (observed), we will have a qualitative verification of configuration performance on extreme precipitation forecast.Statistical error values are summarized in Table 4.Most of the BMJ cumulus parameterizations (E1-E5) produce underestimation of precipitation for the majority of the events in the eastern river basin (Urumea), especially using NTH (E4) and WDM6 (E5) microphysics parameterization schemes.The exception is the WSM3 + BMJ (E1).The KF cumulus parameterizations (E0, E6-E10) perform forecasts more adequately.The configuration E9 with NTH microphysics parameterization makes the best characterization for 300-400 mm episodes.These episodes of highest precipitation correspond to the eastern river basin (Urumea).WSM3 (E6), LIN (E7) and WDM6 (E10) generate very large overestimation for a single episode.
In the western river basin (Kadagua), for events exceeding the 100 mm (not observed events exceeding 200 mm), the model configurations with BMJ cumulus parameterization underestimate precipitation forecast, more noticeable with WSM3 (E1), LIN (E2) and NTH (E4) microphysics schemes.Similar behaviour is observed in the NTH + KF (E9) configuration.The episodes with greater amount of precipitation are properly simulated by the reference configuration WSM3 + KF − Grell (E0).
Analysing indexes coming from contingency table, Proportion Correct (PC) index values are higher in the western river basin than the eastern.All configurations show similar values for each of the river basins.Most of the HSS indexes are higher for KF cumulus parameterization configurations and higher for the western river basin than for the eastern one (see Fig. 4).
For 11-category daily precipitation contingency tables (see Fig. 6), increasing the event severity causes skill indexes to worsen.Probability of detection (POD) of yellow-orangered cases is higher in KF cumulus parameterization (E0, E6-E10) than in BMJ cumulus parameterization configurations (E1-E5), pointing out that the precipitation events in the 0-20 mm day −1 interval are better predicted by BMJ configurations.LIN + BMJ (E2), WSM3 + KF (E6) and NTH + KF (E9) configurations show no skill in red events detection, while KF − GD + WSM3 (E0) and KF + LIN (E7) configurations seem to be the best configurations with a gen- eral behaviour from green to red but with poor results in the eastern river basin (Urumea) with higher NRMSE values.
A preliminary subjective validation was made by comparison of simulated precipitation patterns against observed precipitation fields.Observed maps have been generated using geostatistical techniques (Hernandez et al., 2003).In Fig. 7, some examples of forecasted precipitation patterns vs. observed are showed.Maximum values in eastern Basque Country are correctly located but underestimated, while quantity differences between north and south precipitation patterns (top panels) are properly simulated.Maximums of rainfalls are acceptably simulated in location and quantity, as well as other secondary patterns (medium panels).For maximums located in the east and centre of the Basque Country, the amount of precipitations and its location is correctly forecasted (bottom panels).

Conclusions and future work
Is this work, the performance of different configurations in predicting adverse precipitation episodes on daily operational numerical weather prediction has been studied.
Configurations with KF scheme for cumulus parameterization forecast better the quantity of precipitation than BMJ scheme configurations.The configurations present better performance in the western river basin than in the eastern one.Depending on the severity level and the river basin analysed various microphysical schemes show the best behaviour, implying that a single configuration is not accurate enough to simulate all analysed events.For extreme precipitation events, neither acceptable nor overall results were found.
With these results an optimal setup is not possible for operational usage in the Basque country, although poor results of BMJ cumulus parameterization configurations advise against its use.
We have just begun to analyse simulation results and to extract some preliminary conclusions.To obtain full concussions for parameterizations performance in operational mod- The number of events and the stratification should be increased, including more severe events and other categorization groups (synoptic forcing characteristics, seasonal, weather types, etc.).The subsets should contain enough cases to produce reliable verification results.If not possible, as usual for rare events, we need to include quantitative uncertainty estimations of the verification results.This will allow us to judge whether it is likely that differences in model performance are real or just an artificial outcome of sampling variability.
The number of experiments should be increased, to also test the influence of different planetary boundary layer schemes in the precipitation forecast of extreme events around the selected areas.
Some skill scores related to persistence should be used to put verification results in perspective and to show the usefulness of the analysed options for operational purposes 6 Data availability NWP data used in this study are not publicly available, but they are archived in the Basque Meteorology Agency (Euskalmet).Observed data is available at http://www.euskalmet.euskadi.net/s07-5853x/es/meteorologia/lectur.apl?e=5.

Figure 1 .
Figure 1.Basque Country, River basins and rain-gauge location (red and blue points).

Figure 2 .
Figure 2. Scatterplots of precipitation episodes, forecast versus observed, for the whole set of experiments.

Figure 3 .
Figure 3. NRMSE of precipitation (%) for western river basin (top panel) and eastern river basin (bottom panel) automatic weather stations.

Figure 4 .
Figure 4. PC and HSS (top to bottom panels) for 4-category contingency tables for each experiment in western river basin (blue bars) and in eastern river basin (pink bars).

Table 1 .
Configurations of parameterizations for tested experiments.

Table 2 .
Date (YYYYMMDD)and number of days for heavy/persistent precipitation episodes selected for the study.

Table 3 .
Daily precipitation thresholds for contingency tables and number of observed data for each category.For the 4-category contingency table, based on Euskalmet colour coded warning system, and for the 11-category one each 20 mm.

Table 4 .
Error index values for accumulated precipitation forecast for the full set of experiments, considering all the AWS located at river basins.