ASRAdvances in Science and ResearchASRAdv. Sci. Res.1992-0636Copernicus PublicationsGöttingen, Germany10.5194/asr-14-63-2017Italian codified hashtags for weather warning on Twitter – who is really using them?GrassoValentinav.grasso@ibimet.cnr.ithttps://orcid.org/0000-0002-1433-1674CrisciAlfonsoMorabitoMarcoNesiPaoloPantaleoGianniZazaImadGozziniBernardohttps://orcid.org/0000-0002-0730-095XInstitute of Biometeorology, Italian National Research Council, Via G. Caproni 8, Florence, ItalyLaMMA Consortium, Via Madonna del Piano 10, Sesto Fiorentino (FI), ItalyDISIT Lab, Distributed [Systems and internet/Data Intelligence and] Technologies Lab, Dept. of Information Engineering (DINFO), University of Florence, Italy Via S. Marta, Florence, ItalyValentina Grasso (v.grasso@ibimet.cnr.it)4April201714636917January201713March2017This work is licensed under a Creative Commons Attribution 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0/This article is available from https://asr.copernicus.org/articles/14/63/2017/asr-14-63-2017.htmlThe full text article is available as a PDF file from https://asr.copernicus.org/articles/14/63/2017/asr-14-63-2017.pdf
During emergencies, an increasing number of messages are shared through social
media platforms, becoming a primary source of information for lay people and
emergency managers. Weather services and institutions have started to employ
social media to deliver weather warnings even if sometimes this communication
lacks in strategy. In Twitter, for example, hashtagging is very important to
associate messages with certain topics; in recent years, codified hashtagging
is emerging as a practical way to coordinate Twitter conversations during
emergencies and quickly retrieve relevant information. In 2014, a syntax for
codified hashtags for weather warning was proposed in Italy: a list of
20 hashtags, realized by combining #allertameteo (weather
warning) + XXX, where final letters code the regional identification.
This contribution presents a monitoring of Twitter usage of weather warning
codified hashtags in Italy (since July 2015) and an analysis of different
contexts. Twitter messages were retrieved using TwitterVigilance, a
multi-users platform to crawl Twitter data, collect and store messages and
perform quantitative analytics, about users, hashtags, tweets/retweets
volumes. The Codified Hashtags data set is presented and discussed with main
analytics and evaluation of regional contexts where it was successfully employed.
Introduction
Social media have proved to be essential sources of information during
disasters. Many studies analyzed how social networking sites (SNS) like
Facebook and Twitter have been employed during natural hazards like
earthquakes , wild fires
, floods
,
hurricanes . People use social
media in disasters for a broad range of reasons, as also recognized by recent
studies : to have timely information that no other
media can provide ; to receive
unfiltered information ; to offer and search help and
organize emergency relief ; to seek and offer
emotional support . Social media
like Twitter may contribute to enhance situational awareness during disasters
. The amount of information
exchanged online can be overwhelming making it difficult to retrieve relevant
information. In Twitter the use of hashtags tends to reduce this effect.
Hashtags are words (or any alphanumeric string) prefixed by the symbol #
that are used in Twitter as message label. Hashtags are important to
coordinate public discussion and information sharing .
Hashtags may emerge spontaneously or may be created ad hoc by users
or by organizations as an attempt to create communities of users who discuss
around a topic. In recent years the use of codified hashtags emerged as an
issues in the field of emergency management, as proved by the publication
“Hashtags Standards For Emergencies” . The
publication suggests the adoption of a codified syntax to generate hashtags
during disasters. In Italy, during 2014, a set of codified hashtags to use on
Twitter during weather warnings was proposed. It is a list of 20 hashtags
realized by combining #allertameteo (weather warning) + XXX, where final
letters code the regional identification. The regional reference is due to
the organization of the Italian civil protection system based on the Regions
. Widespread of codified
hashtags had only been based on the commitment of local institutions, weather
services or citizens to improve communication during disasters. The first
consistent use of a codified hashtag is dated November 2013, during Sardinia
floods . In that case it was a completely user driven
adoption, as institutional communication on social media was generally
missing. In Tuscany, instead, it was the regional weather service, Consorzio
LaMMA, to firstly adopt the codified hashtag on January 2014. Hashtag use was
explained in the Social Media Policy and its adoption was prompted when
weather warnings were issued. In other regional contexts codified hashtagging
approach was employed firstly by citizens. In this work, we present an
analysis of one year monitoring of codified hashtags to assess if and where
the proposal was successful and hashtags have been adopted.
Methods
To monitor, retrieve and store all tweets containing the codified hashtags we
used the TwitterVigilance platform, developed by DISIT Lab of University of
Florence. TwitterVigilance is a tool for multi-users collection of tweets and
fast statistical analysis (http://www.disit.org/tv). TwitterVigilance
is based on the concept of “Twitter channel” defined as a set of simple and
complex search queries performed on Twitter platform via crawler. Complex
channels may consist of tens of queries, following the search query syntax of
Twitter APIs, obtained by combining keywords, users IDs, hashtags, citations
with some operators (e.g., And, Or, From).
It is worth to mention that Twitter API do not guarantee the retrieval of the
100 % of published tweets. Both Streaming API and Search API results are
limited by Twitter's rate limits. Such limitations may pose problems to
tweets retrieval for critical events where millions of messages are
published. Even if this is not the case of our study, we underline that
TwitterVigilance has a number of metrics to assess the efficiency of tweets
retrieval at the level of single channel. One of these is linked to the
estimation of retweets with respect to tweets collected. When a retweet is
retrieved, if the reference tweet is missing in the channel the latter is
requested and obtained 99.5 % of the time. For limited volume channels this
allows to have 100 % of efficiency in recall, and for medium/large channels
(over 5 million tweets) the 98 % of efficiency.
A set of monitoring channels was created to retrieve and store all tweets
containing at least one of the 20 hashtags and other tweets useful to
strengthen research assessment. The main channel is the “Codified
Hashtags” (CH) channel where tweets are retrieved following a multiple query
parameter corresponding to the list of codified hashtags: #allertameteoPIE
(Piedmont); #allertameteoVDA (Valle d'Aosta); #allertameteoLIG (Liguria);
#allertameteoLOM (Lombardia); #allertameteoVEN (Veneto); #allertameteoTAA
(Trentino Alto Adige); #allertameteoFVG (Friuli Venezia Giulia);
#allertameteoER (Emilia ROmagna); #allertameteoTOS (Tuscany);
#allertameteoMAR (Marche); #allertameteoLAZ (Lazio); #allertameteoUMB
(Umbria), #allertameteoABR (Abruzzo); #allertameteoMOL (Molise);
#allertameteoCAM (Campania), #allertameteoBAS (Basilicata);
#allertameteoPUG (Puglia); #allertameteoCAL (Calbria); #allertameteoSIC
(Sicily); #allertameteoSAR (Sardegna). Other channels were created to
retrieve: tweets related to bad weather conditions; tweets with the hashtag
#weather; tweets sent by users related to commercial and/or public weather
services or weather forecasters. Monitoring period started on 1 July 2015 and
ended on 30 June 2016. The data volume of the Codified hashtags channel is
much smaller compared to that of channels having common-sense words as query
parameters (see Fig. ). The first one is by definition a channel collecting information
produced only during high impact events, whereas other channels collect
tweets about bad weather in general (1 843 095 tweets published about bad
weather conditions respect to only 25 185 tweets using codified hashtags).
Temporal distribution of the four monitored channels.
The CH data set was analyzed for main metrics: activity pattern over time;
volume of different tweets typology over time, differentiating by original
tweets (original messages sent by user) and retweets; volume of mentions and
replies; volume of URLs in tweets; combined metrics like ratio native
tweets/retweets. For each data set the number of “active unique users” was
also evaluated, defined as the number of unique users sending original
tweets. Respect to temporal distribution of messages, we computed an Activity
Rate. This is defined as the percentage of days, during the
monitored period, in which at least one tweet containing the hashtag was
published. Visibility metrics were also calculated. In particular: number of
favourited tweets; most retweeted users and most mentioned users
. Another component of the analysis
was to identify categories of engaged users in the different regional
contexts. On this purpose we coded manually the data set of active unique
users. The aim was to classify users into
main categories and accordingly verify their participation and active role in
the conversation around the codified hashtags. Coding was performed by
manually annotating accounts depending on their affiliation, as declared in
the profile description available on Twitter. We considered six classes of
unique users deemed as relevant for weather related emergency management and
fitting the purposes of this work. Considered categories were: Institutions
(governments and public agencies); Media (tv, radio, news and online media);
Weather (weather forecasting services or weather enthusiast associations);
Volunteers-Non Governmental Organization (NGOs active in the field of rescue
and emergency management); Citizens (accounts of not affiliated individuals;
not belonging to any of the above). The category “BOT” was also
considered to identify accounts managed by software agents automatically
publishing updates or retweeting users.
Results
During the monitored period a total amount of 25 185 tweets was collected in
the Codified Hashtags channel. Native tweets were 7569 and retweets were 17 616,
corresponding to the 70 % of retrieved messages. This high retweeting
rate is in line with previous studies which recognized it as a typical
pattern of social media use during disasters
. Tweets were published by
6674 Unique Users but only 21 % of them were true “Active users”, as shown by
Table 1. Around 80 %, instead, participated only by sharing tweets. The
majority of retrieved messages was related to few high impact weather events
occurred in summer and fall 2015. The highest daily peak of the channel was
reached on 1 October 2015, during Sardinia floods, with 3189 tweets
collected by TwitterVigilance Codified Hashtags channel.
Main features.
Total tweets25 185Native tweets7569Retweets (RT)17 616Unique users6674Active users on total1402Active users % on total21 %Most used hashtags and pattern of use
Looking at the most used codified hashtags it is very clear that few hashtags
have been thoroughly adopted. As it is showed in Fig. 2, where few long
bars (high occurrence) are followed by many shorter ones (low occurrence).
During the monitored period, in the majority of regional contexts codified
hashtags were poorly adopted or not adopted at all. Among the regions lacking
in usage there are small ones, with limited geographic extension and scarce
population, but also major Italian Regions like Lazio, Lombardia, Campania or
Puglia where the data showed that codified hashtag was not employed.
Number of tweets published for each codified hashtag.
Considering this study, the most used codified hashtags are:
#allertameteoTOS (#TOS), that is the top one
with 7841 tweets collected in the monitored period and 1448 unique users;
#allertameteoSAR (#SAR), the second one with 5977 tweets:
but it counts on the higher numbers of unique users 2302;
#allertameteoCAL (#CAL), that is the third one with
3549 tweets and 1678 unique users;
#allertameteoLIG (#LIG), with 3154 tweets collected in the monitored period
and 774 users;
#allertameteoER (#ER), with 1447 tweets collected in the monitored period
and 479 users;
#allertameteoSIC (#SIC), with 980 tweets collected in the monitored period
and 403 users.
As shown in Table 2, Tuscany, Sardinia and Calabria are the regions where the
codified hashtag was more adopted. The three contexts show different patterns
of use. While in Tuscany regional codified hashtag (#TOS) gained the greater
number of occurrences (more tweets in the given period), Sardinia (#SAR) is
the region were unique users were the most, more people engaged. Sardinia and
Calabria showed also higher retweets occurrence (79 and 80 % of total
tweets), compared to #TOS, but a lower publication rate per authors, less
than 3 tweets per author in #SAR and #CAL, compared to more than 10 tweets
per unique author in Tuscany. Activity Rate as well varied greatly: with a
70 % of active days in Tuscany, compared to 24 and 29 % in Calabria and
Sardinia. For Sardinia and Calabria most of the tweets have been published
along a short period of time characterized by high impact weather events. The
two regions were in fact theater of devastating floods: 1 October 2015 in Olbia,
Sardinia, and 24 August 2015 in Rossano, Calabria. On
the contrary, the amount of tweets in Tuscany is the outcome of a more regular
use of the hashtag during the whole monitored period, as proven by the
activity rate at almost 70 %. The data sets of codified hashtag in Liguria (#LIG),
Sicily (#SIC) and Emilia Romagna (#ER) present a suitable number
of tweets, sign of adoption but less diffused use. The ratio of tweets per
user is around 5 in #ER and 4 in #SIC and #LIG; activity rate is around
30–35 % for #LIG and #ER but only 11 % in #SIC. These results confirm
what reported by about the role played by hashtags
in the formation of ad hoc publics. These networks of people or
communities can be ephemeral and arise in response to emergencies and crises,
like in the case of Sardinia and Calabria, or they can be more stable, like
in the case of Tuscany. In Sardinia, as it is shown also in next section, the
use of the codified hashtag was driven by citizens and the role of
institutions was only marginal and secondary. In Tuscany, on the other hand,
institutions had a primary role in the hashtag diffusion as they used it
every time a weather warning was issued.
Next section on users and contexts may give more insights to explain this difference.
Geo annotation of 100 most active users (authors and mentions).
RegionalUniqueMentionedannotationauthorsusersTuscany3721Liguria1214National1124Sardinia1114Emilia Romagna88Calabria48undefined43Lombardy32Friuli V. G.22Piedmont21Sicily22Basilicata10Campania10Lazio11Puglia10Users and contexts
To explain the difference in use we examined the typology of active unique
users engaged in the hashtags communities. On this purpose within the sub set
of six most used codified hashtags (#TOS, #LIG, #SAR, #SIC, #LIG, #ER)
we selected the first 100 most active unique users, those contributing with
more original tweets. Top 100 authors of the data set were manually annotated
in different categories to better describe the channel and to identify the
communication pattern of different users. Authors were classified into six
categories, as explained in Sect. 2. The accounts of top 100 users were
also manually coded for regional attribution (see Table 3). Almost 37 % of top users
resulted to be from Tuscany. This may be considered a further element
attesting that the adoption of the codified hashtag in Tuscany was more
remarkable. Among the other users, around 12 % resulted to be from Liguria
and 11 % for Sardinia. Accounts with a nation-wide dimension were 11 % and
they were mainly Media accounts and some commercial forecasting service
accounts. They were also the most mentioned users (24 %) in the top
100 authors (Table 4). As second step we annotated those 100 unique users
following the identified categories. As showed in Table 4, Institutions
revealed to be the most active (27 %), followed by Citizens (26 %) and Media
(18 %). Half of these very active Institutional accounts were from Tuscany.
Institutions engagement around the codified hashtag in Tuscany sustained the
formation of a more stable community around the #allertameteoTOS. In this
sense it was important the engagement of Tuscany weather service account
(@flash_meteo) that promoted and widespread the use of the codified
hashtag for weather warnings. Regularly adopted in warnings, codified hashtag
spread to local institutions and volunteers in charge of emergency management
and was integrated into official alert communications. Institutional use
boosted CH adoption by media and citizens accounts.
This pattern of communication, that emerged only in Tuscany, appears to be a
case of “regular use”, as also confirmed by the highest Activity Rate (69 %),
by the highest number of tweets published during the monitored period and by
the highest ratio of tweets per user (10). Institutions were the key players
leading to the formation of a more stable community of users. In
Sardinia, instead, the hashtag adoption followed a bottom-up approach and the
network of users was more accidental and ephemeral. The codified hashtag was
proposed by users and it get spread due to single influencers and media
accounts. Institutions did not showed a specific hashtag strategy during the
emergency or afterwards. Like what happened in Calabria, the hashtag adoption
revealed a kind of “burst use” related to exceptional and isolated situations
linked to the occurrence of a disaster. In these two case, in fact, we find
the highest number of engaged users and high volumes of tweets per day, but
only during the limited period of the emergency. Activity rate was in fact
around 25–30 % during the whole monitored period. In other contexts hashtag
use was much lees rooted. In Emilia Romagna and Liguria activity rate was
about 35 %, an indicator of a weak engagement by the community; temporal
distribution of messages, though, was more regular respect to Sardinia and
Calabria. Institutions were not fully engaged in this adoption but they
started, especially in Emilia Romagna. In Liguria the regional weather
forecasting service created a Twitter account deputed to weather warnings.
The account publishes automated updates about weather alerts but tweets do
not include any specific hashtag. In Sicily adoption appears even weaker,
with only 11 % of activity rate. In other contexts hashtags were poorly used
during the monitoring period; contexts like Piedmont, Basilicata and Lazio
showed a still timid use. In all these cases, institutional communication on
Twitter appeared almost missing. When institutions' role is missing, the
codified hashtag does not spread. In some cases citizens may take the lead in
communication practices with the aim of organizing the community efforts to
tackle the emergency during the response and recovery phase. However, a
bottom-up approach does not allow to work on the preparedness phase, which
needs to be coordinated and organized by the deputy institutions and has to
rely on stable network of users.
Users' category of top 100 accounts (as authors and mentions).
Analyzing the data collected with the help of the TwitterVigilance platform
during the monitored period (from July 2015 to June 2016) it emerges that
only six out of the twenty proposed codified hashtags for weather warning
showed a significant adoption. Highest number of tweets was reached by #TOS,
which accounts for 30 % of the whole Codified Hashtags data set.
Following idea that hashtags may sustain the formation of
ad hoc publics, we may say that in most of the contexts codified
hashtags only created “ephemeral” communities, arising in response to an
emergency; in Tuscany, instead, Institutions engagement sustained the
creation of a more stable network of users, a sort of long-term community of
practice deputed to spread weather warnings, share advice and recommendations
as to be prepared in case of emergency.
The analysis of regional contexts highlighted different pattern of usage.
Tuscany appears to be a case of “regular use”. In Tuscany codified hashtag
was actually adopted and institutions had a primary role in building the
hashtag-community. Sardinia and Calabria revealed on the contrary a kind of
“burst use” related to the occurrence of a disaster. Twitter activity around
the hashtag is triggered by extraordinary circumstances and is largely
sustained by Citizens. Those are also the contexts where Institutions appear
less engaged in hashtag adoption and in Twitter in general. In other Regions
the codified hashtag for weather warning showed to be poorly used, Emilia
Romagna, Liguria, Sicily and Piedmont, or never used, Campania, Lombardia,
Veneto ecc. Many different reasons could help explaining this diversity,
starting from population disparity, geographical digital divide, difference
in social media use in urban and rural contexts. One reason could also be
ascribable to different regional climatic conditions. Regions on the
Tyrrhenian Sea, like Liguria, Tuscany, Calabria, Sardinia, Sicily are more
exposed to exceptional rains and consequent flash floods or major flooding.
In these contexts, institutions and citizens probably have found themselves
to face more often this kind of emergencies and turned to social media as a
new way to cope with high impact weather.
Code used for Twitter analytics is available at: https://github.com/valenitna/EMS_codified.
Data are available at: https://github.com/valenitna/EMS_codified.
The authors declare that they have no conflict of interest.
The article reflects only the authors' view and the European
Commission is not responsible for any use that may be made of the information
it contains.
Acknowledgements
This article is part of Phd research work of Valentina Grasso. It represents
also one of the several results of the CARISMAND project which has received
funding from the European Union's Horizon 2020 research and innovation
programme under grant agreement No. 653748.
Edited by: D. Cotgrove
Reviewed by: F. Comunello and one anonymous referee
References
Bruns, A. and Burgess, J. E.: The use of Twitter hashtags in the formation of
ad hoc publics, in: Proceedings of the 6th European Consortium for Political
Research (ECPR) General Conference 2011, 25–27 August 2011, Reykjavìk, Iceland, 2011.
Bruns, A. and Burgess, J. E.: Crisis communication in natural disasters: The
Queensland floods and Christchurch earthquakes, Twitter Soc., 89, 373–384, 2014.
Bruns, A. and Highfield, T.: Blogs, Twitter, and breaking news: the produsage
of citizen journalism, in: Produsing Theory in a Digital World: The Intersection
of Audiences and Production in Contemporary Theory, Vol. 80, Peter Lang
Publishing Inc., 15–32, 2012.
Bruns, A. and Stieglitz, S.: Towards more systematic Twitter analysis: Metrics
for tweeting activities, Int. J. Social Res. Methodol., 16, 91–108, 2013.
Bruns, A. and Stieglitz, S.: Metrics for understanding communication on Twitter,
Twitter Soc., 89, 69–82, 2014.
Fraustino, J. D., Liu, B., and Jin, Y.: Social media use during disasters: a
review of the knowledge base and gaps, National Consortium for the Study of
Terrorism and Responses to Terrorism, 2012.Grasso, V. and Crisci, A.: Codified Hashtags for Weather Warning on Twitter: an
Italian Case Study, PLOS Currents Disasters, 10.1371/currents.dis.967e71514ecb92402eca3bdc9b789529, in press, 2016.Horrigan, J.: Relief donations after Hurricanes Katrina and Rita and use of the
Internet to get disaster news, Pew Internet & American Life Project, Wahington,
http://www.pewinternet.org/~/media//Files/Reports/2005/PIP_Katrina.DateMemo.pdf.pdf
(last access: January 2017), 2005.
Hughes, A. L., St Denis, L. A., Palen, L., and Anderson, K. M.: Online public
communications by police & fire services during the 2012 Hurricane Sandy, in:
Proceedings of the 32nd annual ACM conference on Human factors in computing systems,
26 April–1 May 2014, Toronto, Canada, 1505–1514, 2014.
Ireson, N.: Local community situational awareness during an emergency, in:
2009 3rd IEEE International Conference on Digital Ecosystems and Technologies,
1–3 June 2009, Istanbul, Turkey, 49–54, 2009.
Kavanaugh, A. L., Fox, E. A., Sheetz, S. D., Yang, S., Li, L. T., Shoemaker, D.
J., Natsev, A., and Xie, L.: Social media use by government: From the routine
to the critical, Govern. Inf. Quart., 29, 480–491, 2012.
Kodrich, K. and Laituri, M.: Making a connection: Social media's key role in
the Haiti earthquake, J. Commun. Comput., 8, 624–627, 2011.
Liu, B. F., Jin, Y., and Austin, L. L.: The tendency to tell: Understanding
publics' communicative responses to crisis information form and source, J. Publ.
Relat. Res., 25, 51–67, 2013.
Merrifield, N. and Panechar, M.: Uncertainty Reduction Strategies via Twitter:
The 2011 wildfire threat to Los Alamos National Laboratory, in: Proceedings
from AEJMC Annual Conference, 9–12 August 2012, Chicago, 2012.
Miglietta, M. M. and Rotunno, R.: An EF3 Multivortex Tornado over the Ionian
Region: Is It Time for a Dedicated Warning System over Italy?, B. Am. Meteorol.
Soc., 97, 337–344, 2016.
OCHA: Hashtag Standards For Emergencies, Tech. rep., United Nations Office for
the Coordination of Humanitarian Affairs, UN OCHA, New York, 2014.
Parisi, L., Comunello, F., and Amico, A.: Social media e comunicazione di
emergenza, chap. #allertameteoSAR: analisi di un hashtag di servizio tra
dinamiche di influenza e nuove forme di engagement, Guerini Editore, Milano, 2014.
Procopio, C. H. and Procopio, S. T.: Do you know what it means to miss New
Orleans? Internet communication, geographic community, and social capital in
crisis, J. Appl. Commun. Res., 35, 67–87, 2007.
Smith, B. G.: Socially distributing public relations: Twitter, Haiti, and
interactivity in social media, Publ. Relat. Rev., 36, 329–335, 2010.Starbird, K. and Palen, L.: Voluntweeters: Self-organizing by digital volunteers
in times of crisis, in: Proceedings of the SIGCHI Conference on Human Factors
in Computing Systems, 7–12 May 2011, Vancouver, Canada, 1071–1080, 2011.
Starbird, K., Palen, L., Hughes, A. L., and Vieweg, S.: Chatter on the red: what
hazards threat reveals about the social life of microblogged information, in:
Proceedings of the 2010 ACM conference on Computer supported cooperative work,
6–10 February 2010, Savannah, USA, 241–250, 2010.
Stephens, K. K. and Malone, P. C.: If the organizations won't give us information:
The use of multiple new media for crisis technical translation and dialogue,
J. Publ. Relat. Res., 21, 229–239, 2009.
Sutton, J., Palen, L., and Shklovski, I.: Backchannels on the front lines:
Emergent uses of social media in the 2007 southern California wildfires, in:
Proceedings of the 5th International ISCRAM Conference, Washington, D.C., 624–632, 2008.
Vieweg, S., Hughes, A. L., Starbird, K., and Palen, L.: Microblogging during
two natural hazards events: what twitter may contribute to situational awareness,
in: Proceedings of the SIGCHI conference on human factors in computing systems,
10–15 April 2010, Atlanta, USA, 1079–1088, 2010.Visconti, G. and Marzano, F. S.: An independent overview of the national weather
service in Italy, B. Am. Meteorol. Soc., 89, 1279, 10.1175/2008BAMS2372.1, 2008.
Yates, D. and Paquette, S.: Emergency knowledge management and social media
technologies: A case study of the 2010 Haitian earthquake, Int. J. Inform.
Manage., 31, 6–13, 2011.