Stakeholder-based evaluation categories for regional climate services – a case study at the German Baltic Sea coast

Abstract. In this study, categories, dimensions, and criteria for evaluating regional climate services are derived by a participatory approach with potential service users at the German Baltic Sea coast. The development is carried out within nine face-to-face interviews conducted with decision makers, working in climate sensitive sectors at the German Baltic Sea coast. Three main groups of categories were localized which seem to matter most to the considered stakeholders and which seem to be crucial evaluation categories for regional climate services: (1) credibility, (2) relevance, and (3) appropriateness. For each of these evaluation categories several dimensions emerged, indicating certain perspectives of stakeholder demands. When summarizing these evaluation categories and their dimensions, 13 evaluation criteria for regional climate services can be derived (see Table 1). The results show that stakeholders do mainly address components other than those found in the literature (e.g. inputs, process, outputs, outcomes, and impacts). This might indicate that an evaluation, following solely literature-based (non-participative) components, is not sufficient to localize deficiencies or efficiencies within a regional climate service, since it might lead to results which are not relevant for potential users.


Introduction
Climate research has the societal obligation to inform society of potential climate-related hazards and changes (von Storch, 2009).However, there may be many other urgent societal realms which might have priority on a short-term perspective (Meinke, 2017).Thus, it is a political decision to develop and establish knowledge transfer activities on longterm issues, e.g. a climate service for a particular region.Regional climate services are one kind of knowledge transfer activity.This knowledge transfer facilitates the two-way exchange of concepts, concerns, questions, and knowledge between the scientific sphere and the regional public sphere, regarding regional climate, regional climate change and impacts (von Storch and Meinke, 2008).Following the objective to make results from regional climate research useful for decision processes, this science-stakeholder interaction entails not only information provision but also contextualization of research findings (von Storch et al., 2015).However while climate services hold the promise of integrating the state of climate science in practice, there has been relatively little practical evaluation of their activities (Vaughan and Dessai, 2014).In this context, organizations find it difficult to make informed decisions on where to devote time and resources to improve the development, delivery, and use of climate knowledge and information for societal benefit (Vaughan and Dessai, 2014).
There is a broad range of literature, addressing, at least partly, theoretical evaluation approaches in the field of regional climate services.Some articles focus on evaluating climate services in general (e.g.Vaughan and Dessai, 2014) or on its value (e.g.Clements et al., 2013).Several studies are focusing on the process of knowledge exchange (e.g.Fazey et al., 2014) or codevelopment (e.g.Kirchhoff et al., 2013;Hegger and Dieperink, 2014).Other articles focus on the evaluation of the outcome, for example of usable (climate) science (e.g.Ford et al., 2013;Wall et al., 2017;Dilling and Lemos, 2011) and some studies are focusing on the evaluation of impacts on decision making and policy (e.g.Cash et al., 2002;Evely et al., 2010).
Published by Copernicus Publications.Appropriateness Comprehensibility Availability of information in national language Availability of information in understandable language Availability of formats to adjust the complexity of the information to user needs Format Easy accessibility of filtered (reduced and tailored) information Interpersonal relations Availability of a long-term contact point, responsible for regional climate change issues in practice Fazey et al. (2014) distinguish between two broad evaluation typologies: (1) summative or (2) formative evaluation.
(1) Summative evaluations aim to provide validation at the end of projects on the merits or successes of a particular activity.In natural science, the success of the work is evaluated by scientists of similar competence for the producer of the work (peers).Peer review is generally considered necessary to ensure academic quality and is used in most major scientific journals.Various indices exist, counting the number of published peer-reviewed articles and their citation frequency (e.g.Hirsch, 2005).These indices are applied to measure and quantify the success of a researcher for evaluation and comparability purposes, e.g. for recruitment, advancement, and award of grants (Hirsch, 2005).Using these quantitative measures of success, summative approaches, however, may have limited capacity to understand the often fluid and dynamic nature of knowledge exchange (Fazey et al., 2014).In addition, universally valid evaluation criteria are missing and, often, external factors have not been taken into account (Meinke, 2017).(2) Formative evaluations, in contrast, aim to enhance reflections to improve and refine project activities by embedding evaluation in a broader knowledge exchange process (Fazey et al., 2014).Evaluations can be either participatory or non-participatory.Participatory evaluations involve multiple stakeholders in setting up questions, identifying indicators, and collecting and interpreting data.Some participatory evaluation approaches are specifically designed to break down distinctions between knowledge "producers" and "end users" (Fazey et al., 2014).Participatory approaches that engage multiple stakeholders in doing evaluations directly contribute to the process of knowledge exchange.This study contributes to a participatory evaluation approach by developing stakeholder-based evaluation categories for regional climate services.
The majority of the studies mentioned above follow a non-participatory approach, since they are solely based on literature research, aiming to develop an evaluation framework and to find suited categories of evaluation criteria (e.g.Vaughan and Dessai, 2014;Clements et al., 2013;Dilling and Lemos, 2011;Fazey at al., 2014;Ford et al., 2013;Hegger and Dieperink, 2014;Lemos et al., 2012).Only a few studies are partly participatory in order to test literature-based theories of evaluation frameworks and criteria in a second step.Cash et al. (2002), for example, develop a framework of knowledge production based on literature from social studies, bolstered by their own empirical work.Then, through consultative workshops with practitioners, managers, scientists, and decision makers, the emerging framework was tested.In later studies this evaluation framework has been applied (e.g.Cash et al., 2006).Wall et al. (2017) identified key principles in coproducing knowledge from the existing literature.To gain insight into coproducing knowledge, interviews have been carried out with climate science integrators.Afterwards the information has been synthesized from these sources to develop an evaluative framework that consists of 45 indicators grouped into input, process, output, outcome, and impact indicators.
Although gaps, missing links, and boundaries in terms of knowledge exchange between science and practice are quite frequently described (e.g.Wall et al., 2017;Lemos et al., 2012;Cash et al., 2006), participatory evaluation approaches are quite rare and not addressed to regional climate services, so far (Bruno-Soares and Dessai, 2016).In this paper we introduce a participatory approach to develop evaluation categories and criteria with contributions from potential users to overcome the gaps, missing links, and boundaries in knowledge exchange on regional climate change.
The interviews were embedded in the EU Project EU-CLEIA (EUropean Climate and weather Events: Interpretation and Attribution, http://www.eucleia.eu),aiming to develop an operational extreme event attribution service.The contribution of the Helmholtz-Zentrum Geesthacht (HZG) to this project was a stakeholder dialogue at the German Baltic Sea coast.Interviews with several stakeholder groups, workshops, and focus groups have been conducted in order to derive stakeholders' perception of the potential usefulness of a regional climate service which focusses on extreme event attribution (Schwab and Storch, 2017).This paper focusses solely on general aspects of regional climate services and related stakeholder-based evaluation criteria.

Method and data
Since regional climate research is dealing with changes in a long-term perspective, it is suggested that evaluation criteria for regional climate services are developed by involving social groups which are assumed to be in charge of longterm decisions in climate sensitive sectors, e.g.coastal defense and spatial planning (Meinke, 2017).It is assumed that these stakeholders might represent the perspective of potential regional climate service users.In order to localize potential users of a regional climate service at the German Baltic Sea coast, stakeholders from various stakeholder groups were selected according to the following criteria: (1) they are engaged in climate change adaptation, mitigation, or in weather-related risk management; (2) there is an indication of the need for regional climate information based on their field of work and its interaction with weather related impacts; (3) they have a defined role and experience as regional decision makers in a field related to climate change adaptation, mitigation, or weather-related risk management, and (4) they are not yet regional climate service users but have envisaged interest on regional climate information.The selected interviewees are engaged in civil society organisations, in ministries, authorities and municipality administrations, in education institutions, and in the economic sector.Stakeholders with these working foci were assumed to be potential users of a regional climate service, since their long-term decisions might need to be based on scientific climate research results.Nine face-to-face interviews with 10 interviewees have been conducted.In order to obtain insights into the individual dynamic nature of knowledge exchange the interviews were non-standardized, meaning that there was no predefined questionnaire.Instead, narrations of the interviewees have been initialized according to the following guidelines: (1) the interviewer introduces herself and explains the frame of the interview.The interviewee is asked to introduce him-/herself with a focus on personal and institutional background and connections to climate change in his/her region.
(2) The interviewee is asked about the perceived role of regional climate information, the sources used and additional demands.(3) The interviewee is asked about criteria which are crucial for him/her according to regional climate information.
The interviews lasted between 30 min and 2 h and were audio recorded, transcribed, and analysed according to grounded theory by Glaser and Strauss (1967).Glaser and Strauss raised the fundamental question of how theories can be founded and developed in social sciences (Lamnek, 2005).They try to encourage scientists to contribute to the development of theories.The motivation of Glaser and Strauss is the deep gap between theory and empirical research which could not be overcome despite various approaches.According to Glaser and Strauss, this gap cannot be filled through improving the method for testing theories.Instead the adequacy of theories is scrutinized, since it cannot be divorced from the process by which it is generated.Thus, one canon for judging the usefulness of a theory is how it was generated (Glaser and Strauss, 1967).To overcome this gap they suggest developing grounded theories based (grounded) on empirical data and connected categories and dimensions.Through comparative analyses (Glaser and Strauss, 1967) it was investigated whether the found categories and dimensions remained constant when the data basis was changing.The comparative analysis aims to generate theories by generalization.According to Glaser and Strauss (1967), the testing and confirmation of existing theories did not lead to more applicable theories, but the systematic development of theories based on empirical data did.The scientist should start the study by collecting impressions and data (here interviews) without biases according to certain categories or hypothesis.After a while the first categories and theoretical framings arise, which help to systemize the empirical data.
Following a participatory evaluation approach, the interviews were analysed according to grounded theory.After transcription and coding a continuous comparison process among the different interviews were carried out.Common relationships, a sampling of similarities, and different perspectives have been conducted.Based on these samples, evaluation categories and certain dimensions emerged.According to Glaser and Strauss (1967) this method it is suited to finding a more appropriate criteria for each category since it emerged from the data and thus fits quite well to real-ity (Lamnek, 2005).According to the Code of Ethics of the German Society for Sociology the anonymity of interviewees must be protected (Deutsche Gesellschaft für Soziologie, 2017).This includes not only the anonymization of the interviewees' names, but also avoids indications of their affiliation or their origin (Flick et al., 2008).Following these guidelines, the names of the interviewees were replaced by codes and indications of their affiliations were avoided.

Stakeholder-based evaluation categories
In the interviews, numerous requirements on useful regional climate information were mentioned and described.They were addressed to the source, the content and the format of the provided information.As a result of the comparative analyses of the interviews, the various raised requirements could be assigned to three main categories: credibility, relevance, and appropriateness.For all three categories several dimensions emerged from the different interviews.In the following sections the found categories and their dimensions are introduced.

Credibility
Credibility is mainly described by four different dimensions, addressing (1) the public knowledge and its scientific proof, (2) transparency, (3) expert knowledge, and ( 4) uncertainties (see Table 1).In the following subsections the different perspectives of these dimensions are described.

Dimension of public knowledge -scientific proof
The credibility of climate change information is perceived as significantly governed by the media and by public discussions.To raise trust on the existence of climate change the media plays an important role by conveying the tenor that climate change exists and has certain impacts.9. GV: For credibility, it plays a role in that the media is conveying the general tenor that climate change is taking place and has certain kinds of impacts.
However, it is assumed that the media is also causing an enviro-scare.In this context several interviewees have articulated a need for these messages and information conveyed by the media to be proven with sound scientific methods.
3. AP: There are also data that are always transported in the media, where it is said, yes, from now on we will only have heavy rainfall events.And then there is a demand on the scientific data.Is this really true?So, are there any investigations that show, yes, from now on we will have precipitation only in a very short time, but very severe?Or (. . . ) or is it perhaps also a bit of scaremongering from the media?And then one goes through these individual topics and says these are the concerns we want to investigate.Also, according to public discussions on climate change it has been stated that subjective opinions need to be proven or disproved by distinct reproducible scientific methods.
2. AD: So, I find there is still a lively public discussion, not much on such a fundamental level, but if one asks the question, to what extent this can really be attributed to this topic?Or to what extent could this have happened without climate change?
No one can answer the question.(. . . ) I think it is always a bit critical: well, climate change -is that true?If there was an instrument which is scientifically serious [and] could answer this question more objectively than these subjective statements, that would be, I believe, already helpful.This shows that an active science-based contribution to the public discussion on climate change, its drivers and its impacts would increase the credibility of certain public knowledge on climate change.In particular, scientific proof of statements in the media, public opinions and subjective perceptions would be supportive in this context (see Table 1).On the other hand this supports the results of Meinke (2017) which identified the media as the main information source of (regional) climate change.This, in turn, could mean that credibility of a regional climate service might be reduced in case climate change disappears from headlines in the media in favour of other issues with higher (short-term) priority.

Methodical dimension -transparency
Various interviewees mentioned that scientific information is often seen as a black box wherein the methods cannot be related to the communicated results.This missing transparency is perceived as a barrier for credibility.4. BQ: So for us, the whole thing behind it is, so to say, a big black box.(. . . ) You do not want to know every detail because you do not understand it anyway.High physics.But it would be good to know at least something: what are the parameters which one can adjust in order to get a different water level, to get a different sea wave, to get other wind conditions.Here we are, so to speak, left alone.It is hard to say, so this little piece is missing.
In this context scientific evidence of the human contribution to climate changes was perceived as supporting credibility in order to fill this gap.
2. AD: . . .when I could perceive serious evidence that certain climatological phenomena is, so to speak, attributable to anthropogenic climate change, then I would have to accept or would accept the projections (. . .).Then for me, engagements which try to limit the problem would be more adoptable.(. . . ) If I assume that all is the same and that it has nothing to do with us, why should I then deal with climate protection and climate change?This is somehow connected.And that would not allow many discussions to arise if there was greater -credibility.That is the wrong word, but it may be -traceability.
Also continuity according to the methods and their reproducibility after several years with sometimes changing staff was mentioned as crucial factor in the context of credibility.4. BQ: You have an order, which you give to a university, there sits a young scientist who is modelling.He (. . . ) turns the parameters a bit.And in the end he has then calculated that the sea weave is so-and-so high.And then that's it.And then after 5 years they want to look (. . . ) what he actually did there.(. . . ) The scientist is no longer at university.He's gone.And the model is still running, but the boundary conditions he used, no one can understand.Instead, one starts again at zero.After 5 years, if you want to run the model.These methodical aspects could be summarized as "transparency" according to the linkage between the communicated climate change information and the underlying methods (Table 1).Transparency can be supported by a scientific proof of human contribution to climate change and by longterm reproducibility of the methods applied.

Dimension of expert knowledge
Several interviewees stated that this science-based regional climate information should be communicated by external experts since they are more trusted than members of their own working environment.
6. DS: And, if we officially open the CO 2 Parkour for the public, we would like to combine this with a press event, and of course, it would be a good idea to have a very good external impulse here.So that we can show, these are really scenarios that are POSSIBLE.And, thus, people cannot say these are the disaster criers of [name of the institution] who always say this is the end of the world.7. ET: So, I often think, when certain information comes from outside, it weighs emotionally a bit more than the information from inside.4. BQ: Or lectures where someone from outside is available.This is just knowledge, because (. . . ) the prophet does not go to his own country and when it is said, this is the [name of the institution] which comes, that has a different significance.And this just transports the knowledge.
In contrast to the articulated need for external experts and transparency according to the underlying methods, scientists are sometimes perceived as arrogant inhabitants of the ivory tower who muzzle practitioners by implying that they are not capable of understanding the background.To obtain credibility, it was perceived as crucial that scientists make the underlying methods more transparent by explaining them on a par with each other.4. BQ: There are a few people at the universities who can explain this so that you understand it.But basically, nobody explains anything, but we have to swallow that.And I mean we're the professionals who need to deal with it, and have to swallow that somehow, all (. . . ) without having the chance to look behind that information.I had it a few years ago in Leipzig.There was another climate conference.And there was someone from [name of the institution] with whom I tried to discuss that.But since there was so much arrogance, arrogance out of this ivory tower that he says, if you do not understand that the sea level can rise this way or that way, then I am sorry.And this is classically the way to make someone silent, saying yes lack of knowledge or skill.But you do not get any further.You do not have the chance to look into any processes.This shows that the interpersonal relationship between the scientist or service provider and the stakeholder plays a crucial role in the credibility of information.Credibility can be increased when scientists are available to contribute to stakeholder events as external experts.This should be accompanied by a science stakeholder dialogue on the underlying concepts and methods, where both groups are on a par with each other (see Table 1).

Dimension of uncertainty
Missing credibility is often connected to large uncertainties in the information.This is often connected to uncertain scientific knowledge caused by the underlying method, e.g.multimodel and multiscenario ensemble methods.A range of possible future climate change scenarios are perceived as not reliable.Instead, distinct information on definite future changes is expected.In this context it was articulated that this information on future climate changes seems to be misused to scare people, while it is not even possible to get information on the present status. 5. CR: No, so the only important thing is reliable information, so really reliable statements that say it [sea level] will definitely rise this and that way.
www.adv-sci-res.net/14/279/2017/Adv.Sci.Res., 14, 279-291, 2017 5. CR: Well, they say we (scientists) can do a great job in the future and we can scare you.This is how it is often translated.But we cannot calculate (and show) how it is today.
This indicates that the impact of a science stakeholder dialogue explaining the underlying methods is limited when the applied method itself causes uncertainties.This is especially the case when regional climate scenarios are used for regional climate change communication.In this context, again, evidence associated to the recent or actual climate might increase credibility (see Table 1).

Relevance
The relevance of regional climate information was another very important issue raised by the interviewees.Many different requirements that emerged from the interviews turned out to be aggregated in a joint category of "relevance".Relevance is mainly articulated according to the respective professional environment but also within social contexts.The category of relevance can be described by four main dimensions, addressing (1) the dimension of scales, (2) the personal dimension, (3) the practical implication, and (4) the uncertainty of regional climate information (see Table 1).

Dimension of scales
The different scales of regional climate information and its potential field of application were mentioned quite often.
8. FU: On the one hand, the climate projections are too coarse, spatially and temporally, and on the other hand they are uncertain, but that is also a bit due to the matter that one does not really know what will actually happen here in 80 years.
One important association regarding climate change is the temporal dimension of the changes compared to the interviewees' own time horizon.This refers to the expected remaining lifetime, which strongly corresponds to the time horizon in which they are concerned about certain issues.
6. DS: Then they say, yes, then I won't live any more.I'm not getting that old.And that's it.5. CR: And there you are sometimes met with a shake of the head.Or they say well, let him go.We still have 20 more years, then I'm not alive anymore.
Another aspect regarding the temporal dimension of climate change information is the perception of temporal changes, which often does not correspond to a weather statistic of 30 years and its long-term changes.The latter is perceived as rather theoretical.
7. ET: And that climate change is something that you have to look at for over 30 years, so that we can determine at all whether or not there is anything that changes.This makes it very theoretical and difficult to understand for normal citizen.And so in this respect, the topic is not so easy to convey.
Several interviewees addressed a temporal mismatch between the short-term practice in their day-to-day working routine and the long-term changes of climate change.9. GV: But the first reactions were, well, climate change is something very slow but we have to deal with our daily problems.9. GV: These are people who have daily tasks of planning and execution of their work.This must be remembered.
5. CR: Such long-term matters are always difficult to deal with, with practitioners.They all live from hand to mouth nowadays.However, even in the case that long-term planning is carried out, for example as part of capital investment decisions, climate change and its possible impacts on these investments are not taken into account.
3. AP: So, now investment decisions are being implemented, which have dimensions of 40 to 50 years.And I think at the moment, no one is really concerned with the question of whether this location is the right one from the point of view of climate adaptation.
One reason for this might be that the planners are not aware of possible future impacts of climate change since they are not yet visible or not perceived so far.
2. AD: And investment decisions -that is the problem.They have the planning horizon of 50 years, right?So for now climate change is not an issue, but what is in 50 years?
Another aspect of relevant climate information is its spatial resolution and its interaction with local features.Even regional climate information is perceived as too coarse.Further processing and analyses are suggested in order to receive place-based climate information.8. FU: Which areas are at risk in the city, so where does the water run, where are sinks and also where is the sewer network perhaps somehow not good.There is a need.But that is, so to speak, already more then pure climate information.It is already the utilization of climate information.
Contrary to the articulated need for localization, a global perspective of climate change is also perceived as making the information relevant.In particular when something irreversible is happening worldwide it might be picked up by nationwide media.This was perceived to support the relevance of the information.
6. DS: Well, it is not so well known, I must say.There are, of course, also more detailed figures for coastal protection, and so on.But, of course, is not like in the world climate, where one says 4 m of sea level rise, or when one says the West Antarctica melts off and can no longer be stopped.This is, of course, something picked up by the nationwide media and the focus here is, of course, only more on the northern German coasts.

Personal dimension
Several interviewees emphasized that suited scales alone would not increase the relevance of regional climate information.In this context, the implication of climate change on private lives, the personal impairment was perceived as inevitable for generating relevance of regional climate information (see Table 1).7. ET: . . .We have a lot of information that really relates to Lübeck or which is broken down on Lübeck, but I believe that people often only wake up when again such a disaster happens.Like a flooding on the Elbe or something.Then people begin to deal with it.It is regrettable, but it is often so that people only consider changing something or preventing damage when something has already happened.
7. ET: And everyone knows it, because he/she somehow reads it in the press, but a personal concern does not actually exist.And that is why the issue is somewhat difficult to convey.
In this context the local risk perception of weather related and thus potential climate-change-related risks is perceived to be connected with the relevance of regional climate information.

AP: Yes, but on the other hand if you look at
Hamburg then of course you even have the tides and the Elbe, storm surges and so on.There it is, of course, a much (. . . ) more dangerous topic than this for us -at least it is perceived as less dangerous, right?In the Baltic Sea there is also water level variability at certain winds, and the water sometimes sloshes back.But yes, it is quite long ago that we had such a storm surge here and thus no one is thinking about it.
In particular, in regions where climate change impacts are perceived as generally rather favourable, regional climate change information is not perceived as relevant.
2. AD: So, those who will be high affected in the future will also be forced to deal with it.The fact that we do not deal with it is also a matter of the fact that there is just the feeling, well, the summers are getting more beautiful, the weather gets better -well, everything is wonderful!Of course!More tourists!Yes, more tourists!We will have the future Adriatic here.And we are also high enough.So, we won't be under water that soon.
In order to increase the relevance of regional climate information, scientifically verified personal impairment was mentioned as supportive.
1. AN: What problems emerge for us, what do we have to face somehow?This is, in my opinion, the first question anyone would ask when they begin to deal with the issue of climate change.And if I want to sensitize the population to the issue, or if I want to interest local politicians, I have to indicate the particular impairment.But I say verified impairment, so not disaster induced, but reproducible verifiable impairment.I believe this is always the first approach.

Dimension of practical implication
Another dimension of relevance is the practical, mainly jobrelated, concretion of the regional climate information.Generally, a direct link to the individual professional context was mentioned to be supportive in order to increase relevance (see Table 1).This means that the information should either be linked to the source of livelihood or to the individual day-today routine in order to directly support their practical work.9. GV: So, for a tourism manager it is only a question in the case that the storm surge has also taken the beach away.

FU: It depends on what the concrete topic is.
And the more practical this is, the more people you get from the administration.
In turn, a missing direct link of the climate information to their particular field of responsibility was perceived as the limit of relevance since many stakeholders, e.g. in administration departments, are not used to being confronted with issues beyond their assignments.8. FU: At the moment I do not see how this would help us in our daily work because we are more concerned about the practical handling of the consequences.
8. FU: So we have seen that it is just not that simple with people who have administrative tasks.It is not easy to confront them with things that go beyond their duties.
In this context an important dimension of relevance is the usage of climate information for the legitimation of personal decisions in the public arena.
4. BQ: Also to the legitimation of coastal protection measures, which in principle are of public interest, but are not always generally accepted.In this context, such basic scientific facts, which you are also working on, must always be integrated.

BQ:
We need to convince everyone who is affected by a coastal defense measure in any way.Convince them that this measure makes sense and that it is right what we do.And if you, of course, have the opportunity to contribute information or whatever to support this, then that is useful.
Another aspect in this context is the order of priority which is often high for tasks of the day-to-day routine, whereas it is lower for other more basic issues, although they are recognized as essential.
6. DS: And the people all say, I would like to know and understand it, but I have so many things to do daily, I have to complete this until next week.The other thing is nice to have and then they postpone it.I also often find myself doing this.If I know, man, today I have to finish this work or I have to answer a request.Although it is insignificant and the other is much more essential, I think, I can still read it or do it in 2 months.
Further aspects of relevant climate information within the professional context are competitive advantages.Information which increases the job-related knowledge of an individual compared to others is perceived as advantageous, which makes this information relevant.9. GV: So, as I said, you should always pay attention to the effects on the local authorities in such a way as to document that such effects are relevant to them, that the knowledge about these facts really has enormous advantages for them, yes.
But even if no link is seen between climate information and practice with regard to content and thus nobody is basically interested, climate information is still considered as relevant if the public awareness of the company or institution is increased through the media.
5. CR: Maybe it is also nice to advertise action somehow and then you can get in the newspaper again, which would be good.But I say, nobody here has a real interest.
Additional aspects of relevant climate information are financial advantages in relation to competitors.This can be information which increases the chance of getting funding for certain applied research.5. CR: What might be interesting too are funding possibilities.This is always interesting.Whether you can wave with money.Whether it is said, this can also be further investigated.You can support us.There is the Federal Government's funding guideline, in this and that area.Or that further project options or relationships develop from this project.
Alternatively the information leads to additional funding which is granted because it causes less impact on the environment.
5. CR: So whether there is funding for such a thing, if we make something more environmentally friendly somewhere or whether we reach an advantage against a competitor somewhere.

Uncertainty
As with the category of credibility (see Sect. 3.1), uncertainty is also a limit for relevance, especially in cases where not even the signs of the trends have the same direction; e.g.possible future changes of storms in northern Germany are perceived as limiting relevance.
7. ET: And when it then comes to the fact that perhaps we do not really know exactly whether the storms are actually influenced by climate change or not and such things.This lets people not say, yes, we have to do something now and in any case.
The reason for the limitation of relevance caused by uncertainties is not intransparency of the method or missing knowledge on how scenarios have to be interpreted.It is rather the missing applicability in practice.4. BQ: I know that too.I have enough to do with the university.So, I understand what it means when one talks about probabilities and about increase.I understand that they all have the same probabilities.Everybody can somehow explain why this is the case.The point however is that it does not help in practice . . .But this often does not help much in practice.
Similarly to the context of credibility, in the context of relevance a scientific proof of the climate information was also perceived to limit the vacuity caused by uncertainties (see Table 1).

BQ:
If you can prove it, we will implement it.Then we show this in our lectures.Then we convey it.

Appropriateness
A third category of requirement addressed by all interviewees according to regional climate information is appropriateness.The category of appropriateness can be described by the three dimensions of (1) comprehensibility, (2) the format and (3) (inter-)personal relations (see Table 1).

Dimension of comprehensibility
Since most climate research results are written and communicated in English, comprehensibility has been addressed in particular according to the national language of the information.It was stated that in order to communicate the research results in municipalities, information in the national language (German) is needed, since the majority of stakeholders in the municipalities does not have any link to English in their day-to-day routine.In combination with the national language, the often-missing practical orientation of scientific publications was also mentioned as a reason for incomprehensibility.
10. GW: But for EU projects, you have to publish a lot in ENGLISH too.Here, this means always being bilingual in any case.Because with English you do NOT come far in the smaller communities.And we have just a very high practice orientation and applied projects, and such scientific results for an EU project in English would be of little use here.
Another aspect in this context is the disciplinary wording within scientific publications, which prevents stakeholders from the acquirement of scientific knowledge on climate change, even if they were trained in the English language.4. BQ: So (. . . ) it is not the case that one could just say, well, you can just read it.Just read the IPCC, read it through.Hundreds of scientists have written it, just read it.Yes (. . . ) one could do.But I do not understand it.(. . . ) on the one hand, I cannot speak fluent English, and if I could, I would not really understand it due to the fact that I don't understand the terminology.
Furthermore, the complexity of the provided information has been addressed in the context of comprehensibility.Although appreciated, basic research is often perceived as too complex.This complexity prevents the implementation of the scientific information in the public sphere.
6. DS: Because there is so much great research at [name of research institution] and so on, but so in the general public it does not matter so much, because it is too complex and has too little luridness for the media.
Very simple presentations of climate change information implemented in familiar surroundings were perceived as supporting a decrease in comprehensibility.
4. BQ: Yes, in principle simplest is best, isn't it?So if one would go into the terrain, for example, and get a couple of people from the village and stretch a red fluttering tape which is about 2 m high and they just say the water can stand so high.This is, for example, the simplest way and everyone understands it.Everybody.Everyone sees this.Everyone knows the height.Everyone knows where he lives.So we do not need to have complex visual materials.Although everything is very modern and works well, it can also be easier (. . . ) VERY simple representations.
However the different requirements of various target groups were also addressed according to the complexity of the provided information.Even within an apparently homogeneous group like students there is a broad range of different demands according to the degree of complexity.This applies to the scientific wording as well as to the degree of complexity of the content.
6. DS: So in the representation it is not that simple, since we have very different target groups here.This starts from primary school up to A-level in physics.So, a very wide range.Also adults who are partly well informed but also are partly absolute laypersons.It is very difficult.So it is impossible to provide the same information for all target groups.For one group it is too complicated because there are five lines of text to read, for the other group it is banal.Also, the articulated need for background information according to the underlying method and its explanation would contradict solely providing scientific information with low complexity.
4. BQ: Just saying that the sea level rises about 4 m when Greenland melts.This is so striking, isn't it?If you want to get a message through to the people, to the citizen, to the tourism or to us from authorities, then one should simply try to explain the matter professionally.One can (. . . ) one should really make an effort to do that.And not rest on saying that no one understands it anyway.

Dimension of format
The dimension of format within the category of appropriateness is mainly governed by stakeholder-related constrains, in particular the lack of time and overload in the day-to-day working routine due to information abundance in a digitalized world.
3. AP: And a lack of information is apparently not the problem.This does not seem to be the case.It is rather a problem that there is such a great abundance of different ways to access this topic.You are likely to be slayed by the masses when you start to look around a bit.How much literature do you think I have on my desk about this subject?And that is only the region -this is just one section.So, if I would need to deal with it a bit more, or if I were forced to deal more intensely with it, I believe that one would not see me anymore underneath the mountain of brochures and books and whatever.
In this context, filtered information is perceived as supportive.
8. FU: One gets the impression that they always have a lot to do and there is always little time and therefore it is important to adjust the information as much as possible to the respective user and to bundle it, so to speak, to save the users' effort.
The filtering should decrease the quantity of information and provide easy access.In this context a newsletter is suggested which summarizes and bundles the main results.8. FU: . . . it may also depend on where this is provided.So, how much effort the respective user has to spend to get to the information.For example, do I have to do my own research now on the internet and do I need to click for a long time until I get to the information or it is sent to me perhaps bundled as information via newsletters, for example.
On the other hand filtering was mentioned in the context of tailoring information to the individual needs of a certain stakeholder, which certainly cannot be reached by one identical newsletter for all stakeholders.8. FU: And, is the information tailored directly to my needs or do I still have to do a lot of my own interpretation.I think these are things that play a role.
Also the adequacy of text-based information was questioned, since it has been observed that nowadays most citizens read less.Instead interactive information in social media has been suggested.4. BQ: I mean we all know it: nobody reads these reports any more.You can give these little booklets to the citizens, as nice as they are, but they will look at the nice photos and that's it.There is hardly anybody left who reads voluminous material, although it has been beautifully prepared.
4. BQ: I think that more content might be conveyed if it was somehow cooler, for example on a mobile phone.Then you could reach a generation which should be sensitized for the future.

Dimension of (inter-) personal relations
All interviewees addressed interpersonal relations in certain contexts.It was seen as essential that the right persons get in contact with each other.According to the interviewees the right stakeholder who should be addressed by a regional climate service provider are persons who are either in charge of prevention or maintenance in climate-change-related contexts.This could be either job related or in a private context.4. BQ: So, first the mayor has the duty to do something.For prevention.For maintenance.Then the county jumps in and then it all goes.But how to prepare?Does every tourist who goes to a hotel need to be aware of what happened outside?
In particular, in a job-related context it was suggested that the hierarchical structure of the company or authority should be in order to find the right person.Otherwise it was assumed to be hard to find the right person.This shows that regional climate service is facilitated when stakeholders' competence and responsibilities according to climate change are clearly assigned and visible from outside. 4. BQ: We went through the whole country.Again and again we addressed certain people until we found out that we actually need to call someone in Bonn.And then that person says you can contact the person from xy.It is the same with the railways and with the energy companies.They all have different areas and each has contact persons.But you will never find them in your life if you try to get through it on your own.You must go through a highly official path.
The personal contact between service provider and stakeholder was perceived as one essential element of a regional climate service.The personal contact should support different methods of knowledge acquirement.This could be making initial contact in order to learn about stakeholders' individual circumstances in which climate change could play a role.The acquirement, the usefulness, and the usage of climate-change-related knowledge was perceived as rather unique among the different stakeholders.
8. FU: So first of all, it is important to talk with the administration or the actors in the administration: how they can play a role at all and how information can be picked up by them at all.So that has a lot to do with it, yes, to conduct conversations and determine what useful information is for them.
However, it could also be a long-term contact person who takes care of all climate-related issues.10.GW: But in any case, this is something we always hear, there is a need for somebody who takes care, an institution, a contact point where some personal support is available.Moreover, ongoing personal support when climatechange-related issues arise was mentioned as quite essential.Rather than putting more information on the web, which would get lost in the overload of existing information, a personal support service could help to develop a suitable orientation and focus the climate information as necessary on the stakeholders' side.10.GW: I believe this inhibition to put oneself in front of the computer and look for some results when I do not even know exactly what I must look for, this inhibition is greater, than if I would just have the right person to ask.
When talking about the needed filtering of the climate information (see Sect. 3.3.2) it was always thought of as a certain person (not a tool) who should carry out the filtering, according to their particular needs.This implies that the climate service provider knows the exact context of the particular stakeholder and the related question.
3. AP: At the moment, we are no longer able to find such information, which is probably well prepared, on the web.We cannot make it manageable for us.If I now get the task: so, please take care of it and deal with it -then I would need an assistant -perhaps the word fits quite well -to compose a short document in which relevant information is filtered and bundled in order to make it manageable according to a certain question.
1. AN: So, I think at the moment we would rather need someone who filters.So someone who does the service that relates information to a particular question.
The filtered information should include options for different strategies in order to support decisions.9. GV: We expect that we will get information from the scientists which can support our decisions.So we want to decide, but we want to be able to rely on the help of science in our decisions.
Another important aspect of interpersonal relations is the dialogue among different stakeholder groups in order to facilitate the exchange of practical experience.9. GV: It would be interesting to know how other European cities are dealing with it.This is something that is always demanded, good examples.8. FU: To enable an exchange, this is of course always something interesting for us and for every municipality.To exchange ideas with each other about this issue.
All aspects mentioned in the context of interpersonal relations had a strong demand of continuity in common.In particular it was stated that a long-term contact person is perceived to reach a better linkage between climate information and a particular region.Long-term responsibility and reachability should be assured in combination with sustained returning activities which avoids climate change disappearing from the political agenda and which avoid it being forgotten in the public arena.
8. FU: And also important is otherwise a certain continuity with the information.That you then have regular contact, so that the topic does not disappear from everyday life again.But that it gains certain significance.
7. ET: That often means you have to convince people within the city administration, you have to convince the policy and there I basically think constant dripping wears the rock away.Always try again.4. BQ: So, one has to do that again and again.It must not be forgotten.This is, I believe, what is very important.
One essential aspect in this context is to raise awareness of climate change.It was suggested to raise awareness by linking climate change to periodic events (e.g.storm surges) in order to increase public awareness of a potential increasing risk.
4. BQ: Prevention.Awareness raising for certain events.One needs to do that again and again.For storm surges one should do it when the season starts again.That means in October every year one should prepare oneself and the other citizens to what could happen and about what to do in that case.
On the other side, it was doubted that solely providing additional information would increase attention.Instead it was suggested to link climate information to personal perceptions (emotions, values, and familiar place-related customs).
7. ET: Because there are really many meetings and there are also many people who go there.So, there is actually more than enough information.But this alone does not make it.I think you have to work a bit more with emotions -not with fears, but with emotions.That we represent the value of our region more clearly, also the particular features about Lübeck and so on.
In this context, the personal responsibility was addressed.As consequence of a certain fatigue from solely receiving natural scientific information on climate change, it was perceived as helpful to attribute climate change to citizens' individual consumerism and environmental awareness.4. BQ: And especially when it comes to climate change, one really has to be careful.As I said, the term in itself has died. . . .It is difficult, of course, to give a talk or raise awareness about climate change.You really have to look somehow at how to get the citizens to self-commit.So to say that someone points at me with their finger, you, you are responsible, you and your garbage production.You and your shopping behaviour, your consumption behaviour.You.You.You.

Conclusion and discussion
From the comparative analyses of the conducted interviews with potential regional climate service users, three main categories evolved, which should be accounted within an evaluation of regional climate services: (1) credibility, (2) relevance, and (3) appropriateness.For each of these evaluation categories several dimensions emerged, indicating certain perspectives of stakeholder demands.When summarizing these evaluation categories and their dimensions, 13 criteria can be derived (see Table 1).It cannot be excluded that additional criteria emerge when further stakeholders are involved, especially when they are not working in a regional or local context but in a more national or global one.However, according to regional climate services it is assumed that possible additional criteria could be allocated to one of these three main categories.
The results of this study show that, contrary to nonparticipative approaches, stakeholder-based evaluation approaches do not necessarily address the component inputs, process, outputs, outcomes and impacts, and their associated indicators (e.g.Wall et al., 2017).This might indicate that an evaluation, following solely literature-based (nonparticipative) components, might not be sufficient to localize deficiencies or efficiencies within a regional climate service, since it might lead to results which are not relevant for potential users.
The stakeholder-based criteria derived in this study may help organizations to make informed decisions on where to devote more time and resources according to a regional climate service in order to improve its social benefit.In order to evaluate to what extent the promise of integrating the state of regional climate science in practice has been fulfilled, it is suggested to survey the perception of regional climate service users following the three main categories.

Table 1 .
Stakeholder-based evaluation categories, dimensions and criteria according to a regional climate service.