On the comparability of knowledge transfer activities – a case study at the German Baltic Sea Coast focusing regional climate services

Abstract. In this article the comparability of knowledge transfer activities is discussed by accounting for external impacts. It is shown that factors which are neither part of the knowledge transfer activity nor part of the participating institution may have significant impact on the potential usefulness of knowledge transfer activities. Differences in the potential usefulness are leading to different initial conditions of the knowledge transfer activities. This needs to be taken into account when comparing different knowledge transfer activities, e.g., in program evaluations. This study is focusing on regional climate services at the German Baltic Sea coast. It is based on two surveys and experiences with two identical web tools applied on two regions with different spatial coverage. The results show that comparability among science based knowledge transfer activities is strongly limited through several external impacts. The potential usefulness and thus the initial condition of a particular knowledge transfer activity strongly depends on (1) the perceived priority of the focused topic, (2) the used information channels, (3) the conformity between the research agenda of service providing institutions and information demands in the public, as well as (4) on the spatial coverage of a service. It is suggested to account for the described external impacts for evaluations of knowledge transfer activities. The results show that the comparability of knowledge transfer activities is limited and challenge the adequacy of quantitative measures in this context. Moreover, as shown in this case study, in particular regional climate services should be individually evaluated on a long term perspective, by potential user groups and/or by its real users. It is further suggested that evaluation criteria should be co-developed with these stakeholder groups.


Introduction
In natural science, various indices exist, counting the number of published peer reviewed articles and their citation frequency (e.g.Hirsch, 2005).These indices are applied to measure and quantify the success of a researcher for evaluation and comparability purposes, e.g. for recruitment, advancement and award of grants (Hirsch, 2005).Among others, these indices are also an important basis of research program evaluations.They support the state and federal governments in making decisions on how much funding to give each research program (e.g.Helmholtz Association, 2017a).
Besides funding of pure research a substantial investment is made each year in applied research and knowledge transfer activities to support policies and decisions (Bell et al., 2011).In this context, knowledge transfer activities are also subject of program evaluations (Bell et al., 2011).In order to set priorities and to allocate funding, different knowledge transfer activities would need to be compared with each other.Universally, valid evaluation criteria are needed for these program evaluations, which could not be found, so far (e.g.Helmholtz Association, 2017b).Unlike in natural science (Hirsch, 2005), countable standard formats like e.g.peer-reviewed publications and their citation do not exist for knowledge transfer activities (e.g.von Storch et al., 2011;Weisse et al., 2015).In addition, external factors, like Published by Copernicus Publications.
(1) the perceived priority of the particular topic, the knowledge transfer activity is focusing on as well as (2) the used information channels, (3) gaps between information demands and the research agenda and (4) the spatial coverage of the knowledge transfer activity could influence its potential usefulness.This would lead to different initial conditions and, thus, affect their comparability within program evaluations.In order to assess the comparability of knowledge transfer activities, this paper analyses the role of external impacts on the comparability of knowledge transfer activities.The study is carried out in a case study of regional climate services at the German Baltic Sea coast.Regional climate services are one kind of knowledge transfer activities.The knowledge transfer facilitates the two-way exchange of concepts, concerns, questions and knowledge between the scientific sphere and the regional public sphere regarding regional climate, regional climate change and impacts (von Storch and Meinke, 2008).Following the objective to make results from regional climate research useful for decision processes, this sciencestakeholder interaction entails not only information provision but also contextualization of research findings.This enables stakeholders to integrate the state of climate science in their understanding and decisions (von Storch et al., 2015).
In Germany, and here especially at the German Baltic Sea coast, several evaluation studies of (regional) climate services have been conducted (e.g.Bray and Martinez, 2011;Stelljes, 2012;Kiresiewa et al., 2013).The regional specific external conditions of the services as well as the comparability of the service among each other have however not been analyzed or discussed, so far.In the meantime, several studies have been carried out in order to localize on what an evaluation of knowledge transfer services should focus on and which criteria might be suited in this context (e.g.Cash et al., 2002;Ford et al., 2013;Kirchhof et al., 2013;McNie, 2007or Meadow et al., 2015).Arnott et al. (2016) describe different purposes of evaluation, which are either to decide about the existence or funding of a program or to improve their efficiency.Most recently, Wall et al. (2017) has conducted a broad review of corresponding literature and synthesize the metrics currently used to evaluate usable or actionable climate science.It is suggested that six different components should be evaluated.Besides the five internal components, (1) input, (2) process, (3) output, (4) outcome and (5) impact, this study also accounts external factors.These external factors address components like (a) turnovers in agency staff, (b) the technical capacity to manage new information, (c) the political will for action/change and (d) the financial capability for change/action.
These external factors described in Wall et al. (2017) are not part of the knowledge transfer activity itself but they are still connected with the conditions of one or more institutions involved in the knowledge transfer activity.Since these institutions are already involved in the process, it is assumed that they generally agree in the usefulness of the knowledge transfer activity.However, to compare different knowledge transfer activities, it is also important to account external factors outside the process and beyond the involved institutions since they could have impact on the potential usefulness.It is assumed that the potential usefulness is driven by external factors leading to different initial conditions of knowledge transfer activities.In this context, political, social and personal convictions, priorities and targets as well as the usage of information channels are crucial factors (e.g. von Storch, 2009;von Storch et al., 2011von Storch et al., , 2015)).They have impact on the potential usefulness of a service transfer activity, leading to a certain very individual initial condition of that particular activity.Lemos and Morehouse (2005) distinguish usefulness and usability following the process of knowledge transformation.Information needs to be transferred from useful to usable in order to fit user's needs.Thinking about climate service or other knowledge transfer activities, this transformation is addressing internal, service immanent factors.This paper is focusing on external impacts and their influences.Following the concept of knowledge transformation it is focusing on the initial state of the process, namely the potential usefulness.
After describing the methods and data used in this study (see Sect. 2), external impacts on the potential usefulness of knowledge transfer activities (regional climate services) are described and analyzed in Sect.3. Here, aspects like the general perception of the topic, the knowledge transfer activity is focusing on, the priority of this topic compared to other issues, the information channels used as well as the information needs and the role of a regional focus are analyzed and interpreted in the context of regional climate services and their potential usefulness.

Method and data
To analyze external impacts which may influence the potential usefulness of knowledge transfer activities (here, regional climate services), two surveys were carried out at the German Baltic Sea coast.It is assumed that the potential usefulness of a certain topic is mainly guided by its general perception in the regional public sphere.As substantial component of the regional public sphere, regional politicians and the general public were analyzed within two surveys.One survey was carried out among mayors of municipalities at the German Baltic Sea Coast.Personal letters were sent to all 1109 municipality mayors inviting them to participate in the survey.In the questionnaire personal opinions about general aspects of climate change, preferred information channels and information demands were addressed.The response rate was about 15 % with 165 answers (N = 165).
The general public was interviewed within a representative telephone survey of the German Baltic Sea region, carried out by a subcontractor.The questionnaire of the telephone survey addressed the same aspects as in the survey of the municipality mayors.The survey population represents the whole range of age groups, older than 16 years and the whole range of education levels.In total, 630 people were interviewed (N = 630).
The data collected within the two surveys were analyzed regarding the general perception of climate change (see Sect. 3.1), its priority compared to other issues (see Sect. 3.2), the information channels used (see Sect. 3.3) and information needs (see Sect. 3.4)

General perception of the theme
One important prerequisite of potential usefulness of a knowledge transfer activity is the general perception of the content the activity is focusing on.In this context the perceived existence in general and in a specific region is crucial.Nobody would question the usefulness of a weather service or a repair shop.It is self-evident that we have to prepare for certain weather and sometimes need aid with a broken car.However, this might be different with a regional climate service where some people generally even deny the existence of anthropogenic global climate change and others are not aware of climate change in their particular region.Since nobody would request a service for something that does not exist, this would certainly reduce the potential usefulness of a knowledge transfer activity and the received attention.However, the survey among the general public at the German Baltic Sea coast shows that there is a broad majority of almost 90 %, believing that climate change is partly caused by human activities (Fig. 1a).Only 4 % of the respondents have reasonable doubts that human activities contribute to climate change and 6 % believe that human activities do not play a role referring to climate change.The survey among the municipality mayors shows similar results: almost 70 % of the mayors think that climate change is partly caused by human activities.Only about 14 % have reasonable doubts that human activities play a role and 8 % think that human activities do not contribute to climate change.It is often stated that the knowledge about human impacts on climate change is an important requirement for mitigating greenhouse gas emissions and for developing measures to adapt to a changing climate (e.g.Jones et al., 2014).If this is the case it also could be assumed that regional climate services might have potential usefulness to support these activities and thus, that there is potential attention among the general public and mayors regarding information on regional climate change and a correspondent service.However, also other drivers seem to be relevant.

Priority setting among other issues
In the public arena various issues are competing for attention (e.g. von Storch, 2009).According to Eisenhower's matrix, it is suggested to set priority on urgent and important cases and to postpone issues which are neither important nor urgent (Krogerus and Tschäppeler, 2008).In terms of the potential usefulness of a knowledge transfer activity (here regional climate services) this means that it is strongly connected with the perceived urgency and importance of the content the activity is focusing on (here regional climate change).The survey among the general public shows that issues like unemployment, security, health and poverty were mentioned most frequently as most important issues.Only 2 % of all participants mentioned climate change as an important issue.Even when asking explicitly about environmental issues, less than 6 % mentioned climate related themes.The survey among the community mayors at the German Baltic Sea region shows similar results (Fig. 2).About 17 % of the mayors participating in the survey stated that climate change does not play any role in their region; almost half of the respondents (48 %) answered that climate change issues have only minor priority in their region.For about 12 % of the mayors climate change has priority only in combination with other issues.This clearly shows that the majority of the interviewed mayors does not see any urgency or need for action regarding climate change in their region.Only less than a quarter of the participating mayors (23 %) stated that climate change has high priority in their region (Fig. 2).
Although there is a broad agreement that human activities have contributed to climate change (see Sect. 3.1), the results of the two surveys show that other issues with more direct impact to society are more important or have higher priorwww.adv-sci-res.net/14/145/2017/ity.This means that before a community is getting involved with climate change issues (mitigation or adaptation) other problems which are perceived as more urgent will be tackled, first.This shows that regardless of the scientific quality and the perceived trustworthiness of a knowledge transfer activity the potential usefulness of a service which is focusing on a prioritized content (e.g. a health service, an anti-terror service or a come-back to the job service) would receive much higher attention and would perceived as much more useful than a service which is focusing on an issue which people decided to postpone.Consequently, there are different initial conditions for knowledge transfer activities which hinder a comparison based on quantitative measures.

Usage of information channels
In this context the use of information channels is analyzed since this decides how the information -here on regional climate change -comes to the individual and which channel is assumed to be qualified to communicate climate change issues.Both surveys show that media (print, radio and TV) are the most important information sources on climate change issues for most of the mayors and for the majority of the general public.Only about 4 % of both groups stated that they use scientific information services as information source for climate change.This might correspond to the low importance and priority of climate change compared to other topics as shown before.Using the media as main information source may indicate a missing target-orientation, meaning that the information on climate change comes more or less randomly to the people as part of the everyday general information flow.Analyzing only the group of those respondents of the general public, which mentioned climate change as one of the most important issues, the most frequently used information channels were rather similar.In this group again the media are most frequently used and the usage of scientific information services is only slightly higher (6 %).In a representative online survey in Germany, Taddicken and Neverla (2011) show that the use of classical media is mainly correlated with increased climate related knowledge, but is less correlated with increased awareness of climate change and fewest correlated with intentions of climate related action.This agrees with the findings described above.As regional climate services aim to support decisions related to regional climate change a general insight in the need of climate change related action is required before they can become useful.The fact that the general public and regional politicians are mainly using classical media as information source for climate change indicates a gap between knowledge and climate related action which is limiting the potential usefulness of regional climate services in that region.Before relevant scientific information can serve as basis for decisions, awareness and intention for action needs to be increased, first.This, in turn, is connected with the priority attached to this theme which entails a certain choice of information channels.

Relevant information and information needs
Besides the priority and the used information source, the perceived relevance of existing climate information and further information needs might have additional impact on the potential usefulness of a regional climate service.
In the survey among the general public of the German Baltic Sea coast the most relevant climate change information was requested (Fig. 3a).Among the four suggested categories of answers two are focusing on natural scientific issues -(1) regional and local climate change; (2) human contribution on climate change.The other two categories provided as answers do not base on natural scientific research.Figure 3a shows that only about half of the respondents (54 %) indicate information, based on natural scientific research as most relevant fields (human impact 35 % and regional/local climate change 19 %).The impact of climate change on society was perceived by 23 % of the respondents as most relevant and 11 % thinks that funding of mitigation and adaptation are most relevant information regarding climate change.Since the answer "other" is addressed by only 5 % of the respondents it seems that the provided answers cover the relevant fields quite well.These results show that a regional climate service based solely on natural scientific research does not cover all relevant climate information.
In the survey among the municipality mayors at the German Baltic Sea coast 11 different categories of answers regarding their additional information needs were selectable.Seven of them were research fields of natural science, three of them were research fields of social sciences and another addressed funding for mitigation and adaptation of climate change (Fig. 3b).The category "other" and "nothing else" were the least frequently chosen answers.Although about twice as much categories of natural science topics were offered, the survey among the mayors shows that less than 50 % of the most requested information matches natural science topics.Among the five most requested information categories only two are from natural science.These categories are (1) regional and local impacts of climate change and the (2) sea level rise.Mitigation of greenhouse gas emissions, the  consequences of climate change for society and options for action as well as funding options for mitigation measures are other most frequent requested information by municipality mayors.Categories addressing social sciences and funding information contribute more than 50 % to the most requested information (Fig. 3b).
Since these most frequently requested information categories are not part of the research agenda of those institutions which provide regional climate services, the potential usefulness of the provided natural science based regional climate service seems to be limited in advance since the scientific basis can only partly cover the articulated information needs.

Regional focus and spatial coverage
Besides the external impacts described above, the spatial coverage of a regional climate service may limit the potential usefulness of the service, since such a service excludes the public and stakeholders which are not living or acting in the focus region.Web tool providers often count the number of visitors on their websites in order to estimate their success.In this context two web tools on regional climate change are compared regarding their number of visitors.The northern German climate atlas (http://www.norddeutscher-klimaatlas.de; Meinke and Gerstner, 2009) provides information on possible future climate change in Northern Germany, including the federal states and physical regions within Northern Germany.Another webtool, the regional climate atlas has been designed after the same concept and offers the same climate information, with the same spatial resolution, but it is covering all federal sates of Germany (http://www.regionaler-klimaatlas.de; Meinke et al., 2010).This means that the concept, the degree of regionalization and the spatial resolution of climate change information is the same for both tools.The only difference is the spatial coverage, here the number of federal states.The comparison of the numbers of visitors of the two websites shows (Fig. 4) that there are about twice as much visitors at the regional climate atlas than at the northern German climate atlas.Since there is no other difference than the spatial extension, this shows that the potential usefulness of regional climate service is increasing with spatial coverage, given that the degree of contextualization and the spatial resolution is equal.

Conclusion
In this study examples for external impacts on the potential usefulness of knowledge transfer activities are described and analyzed.It has been shown that the potential usefulness of knowledge transfer activities (here regional climate services) is strongly governed by external impacts.These are (1) the priority of the theme (climate change) in the particular region, (2) the mainly used information channel, (3) the consistency between public information needs and the research agenda of service providing institutions and (4) the spatial coverage.In Germany some evaluations of climate services were carried out in the frame of the advancement of the German Strategy for Adaptation to Climate Change (Federal Government, 2008).These studies applied rather general quantitative criteria which suggest a quantitative measure of their success (Bray and Martinez, 2011;Stelljes, 2012) and different climate services are compared by rating their general quality (Kiresiewa et al., 2013).However, without accounting the external impacts, described in this paper, it is not possible to allocate the findings to the right reasons.As shown in this study, different priorities of the regional public sphere, a preference on particular information channels and a mismatch between research agenda and information demand offer important approaches for interpretations, especially in order to evaluate success of different knowledge transfer activities.
It is shown that there is a broad agreement among the population and the politicians at the German Baltic Sea coast, regarding a human contribution to climate change.Nevertheless, compared to other social realms, climate change has only low priority in this region.Thus, the potential usefulness of a regional climate service is perceived lower, for example compared to a service, providing information on a topic with higher priority (e.g.security, health, employment).Analyzing the information channels, the surveys show that inhabitants and politicians inform themselves upon climate change mainly in a passive way by using mass media.Consequently, the perceived awareness of regional climate change keeps low and there is a missing link between climate change knowledge and intention for action need which would entail a demand of regional climate services.Additional information needs do mainly not match the research agenda of those institutions which provide regional climate service.Thus, the potential usefulness of a solely natural science based regional climate service seems to be limited in advance since the scientific basis can only partly cover the articulated information needs.A regional focus is needed to build the missing link described above, but in turn, the limited spatial coverage of this focus excludes potential users which are not part of the particular regional community.This is challenging quantitative measures as evaluation criteria for a regional climate service web tool.
It is suggested to account for these external impacts within program evaluations of knowledge transfer activities.The results show that there is only little comparability among knowledge transfer activities of other research fields, like e.g.health, terrorism or security since these are perceived as more urgent social problems.Consequently, solely quantitative measures like the number of requests or the number of people knowing a particular service, the number of people considering this service as useful and the number of visitors of a web tool seem not suited as uniform evaluation criteria for all knowledge transfer services.
Since climate research has the societal obligation to inform society on potential climate related hazards and changes (von Storch, 2009) it is a political decision to establish regional climate services although there may be many other urgent societal realms which might have priority on a short term perspective.Since climate research is dealing with changes in a long term perspective, it is suggested that the evaluation of climate service is carried out by social groups which are involved with long term decisions (e.g.coastal defense and spatial planning) as potential users (Meinke and Schwab, 2017) and/or by the real regional climate service users.Within these groups suited criteria for the evaluation of climate services need to be co-developed.

Figure 1 .
Figure 1.Do you think that human activities contribute to climate change?(a) Survey general public (percentage of N = 630); (b) survey municipality mayors (percentage of N = 165).

Figure 2 .
Figure 2. What priority has climate change in your region (survey municipality mayors, percentage of N = 165).

Figure 3 .
Figure 3. (a) Which of the following climate change information are most relevant for you?(Survey general public, percentage of N = 630); (b) on which research field would you like to have additional information?(Survey municipality mayors, percentage of N = 165).

Figure 4 .
Figure 4. Numbers of visitors at northern German climate atlas (black) and regional climate atlas (grey).