Cyber Security and Privacy R&D - Delivering Impact

Full text

Theme [ICT-2011.1.4]
Trustworthy ICT
SECurity and trust COoRDination and enhanced collaboration
Project Nº 316622
Cyber Security and Privacy R&D
Delivering Impact
Massimo Felici, Dharm Kapletia, Nick Wainwright
Hewlett-Packard Laboratories
A White Paper based on
- Stakeholder engagements in the United Kingdom and Europe (2014)
- Support by the SecCord project funded by the European Commission FP7
For more details, contact: massimo.felici@hp.com

1
Executive Summary
Research and Development (R&D) Investment in cybersecurity continues to rise yet the scale of
attacks and severity of security breaches is also increasing. This calls into question how Europe’s
innovation ecosystem can respond the cybersecurity problem. There are a number of
considerable challenges associated with supporting innovation and bringing new security and
privacy technologies to market. Few studies have engaged key stakeholders to understand their
views on what is and is not working and also how we can collectively go forward to address
challenges associated with making R&D more successful.
This white paper explores a range of issues at the level of markets, firms and individuals, ranging
from new skill-sets required for commercialisation, differences in timeframes between research
and practice, national initiatives and funding for technology and market development activities,
and integration into existing infrastructure.
This white paper presents a roadmap of expert on-the-ground views from innovators, investors
and other key stakeholders involved in bringing new cyber security technologies to market. A
total of 74 expert responses were collected over four months during 2014, which consisted of
mainly SMEs, followed by academia/research, large enterprise and government. The method
used involved asking 24 questions (6 open, 18 using a Likert scale) and securing participation
using a survey and semi-structured interviews. The results were analysed by a roadmap
structured into three interrelated priority research areas that require future interventions: (1)
Technology Readiness and Maturity, (2) Technology Transfer, and (3) Market and Policy. This
paper reports the key findings clustered into seven recurring key themes:
 Data Sharing – sharing information of incidents, risks and threats can be usefully applied for
the purpose of developing stronger technological responses to threats and creating
analytical models to demonstrate the value of new technology
 Financial – funding mechanisms can be tuned to help advance technology development and
incentivise talented researchers into commercial activities
 Business Case – evidence-based value proposition supports the need to quantify return on
investment, qualifies security claims and demonstrates how outcomes and benefits will
impact on businesses
 Entrepreneurship – third-party organisations can help developing commercial relationships,
by gathering stakeholder feedback during technology developments and finding potential
partners and buyers for commercialising mature technologies
 Performance of New Technology – addressing the issue of limited access to resources (in
particular for Academia and SMEs) for testing and validating new technologies in order to
simulate actual operations within or across large complex organisations
 Standardisation – improved government-led legislative interventions and market-led
interventions (in close consultation with experts) can support effective risk management
 R&D Competitiveness – managing intellectual property can overcome barriers to effective
commercialisation.
This white paper delivers practical insights designed to assist policy makers, industry and
academic leaders to achieve the goal of increasing R&D impact in cyber security and privacy.

2
Contents
Executive Summary ....................................................................................................................... 1
Introduction .................................................................................................................................. 3
Research method .......................................................................................................................... 4
Sample Stakeholder Population .................................................................................................... 5
Roadmapping stakeholder views .................................................................................................. 7
Themes in Cybersecurity R&D ....................................................................................................... 9
Data Sharing .......................................................................................................................... 9
Financial ............................................................................................................................... 12
Business Case ....................................................................................................................... 15
Entrepreneurship ................................................................................................................. 17
Performance of new technology .......................................................................................... 19
Standardization ................................................................................................................... 22
R&D competitiveness ........................................................................................................... 25
Concluding Remarks .................................................................................................................... 29
References ................................................................................................................................... 30

3
Introduction
Europe’s innovation is affected by a number of critical barriers in its science and innovation
system according to a report on impact assessment [1] by the European Commission (EC). This
contributes to the problems of low productivity, declining competitiveness, inadequate
response to societal challenges and inability to move to a new sustainable economic model [1].
Whilst capturing macro-aspects of innovation in general, these challenges are also particularly
pertinent to Europe’s science and innovation system in cyber security and privacy. In particular,
the European Commission’s Network and Information Security Directive [2] presented in
connection with the European Cybersecurity Strategy points out that there is an “insufficient
level of protection against network and information security incidents, risks and threats across
the EU are undermining the proper functioning of the internal market” [2].
R&D in Cyber security and privacy like other domains faces difficulties when transitioning from
research into practice. Recent work highlights as main factors [3]: insufficient awareness of the
complexity of cyber security transfer, a scattershot approach to R&D, and a mismatch between
the market and threat environment. The US Department of Homeland Security identified other
issues that point out differences in personality types, skills and expertise between research and
practice, and other factors such as the lack of financial incentives to commercialise research
results, complexity in the transition process, as well as differences in goals and timeframes
[4].This gives rise to a situation where many investments lead to security technologies that
never see the light of day. This situation is often depicted as the “valley of death” (Figure 1),
which on the one hand usefully filters out poorly conceived propositions and on the other hand
requires specific policies, strategies and skills to navigate a pathway to industrial application and
real world impact [5].
Figure 1 Transitioning Cyber Security Research into Practice [6]
Whilst from a top-down perspective there are a number of broad generic responses to help
support promising new technologies (e.g. increased funding to R&D schemes, support further

4
collaboration between research and practice, etc.), our research takes into account a
contrasting bottom-up view in order to understand the first-hand operational realities faced by
innovators and stakeholders involved in cyber security and privacy R&D. This work should be
considered alongside other pertinent research agendas, which are for instance investigating
relationships between the technical, socioeconomic and legal perspectives as well as
advancements of business models and innovation paths in cyber security and privacy [7]. The
rationale is to gather observational evidence from which to derive insights and to formulate
recommendations. This research delivers practical insights on innovation in cyber security and
privacy that inform future European policies as well as research and innovation practices.
Research method
With a focus on the innovation ecosystem, this research sought to capture ‘stakeholder
experiences’ of bringing new cyber security technologies to market. To achieve this, as indicated
in Stage One of Figure 2, the HP Labs research team carried out early stage engagements with
government and private sector stakeholders and also reviewed relevant academic and
practitioner innovation and cyber security literature [7]. The combination of expert views, a
review of the state-of-the-art, and documented lessons learned from industry assisted with the
development of a research framework to help shape the direction of the investigation and
formulate questions employed in the research protocol.
Figure 2 Research process
The research framework targeted three interrelated priority areas for further investigation:
A. Technology maturity and readiness: decisions and processes for new technology
development and testing (pre-commercialisation) – with emphasis on technology usages
and economic incentives and investments
B. Technology transfer: the challenge of transitioning from research to practice – with
emphasis on stakeholder collaboration and commercialisation and process
C. Market and policy: the operating environment and context – with emphasis on market and
technology alignment and market regulations.
The research framework and questions within the research protocol proved to be highly
effective in eliciting stakeholder views on what current innovation practice looks like, as well as
its perceived effectiveness and what can be done to improve the situation. The questions

5
outlined in Table 1 were initially tested with the UK Malvern Cyber Security Cluster and two
global enterprises before being refined and fully implemented. The protocol provided
stakeholders with opportunities to feedback on issues and be interviewed by phone/face-to-
face. The interim results of this study were presented for review in a dedicated workshop (as
part of the stage two of the research process) hosted at the Digital Catapult Centre in London
(November 2014, attendants were mainly SMEs and representatives from governmental
organisations and R&D labs of large enterprises).
Table 1 Online survey and interview questions
TECHNOLOGY READINESS
AND MATURITY
TECHNOLOGY
TRANSFER
MARKET AND
POLICY
To what extent do you agree or disagree – on a
scale of 1-7
1. Cyber security technologies with a
strong business case still lack
opportunities to access capital to follow
through into application
9. Integration between new cyber
security technologies and current
infrastructure presents a significant
barrier to technology transfer
17. Publicly-funded research is
effectively addressing current and future
capability gaps needed by industry and
public services
2. Further support mechanisms are
needed to help demonstrate utility in
large scale systems environments
10. The lack of shared data on security
incidents and industry benchmarks is a
major obstacle to finding potential
applications
18. Stakeholder forums and initiatives
are effective for pooling and sharing
requirements across industry
3. Access to actionable test feedback
from end users is hard to achieve for
new cyber security technologies
11. Geographical proximity to a single or
group of large ICT clients is a critical
factor at the technology transfer stage
19. The national talent base in cyber
security R&D is capable of producing
game-changing technologies to meet
current and future threats
State five
most
important
4. What factors are critical to the
development of competitive business
models for new technologies in cyber
security and privacy?
12. What collaborative initiatives would
help support opportunities for
technology transfer between the
scientific cyber security community and
potential buyers and/or users?
20. What national or European level
initiatives would help industry and
academia work towards addressing
future demands in cyber security and
privacy?
To what extent do you agree or disagree – on a
scale of 1-7
5. Industry recognised metrics specific to
cyber security and privacy are already
widely used in commercial documents to
demonstrate the efficacy of new
technologies against threats
13. The lack of effective marketing
expertise is a significant barrier to
technology transfer, particularly in the
context of cyber security and privacy
21. Government and agencies are ahead
of the curve in terms of supporting
industry to find solutions to emerging
cyber security and privacy threats
6. Large enterprises should play a
greater technical and economic role in
supporting new cyber security
technology ventures in the wider
marketplace
14. Concerns over Intellectual Property
Rights (IPR) prevent exploration of the
full range of commercial options for new
technologies in cyber security
22. Government incentives accelerate
the adoption of new technology in cyber
security and privacy
7. The effective application of new cyber
security technologies is significantly
affected by exogenous factors, such as
legal frameworks, insurance and
taxation
15. Adapting and responding to new
market requirements presents a major
barrier to commercialisation for new
cyber security technologies
23. There are coherent industry
regulations in place for effective security
controls in my area of operation
State five
most
important
8. What can be done to decrease risk to
investments for technologies that have
demonstrated potential in laboratory
environments?
16. What can be done to clarify the value
proposition of new cyber security
technologies between suppliers and
buyers?
24. What kind of policies or legislation
are important for stimulating innovation
in cyber security and privacy?
Sample Stakeholder Population
The empirical study involved a small HP team that collected 74 valid responses from
stakeholders over a short period of time of around four months in 2014. The majority of

6
respondents were from the UK, followed by Italy, France and Ireland, and the remaining being
mainly singular contributions from Switzerland, Sweden, Lithuania, Germany, Finland and
Belgium. Figure 3 and Figure 4 provide a snapshot of the sample population who participated in
the study, combining those who completed the both the survey and interviews. They illustrate
the range and professional authorities, the types of organisations, their roles and primary
interests of the stakeholders consulted for this study.
Figure 3 Types of Organisations
Figure 4 Organisational roles
The overall distribution over the different types of organisation is characteristic and typical of
many countries with large presences of SMEs and research institutions and few large
enterprises. Stakeholder professions included both managerial and consulting roles (such as

7
Chairs, CEOs, Partners, Commercial Directors, Project Managers) as well as technical roles (such
as CTOs, Professors of Cryptography and Information Security, R&D Managers, Security
Officers). Whilst notable contributions were secured from academic professors and
departmental staff involved in commercialisation (28%), the largest respondent group (as
expected) was from Small-to-Medium Enterprises (45%). Large enterprises made up 18% and
government respondents made up 8%, and other organisations were 1%. Around 37% of
respondents reported R&D as the major part of their business. Some SMEs (including venture
capitalists), governmental organisations and a large enterprise too stated that they had
interests in funding/investing in new technologies. Others stated that were technology owners
or operators (19%) and a few stated that they were concerned with policy and regulation (5%).
Some participants provided the following general comments capturing the overall perception
about the study:
 A multi-disciplinary effort will be required to address future challenges
 The main challenge is to help clients define their cyber security requirements
 SMEs spend too much time chasing funding, diverting them from conducting R&D
 Questions raised in this survey are critical and get respondents to actually think about
answers from technical and business perspectives
 The goal is secure applications, start with the need and focus on the technology – if you
get this right there is less need for support initiatives.
With the right incentives (and with additional time and resources) for active participation from
key European institutions, networks and firms, the investigation can be extended further and
the survey has the potential to be refined and repeated on a larger scale. This would help
develop a comprehensive picture for comparison across states and a powerful analysis of how
key actors and policies within different national innovation ecosystems are performing.
Roadmapping stakeholder views
There are a variety of visual tools that can assist in the analysis of stakeholder views and
experiences. Roadmapping has been used for decades to capture ideas and illustrate an
integrated picture of future pathways for either new technologies or industry wide
development [9]. A highly detailed stakeholder roadmap was generated to help map and
summarise findings. The scope of the roadmapping aligns well with generic technology/strategic
perspectives, which involve the three focal areas outlined by the research framework [7]: (A)
Technology Readiness and Maturity covers resources (includes technology) and enablers to be
marshalled, integrated and applied, (B) Technology Transfer typically covers products, services,
systems, requirements, and (C) Market and Policy is normally concerned with trends and
drivers. Responses to open questions (question numbers 4, 8, 12, 16, 20, 24 in Table 1) went
through a process of filtering, clustering and mapping to synthesise the findings. Conclusions
drawn from a statistical analysis of the remaining ‘agree vs. disagree’ questions were also
incorporated into the stakeholder roadmap and are presented in graphs throughout the
following sections. Table 2 presents the major clusters that emerged from analysis of responses.

8
Table 2 Analysis and classification of source data
FRAMEWORK
CLUSTERS IDENTIFIED FROM
ROADMAPPING SOURCE
DATA
THEMES IDENTIFIED FROM CROSS-CLUSTER ANALYSIS
A.
Technology
Readiness
and Maturity
A1) Customer Focus
1
2
2
A2) Industry Standards
2
A3) Intermediation and
Funding
2
5
1
4
A4) Performance in
Operational Systems
4
2
1
1
A5) Multidisciplinarity and
Skills
1
2
1
3
B.
Technology
Transfer
B1) Technology Assurance
3
1
1
B2) Industry Coordination
and Communication
2
2
3
1
4
B3) Benefits and Outcomes
1
5
1
B4) Validation and
Application
1
1
1
4
1
C.
Market and
Policy
C1) Technological
Environment
1
1
1
1
5
3
C2) Funding Mechanisms
1
6
C3) Growth and Access
1
1
1
1
C4) Commercial
Exploitation of R&D
3
2
2
Data Sharing
Financial
Business Case
Entrepreneurship
Performance of
New Technology
Standardisation
R&D
Competitiveness
12
19
13
11
8
10
23
The stakeholder roadmap can be analysed both horizontally (individually within each of the
three focal areas of the framework) and vertically (commonalities across all focal areas). Looking
horizontally in Table 2, with regards to Technology Readiness and Maturity, feedback centred
on the benefits that could be derived from the early adoption of customer focus (A1);
considering dimensions related to industry standards (A2); how to improve and exploit
opportunities for third party intermediation and funding (A3); demonstrate the application of
new technology and its performance in operational systems (A4), and lastly, human talent
related benefits from a focus on multidisciplinarity and skills (A5). Responses for Technology
Transfer focused on the analysis and expression of strategic information that could assist in
providing technology assurance (B1); feedback on where to focus efforts for improved industry
coordination and communication (B2); increasing the emphasis on the benefits and outcomes,
or utility of cyber offerings, particularly from the client’s perspective (B3); and finally, exploring
various approaches to demonstrate confidence in the validation and application of new cyber
security technologies.

9
Stakeholder responses for Market and Policy focused on macro level views about the
technological environment (C1) such as measures to improve standards, coherence and
compliance; improvements to the diffusion and type of funding mechanisms (C2) available for
cyber R&D; specific channels to assist with growth and access (C3) to clients and markets, and
lastly, support to research communities to drive future ‘commercial exploitation of R&D’ (C4).
Themes in Cybersecurity R&D
Table 2 also presents themes emerging from a cross-cluster analysis. It is interesting to note the
clear concentration around increasing R&D competitiveness and financial considerations.
However, increased R&D funding and better technology will not necessarily automatically
equate to more successful innovations and safer cyberspace; thus when thinking about the
impact of R&D it is useful to take a systems point of view, to understand the linkages and
dependencies across both the horizontal clusters and vertical themes (several themes are
relevant, but the analysis is concerned with those themes identified by the sample stakeholder
population). Looking vertically in Table 2, seven themes were identified by clustering
stakeholder feedback that ran through the all three layers of the stakeholder roadmap:
 Data Sharing
 Financial
 Business Case
 Entrepreneurship
 Performance of New Technology
 Standardisation
 R&D Competitiveness.
To ensure future innovation-related decisions and implications are considered in an integrated
way, this paper proceeds to explore each theme in-turn. Diagrams are presented in each theme
which contain a synthesis of the source data from the stakeholder roadmapping exercise. Direct
quotes from a variety of stakeholders are also presented in most themes.
Data Sharing
As indicated by the European Commission, Action 124 of the EU Cyber-security strategy points
out that “Efforts to prevent, cooperate and be more transparent about cyber incidents must
improve”, and “Previous efforts by the European Commission and individual Member States have
been too fragmented to deal with this growing challenge” [2]. As part of Europe wide initiative,
the Network Information Sharing (NIS) Platform currently has a Working Group (WG2) looking
at “information exchange and incident coordination, including incident reporting and risks
metrics for the purpose of information exchange”. At the national level, in the UK a joint
industry-government initiative called the Cyber-security Information Sharing Partnership (CiSP)
has been set up for all types of organisations to exchange cyber threat information in real time,
in a secure and dynamic environment, whilst operating within a framework that protects the
confidentiality of shared information. Whilst these initiatives are at an early stage, very few
respondents were aware of them (in interviews) and those who knew were unclear about the
tangible benefits arising from participation.

10
Clearly, from the findings outlined in Figure 5, there is support for trusted intermediary(s) to
take on the role data collection and sharing, and advising on collective responses. The major
trend running through the framework in Figure 5 relates to the utility of data sharing. In other
words, how the sharing of incidents, risks and threats can be usefully applied for the purpose of
developing stronger technological responses to threats and creating analytical (technical and
commercial) models to demonstrate the value of new technology.
Figure 5 Key themes related to data sharing
In terms of increasing cyber security collaboration within the operating environments of
respondents, there is clearly still room for improvement based on the results from Figure 6,
where it is not clear if pooling and sharing requirements through stakeholder forums and
initiatives are effective. It may be the case that recent efforts by European and US Governments

11
to improve the security of critical infrastructures will provide exemplars for other industries on
how to pool and share requirements across industries.
Figure 6 Survey response to Question 18 within Market and Policy
An SME respondent argued that cross-industry events are useful and allow collaboration
between parties who are not in direct competition. Another SME proposed that competing
technology suppliers could still collaborate on issues like educating clients about current and
future cyber security and privacy challenges. A government respondent mentioned the wider
use of pre-procurement initiatives such as the UK’s Small Business Research Initiative (SBRI),
designed to connect public sector challenges with SMEs involved in R&D. Whilst promising
European stakeholder forums and networks exist such as the SecCord CSP Forum, the SINET
Global Cybersecurity Innovation Summit and events runs by European Network and Information
Security Agency (ENISA), there is the potential to further leverage their contributions by taking
on agendas to tangibly help address some of the data sharing issues highlighted in Figure 5.
At an earlier stage of technology development, ideas should be tested in the market in order to
refine them. However, as indicated in Figure 7 within the context cyber security, it appears that
access to actionable test feedback from users is difficult to achieve in practice. As expected
there is a slight indication that academia/research and SMEs find this particularly challenging.
Other interviewee comments argued that current approaches were limited and expensive, real
world testing carried high risks for the technology developer and users are still failing to realise
their role in enhancing security.

12
Figure 7 Survey responses to Question 3 within Technology Readiness and Maturity
As consistent with qualitative responses in Figure 5, there is also consensus from most survey
respondents in Figure 8 indicating a pressing need to address the lack of shared data on security
incidents and industry benchmarks in the context of finding applications for new cyber security
and privacy technologies. This particular issue has a significant bearing on the other themes
discussed in the study, particularly business case, performance of new technology,
standardisation’ and ‘R&D competitiveness.
Figure 8 Survey responses to Question 10 within Technology Transfer
Financial
At the European level, financial investments and incentives for R&D in cybersecurity and privacy
have been concentrated on Framework Programme 7 and the Horizon 2020 programme. At the
national level, the UK allocates funding through its Department for Business, Innovation and
Skills, which through its various bodies employs mechanisms such as national and regional
funding competitions for SMEs, academic research centres and collaborative government-
university programmes. Other activities include the funding of international academic research
partnerships such as the recently funded 2015 bilateral cyber research programme between the
UK and Israel.

13
Figure 9 Key themes related to financial
Whilst Figure 9 presents useful feedback across a multitude of financial dimensions, the major
trend running through the framework in Figure 9 relates to financial mechanisms to support
new ventures. This trend applies to academia/research and SMEs in particular, and how
mechanisms can be better focused to help advance technology development and incentivise
talented researchers into commercial activities. When compared to responses in Figure 10,
there is a slight perception (particularly from academia) that public-funded research will be
ineffective unless changes are made.

14
Figure 10 Survey responses to Question 17 within Market and Policy
A large enterprise respondent argued that because Universities lacked access to industry data
they were often rediscovering old technologies and doing so unconvincingly. Both government
and university based respondents argued the need to transform academic institutional drivers,
the recruitment criteria and how impact is recognised; as well as increase support for higher
TRL level funding, and encourage wider adoption of successful models of university-industry-
end-user engagement. Overall, respondent’s views in Figure 11 relating to the effectiveness of
government incentives are clearly mixed, with a slight indication that more needs to be done.
Interview feedback from respondents also pointed out the need for greater publicly-funded
investment in cyber security and privacy R&D, greater collaboration between academia and
industry as well as end-beneficiaries, and shifting research incentives towards industry impact.
Figure 11 Survey responses to Question 22 within Market and Policy
As indicated in Figure 12, whilst there the picture is unclear, overall it appears that access to
capital from the perspectives of SMEs and academics in particular is an issue that may prevent
progression into the applications space. Those that disagreed tended to be geographically based
within a stronger local innovation ecosystem, which might explain their response.

15
Figure 12 Survey responses to Question 1 within Technology Readiness and Maturity
In the context of funding, one particular SME stated the follow:
“The existing cyber security technology companies have a huge advantage when
compared to specialist start-ups. Furthermore, US start-ups have a huge financial
advantage compared to UK / EU start-ups. It is clear to me that as a US tech start-up
it would be trivial to directly access relatively large sums from VCs in order to create a
runway of 18 - 24 months for the business.”
The perception that both angel funding and venture capitalist (VC) funding is easier to find and
secure in the US than it is in Europe, was a widely held belief across research participants. This
is potentially an area for further research, where there could be merits in understanding how
and why investments in cyber security technology differ (quantitatively and qualitatively)
between the US and European states. Respondents from other groups made some further
arguments regarding the issue of funding:
“Funding is not the biggest problem, new ventures struggle more with market
exploitation” (Large enterprise)
“For SMEs it is challenging, competing with established players in existing markets.
Quite often they have not thought through the business case for market application”
(Government)
Business Case
Recent technology management research examining the key features business cases for new
technologies, proposes the need to consider the following:
1. Identify the technology/problem combination – finding applications based on client need
2. Select potential customers and sales strategies – understand the extent of client demand
and whether they can afford it
3. Understand the target customers’ needs – focus on end users, technology seller must
understand the buyer and their requirements, not vice versa
4. Develop the business case with the customer [10].

16
Figure 13 Key themes related to business case
Respondent feedback in Figure 13 helps to further articulate what is specifically needed to
strengthen the business case for new technologies in the context of cyber security and privacy.
This includes overcoming challenges such as having limited access to client security breach data
(relates to Data Sharing section) which makes it difficult to evidence the impact of new
technologies: and also, the technical complexity associated with some cyber security
technologies makes it harder to demonstrate benefits to buyers and users. Linking back to the
financial section, one SME argued that:
“What we need is the resources to develop a strong business case, then the funding
will follow”
One of the major trends running through the framework in Figure 13 relates to evidencing the
value proposition. A reoccurring story throughout research interviews was the need to quantify

17
return on investment, qualify security claims and demonstrate how outcomes and benefits will
impact on the client’s business.
Figure 14 Survey responses to Question 2 within Technology Readiness and Maturity
One fundamental aspect of strengthening business cases for new technologies and evidencing
the associated value proposition is the ability to demonstrate scalability, interoperability,
competitive disruptiveness and a smooth transition into existing infrastructure. Some SME
respondents believed that universities could do more to help create synthetic environments,
yet university respondents counter argued that this was difficult to achieve in practice. As
indicated in Figure 14, there was strong consensus that further support mechanisms are needed
to help demonstrate utility in large scale systems environments. In terms of large enterprise,
one respondent argued that they needed to be more tolerant of the disruption impact potential
of new technologies and that a cultural attitude shift was needed to help make this happen.
Entrepreneurship
The process of creating innovative new ventures (independently or spinning out of an existing
firm or institution) has received much attention recently from governments and academic
researchers. Looking at all sectors covered by Europe’s Innovation Union Scoreboard 2014,
there is reported to be high degree of variance between member states in terms of linkages and
entrepreneurship as well as innovators outputs [11]. Amongst other things, this involved
measuring SMEs that innovate in-house and collaboration efforts between innovating firms,
research collaboration between the private and public sector, exports of knowledge-intensive
services, and intellectual property. As discussed in the Introduction, if the research were
extended into a larger study, a country-by-country comparison could offer exemplar practices
to help address variances that exist for cyber security and privacy innovation.

18
Figure 15 Key themes related to entrepreneurship
The major trend running through the framework in Figure 15 relates to third-party facilitation
of opportunities to develop commercial relationships. Sellers of cyber security and privacy
technologies are seeking support to gain customer inputs (I) during the technology development
process and (II) at the point of finding potential partners and buyers of mature technologies.
The UK is fast developing further support mechanisms particularly aimed at SMEs with its 15
cyber security clusters, Catapult commercialisation centres and funding competitions through
Innovate UK.
Academic researchers however are not well incentivised to pursue entrepreneurial activities
and a perceived disconnect between research and application needs to be addressed by both
funding bodies and policies within academic institutions. Transitioning research into practice
should be encouraged at the institutional level and where applicable commercial training should

19
be provided. An alternative school of thought suggests researchers/technologists should
instead partner up with skilled and experienced commercial professionals. This would allow
each party to play to their strengths, which as Figure 16 suggests will be critical to ensure Europe
fully exploits the technical skills of talent base in cyber security.
This is an approach successfully demonstrated by staff at the Centre for Secure Information
Technologies (CSIT) at Queens University Belfast. CSIT is one of seven funded UK Innovation and
Knowledge Centres (IKCs), which are university based, led by an expert entrepreneurial team,
and focused on the commercialisation of emerging technologies, in an area of high innovation
potential. The wider adoption of IKCs is recommended in Figure 15. CSIT staff argued that
commercial teams bring three important capabilities: (1) industry networks that provide
feedback for technologists, (2) horizon scanning for R&D, and (3) discovery of applications
beyond intended use and market.
Figure 16 Survey responses to Question 19 within Market and Policy
Whilst it is encouraging that most respondents believe that the national talent base is capable
of producing game-changing technologies (some interviewees added ‘thought leadership’ also),
some further questions were raised. It is not clear which researchers and institutions are leading
in Europe in terms of game changing technologies – where has progress been made and what
can be learned? It is also difficult to measure whether the available talent is meeting current
demands – what more needs to be done and are industry and academia working effectively
together?
Performance of new technology
Feedback from venture capitalists and potential buyers of new technologies in cyber security
and privacy argue the importance of being able to evidence technology performance. This is
closely linked to the trend identified in Figure 13 in the Business Case section concerned with
the subsequent task of evidencing the value proposition which is concerned with the impact a
buyer’s offering makes to the client’s organisation. Another perspective within this trend is to
simultaneously consider both technology readiness and environmental readiness for the
adoption of new technology. This may require the technology developer to consider levels of
risk and investment associated with either market stimulation/development or adapting the
technology to meet market requirements as they stand.

20
Figure 17 Key themes related to performance of new technology
The major trend running through the framework in Figure 17 relates to the testing and
validation of new technologies. Feedback from academia and SMEs in particular focused on the
issue of limited access to facilities that can realistically simulate actual operations within or
across large complex organisations. As discussed in the Data Sharing section, feedback in this
area also raised the need to create a new body or support an existing body(s) to provide shared
test beds for technologists or independent testing and validation services.

21
Figure 18 Survey responses to Question 9 within Technology Transfer
In the context of technology readiness and maturity, one respondent argued the need to “pay
attention to the infrastructure and service providers that set the context for security” and to
watch out for upcoming organisations that will set future standards and rules. In this context,
as represented in Figure 18 it was also counter argued by SME and academic respondents that
they had limited visibility and access to potential client systems, thus making integration with
current infrastructure difficult. Figure 19 also clearly confirms the issue of integrating new
technology into existing infrastructure at the point of technology transfer. These points raise
the question of how technology infrastructure owners and large enterprise can play more of a
supporting role.
Figure 19 Survey responses to Question 6 within Technology Readiness and Maturity
According to Figure 19, there is a clear consensus that large enterprises should play a greater
technical and economic role in supporting new ventures in cyber security – a point that most
large enterprise respondents also concur with. Although two separate large enterprise
respondents argued “If SMEs could convey their propositions better, large enterprise would take
more of an interest”, and “Large enterprise could play a greater role in thought leadership,
mentoring and partnering with new ventures”. Given that much of the focus of national
innovation initiatives are aimed at SMEs, it may be necessary to clarify the role of large
enterprise and how they can be incentivised to support more collaborative approaches to R&D
in cyber security and privacy. Other respondent feedback in this area included the following:

22
“Large enterprise could help more with validation of new cyber security technologies”
(Government)
“SMEs lack access to computing power and models that offer real world scale
scenarios” (SME)
“It would be helpful to have continuity of support over longer periods” (SME)
“Government should play a greater role in creating the right conditions where SMEs
can stand up on their own merits, it is not the responsibility of large enterprise”
(Academia/Research)
Figure 20 Survey responses to Question 11 within Technology Transfer
Another perspective that is relevant to the performance of new technology theme is the
question of newly emerging cyber security technology clusters and their geographical proximity
to major ICT clients. The responses however in Figure 20 indicate that there is some uncertainty
around this question. More data over time on the performance of technology firms within
clusters may provide further insights. Interviews also reflected divergent views leaning both
ways. On one hand the strengths of the technology and thinking internationally were more
important. On the other hand it was argued that close geographic relationships and a critical
mass of suppliers, scientific talent were needed.
Standardization
Standards are important across industry boundaries to help prevent or mitigate cyber security
attacks. The 2013 UK government sponsored research report on standards revealed the extent
of the challenge in this area, suggesting poor perceived incentives and benefits from investing
in standards and external certification [12]. As illustrated in Figure 20, a number of factors were
identified relating to standardization (broadly defined). These include technical and service
standards, legislation, licensing and skills, procurement rules, performance metrics and
compliance.

23
Figure 21 Key themes related to Standardization
One of the trends running through the framework in Figure 21 relates to the need for clearer
criteria for effective risk management. As consistent with the aforementioned government
report on standards, respondent feedback suggests that organisations require improvements
to both government-led legislative interventions (in close consultation with experts) and
market-led interventions to support effective risk management. These sentiments are echoed
in Figure 22 and Figure 23, where there is clear agreement for the need to make industry
regulations more coherent and metrics more effective.

24
Figure 22 Survey responses to Question 23 within Market and Policy
Figure 23 Survey responses to Question 5 within Technology Readiness and Maturity
Respondents from different groups had the following feedback on the issue of metrics:
“Existing metrics are not fit for purpose, yet we still use them – we really need better
metrics” (Large enterprise)
“Metrics relevant to new technology in cyber security are not widely used”
(Government)
“Badly constructed business plans often contain poor use of metrics and evidence”
(Academia/Research)

25
Figure 24 Survey responses to Question 7 within Technology Readiness and Maturity
As illustrated in Figure 24 respondents are mostly agree that the application of new cyber
security technologies are affected by exogenous factors. Also, specific industries may have their
own unique constraints to consider, such as payment security standards in the financial sector.
When comparing Figure 24 with Figure 10 and Figure 11 on the effectiveness of public funding,
a large enterprise respondent argued that external factors such as policy, economics and culture
had a more significant impact. Other respondents provided further comments:
“Agree with the question but […] the most important factors are more commercial”
(Large enterprise)
“If the client's system fails, who is responsible - fear of being sued is an issue” (SME)
“Factors are more 'export control', 'compliance law' and insurance”
(SME – Venture Capitalist)
“Exogenous factors have a strong influence, regardless of the merits of the technology”
(Government)
“In future, insurance premiums might be reduced where compliance with accepted
standards has been met” (Government)
“There is currently no liability for poor quality cyber security software” (Government)
These comments raise further questions and as well as potential knowledge gaps (e.g. legal
liability factors and the link between standards and insurance), which may benefit from further
research.
R&D competitiveness
This section discusses the broadest theme, encompassing a range of issues from enhancing
commercial relations between university-industry, the role of supporting government initiatives
and better utilisation of talent to achieve commercial benefits.

26
Figure 25 Key themes related to R&D competitiveness
One of the key trends running through the framework in Figure 25 relates to managing
intellectual property. Concerns were raised on both sides of the equation in terms of SMEs
being concerned about loss of IP to large enterprises and vice versa. This is clearly reflected in
Figure 26 where concerns about IPR is a barrier to effective commercialisation. Interviews with
academia/research raised similar concerns particularly around the difficulties of enforcing legal
protection and loosing IP to partners who can in future become competitors. Although
technology transfer staff were more confident than their research colleagues stating that IPR
issues (beyond cyber security) tend to managed effectively. Whilst SMEs look to government to
support IP protection during, some government respondents argued IPR issues did not prevent
exploration, rather IP related problems tend to occur where R&D partnering and collaboration
is involved.

27
Figure 26 Survey responses to Question 14 within Technology Transfer
Figure 27 Survey responses to Question 15 within Technology Transfer
Given the pace of change and importance of responding to the client’s operational demands (as
mentioned in Figure 25), it is interesting to note the level of uncertainty indicated in Figure 27
as to whether adapting to new market requirements presents a major barrier to
commercialisation of new cyber security technologies. The uncertainty is notable across all
organisational types. A government respondent argued that on balance, academia/research
need to be more focused on long-term questions and challenges and avoid competing with
SMEs who typically operate in shorter time-scales.
In a related question, Figure 28 indicates that most respondents believe government and
agencies are lagging behind in terms of supporting industry to find solutions to emerging cyber
security and privacy threats. Government and agency interviewees argued that were now
putting the right mechanisms in place and whilst there was room for improvement, time would
tell as to the effectiveness of specific policies. For example, positive feedback was made about
the approach taken by the Horizon 2020 research programme and increasing engagement
(including seed funding) of the UK’s Government Communications Headquarters (GCHQ) with
industry and academia/research.

28
Figure 28 Survey responses to Question 21 within Market and Policy
At the level of market and policy, two different SME respondents argued:
“Much of government is still focused on Information Assurance (IA) rather than Cyber
in its widest context. Cyber is more than the Internet. Acceptance that current IA
activities are not fit for purpose in the future SMART Society allows work to be done to
create the environment where technology can thrive and be forward looking. This
message has also to be pushed out to Industry.”
“I don't really think general policies and legislation are of much importance. The
problem is that they are typically formulated by people with little insight into the
problems and issues that needs to be addressed”
Figure 29 Survey responses to Question 13 within Technology Transfer
One of the key issues brought up from a facilitated Advisory Focus Group session at the SecCord
2014 Cyber Security & Privacy (CSP) Innovation Forum in Athens was the lack of marketing
expertise to ensure effective technology transfer. As indicated in Figure 29, respondents also
agree with the proposition that there is a significant issue with the lack of effective marketing
expertise. Looking back to Figure 25, there are a number of marketing oriented challenges that
include: gaining access to the right channels and potential buyers, being able translate the
benefits of technology adoption into the client’s language, demonstrate readiness for entering
into the client’s procurement processes and mapping technology impact to customer problems.

29
Concluding Remarks
This paper presented expert on-the-ground views from innovators, investors and other key
stakeholders involved in bringing new cyber security technologies to market. A roadmap was
developed encompassing the key areas of market and policy, technology transfer and
technology readiness and maturity. Whilst the sample was small and most of the responses
were UK oriented, the research provides some valuable stories of the realities faced by
innovators and supporting bodies. The proven methodology could also provide the basis for
further larger studies. The results were synthesised into seven focal areas, designed to help
policy makers, industry and academic leaders to focus their efforts towards increasing the
impact of R&D. These are discussed in-turn below.
 Data sharing. There are clear drivers to address the challenges involved in collecting, sharing
and analysing useful information on incidents, test feedback, and security threats. The major
trend reported in this theme relates to how data can be usefully applied for the purpose of
developing stronger technological responses to threats and analytical (technical and
commercial) models to demonstrate the value of new technology. Stakeholder forums and
initiatives can do more to address data sharing issues and the role and benefits of engaging
with independent authorities needs to be more prominent.
 Financial. A diverse number of issues and recommendations were raised in this theme,
particularly in the context of market and policy, where new and existing financial
instruments should be aimed at stimulating (higher TRL) technology development
opportunities. One of the major trends in this theme was around financial mechanisms to
support new ventures as well as access to capital to follow through into application. For
government, the status quo is not an option as most respondents believe that more needs
to be done to ensure publicly-funded research addresses current and future capability gaps
and the right incentives are in place to support the adoption of new technologies.
 Business case. This theme presented issues around the technology-problem/opportunity
combination space. Whilst useful ways forward are identified, one of the major challenges
was around the resources needed to evidence the value proposition. This means quantifying
the return on investment, qualifying security claims and demonstrating how outcomes and
benefits will be realised by potential customers. It is clear however, to achieve this kind of
evidence it will be necessary to introduce further support mechanisms to help demonstrate
utility in large scale systems environments.
 Entrepreneurship. New ventures in cyber security and privacy appear to be seeking greater
assistance in third-party facilitation of commercial relationships, at both early and mature
stages of the technology development process. Whilst supporting mechanisms and bodies
are in place in the UK, further research might illuminate their contributions and bring out
exemplar practices for comparison across Europe. Whilst the talent base is perceived to be
strong, questions remain around if and how academic institutions can be reformed to
encourage entrepreneurship.
 Performance of new technology. This theme explored challenges related to proving
technologies in operations. In this context it can be useful to simultaneously juxtapose
technology readiness with market readiness and what may be needed in terms of further
investment and managing risk. Integration between new cyber security technologies and

30
current infrastructure was considered to be a significant barrier to technology transfer.
Furthermore, it will be necessary to review how large enterprises can be incentivised to
support more collaborative approaches to R&D, given their key role in the innovation
ecosystem.
 Standardization. A number of factors related to standards were brought up, calling for both
government and market led interventions. This included a trend related to the need for
clearer criteria for effective risk management and other factors including technical and
service standards, legislation, licensing and skills, procurement rules, performance metrics
and compliance. The coherency of industry regulations in respondent areas of operation and
metrics demonstrating the efficacy of new cyber security technologies were both
considered to be areas for attention and improvement. In a related question, there was a
clear perception that the application of new cyber security technologies were significantly
affected by exogenous factors, such as legal factors and insurance, however this was an area
that would benefit from further research.
 R&D competitiveness. This theme provided a broad category for a range of issues including
the role of government schemes and initiatives, enhancing commercial relations between
industry and Universities, and exploiting the talent base. A key trend from both the survey
and interviews was around roles and responsibilities for improving intellectual property
rights (IPR) issues; which was considered to be barrier for exploring the full range of
commercial options at the point of technology transfer. Another issue affecting the
performance of innovators, was around the need to develop greater marketing expertise to
ensure a smoother transition transfer process, particularly in terms of meeting client
requirements. Given the pace of change in cyber security and privacy, it was not clear
whether adapting and responding to new market requirements presented a major barrier
to commercialisation for new technologies. In the case of government and agencies
however, there was a reasonable consensus that more needed to be done to support
industry with finding solutions to emerging threats.
In conclusion, this white paper has reported an analysis of a stakeholder consultation concerned
with the impact of R&D in cyber security and privacy. The roadmapping of stakeholder feedback
(structured according to three main research areas: Technology Readiness and Maturity,
Technology Transfer, and Market and Policy) points out insights across seven emerging focal
areas of intervention for enhancing the impact of R&D in cyber security and privacy.
References
1. European Commission (2011) Impact Assessment, Communication from the Commission 'Horizon
2020 - The Framework Programme for Research and Innovation, EC Brussels, 30.11.2011
2. European Commission (2011) Impact Assessment, Proposal for a Directive of the European
Parliament and of the Council, Concerning measures to ensure a high level of network and
information security across the Union, EC Strasbourg, 7.2.2013
3. Anderson, R; Boehme, R; Clayton, R; and Moore, T. (2008) Security Economics and the Internal
Market, ENISA.
4. Maughan, D; Balenson, D; Lindqvist, U; and Tudor, Z. (2013) Crossing the Valley of Death:
Transitioning Cybersecurity Research into Practice, IEEE Security & Privacy, March/April
5. Downey, F. (2012) Bridging the “valley of death”: improving the commercialisation of research,
Engineering the Future

31
6. Felici, M. (2014) Economics, Security and Innovation. In: Altmann, J; Vanmechelen, K; Rana, O.F.
(Eds.), Economics of Grids, Clouds, Systems, and Services, Proceedings from 11th International
Conference, GECON 2014, Springer, LNCS 8914, pp. 3-15.
7. Kearney, P; Dooley, Z. (Eds.) (2015) Business Cases and Innovation Paths, NIS Platform Working
Group 3
8. Kapletia, D; Felici, M; and Wainwright, N. (2014). An Integrated Framework for Innovation
Management. In Francis, C., Felici, M. (Eds.) Cyber Security and Privacy, Springer, CCIS 470, pp 135-
147.
9. Groenveld, P. (2007) Roadmapping integrates business and technology. Research Technology
Management, Industrial Research Institute, 50(6) pp 49-58
10. Centre for Technology Management (2009) Making the business case for new technologies, IfM
Briefing, Vol 1 (3), Institute for Manufacturing, University of Cambridge
11. Hollanders, H; and Es-Sadki, N. (2014) Innovation Union Scoreboard. European Commission, Belgium
12. PricewaterhouseCoopers (2013) UK Cyber Security Standards, A research report commissioned by
the UK Department for Business Innovation and Skills