User-driven innovation and technology-use in public health and social care: A systematic review of existing evidence

Abstract

This systematic review revealed and discussed empirical evidence generated from 21 international user-driven innovation studies in the public health and social care. We used PRISMA guideline to ensure a transparent and replicable research process. With the guide of relevant theoretical models, we identified the distinct characteristics of user-driven innovation in current public social care sectors, in respect of its strategic innovation process and user-oriented, empower-based objectives. We categorized different end- and intermediate user groups and discussed how them directly and indirectly engaged in various innovation phases via the support of different methods, and how their participation helped to fulfill users’ roles for exploration, experiments, test, and innovation. We also took a closer look at technological options addressed via included studies and in particular discussed how technologies interact with users in the innovation process.

Full text

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
User-driven innovation and technology-use in public
health and social care: A systematic review of existing
evidence
Hong Zhu
zhu.hong@uit.no | Faculty of Child Welfare and Social Work, Department of Humanities, Social
Sciences and Education, UiT The Arctic University of Norway, Follumsvei 39, 9510 Alta, Norway
Synnøve Thomassen
synnove.t.andersen@uit.no | Faculty of Child Welfare and Social Work, Department of Humanities,
Social Sciences and Education, UiT The Arctic University of Norway, Follumsvei 39, 9510 Alta, Norway
Abstract. This systematic review revealed and discussed empirical evidence generated from 21 inter-
national user-driven innovation studies in the public health and social care. We used PRISMA guideline
to ensure a transparent and replicable research process. With the guide of relevant theoretical models,
we identified the distinct characteristics of user-driven innovation in current public social care sectors, in
respect of its strategic innovation process and user-oriented, empower-based objectives. We categorized
different end- and intermediate user groups and discussed how them directly and indirectly engaged in
various innovation phases via the support of different methods, and how their participation helped to
fulfill users? roles for exploration, experiments, test, and innovation. We also took a closer look at
technological options addressed via included studies and in particular discussed how technologies interact
with users in the innovation process.
Keywords. Innovation; Technology; Systematic Review; Health Care; Child Care; Care for the Elderly.
Cite paper as: Zhu, H., Thomassen, S., (2018). User-driven innovation and technology-use in public health
and social care: A systematic review of existing evidence, Journal of Innovation Management, www.open-jim.org,
6(2), 138-169. http://hdl.handle.net/10216/113222
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 138
HANDLE: http://hdl.handle.net/10216/113222
SM: Sep/2017 AM: May/2018

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
1 Introduction
For decades, public health and social care in most industrialized societies have encountered sig-
nificant challenges in delivering better and high-quality services. Most countries, for example,
experience substantial demographic changes and the resultant projected increase in need of el-
dercare combined with a concurrent decline in the number of professional careers (Couture et al.,
2016; Huber et al., 2017). Also, health agencies today have to address the sophisticated threats,
including morbidity and mortality associated with chronic, infectious and lifestyle diseases, in
the context of flat or constrained funding and new and changing healthcare legislation (Lister
et al., 2017; Tursunbayeva et al., 2017). Nevertheless, due to the high tax level, citizens urge to
receive public services shaped for their needs and of the best quality (Granier and Kudo, 2016).
At the same time, technologies open for new services that lead to social, political, economic, and
legal transformation. Such transformation accelerates citizens’ expectation on states in harness-
ing technology advances for value creation and social inclusion (Chan and Holosko, 2016; Keys,
2016; Huber et al., 2017).
Innovation is high on the agendas in most developed economies to meet undergoing challenges
in public health and social care. In classic innovation theory, technology is an essential driver for
innovation (Schumpeter, 1934). It has also been the same situation in the public sector where
the introduction of technologies lead to the change in services (Marsan and Paré, 2013; Pang
et al., 2014; Huber et al., 2017). However, the development of technologies in public domain
conforms to an expert-driven and top-down process where users of those technologies, including
service recipients and employees, are not actively involved (e.g., Hill and Shaw, 2011; Andersen
and Jansen, 2012; Moen, 2012). This brought common problems such as dehumanized and
undemocratic technical services and the loss of valuable views, resources, and knowledge from
users in the innovation process (Hill and Shaw, 2011; Baker et al., 2014). Furthermore, due
to the mass implication of new technologies, primarily increasingly digitized production/service
process, citizens gradually grew their demand for a better and open system where they can be
part of service design and delivery (Von Hippel, 2009; Sørensen and Torfing, 2012; Andersen,
2013). All these present issues make the public health and social care sector to further explore
an innovative approach to work with different user groups, exchange their views and knowledge,
and eventually facilitate the value co-creation process in service design and delivery (Chan and
Holosko, 2016; Palumbo, 2016).
User-driven innovation (UDI), in this light, has been gaining attention over the years, especially
in the welfare sector of Northern Europe (e.g., Pässilä et al., 2013; Hvenegaard Rasmussen,
2016; Puig-Pey et al., 2017). It refers to a process of "tapping users’ knowledge" to develop
new products, services, and concepts"(Von Hippel, 2009, p. 30). Users, instead of technologies,
become the primary driver for innovation. UDI concerns both an understanding of users needs
and a systematic involvement of users at different stages of the innovation process. Its strengths
lie in the fact that both service recipients and professionals working in the public sector perform as
crucial players in the provision of solutions for defined problems in the field. Those individuals
thus will have better insight to innovate and spur development of precisely what they want
(Andersen and Jansen, 2012; Wihlman et al., 2014; Scupola and Zanfei, 2016).
Since user knowledge is gradually seeing as a crucial source of innovation in the public sector,
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 139

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
there is a rasing need to discuss questions such as what ultimate goals of UDI in public health
and social care, how to support user participation in the innovation process, and how to gather
accurate user knowledge in the practice for UDI. However, there has been relatively little research
discussed those questions (Andersen, 2013; Kallio et al., 2013; Szymańska, 2017). Further, some
previous studies in the public sector have argued that although users, instead of technologies,
became the primary drive in UDI, we cannot neglect the fact that technology is still the force
of all kinds of innovation in today’s digitalized world. It can, for example, help gather, store
and sort out user knowledge for innovation, supporting the institution access user knowledge and
making the access cost inexpensive (Hill and Shaw, 2011; Barlott et al., 2016). However, even
fewer studies investigated if technologies can facilitate UDI by providing easy and inexpensive
access to user knowledge. To address current literature gap, we conducted a systematic review of
existing international evidence related to UDI in public health and social care. Our study aims
to support an in-depth understanding towards research topic by providing answers to following
three essential questions:
1. What are distinct characteristics of UDI in public health and social care, in respect of its
objective and process?
2. Who are users of public health and social care, and how different groups of users can be
engaged in UDI process?
3. What roles of technologies play across UDI process, and what their relationship with users?
2 Theoretical Framework
Previous studies are valuable resources shaping a fundamental view towards the study topic.
Some of these useful theoretical perspectives on innovation, UDI, the process of UDI, the role
of users, and interaction between technology and user, are briefly presented in the following
paragraphs.
2.1 Innovation
Early in 1934, Joseph Alois Schumpeter (1934) defined innovation as "combinations of resources"
that drive the economic development substantially (p.66). Technology plays a significant role
in innovation process - leading to the launch of a new product, method, market, sources, or
industry structure (Joseph Alois Schumpeter, 1934). Over decades, Schumpeter’s classic innova-
tion theory has inspired practitioners and scholars to understand the meaning and the process
of innovation in private and market-based sectors. Now innovation is often viewed as the new
and improved application of the product, process, marketing and organizations (OECD 2005).
Indicative of growing cross-sector exchange, the proliferation of innovation has been developed
in public domain (Phills et al., 2008). However, it is also subject to varying definitions. In
this article, we operate on the definition that innovation is the creation and implementation
of new products, services, process, and methods, which helps to meet new requirements and
unarticulated needs in public health and social care (Albury, 2005).
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 140

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
2.2 User-driven Innovation
The user-driven approach is an interdisciplinary concept, which derived from fields such as ar-
chitecture and design, IT and engineering, business management and social sciences. In archi-
tecture and design, for example, UDI refers as "systematic approach to develop new products
and services, building on the investigation of adoption of users’ life, identity, praxis, and needs"
(Christiansson et al., 2008, p. 249). Shaped within multiple disciplines, in this article, we use
Von Hippel’s definition on UDI that mentioned in the Introduction. UDI is a process of draw-
ing on users’ knowledge" to develop innovation (Von Hippel, 2009). It often links with terms
such as co-design (Dekelver et al., 2011; All et al., 2013; Habicht and Thallmaier, 2017) and
co-production (Pestoff, 2012; Durose et al., 2017). UDI is open in character and focuses on iden-
tifying existing and potential users, systematic searching and understanding users’ explicit and
implicit needs, knowledge, and ideas, and intensive involving users in co-creation and innovation
process (Tuomi, 2002; Wise and Høgenhaven, 2008; Von Hippel, 2009). Users in this definition
include intermediate users (e.g., user organizations) and consumer users (individual end-users
or user communities), who are distinguished from suppliers (e.g., producers or manufacturers)
(Trott et al., 2013).
2.3 UDI Process and Users’ Roles
The Innovation Wheel (Fig.1) is a model for UDI described by Wise and Høgenhaven (2008),
which has been applied to describe an organization’s innovation process and the involvement of
users throughout the process (e.g., Røtnes and Staalesen, 2010; Pugh, 2014). As Fig.1 indicates,
to launch a new UDI, the first crucial phase is to focus on what to produce, referred as WHAT
periods, including stages of opportunity identification, data collection, pattern recognition, and
concept ideas that meet users’ needs. The second phase addresses how innovative ideas can be
implemented, referred as HOW stages, including steps of conceptualization, prototype, test, and
implementation.
WHAT
HOW
Opportunity
Identification
Data
Collection
Pattern
Recognition
Concept
Idea
Conceptulization
Implementation
Test
Prototyping
Fig. 1. The Innovation Wheel - cited from Wise and Høgenhaven (2008, p. 17)
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 141

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
To support The Innovation Wheel, Wise and Høgehaven (2008) outline another framework (Fig.2)
which was modified in this study to help to understand the users’ four different situations,
namely observation, experiments, tests, and innovation, across WHAT and HOW phases in the
innovation. These four categories suggest users’ direct/indirect involvement in innovation steps,
together with their various roles and responsibilities. Innovation teams often applied different
methods and strategies to fulfill specific functions of user in particular innovation stages (Wise
and Høgenhaven, 2008).
Above two models are beneficial to our review study especially in a situation where there is
rare literature offers a systematic theoretical view to understanding UDI process and users’ roles
inside (e.g., Røtnes and Staalesen, 2010; Paiva et al., 2016; Szymańska, 2017). Both models are
valuable frameworks developed through many scientific case studies upon UDI. Also, they have
been continuously applied and further developed via other innovation practitioners and scholars
who dedicate to identify new and better ways to facilitate UDI (e.g., Røtnes and Staalesen, 2010;
Pugh, 2014). In this study, therefore, we use these two models as conceptual tools to help us
extracting, categorizing, comparing and summarizing existing evidence regarding UDI process,
users’ roles, and methods applied to encourage user participation. It worth to note that these
two models lack a particular focus on technologies that can be the force in supporting all kinds of
innovation (Burgelman et al., 1996; Lundvall and Borrás, 2005) and the tool of helping tapping
user knowledge (Bhatt, 2001; Wang et al., 2017). In next paragraphs, thus, we placed attention
on previous studies that address the interaction between technology and users.
 
   
   

   
   

 
   
    
 
  
  
 
 
   
  
  
 
 
  
     

 
   
    
  

Direct
Indirect
Acknowledged Unacknowledged
Fig. 2. Map of Users’ Roles in Innovation Process - Adapted from Røtnes and Staalesen
(2010)
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 142

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
2.4 Interaction between Technologies and Users
The need to create and deliver innovation at an ever-increasing rate is requiring organization
today to apply a more deliberate and systematic approach to manage and utilize collective
knowledge stemming from users (Jenny and Rod, 2002; Wang et al., 2017). In previous studies
on technology and knowledge management, we found that efficient technology appliances can help
users with the facilitation of knowledge creation and diffusion (Jenny and Rod, 2002; Marina,
2007; Wang et al., 2017). For example, information and communication technologies (ICTs),
particularly social media, may accelerate the speed of knowledge transfer and creation by afford-
ing new types of behaviors that were not possible with the previous form of computer-mediated
communication (Lewis et al., 2010; David et al., 2014; Hussain, 2017). Favorite technological
tools such as employee competence databases, online search systems, expert networks, workflow
software, decision support systems, data warehouse and so the forth, all can enable organization
to gather, store, response, sort out and utilize user knowledge more efficiently and economically
(Sher and Lee, 2004; Park et al., 2015).
While technologies are used for knowledge management, their adoption in practice sometimes
meets resistance from users. Many existing studies focus on the interaction between human
and technology, which aims to make technologies more usable and useful to provide people with
experiences fitting their specific background knowledge and objectives. For example, technol-
ogy acceptance theory identifies two main factors that affect users’ acceptance of technological
appliance, namely perceived usefulness and perceived ease of use (Davis, 1985). Also, there
are some key determinants of perceived usefulness and usage intention constructs, which can
be divided into two groups as social influence processes (e.g., subjective norm, voluntariness,
and imagination) and cognitive instrumental processes (job relevance, output quality and result
demonstrability)(Venkatesh and Davis, 2000).
From performance-based perspectives, on the other hand, the utility and usability of technical
systems can be assessed by effectiveness, efficiency (Bevan, 1995) and user satisfaction (Dillon,
2002). More recently, increasing studies highlight that user emotion and intrinsic motivation,
influenced by personal experience with technology, preferred working style, and the aesthet-
ics of system design,can be more crucial for users to adopt a particular technological system
(Venkatesh, 2000; Beaudry and Pinsonneault, 2010; Cohen, 2014). Further, from an engage-
ment aspect, factors such as attributes of challenges, playfulness, endurability, aesthetic and
sensory appeal, perceived user control, etc., might also influence people’s interactive relationship
or experience with the use of technologies (O’Brien and Toms, 2008; Kim et al., 2013).
3 Methodology
We used the systematic review as the method in this research. In this section, we elaborate on
the research strategies that were used to select studies and the detailed review process across
research stages.
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 143

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
3.1 A Systematic Review
A systematic review of previous research can bring together the results of existing evidence
and provide a reliable answer "how" and "why" of a particular phenomenon (Booth et al.,
2016). This study used a systematic review as a methodology because this form of review differs
from traditional narrative review by adopting a replicable, scientific and transparent process
(Thomas and Harden, 2008). It helps us to minimize bias through exhaustive literature searches
of both quantitative and qualitative studies and by offering an audit trail of the reviews decision,
procedures, and conclusions (Tranfield et al., 2003). During the review, we followed Proffered
Reporting Items for Systematic Review and Meta-analysis (PRISMA) guidelines developed by
Moher et al. (2009). This guideline has been widely used in various systematic review studies
related to public services across countries (e.g., Lopez-Hartmann et al., 2012; Seys et al., 2013;
Elia et al., 2016). It helped us to ensure a transparent and replicable research process.
3.2 Search Strategies
This review searched two international electronic databases including ISI Web of Science and
Scopus. To focus on very current studies, search centered in January 2013- December 2017 texts.
Only articles published in peer-reviewed academic journals in English were included. The search
strings we applied include "user-driven" OR "co-design" OR "co-production" AND "innovat*"
in the title/abstract/keywords. With ISI Web of Science, we further refined results by choosing
exiting categories including Health Care Sciences Services, Health Policy Services, Public En-
vironmental Occupational Health, Social Sciences Interdisciplinary, Social Work, Sociology, and
Rehabilitation. Similarly, for Scopus, we refined results by selecting categories such as Social
Sciences, Medicine and Health Professions.
3.3 Sample Selection
The literature search was conducted from 1st to 12th March 2018. Initially, the database yield
197 hits based on our search keywords (45 from Web of Science and 152 from Scopus). Scopus
included more studies in our case because it has a relatively less extensive category system
that failed to help us further refine results as Web of Science did by narrowing down research
subject areas. All outputs from both databases were exported in Endnote X8 software for
reference management. We deleted manually 25 duplicates and led to 172 records for further
reviewing.
To examine the most rigorous studies and to adequately address the research questions, we
applied following inclusion and exclusion criteria. As these criteria suggest, we only want to
select studies that are empirical, and directly addressed the concept of UDI by offering a precise
definition of it. Since this research target exclusively on public health and social care, we want
to particular include studies within public fields of healthcare, eldercare, childcare, community
services, etc. Also, to answer research questions, we also need to select studies that have a clear
description towards UDI objectives and process, together with roles of users and technologies in
UDI.
Table 1. Inclusion and Exclusion Criteria
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 144

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
Inclusion Criteria Exclusion Criteria
1) The empirical study directly addresses
UDI in public health and social care,
which may include fields of healthcare,
community services, work welfare,
eldercare, childcare, etc.;
2) It has a description towards innovation
concept, process, and outcomes;
3) It has a precise categorization of users in
innovation process;
4) It has a clear explanation towards the
roles of technology in innovation process;
5) It is theory-guided.
1) The study is not empirical;
2) It is irrelevant to public health and social
care;
3) It lacks detailed description towards
innovation concepts, targets, and process;
4) It has no explanation towards the roles of
users in innovation process;
5) It has no focus or lack of detailed
information towards technologies;
6) Lack of full-text access to the article.
After screening by title, keywords, and abstracts, we deleted 105 studies since 77 studies are irrelevant
to public health and social care while 28 hits are not empirical studies. Further, 67 studies went through
a full-text assessment, and 46 were removed. In total 21 studies were selected for this review as they
fully meet all inclusion criteria. Each study was then independently reviewed using our inclusion criteria
by both authors, to ensure their election eligibility. The stage of selection is illustrated in Fig.4, using a
PRISMA flow chart.
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 145

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
Rec o rds identified through da tabase searching (n=197)
ISI Web of Science (n=169); SCOPUS (n=207); ProQuest (n =14 6)
Rec o rds after duplicates removed (n=172)
41%(n=189)duplicates removed
Tittl e, Key
Words, Abs tra ct
screening for
eligibility
(n=111)
Art i cl es excluded, wi t h reasons (n=105)
73%(n=77)were deleted because th ey are
technology irrelevant;
27%(n=28)were deleted because th ey are not
empirical studies
Screening
Ful l -text as sessed
for elig ibility
(n=67)
Art i cl es excluded,wi t h reasons (n =46)
45%(n=20 )wer e deleted si n ce they are not
technology relevant;
27%(n=13)were deleted as they failed to
present inno v atio n concept,process and
outcome;
24%(n=11)were deleted as they are not
empirical studies;
4% (n=2) wer e deleted since no full-text
access;
Eligibility
Included
Art i cl es inc l uded
(n=21)
Fig. 3. PRISMA Flow Chart
3.4 Critical Appraisal of Study Quality
In order to ensure quality of included studies, we rated each study using the Critical Appraisal Skills
Program (CASP, 2017) checklist (Appendix A), which was slightly modified by adding a "not clear" (1)
option for each questions and to the standard "yes" (2) or "no" (0) . There are ten questions on CASP
checklist, so the final score of a study can be between 0 and 20, where 20 was most applicable while 0 was
least applicable. The first author assessed all the eligible studies, while the second author independently
assessed a random sample to appraise inter-rater consistency and resolve any ambiguities.
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 146

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
3.5 Qualitative Synthesis
The systematic review helped us to collect and assess both qualitative and qualitative evidence related to
the investigated topic. The qualitative synthesis was further used to interpret the meaning of the received
data. The strength of qualitative synthesis is to provide in-depth focus and contextualized details towards
phenomenon under the review (Bearman and Dawson, 2013).
We first extracted and coded information from all eligible studies with the support of a structured form.
It contains the following items: the study’s publication features; description of technologies applied;
the categorization and description of users; the innovation objectives, process, and outcomes; research
questions; theoretical frameworks; study methodology; and main findings and conclusion. Followed, with
the support of theoretical underpinnings, we categorized and summarized the extracted information into
key ’themes’ to answer research questions.
4 Finding
4.1 Publication Characteristics
Table 2 reports the essential publication characteristics of selected studies (n=21). All eligible studies were
published between January 2013 and December 2017. Data collection for those studies was undertaken
during 2006 and 2016. The length of data collection period of selected studies was varied: seven studies
less than two years (S1, 6, 7, 8, 17, 19, 21); while ten within two to five years (S2, 3, 5, 9, 10, 12, 14,
15, 16, 20). Also, there are four papers (S4, 11, 13, 18) did not specify the timeframe of their data
collection. We observed that, on average, almost one or two years typically passes during the period of
data collection and the publication of results, although research on innovation should, represent a rapidly
changing landscape. It indicates that the conventional academic literature may be lagging behind as a
source of relevant information on UDI in studied sectors.
Chosen articles targeted different areas in public health and social care: eleven in healthcare (S4, 5, 6,
7, 11, 12, 16, 17, 18, 19, 21), five in eldercare (S1, 2, 10, 13, 15), three related to community service or
protection (S8, 9, 20), and two address childcare and service (S3, 14). Our review supports the current
assumption - there is growing literature to discuss UDI in health- and eldercare, while such discussion
in other social care sectors, such as community work and childcare remain less (e.g., Adedoyin, 2016;
Szymańska, 2017).
Empirical materials from chosen studies were collected from 16 countries. There are 20 studies targeting
high-income countries: thirteen from Western Europe (S1, 2, 3, 4, 5, 6, 8, 13, 15, 17, 18, 19, 21), seven
from Scandinavia countries (S2, 3, 7, 9, 10, 14, 21), and two from Canada and USA (S12, 21). There
is only one study addressing lower-middle-income state, India in particular (S16). No studies meeting
our inclusion criteria took place in low-income countries, despite there being a high need for bottom-up
technical innovation as the leading force in sustainable development and establishment of universal health
and social services for citizens in those regions (UN, 2016).
Regarding the methodology of selected studies, only one adopted quantitative approach (S3) and one with
mixed methods (S21). The rest of studies were qualitative, using mainly a single or multiple case study
approach. In addition, there were four articles applied comparative studies across different countries (S2,
3, 13, 21). As we notice, a qualitative bias may prevail among studies regarding UDI in public health and
social care sector. It might because qualitative have an in-depth and detailed focus on user-innovation
processes including different actors (Konsti-Laakso, 2017).
According to CASP assessment, none of the qualifying studies received a maximum score of 20 on the
quality evaluation scale, although all of them were ranked as being of good quality (scoring 14+) and 14
as high quality (scoring 16+).
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 147

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
Table 2. Basic Publish Characteristics of Selected Studies
No. Authors
and Year
Study Topics Related
Sectors
Study
Coun-
tries
Study
De-
sign
Journal
Name
Data
Col-
lec-
tion
Year
CASP
Score
S1 Angelini et
al., 2016
Co-develop
products and
services for the
elderly
Eldercare Switzerland Qual. Future
Internet
2015 18
S2 Bugge et
al., 2017
Co-design in
assistive living
Eldercare UK &
Norway
Qual. European
Planning
Studies
2014-
2016
18
S3 Calvo-
Lerma et
al., 2017
Mobile
appliance for
childcare
Childcare Spain,
Germany,
Italy,
Norway,
Belgium,
Portugal
& Nether-
land
Quant. BMJ Open 2015-
2017
14
S4 Cosma et
al., 2016
Co-design game
for cancer
evaluation and
education
Healthcare UK Qual. Journal of
Assistive
Technolo-
gies
Unclear 16
S5 De Souza
et al., 2017
Patient
involvement in
service design
Healthcare UK Qual. Health Ex-
pectations
2014-
2016
19
S6 Devlin et
al., 2015
Digital
healthcare
Healthcare UK Qual. Research
and Appli-
cations
2014 18
S7 Dithmer et
al., 2016
A telerehabili-
tation game
program for
heart patients
Healthcare Denmark Qual. Games for
Health
Journal
2015 18
S8 Gasco,
2017
Living labs in
supporting
citizens’
participation
Community
service
Spain Qual. Government
Informa-
tion
Quarterly
2014 15
S9 Kallio et
al., 2013
Co-innovation
in public
service
Community
service
Finland Qual. The
Innovation
Journal
2010-
2012
16
S10 Lehto, 2013 Assistive
technology for
aging at home
Eldercare Finland Qual. Australasian
Medical
Journal
2006-
2010
16
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 148

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
No. Authors
and Year
Study Topics Related
Sectors
Study
Coun-
tries
Study
De-
sign
Journal
Name
Data
Col-
lec-
tion
Year
CASP
Score
S11 Lindgren,
2013
End-user
development of
health record
system
Healthcare Sweden Qual. Studies in
Health
Technology
and
Informatics
Unclear 14
S12 Luck et al.,
2015
User-driven
design for
cancer care
Healthcare USA Qual. Journal of
Oncology
Practice
2009-
2013
17
S13 Mort et al.,
2013
Aging with
telecare
Eldercare UK,
Spain,
Nether-
lands &
Norway
Qual. Sociology
of Health
and Illness
Unclear 16
S14 Pihlainen
et al., 2016
Co-
development of
technology for
children with
special needs
Childcare Finland Qual. International
Journal of
Child-
Computer
Interaction
2009-
2013
18
S15 Pratesi et
al., 2013
Co-design of a
smart
monitoring
system
Eldercare UK Qual. Innovation
Journal
2009-
2011
18
S16 Purkayastha
et al., 2015
Care-
coordination
Healthcare India Qual. Yearbook
of Medical
Informatics
2012-
2014
16
S17 Rubinelli et
al., 2013
Co-design of
health websites
Healthcare Switzerland Qual. Patient
Education
and
Counseling
2013 15
S18 Savory and
Fortune,
2016
Open
technology
innovation
systems within
a teaching
hospital
Healthcare UK Qual. Journal of
Health Or-
ganization
and Man-
agement
Unclear 15
S19 Sugarhood
et al., 2014
Co-design of
assistive
technology
Healthcare UK Qual. Disability
and Reha-
bilitation:
Assistive
Technology
2012 18
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 149

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
No. Authors
and Year
Study Topics Related
Sectors
Study
Coun-
tries
Study
De-
sign
Journal
Name
Data
Col-
lec-
tion
Year
CASP
Score
S20 Southern et
al., 2014
Co-designing
prototypes
with vulnerable
communities
Community
service
UK Qual. Technological
Forecasting
and Social
Change
2009-
2012
19
S21 Timmings
et al., 2016
End-user
driven
approach to
develop
telecare
Healthcare Canada,
Sweden &
Switzer-
land
Mix BMC
Medical
Informatics
and
Decision
Making
2014 18
4.2 The Distinct Characteristics of UDI in Public Health and Social Care
We identified the distinct characteristics of UDI in public health and social care by addressing
two important elements of innovation: objectives and processes. Through comparing, summa-
rizing and discussing those two elements among different UDI, we can, in general, understand
where the UDI in public health and social care is going, how it distinguishes itself from other
types of innovation in the studied sector, and what kinds of action and effort needed for goals
achievement.
4.2.1 UDI objectives
Specific motivation for each UDI varied across projects and fields in public health and social care.
However, in general, there are three common objectives of involving users to generate bottom-up
innovation, including creating a user-centric service/product, to democratize service-designing
process and support a partnership, and to empower service recipients. In details, among studies
in healthcare, the most mentioned objective of UDI is to create a user-centric service, expressed
in terms such as "tailoring service", "customized treatment", and "people-oriented" healthcare
(S 5, 7, 11). This target is quite closely followed by another two objectives - changing the "culture
of care" by bringing a relatively equal cohort between patients and care provider (S5, 17) and
increasing services’ effectiveness through better-addressing patients’ needs (S7, 11, 17).
In eldercare, the principal purpose of involving different user groups in the innovation process
is to develop user-centric, value-added service/products, and to support a partnership between
the elderly and healthcare professionals (S1, 2, 10, 13, 15). By achieving those objectives,
the innovation can further empower the elderly by enabling their "security" "independence"
"mobility" "self-management" and "social interaction", which in general increase the elderly’s
quality of life, expressed in terms such as aging well and active aging (S1, 10, 13). Likewise,
for childcare, the use of UDI is to create service that addresses children’s particular needs and
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 150

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
eventually improve children’s quality of life and potentials for development (S3, 14).
Within three studies related to community service, we found that empowering citizens (S8, 20)
and supporting citizen engagement (S9) were primary reasons for adopting UDI. Additionally,
developing the ailment among different stakeholders in the community for a better social, culture
and economic return was also an essential aim (S8, 20).
4.2.2 UDI process
With the guide of the Innovation Wheel (Fig.1), we categorized UDI process in public health
and social care into eight progressive stages: opportunity identification, data collection, pattern
recognition (data analysis), concept ideas, conceptualization, prototypes, test, and implication.
Not all included studies described this eight-step innovation process in detail, nor did an inno-
vation go through those stages consecutively.
First, a UDI often starts with opportunity identification. Such step aims to identify an excit-
ing field where organizations might have a chance to offer a better or improved service in the
future. Illustrated in S7, the innovation opportunity and objectives were initiated collectively
by Department for Health Science and Technology at Aalborg University and Vendsyssel Hospi-
tal, Denmark, who decided to develop a new application, "The Heart Game", to support heart
patients’ participation in rehabilitation activities. According to S7, only 13 percent of heart
patients engage in rehabilitation activities in Denmark. The reasons for low participation are
manifold, including long transportation times, lack of information about activities, emotional
instability, lack of motivation among the patients, and the time of day due to patients having a
job. To better address existing problems, relevant organizations decided to apply UDI to develop
a more individualized rehabilitation program (Dithmer et al., 2016).
After identifying the innovation direction, steps of data collection and analysis (pattern recog-
nition) have to be taken to generate a better understanding towards targeted users’ demands.
Data collection and pattern recognition usually happen consecutively, and they are two essential
stages across UDI since all included studies have addressed them particularly. Different ap-
proaches usually applied to ensure the richness and quality of collected data. S15, for instance,
demonstrated a UDI project that used older people as co-innovators in the design, development,
and implementation of an intelligent activity monitoring system. Abundant data were collected
through multiple methods including in-depth interviews, telephone interviews, focus groups, and
workshops. This led to an intensive analysis of older people’s technological, psychological and
social needs and preferences, as well as their requirements for an activity monitoring system for
use in homes and residential care setting (Pratesi et al., 2013).
As the outcome of pattern recognition, organizations or innovation teams usually can better
present innovation concepts in detail and evaluate the possible solution regarding how well they
can be adapted to users’ requirements. The Innovation Wheel (Fig. 1) recognize these as two
stages, concept idea and conceptualization, which were perceived as difficult to separate from
each other in our included papers. The UDI project in S11, for example, concept ideas and
conceptualization were completed together with the participation of 176 elderly and 105 experts
in the selected municipality. The outcome of conceptualization led to defining six important
themes for further developing interactive programs for eldercare. Likewise, S14 presented a UDI
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 151

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
program that supports parental co-development of technology for children with special needs.
This project included nine parents, eleven children, three researchers, and four student teaching
assistants in data collection when 79 ideas related to technology development were generated.
After pattern recognition, five themes related to created concepts were identified and presented
to fuel the further growth of technologies.
In the prototype stage, the first concrete example of a new product/service/idea is created.
This stage was emphasized via all qualifying studies as it helps to visualize the original idea
and improve the innovative concepts. Prototypes addressed via qualifying studies are most
physical products or models, as the Cancer Evaluation and Education game presented by S4,
new home-based telecare solutions in S13, and an ICT-based decision support tool in S20. Few
articles presented non-physical prototypes, as the form of descriptions or experiments. S17, for
example, discussed how an interactive health customer website could facilitate user-driven self-
care solution. The prototypes, as a description of the new solution, were pitched from different
online scenarios at this site. Such user-generated solutions were further discussed and expanded
by various other end-user groups (Rubinelli et al., 2013).
Followed by the prototype, the next stage of UDI is testing. Not all studies (e.g., S5, 6, 9,
11, 14, 17, 20) have detailed information towards this stage although it is crucial to provide
an opportunity for users to react and further provide input, and for the further adjustment
to the innovation. Presented via S2, a Norwegian UDI conducted its testing via involving 31
municipalities (out of 426 nationally) to generate feedback and further help for the improvement
of methodologies and practical tools and service models (Bugge et al., 2017). It is important to
mention that the innovation process is not never linear, and often there are loops in which the
project circled. In S2’s scenario, the new problems or needs from users were addressed after the
test, which created a deep collaborated loop within prototype and test.
In the final stage of implementing, new products or services are launched in the market, or
the new concepts are adopted. In S12, an innovative product, the Lung Cancer Care Toolkit,
was launched nationwide after the one-year development process. During the implementation, a
social marketing approach was applied to accelerate dissemination to target end-users (Luck et
al., 2015). There is often another loop between implementation and test, where the innovation
process may jump back and forth. As S15 stated, during the first year of implementation the
new monitoring system for the elderly care, the advisory user group continually reported their
feedback towards the system and helped the technical team to improve how the system affects
the everyday lives of end-users in different settings (Pratesi et al., 2013).
4.3 Categorization of Users and Their Roles in UDI
Guided by previous literature, we categorized user groups in public health and social care into
two main types: end- and intermediate users (Von Hippel, 2009; Szkuta et al., 2014). End-users
referred to those who use innovation directly (Von Hippel, 2009). In our selected studies, they are
groups of health and social care recipients (e.g., citizens, the elderly, patients, as well as children
and their families); health and social care professionals (e.g., clinicians, nurses, social workers,
researchers, and others); and relevant organizations’ administrators. Intermediate users referred
to users’ organizations (Bogers et al., 2010). They included public bodies (e.g., health authority
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 152

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
and institutions, municipalities, research agencies, and universities); private sectors; and NGOs.
All selected studies have presented both end- and intermediate users in their innovation. However,
recipients and professionals of health and social care are two primary end-user groups while
public organizations are dominated intermediate users in our reviewed studies. Also, both end-
and intermediate users were involved intensively in eight innovation processes. However, in the
first stage of the process, opportunity identification, intermediate users play the central role in
identifying an innovation direction. In stages of concept ideas, conceptualization, or prototype,
there are some cases (e.g., S7, 11, 17) that only involved the participation of technical developers;
users were excluded. In the project of S7, for example, application designers analyzed alone the
qualitative data from interviews with end-users and developed an initial architecture of prototype
for addressing identified needs.
With applying the framework for Mapping of Users’ roles (Fig.2), we grouped different end-users’
activities and responsibilities across UDI into four generic categories: exploration, experiments,
tests, and innovation. Different groups of end-users may be involved in different stages of UDI
and play various roles in the innovation process. At the early stage of UDI, especially in data
collection, some end-users can play an indirect role in contributing knowledge towards their un-
acknowledged needs (exploration). In this scenario, the innovation team often apply a range of
ethnographic methods, such as shadowing, participation observation, focus group, workshops,
personal interviews, and user dairies, to obtain information and perspectives towards users’ un-
acknowledged needs (e.g., S1, 7, 13, 14, 17, 21). In S1’ UDI, for instance, three ethnographic
methods were adopted including shadowing technique, focus group, and world café. The shad-
owing technique allowed discovering the latent needs of the users (the elderly), by following them
during their daily activities and doing observations. Focus group was a method to debate about
open topics, such as nutrition, mobility, and interpersonal communication; it also helped to in-
vestigate the users’ perception of existing technological options. The World Café was a gathering
that involved the users in discussing prepared topics, such as the most valuable elements that
relevant to the older adults’ health aging and the innovative ways to obtain them (Angelini et
al., 2016).
In many cases, end-users can also directly involved in innovation process together with the project
team (experiments) throughout the WHAT phase of UDI; however, they act as participator
rather innovators. Those individuals’ direct participation aims to help innovation team to fully
understand their unexpressed requirement through a range of participatory approaches, such as
co-design workshops and participatory research/design (e.g., S6, 8, 9, 11, 14, 15, 16). In S15’s
program, end-users (old adults) have been invited as co-designers who has a partnership with
technical designers. The co-designers’ role is both to advise the technical team about the research
methods, and to participate in collaborative design workshops where their input and ideas will
assist to "design" assistive technologies for themselves (Pratesi et al., 2013).
When coming to the WHEN phase of UDI, end-users are mostly involved in testing step. Common
methods applied to involving end-users in testing include interview, focus group, questionnaires,
clinical trial, usability testing, and user acceptance test (e.g., S2, 3, 4, 12, 14, 16, 19, 20, 12).
Usability testing is a most common technique used in UDI to evaluate a prototype by testing it
on users. It is a crucial usability practice because it offers direct input from real users who use
the system (Bugge et al., 2017). Usability testing focuses on measuring a human-made product’s
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 153

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
capacity to meet its intended purpose. In contrast with other inspection methods where experts
adopt different tools to evaluate a user interface, usability testing empowers users to modify and
improve new product/service/concepts to ensure a successful diffusion in the future (Devlin et
al., 2016; Timmings et al., 2016).
There are some UDI programs addressed via selected articles involved particular groups of the
end-users as co-innovator who have specific control over whole innovation process. Those users
can be part of expert teams, who are highly qualified in their area of expertise and possess specific
knowledge that will contribute most to the innovation. The lead-user method, such as seminars,
workshops and dialogue conferences, is most often applied to support the involvement of this
group of users in UDI (e.g., S2, 8, 9, 12, 17, 18). Beside of experts, advanced end-users can also
be co-innovator in UDI. They refer to those who often stay enthusiasm when it comes to specific
products/service/concepts, and they are more knowledgeable than the average users. The living
lab is an emerging approach used to support advanced users as co-innovator into UDI process
(e.g., S1, 8). It offers a real-life setting and platform for learning and experimenting, facilitates the
management of innovation process, and actively encourage continuous and meaningful interaction
among user groups (Angelini et al., 2016).
4.4 Technologies’ Roles with Users in UDI
All selected studies have a particular focus on the technology-use in UDI. However, the purpose
of these technological appliances varied across different sector and projects. On the one side,
many studies focus on how to adopt a user-driven approach to develop new technologies. In
eldercare, for example, a dominative discussion was draw upon the engagement of the elderly into
co-development of assistive technologies, such as monitoring, electronic sensors, and computer-
based cognitive stimulation, which are likely to make significant contribution to the care of
end-users in institutions and at home (e.g., S2, 10, 13, 15, 19). By involving end-users into
co-production, those new assistive technologies can better meet individuals’ special requirements
on safety, security, mobility, and social contact. In healthcare, likewise, many studies examine
the use of co-design approach for developing new health technologies, such as clinical information
system and ICT-enhanced treatment, to fulfill end-users’ ever-changing demands (S5, 7, 11, 16,
18).
On the other side, some studies locate their attention on the issue how to use different techno-
logical options to facilitate UDI (S4, 8, 14, 16, 17, 20). According to those studies, ICTs, such
as the internet (user websites), social media, and mobile applications, are most popular options
to support UDI. Those tools can help with reaching out new users, encouraging their partici-
pation and communication, and empowering individuals in knowledge contribution for self-care,
rehabilitation, or other social, cultural and economic development.
Users’ relationship with technology is heterogeneous, depending on characteristics of user groups,
type and features of technology, and many other tech-social factors. Therefore, at the initial
stages of UDI, users’ technology acceptance, preference and expectation are often examined (S1,
2, 3, 5, 8, 10, 12, 14, 19). As selected studies argue, the perceived usefulness of technologies
is not enough to make such technical options acceptable. However, personal experience with
technologies (familiarity) can greatly decide users’ preference and acceptance (S1, 3, 5, 7, 12). In
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 154

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
both projects of S3 and S5, the choice of the technical solution was made by collective decision
among different user groups. Through workshops and meetings, users considered mobile tech-
nology serve as a most appropriate platform for support designed health service. Such platform
leads to a reduction in traveling and operation cost. More importantly, it already owns large
scale of installed base - most end-users have familiar with such technology and use it in daily life.
This user-friendly, familiar mobile phone network can offer a technical platform making it easier
for the users to participate in the innovation development and implementation (Calvo-Lerma et
al., 2017; de Souza et al., 2017).
Further, if the innovation team wants to reduce impact of familiarity on users’ adoption towards
technology, it should consider applying invisible technological options (technologies that are
transparent to the users’ eyes) or options that easy to learn and operate reduce users’ fear of novel
solution (Pratesi et al., 2013; Angelini et al., 2016; Dithmer et al., 2016). Few studies also argued
that some intrinsic motivation, such as playfulness, pecuniary interests, as well as aesthetic and
sensory appeal, could also attract users to participate in the innovation process. In this sense,
a widely adopted technology in the society that encourages that intrinsic motivation may be a
promising way to support user engagement (S4, 7, 8, 12, 17). By using user websites and social
media, S17 demonstrated how creating an online entertainment environment can facilitate user
engagement, interaction, and knowledge sharing and learning (Rubinelli et al., 2013). Similarly,
in the S7 project, mobile technology and gamification work tother as useful tools to motivate end-
users in rehabilitation. Digital games apparently offer a promise for stimulating and entrainment,
while mobile platform creates a connection between players and offer the chance for social and
dynamic experience while also allowing these individuals to access the game independent of time
and place (Dithmer et al., 2016).
No double, in some cases, the groups of end-users might not be cognizant of or automatic users of
much of the more basic technologies that are fundamental for innovation. Therefore, suggested
via included papers (S2, 8, 18), involving those people directly as a participator in WHAT
phase of UDI are crucial. Such direct participation helps to explore appropriate technological
options and methodologies to users; simultaneously; it can empower end-users by enhancing their
technical knowledge and innovation competencies (Bugge et al., 2017; Gasco, 2017). In another
word, UDI in health and social care with a technology end goal must apply deliberate methods
for experiencing the potentiality of technologies for more significant innovation by co-designers
themselves to made possible (Savory and Fortune, 2015).
5 Discussion
The most distinct characteristic of UDI in public health and social care is to involve wide ranges
of the end- and intermediate users into both WHAT and HOW innovation stages to meet specific
goals in the fields. Unlike the private sectors where the concept of UDI was often discussed within
the last decade, UDI in the public sector is still a emerging theme which has been brought into
focus very recently (Andersen and Jansen, 2012; Lassen et al., 2015; Dithmer et al., 2016;
Hennala and Melkas, 2016). In general, UDI in public and private sectors pursue very different
objectives. Where the private companies usually apply user-driven approaches to create value-
added products/services for maximizing the profit and enhance the competitive position in the
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 155

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
market, the public institutions are engaged in UDI aiming at increased welfare and democracy
(Waugh et al., 2013; Savory and Fortune, 2015; Paiva et al., 2016; Bugge et al., 2017). Also,
the health and social care sector may different from other public sectors, with regard to a wider
range of user groups with different interests and a more complex dimensions of user involvement
(Rubinelli et al., 2013; Bugge et al., 2017; Gasco, 2017; Pihlainen et al., 2017).
Generally speaking, all included studies demonstrate a strong believe that UDI in public health
and social care is far more sustainable than innovations generated by a single institution. By
focusing on the end-users, health and care recipients and professionals, UDI first unfolds welfare
principles, such as empowerment and client self-determination (Humphreys, 2015; Lassen et al.,
2015; Angelini et al., 2016); it also facilitates changes of culture of care, and organizational struc-
ture where top-down innovation process was dominate (Sugarhood et al., 2014; Wihlman et al.,
2014; Luckock et al., 2017; Valaitis et al., 2017). Through involving different intermediate users,
UDI brings closer cooperation among different health and social care stakeholders and facilitate
the change of existing system for a better solution to addressed revealed challenges (Lindgren,
2013; Purkayastha et al., 2015; Savory and Fortune, 2015; Bugge et al., 2017). However, despite
benefits of UDI, many user-driven projects presented via include articles encountered challenges
during the operation, and the management of which was not always successful. We will further
explain this in the following two section.
5.1 Challenges of Continuous and Adequate Engaging Users
Our findings indeed suggest that end-users can play various roles across UDI process, and there
are comment methods available to apply to support user participation. However, we also noticed
that challenge exist concerns the need for the continuous and adequate engagement of different
groups of end-users in public health and social care. Many UDI projects in our review only
involved end-users in specific innovation steps, and there is still rare case that users are seen as
co-innovators from beginning to the end and take significant control of whole innovation process.
Moreover, several dominative barriers can hinder user participation in UDI: namely differences
in willing of involvement, inefficient communication, and constrained resource.
We know that the willingness to innovate and learning are crucial to stimulate individuals to
innovate (e.g., Savory and Fortune, 2015; Bugge et al., 2017). However, at beginning of many
UDI projects, it is often to have some end-users were not necessarily enthusiastic, not with the
same level of intensity at least, to involve the innovation process, and to learn and share relevant
knowledge or experience (Pratesi et al., 2013; Pihlainen et al., 2017). To address such situation,
early relationship-building with the members of innovation team beyond the innovation context
may be necessary for identifying most enthusiastic participators, and cultivating a genuine part-
nership based on understanding, respect, and trust (Pratesi et al., 2013). Also, a more creative,
flexible and interactive work approach needs to be adopted during the innovation process, and
the presentation of complex information should be easy to understand. It might be beneficial
to emphasizing the importance of participation as a democratic form of inquiry and grounding
the outputs in the experiential aspects of users themselves (Pihlainen et al., 2017). Above two
solutions, however, are time-consuming which might not be realist within a planned UDI within
the constrained timeframe.
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 156

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
When there are many end-user groups involved in UDI process, inefficient communication be-
tween different parties can quickly rise because of diversity in individuals’ interests, knowledge
background, and purposes of participation (Purkayastha et al., 2015; Gasco, 2017). Ensuring
that channels of communication are opened up and transparent across all partners, therefore, is
essential to avoid significant conflicts and misunderstanding (Gasco, 2017). Further, reducing
geographical distance could be promising in supporting the open discussion on the innovation
progress and share ideas in real time (Purkayastha et al., 2015). Also, exploring new and in-
terdisciplinary methodological tools to address communication issues is necessary (Savory and
Fortune, 2015).
Finally, facing constrained innovation resource might be the most critical challenging of UDI.
Budget and time constraints all limit the number of users can be engaged and the degree of their
participation (Pratesi et al., 2013; Sugarhood et al., 2014; Angelini et al., 2016). In addition,
further efficiently tailoring user-driven solution requires sustained assessment, support, and re-
view over a period. However, in reality, lack of a financial and human resource often leave this
continued work unclear (Sugarhood et al., 2014). Argued via Timmings et al., (2016), given the
complexity of UDI, preparatory work to avoid costly implementation errors should be considered
including assessing barriers and facilitators to change, developing a detailed budget plan, and
assessing and establishing organizational readiness for change. Resources should be adequately
assessed and evaluated from the outset and built into the development of the innovation pro-
posal. During the UDI process, it is also vital to assess resource in real time to relocate resource
if challenges arise (Pratesi et al., 2013).
5.2 Challenges of Creating Synergy between Users and Technologies
Included studies suggested that users’ relationship with technology in UDI, is heterogeneous,
depending on users’ characteristics, the types and features of technology, and availability of
technology facilitator. Technologies can be the outcome of a UDI process or a supportive tool
for innovation. No matter of which is the case, users’ perception towards specific technological
options more or less affect their motivation, the kind of, and the degree of participation in UDI
process. Such finding rises our concern about the issue upon how to create synergy between
users and technologies across UDI process. Firstly, although it is essential that user insights
and knowledge generated find their way into the development process, it is still challenging
to adequate translation and transformation of users information and requirements into more
technical requirements (and vice versa)(Joseph et al., 2010; Durose et al., 2017).
For example, when health professionals need to translate their medical knowledge into formal
computer-interpretable formats, they might face the challenge of switching their role from clini-
cian to a knowledge engineer, because such process requires learning the relevant tools, in addition
to time resources. Further, the laws and regulations in health and social care often restrict the
members of the care providing organizations in how they can use information and system (Lind-
gren, 2013). Some selected studies suggest using deliberate methods to support experiencing the
potentiality of technologies by participated end-users for knowledge translation and transforma-
tion. For instance, through the live lab approach, end-users were free to experience different
technical systems in a real-life setting, and they eventually turned to knowledge engineer who
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 157

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
can develop new ideas via the support of technologies they have tried (Gasco, 2017). Likewise,
by inviting end-users to join different technology clubs where workshops, seminars, and product
presentations were frequently organized across a period, those people were encouraged to try the
different technological options, to learn relevant knowledge, and eventually they could contribute
their ideas for further technology improvement (Pihlainen et al., 2017).
Although above studies to some extent presented and discussed methodologies for creating syn-
ergy between users and technologies in UDI process, there are still problems remain. First, those
methods are primarily on certain technologies and new applications, instead on the way users in-
teract with different access network in their natural context. In addition, we lack enough evidence
to see if those methods in UDI can fulfill the promoting objectives of projects and further lead
to radical and sustainable innovation in public health and social care. As we can see, indeed,
the living lab approach, interactive workshops and other methods, might support knowledge
transfer and empower end-users to innovate. However, there is no study conclude those methods
are better than other methods or innovation system in leading a better innovation outcome and
more sustainable impact.
6 Conclusion
In summary, through a systematic review of existing evidence, we illustrated how current UDI
in public health and social sectors operates with distinct characteristics in respect of its strategic
innovation process and user-oriented, empower-based objectives. Both end- and intermediate
users in directly and indirectly engage in different innovation stages and fulfill their roles for
exploration, experiments, test and innovation. Exploring users with ethnographic tools and par-
ticipatory methodologies are very popular at WHAT stages to in-depth understand what problem
user face and therefore what solution can be offered. During HOW phases, however, living lab
and lead users approaches often operate to encourage users as innovator or co-innovator for the
innovation conceptualization and prototype, while various user testing tools are available for
helping users to experience and evaluate designed prototypes. However, users as completely
innovators seem less popular than users as co-innovators who only contributed partially to in-
novation teams. It may because the former is a relatively new field regarding its application
in innovation processes and organizations in public health and social still face challenging to
learn how to achieve the best results when the users have the most powers in the innovation
process.
6.1 Further Implementation
With an understanding of how diverse the role of users is, it is likely that users will participate
in UDI in many different ways. However, in some cases in practice, there is still a discrepancy
between theory and practice. We discussed two crucial challenges among UDI in public health
and social care: the need for continuous engagement with users cross each step of the innovation
process, and the demand for creating mechanisms to integrate users and technologies to support
a better innovation outcome. For addressing those challenges, we have to continuously explore
and test new interdisciplinary methodologies or tools to support user participation in a natu-
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 158

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
ral setting, to encourage a smooth interaction and communication, and to balance the power
relationship between human and technologies. Further, we have to continuously and critically
evaluate UDI outcome in respect of their promising objectives with a large scale of research evi-
dence. By focusing on how UDI took place in public health and social care is not enough; we need
to further systematically evaluate if UDI can achieve better results than other innovation types,
can help to solve the confronted challenges in the field, and eventually change organizational
culture towards innovation or the innovation system in policy network.
6.2 Limitation and Further Research
Since UDI is a somewhat new concept in public health and social care, which leads to this
article become one of the first systematic reviews on this topic in which has a limited number
of empirical evidence available. It decides the nature of this review to be more explorative and
be descriptive in general and lack of an in-depth critical reflection due to constrained materials.
Further, although we developed a research protocol in advance and used PRISMAS to guide
the whole review process, there might be still bias in the selection of publications included and
inaccuracy in data extraction. If we want to fully understand of UDI in the public health and
social care, it is necessary to develop a theoretical and methodological framework, which is based
on the concept of innovation but re-defined in a way that takes into account that the public
health and social care is operating on other conditions than private companies. At the same
time, the framework must take into account that users of present public services are a variety
of different actors, who play different roles and participate in the process of innovation in very
different ways. Also, more evidence needed to conduct a proper evaluation of the effect of UDI
in the studied sector, as well as an assessment towards various methods applied to support user
participation.
7 Reference
Adedoyin, A. C. A. (2016). Deploying virtual communities of practice as a digital tool in social
work: a rapid review and critique of the literature. Social Work Education, 35 (3), 357-370.
doi:10.1080/02615479.2016.1154660
Albury, D. (2005). Fostering innovation in public services. Public money and management,
25 (1), 51-56.
All, A., Van Looy, J., & Castellar, E. P. N. (2013). An Evaluation of the Added Value of Co-
Design in the Development of an Educational Game for Road Safety. International Journal of
Game - Based Learning, 3 (1), 1.
Andersen, S. T. (2013). User-centered Innovation in Psychiatric Health Care By Using Mobile
Technology: The role of the installed base. (Doctor), University of Oslo, Oslo.
Andersen, S. T., & Jansen, A. (2012). Installed base as a facilitator for user-driven innovation:
how can user innovation challenge existing institutional barriers? Int J Telemed Appl, 2012,
673731.
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 159

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
Angelini, L., Carrino, S., Khaled, O. A., Riva-Mossman, S., & Mugellini, E. (2016). Senior living
lab: An ecological approach to foster social innovation in an ageing society. Future Internet,
8(4).
Baker, S., Warburton, J., Hodgkin, S., & Pascal, J. (2014). Reimagining the Relationship
between Social Work and Information Communication Technology in the Network Society. Aus-
tralian Social Work, 67(4), 467-478.
Barlott, T., Adams, K., & Cook, A. (2016). Increasing participation in the information soci-
ety by people with disabilities and their families in lower-income countries using mainstream
technologies. Universal Access in the Information Society, 15 (2), 189-198.
Bearman, M., & Dawson, P. (2013). Qualitative synthesis and systematic review in health
professions education. Medical Education, 47(3), 252-260.
Beaudry, A., & Pinsonneault, A. (2010). The other side of acceptance: studying the direct and
indirect effects of emotions on information technology use. MIS Quarterly, 689-710.
Bevan, N. (1995). Measuring usability as quality of use. Software Quality Journal, 4 (2), 115-
130.
Bhatt, G. D. (2001). Knowledge management in organizations: examining the interaction
between technologies, techniques, and people. Journal of Knowledge Management, 5 (1), 68-
75.
Bogers, M., Afuah, A., & Bastian, B. (2010). Users as innovators: a review, critique, and future
research directions. Journal of Management, 36 (4), 857-875.
Booth, A., Sutton, A., & Papaioannou, D. (2016). Systematic Approaches to a Successful Liter-
ature Review: SAGE.
Bugge, M., Coenen, L., Marques, P., & Morgan, K. (2017). Governing system innovation:
assisted living experiments in the UK and Norway. European Planning Studies, 25 (12), 2138-
2156.
Burgelman, R. A., Maidique, M. A., & Wheelwright, S. C. (1996). Strategic management of
technology and innovation (Vol. 2): Irwin Chicago, IL.
Calvo-Lerma, J., Martinez-Jimenez, C. P., Lázaro-Ramos, J. P., Andrés, A., Crespo-Escobar, P.,
Stav, E., . . . Ribes-Koninckx, C. (2017). Innovative approach for self-management and social
welfare of children with cystic fibrosis in Europe: Development, validation and implementation
of an mHealth tool (MyCyFAPP). Bmj Open, 7(3).
Chan, C., & Holosko, M. J. (2016). A Review of Information and Communication Technology
Enhanced Social Work Interventions. Research on Social Work Practice, 26(1), 88-100.
Christiansson, P., Sørensen, K. B., Rødtness, M., Abrahamsen, M., Riemnann, L. O., & Alsdorf,
M. (2008). User driven innovation in the building process. Tsinghua Science & Technology, 13,
248-254.
Cohen, E. L. (2014). What makes good games go viral? The role of technology use, efficacy,
emotion and enjoyment in players’ decision to share a prosocial digital game. Computers in
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 160

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
Human Behavior, 33, 321-329.
Couture, M., Sasseville, M., & Gascon, V. (2016). Facilitators and Barriers to Implementing
Transitional Care Managers Within a Public Health Care System. Journal of Gerontological
Social Work, 59 (4), 364-377.
David, W., Gabriele, V., & Heinz-Theo, W. (2014). The impact of information technology on
knowledge creation: An affordance approach to social media. Journal of Enterprise Information
Management, 27 (1), 31-44.
Davis, F. D. (1985). A technology acceptance model for empirically testing new end-user infor-
mation systems: Theory and results. Massachusetts Institute of Technology.
de Souza, S., Galloway, J., Simpson, C., Chura, R., Dobson, J., Gullick, N. J., . . . Lempp,
H. (2017). Patient involvement in rheumatology outpatient service design and delivery: a case
study. Health Expectations, 20 (3), 508-518.
Dekelver, J., Van Den Bosch, W., & Engelen, J. (2011). Supporting social inclusion of youth
at risk using social software: Impact, sustainability and evaluation, one year after pilot testing.
Housing, Care and Support, 14(2), 61-66.
Devlin, A. M., McGee-Lennon, M., O’Donnell, C. A., Bouamrane, M. M., Agbakoba, R.,
O’Connor, S., . . . Dallas Evaluation, T. (2016). Delivering digital health and well-being at
scale: lessons learned during the implementation of the dallas program in the United Kingdom.
Journal of the American Medical Informatics Association, 23 (1), 48-59.
Dillon, A. (2002). Beyond usability: process, outcome and affect in human-computer interactions.
Canadian Journal of Library and Information Science.
Dithmer, M., Rasmussen, J. O., Grönvall, E., Spindler, H., Hansen, J., Nielsen, G., . . . Dinesen,
B. (2016). "The Heart Game": Using Gamification as Part of a Telerehabilitation Program for
Heart Patients. Games for Health Journal, 5 (1), 27-33.
Durose, C., Needham, C., Mangan, C., & Rees, J. (2017). Generating ’good enough’ evidence
for co-production. Evidence & Policy, 13 (1), 135-151.
Elia, M., Normand, C., Norman, K., & Laviano, A. (2016). A systematic review of the cost and
cost effectiveness of using standard oral nutritional supplements in the hospital setting. Clinical
Nutrition, 35 (2), 370-380.
Gasco, M. (2017). Living labs: Implementing open innovation in the public sector. Government
Information Quarterly, 34 (1), 90-98.
Granier, B., & Kudo, H. (2016). How are citizens involved in smart cities? Analysing citizen
participation in Japanese "Smart Communities". Information Polity, 21 (1), 61.
Habicht, H., & Thallmaier, S. R. (2017). Understanding the customer value of co-designing indi-
vidualised products. International Journal of Technology Management, 73(1-3), 114-131.
Hennala, L., & Melkas, H. (2016). Understanding Users’ Collective Voice in Public Service
Innovation. Knowledge and Process Management, 23 (1), 62-72.
Hill, A., & Shaw, I. (2011). Social Work & ICT. London: United Kingdom, London: SAGE
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 161

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
Publications Ltd.
Huber, J. M., Schneider, H., Pfister, V., & Steiner, B. (2017). Use and Development of New
Technologies in Public Welfare Services: A User-Centred Approach Using Step by Step Commu-
nication for Problem Solving. In I. Kollak (Ed.), Safe at Home with Assistive Technology (pp.
109-135). Cham: Springer International Publishing.
Humphreys, A. (2015). Really, really rapid prototyping: Flash builds and user-driven innovation
at JSTOR Labs. Information Services and Use, 35 (1-2), 71-75.
Hussain, A. F. (2017). The acceptance of corporate wiki use for knowledge diffusion purposes.
Aslib Journal of Information Management, 69 (6), 642-659.
Hvenegaard Rasmussen, C. (2016). The participatory public library: the Nordic experience.
New Library World, 117 (9-10), 546-556.
Jenny, D., & Rod, M. (2002). Examining the link between knowledge management practices and
types of innovation. Journal of Intellectual Capital, 3(3), 210-222.
Joseph, W., Deryckere, T., & Martens, L. (2010). User-driven innovation? Challenges of user
involvement in future technology analysis (Vol. 37).
Kallio, K., Lappalainen, I., & Tammela, K. (2013). Co-innovation in public services: Planning
or experimenting with users? Innovation Journal, 18 (3).
Keys, M. (2016). Evaluating the Impact on Practice of Online Child Protection Education at
Master’s level. Social Work Education, 35 (4), 444-456.
Kim, Y. H., Kim, D. J., & Wachter, K. (2013). A study of mobile user engagement (MoEN):
Engagement motivations, perceived value, satisfaction, and continued engagement intention.
Decision Support Systems, 56, 361-370.
Konsti-Laakso, S. (2017). Stolen snow shovels and good ideas: The search for and generation
of local knowledge in the social media community. Government Information Quarterly, 34 (1),
134-139.
Lassen, A. J., Bonnelycke, J., & Otto, L. (2015). Innovating for ’active ageing’ in a public-private
innovation partnership: Creating doable problems and alignment. Technological Forecasting and
Social Change, 93, 10-18.
Lewis, S., Pea, R., & Rosen, J. (2010). Beyond participation to co-creation of meaning: mobile so-
cial media in generative learning communities. Social Science Information, 49(3), 351-369.
Lindgren, H. (2013). Sociotechnical systems as innovation systems in the medical and health
domain. Studies in health technology and informatics, 194, 35-40.
Lister, C., Payne, H., Hanson, C. L., Barnes, M. D., Davis, S. F., & Manwaring, T. (2017). The
Public Health Innovation Model: Merging Private Sector Processes with Public Health Strengths.
Frontiers in Public Health, 5.
Lopez-Hartmann, M., Wens, J., Verhoeven, V., & Remmen, R. (2012). The effect of caregiver
support interventions for informal caregivers of community-dwelling frail elderly: A systematic
review. International Journal of Integrated Care, 12(JULY-SEPTEMBER 20).
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 162

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
Luck, J., York, L. S., Bowman, C., Gale, R. C., Smith, N., & Asch, S. M. (2015). Implementing a
User-Driven Online Quality Improvement Toolkit for Cancer Care. Journal of Oncology Practice,
11 (3), E421-E427.
Luckock, B., Barlow, J., & Brown, C. (2017). Developing innovative models of practice at the
interface between the NHS and child and family social work where children living at home are
at risk of abuse and neglect: a scoping review. Child and Family Social Work, 22, 62-69.
Lundvall, B.-Å., & Borrás, S. (2005). Science, technology, and innovation policy Oxford handbook
of innovation (pp. 599-631): Oxford University Press.
Marina, d. P. (2007). The role of knowledge management in innovation. Journal of Knowledge
Management, 11 (4), 20-29.
Marsan, J., & Paré, G. (2013). Antecedents of open source software adoption in health care
organizations: A qualitative survey of experts in Canada. International Journal of Medical
Informatics, 82 (8), 731-741.
Moen, E. (2012). Forsknings-og innovasjonspolitikk: Norske utfordringer i lys av internasjonale
endringer.
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Group, P. (2009). Preferred reporting
items for systematic reviews and meta-analyses: the PRISMA statement. PLoS med, 6 (7),
e1000097.
O’Brien, H. L., & Toms, E. G. (2008). What is user engagement? A conceptual framework for
defining user engagement with technology. Journal of the Association for Information Science
and Technology, 59(6), 938-955.
OECD (2005). Oslo Manual: Guidelines for Collecting and Interpreting Innovation Data, 3rd
Edition, The Measurement of Scientific and Technological Activities, OECD Publishing, Paris,
https:/ https://doi.org/10.1787/9789264013100-en/.
Paiva, T., Domingues, C., & de Andrade, L. P. (2016). Innovation and knowledge transference in
a cluster user-driven innovation perspective - the Inovcluster experience. International Journal
of Food Studies, 5(1), 54-60.
Palumbo, R. (2016). Contextualizing co-production of health care: a systematic literature review.
The International Journal of Public Sector Management, 29(1), 72-90.
Pang, M. S., Lee, G., & DeLone, W. H. (2014). In public sector organisations: a public-value
management perspective. Journal of Information Technology, 29(3), 187-205.
Park, H. Y., Cho, I.-H., Jung, S., & Main, D. (2015). Information and communication technol-
ogy and user knowledge-driven innovation in services. Cogent Business & Management, 2 (1),
1078869.
Pässilä, A., Oikarinen, T., Parjanen, S., & Harmaakorpi, V. (2013). Interpretative dimension of
user-driven service innovation: Forum Theatre in facilitating renewal in Finnish public health
care. Baltic Journal of Management, 8 (2), 166-182.
Pestoff, V. (2012). Co-production and Third Sector Social Services in Europe: Some Concepts
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 163

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
and Evidence. Voluntas, 23 (4), 1102-1118.
Phills, J. A., Deiglmeier, K., & Miller, D. T. (2008). Rediscovering social innovation. Stanford
Social Innovation Review, 6(4), 34-43.
Pihlainen, K., Suero Montero, C., & Kärnä, E. (2017). Fostering parental co-development of
technology for children with special needs informal learning activities. International Journal of
Child-Computer Interaction, 11, 19-27.
Pratesi, A., Sixsmith, J., & Woolrych, R. (2013). Technological and methodological innovation:
Working "with" and "for" older people to develop a smart activity monitoring system. Innovation
Journal, 18 (3).
Pugh, R. (2014). Regional innovation policy and economic development: the case of Wales.
Cardiff University.
Puig-Pey, A., Bolea, Y., Grau, A., & Casanovas, J. (2017). Public entities driven robotic inno-
vation in urban areas. Robotics and Autonomous Systems, 92, 162-172.
Purkayastha, S., Price, A., Biswas, R., Jai Ganesh, A. U., & Otero, P. (2015). From Dyadic
Ties to Information Infrastructures: Care-Coordination between Patients, Providers, Students
and Researchers. Contribution of the Health Informatics Education Working Group. Yearbook
of medical informatics, 10(1), 68-74.
Røtnes, R., & Staalesen, D. P. (2010). New methods for user driven innovation in the health
care sector: Nordic Council of Ministers.
Rubinelli, S., Collm, A., Glässel, A., Diesner, F., Kinast, J., Stucki, G., & Brach, M. (2013). De-
signing interactivity on consumer health websites: PARAFORUM for spinal cord injury. Patient
Education and Counseling, 93(3), 459-463.
Savory, C., & Fortune, J. (2015). From translational research to open technology innovation
systems. Journal of Health Organization and Management, 29(2), 200-220.
Schumpeter, J. A. (1934). Imperfect Competition. American Economic Review, 24 (1), 21-
32.
Schumpeter, J. A. (1934). The theory of economic development: An inquiry into profits, capital,
credit, interest, and the business cycle (Vol. 55): Transaction publishers.
Scupola, A., & Zanfei, A. (2016). Governance and innovation in public sector services: The case
of the digital library. Government Information Quarterly, 33 (2), 237-249.
Seys, D., Scott, S., Wu, A., Van Gerven, E., Vleugels, A., Euwema, M., Vanhaecht, K. (2013).
Supporting involved health care professionals (second victims) following an adverse health event:
A literature review. International Journal of Nursing Studies, 50 (5), 678-687.
Sher, P. J., & Lee, V. C. (2004). Information technology as a facilitator for enhancing dynamic
capabilities through knowledge management. Information & Management, 41 (8), 933-945.
Sørensen, E., & Torfing, J. (2012). Introduction: Collaborative innovation in the public sector.
Innovation Journal, 17 (1), 1-14.
Sugarhood, P., Wherton, J., Procter, R., Hinder, S., & Greenhalgh, T. (2014). Technology
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 164

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
as system innovation: A key informant interview study of the application of the diffusion of
innovation model to telecare. Disability and Rehabilitation: Assistive Technology, 9 (1), 79-
87.
Szkuta, K., Pizzicannella, R., & Osimo, D. (2014). Collaborative approaches to public sector
innovation: A scoping study. Telecommunications Policy, 38 (5-6), 558-567.
Szymańska, E. (2017). User-Driven Innovation-the concept and research results. Procedia Engi-
neering, 182, 694-700.
Thomas, J., & Harden, A. (2008). Methods for the thematic synthesis of qualitative research in
systematic reviews. Bmc Medical Research Methodology, 8 (1), 45.
Timmings, C., Khan, S., Moore, J. E., Marquez, C., Pyka, K., & Straus, S. E. (2016). Ready, Set,
Change! Development and usability testing of an online readiness for change decision support
tool for healthcare organizations. Bmc Medical Informatics and Decision Making, 16 (1).
Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a Methodology for Developing Evidence-
Informed Management Knowledge by Means of Systematic Review. British Journal of Manage-
ment, 14 (3), 207-222.
Trott, P., Duin, P. V. D., & Hartmann, D. (2013). Users as innovators? Exploring the limitations
of user-driven innovation. Prometheus, 31 (2), 125-138.
Tuomi, I. (2002). Networks of innovation: Oxford University Press Oxford.
Tursunbayeva, A., Franco, M., & Pagliari, C. (2017). Use of social media for e-Government in
the public health sector: A systematic review of published studies. Government Information
Quarterly.
Valaitis, R. K., Carter, N., Lam, A., Nicholl, J., Feather, J., & Cleghorn, L. (2017). Implementa-
tion and maintenance of patient navigation programs linking primary care with community-based
health and social services: a scoping literature review. Bmc Health Services Research, 17 (1),
1-14.
Venkatesh, V. (2000). Determinants of perceived ease of use: Integrating control, intrinsic
motivation, and emotion into the technology acceptance model. Information Systems Research,
11 (4), 342-365.
Venkatesh, V., & Davis, F. D. (2000). A theoretical extension of the technology acceptance
model: Four longitudinal field studies. Management Science, 46(2), 186-204.
Von Hippel, E. (2009). Democratizing innovation: the evolving phenomenon of user innovation.
International Journal of Innovation Science, 1(1), 29-40.
Wang, Y., Yu, S., & Xu, T. (2017). A user requirement driven framework for collaborative design
knowledge management. Advanced Engineering Informatics, 33, 16-28.
Waugh, A., Austin, A., Manthorpe, J., Fox, C., Stephens, B., Robinson, L., & Iliffe, S. (2013).
Designing a complex intervention for dementia case management in primary care. Bmc Family
Practice, 14.
Wihlman, T., Hoppe, M., Wihlman, U., & Sandmark, H. (2014). Employee-driven Innovation
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 165

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
in Welfare Services. Nordic Journal of Working Life Studies, 4(2), 159.
Wise, E., & Høgenhaven, C. (2008). User-Driven Innovation-Context and Cases in the Nordic
Region: Nordisk ministerråd.
Appendix A: Quality Assessment Criteria
This assessment is developed based on the methodology quality assessment criteria (CASP,
2017)
Category
(Questions)
Considerations Yes Not
clear
No
Is there a clear
statement of the
aims of the
research?
What is the goal of the research?
Why it is thought important?
Its relevance.
S1-21
Is a research
methodology
approriate?
If the research seeks to interpret or
illuminate the action and/or subjective
experiences of research participants?
Is the right methodology for addressing the
research goal?
S1-21
Is the research
design approriate
to address the
aims of the
research?
If the researcher has justified the research
design?
S1-21
Is the recruitment
strategy
approriate to the
aims of the
research?
If the researcher has explained how the
participants were selected?
If they explained why the participants they
selected were the most appropriate to
provide access to the type of knowledge
sought by the study?
If there are any discussions around
recruitment (e.g. why some people chose not
to take part)?
S1-21
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 166

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
Category
(Questions)
Considerations Yes Not
clear
No
Is the data
collected in a way
that addressed
the research
issue?
If the setting for data collection was
justified?
If it is clear how data were collected (e.g.
focus group, semi-structured interview etc.)?
If the researcher has justified the methods
chosen?
If the researcher has made the methods
explicit (e.g. for interview method, is there
an indication of how interviews were
conducted, or did they use a topic guide)?
If methods were modified during the study.
If so, has the researcher explained how and
why?
If the form of data is clear (e.g. tape
recordings, video material, notes etc)
If the researcher has discussed saturation of
data?
S1,2,
4-10,
12-15,
18-21
S3, 11,
17
Has the
relationship
between
researcher and
participants been
adequately
considered?
If the researcher critically examined their
own role, potential bias and influence during
(a) Formulation of the research questions
(b) Data collection, including sample
recruitment and choice of location?
How the researcher responded to events
during the study and whether they
considered the implications of any changes in
the research design.
S1, 2, 6,
7, 11,
12, 14,
15, 21
S5, 20 S3, 4, 7,
8, 9, 10,
13, 16,
17, 18,
19
Have ethical
issues been taken
into
consideration?
If there are sufficient details of how the
research was explained to participants for
the reader to assess whether ethical
standards were maintained?
If the researcher has discussed issues raised
by the study (e.g. issues around informed
consent or confidentiality or how they have
handled the effects of the study on the
participants during
and after the study)?
If approval has been sought from the ethics
committee?
S5, 7,
11, 12,
20, 21
S1, 2, 6,
14, 15
S3, 4, 8,
9, 10, 13,
16, 17,
18, 19
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 167

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
Category
(Questions)
Considerations Yes Not
clear
No
Is the data
analysis
sufficiently
rigorous
If there is an in-depth description of the
analysis process?
If thematic analysis is used. If so, is it clear
how the categories/themes were derived from
the data?
Whether the researcher explains how the
data presented were selected from the
original sample to demonstrate the analysis
process?
If sufficient data are presented to support the
findings?
To what extent contradictory data are taken
into account?
Whether the researcher critically examined
their own role, potential bias and influence
during analysis and selection of data for
presentation?
S1-21 S3, 8,
11, 18
Is there is a clear
statement of
findings?
If the findings are explicit?
If there is adequate discussion of the
evidence both for and against the researchers
arguments?
If the research has discussed the credibility
of their findings?
If the findings are discussed in relation to the
original research question
S1-21
How valuable is
the research?
If the researcher discusses the contribution,
the study makes to existing knowledge or
understanding?
If they identify new areas, where research is
necessary?
If the researchers have discussed whether or
how the findings can be transferred to other
populations or considered other ways the
research may be used?
S1-21
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 168

Journal of Innovation Management
JIM 6, 2 (2018) 138-169
Zhu, Thomassen
Biographies
Hong Zhu. Hong Zhu is a PhD research fellow at Faculty of Child Welfare and Social Work
at the Arctic University of Norway. Her main area of interest is the study of innovation,
technology, and their collective implication on today?s social work and child service. She ded-
icated to find out how social work professionals can leverage technology advance in digitalised
society to offer a better service to users.
Synnøve Thomassen. Synnøve Thomassen Andersen, PhD, The Arctic University of Nor-
way, campus Alta. Thomassen Andersen holds a PhD in Computer Science, but also a Bachelor
degree in child welfare. She has since 2001 worked at the Department of Child Welfare and
Social Work. Since 2015, she has been the Vice Dean of research and education at the Faculty.
Thomassen Andersen’s scientific publications cover a number of areas, focusing on information
infrastructures, user-centered processes, digitization, flexibility and studies of innovation and
welfare technology related to health and social work.
ISSN 2183-0606
http://www.open-jim.org
http://creativecommons.org/licenses/by/3.0 169