How To Transfer Tacit Knowledge for Living Lab Practice -Consideration on Tacit Knowledge Representations

Abstract

Despite strong attention and wide ranges of practice, living labs have hardly scaled up for a decade as expected. For living labs as one of the innovative methods for social challenges in cities with high complexity and uncertainty, one of the critical challenges is in knowledge transfer of its practice. Practical tacit knowledge for living lab is often embedded within process, organization and operation, and hardly externalized. The inexperienced practitioners face a lot of uncertainty in implementation of living lab without clues where to tackle. Aiming at promoting the living lab practice widely, this paper investigates tacit knowledge externalization with three different representations. The analysis and comparison of external representations indicate a strong compatibility pattern between representation styles and practitioners' maturity level on the relevant field. In the living lab practice, how to convey tacti knowledge should be considered carefully, depending on practitioners' maturity level.

Full text

How To Transfer Tacit Knowledge for Living Lab Practice
- Consideration on Tacit Knowledge Representations
Mika Yasuoka
Roskilde University
mikaj@ruc.dk
Abstract
Despite strong attention and wide ranges of
practice, living labs have hardly scaled up for a
decade as expected. For living labs as one of the
innovative methods for social challenges in cities with
high complexity and uncertainty, one of the critical
challenges is in knowledge transfer of its practice.
Practical tacit knowledge for living lab is often
embedded within process, organization and operation,
and hardly externalized. The inexperienced
practitioners face a lot of uncertainty in
implementation of living lab without clues where to
tackle. Aiming at promoting the living lab practice
widely, this paper investigates tacit knowledge
externalization with three different representations.
The analysis and comparison of external
representations indicate a strong compatibility pattern
between representation styles and practitioners’
maturity level on the relevant field. In the living lab
practice, how to convey tacti knowledge should be
considered carefully, depending on practitioners’
maturity level.
1. Introduction
Living lab have been attracting global attentions as
one of the innovative methods for social challenges
with high complexity and uncertainty [1], [2]. In 2006,
European Commission supports living labs as one of
the common European innovation systems, and the
European Network of Living Labs (ENoLL) was
established. This EU Living Lab framework provided
an impact in promotion of living lab in EU countries,
and vast number of related projects were initiated and
conducted, and success stories were reported widely
[3], [4]. Since the launch of the initiative, wider
interests were expressed also outside EU borders [5].
However, majority of practitioners and projects were
still in EU and living lab practices outside Europe
seems rather slow and a sign of scaling up is hardly
seen. It is hard to say that knowledge about living lab
practice as an innovation method is transferred, yet.
As factors that hinders scaling up, a lack of general
consensus in a framework, a process and a definition
has been pointed out [2], [6]. The lack of definition, for
example, makes practitioners and decision makers feel
living lab as an incomprehensible method in the
immature stage. Towards this challenge, last couple of
years, a structural and systematic analysis of the living
labs literature have increased and discussed widely to
consolidate its definition, process and framework [2],
[7], aiming at supporting descriptive knowledge
acquisition [8].
Supposing that the consensus was made, there is
still another challenge to implement the living lab
method in practice when there is no experienced
practitioner in a project. Despite the fact that living
labs have been around for nearly a decade, in terms of
practical field knowledge, the current living lab
researches have not provided a proper knowledge
guidance to practitioners.
For example, to conduct a living lab, it is important
to utilize not only descriptive knowledge such as
frameworks and processes but also a wide variety of
tacit knowledge to support long term co-creation with
varied stakeholders. They are for example such
knowledge as how to design co-creation environment
for living labs, how to involve which stakeholders
when, how to conduct co-creation to reach results, and
how to support stakeholders’ motivation for
sustainable commitments. Such down to the earth
knowledge about how things are practically done is
called "procedural knowledge", also known as
imperative knowledge or tacit knowledge [9], which is
acquired through practice and exercised in the
performance of some tasks. The tacit knowledge is
hardly mentioned in the living lab reports because it is
often buried in experience and practice. For that reason,
unexperienced individuals and organizations who
intends to conduct a living lab for the first time face a
lot of practical problems even if they have prepared by
acknowledging with many frameworks and cases.
Experienced practitioners have accumulated
empirical knowledge to achieve successful living labs.
Through the process of internalization, the knowledge

become common sense to them, and disappear behind
the scene. Therefore, the tacit knowledge of living labs
has never consciously been externalized, or hardly
targeted for research. The living lab method has wider
applicability, flexibility and interpretability as the
emerging method, which make it harder for
practitioners to externalize the knowledge. However, in
order to support Living Lab practice, it is an urgent
matter to elicit consciously and share tacit knowledge
for living lab practice through externalization process
with wider practitioners.
In this article, we call a collection of knowledge for
long-term co-creation acquired through experiencing
living lab as "tacit knowledge for living lab practice".
In order to externalize and share the tacit knowledge
with wider living lab practitioners, and to contribute to
scale up current living lab practice, we exercise
external representations of tacit knowledge for living
lab practice. More precisely speaking, through
conscious process, we fist extract, then formalize, and
finally visualize in knowledge presentation formats for
conveying the living lab practical essence easier and
wider.
This paper is organized as follows. First, some
related research concerning about living labs and
knowledge presentation are introduced. Among the
various knowledge presentations, this article deals with
three knowledge externalization methods: multimedia,
pattern language, and a question-based representation.
In Section 3, our approach to collect and externalize
practitioners’ tacit knowledge for living lab practice is
explained, and evaluation, analysis and discussion
about our external representations are conducted from
"tacit knowledge for living lab practice" perspective.
Finally, in Section 7, the conclusion is drawn.
2. Related Research
This section describes related works of this
research. First of all, an overview of the current living
lab research, and then several ways to codify
potentially reusable tacit knowledge of the experienced
living lab practitioners and these characteristics are
reviewed. As knowledge externalization approaches,
this article focuses on a use of multimedia, pattern
language, and a question-based method.
2.1. Living lab characteristics as an innovation
method
Living labs as a design innovation method involves
stakeholders in their living contexts, with a long-term
perspective, and seeks solutions through co-creation
for the complexed societal problems (for example.
[10], [11]). Since no single professional has coherent
or comprehensive knowledge to solve [1], co-creation
is of critical importance. And since our modern society
is highly complex and a full of uncertainty,
stakeholders’ living context becomes an important
field of experiments, where iterative trials and
developments are conducted.
The term, living lab, began to be used around 1998
[12], and living labs gradually started to attract
attentions widely and many projects were conducted in
Europe. Because of its wide applicability and
interpretability, living labs were accepted and applied
in varied domains. Leminen [13] analyzed the benefits
of deploying living labs in three applied fields,
innovation, living context and business chance, while
Førlstad [12] investigated another fields such as R&D,
living context and research context. Living labs
nowadays have a wide range of applied fields, and
various definitions as well.
The definition of living lab is not settled, yet.
ENoLL defines living labs as “user-centered, open
innovation ecosystems based on systematic user co-
creation approach, integrating research and innovation
processes in real life communities and settings”. Living
lab practitioner and researcher, Bergvall-Kåreborn [14]
defines living lab as “a user-centric innovation milieu
built on every-day practice and research, with and
approach that facilities user influence in open and
distributed innovation processes engaging all relevant
partners in real-life contexts, aiming to create
sustainable values”.
Although it has been criticized that the mainstream
of current Living lab research has been practice-based,
researchers on theoretical frameworks and analysis
have been developed. Schuurman et.al. [2] analyzed
scientific papers collected from databases of Google
Scholar and Web of Science, and argued the need of a
solid foundation of the field based on a concrete
theoretical framework such as innovation theory.
Dell'Ella [7], similarly, argued the importance of
clarification of living lab methods and conciseness on
its definition to position living lab in a scientific
context. The increased number of papers reviewing the
literature and cases have pointed out important
common features. As also shown in the above living
lab definitions, currently practitioners and researchers
agreed, to some extent, that Living lab has
characteristics such as co-creation, real-life settings,
awareness of user [7], and technological
implementation [15]. Although an ultimate definition
of living lab has not been made yet [2], it has been
widely agreed that a theoretical framework is important
for the advancement of the field. Last couple of years,
the core of living lab method has gradually condensed.

However, living lab as practice-based approach, there
are still many challenges to conduct living labs as its
core tacit knowledge is not easily externalized in
theories, definitions, and processes. Case descriptions
have been strongly influenced with regional socio-
cultural characteristics, which is difficult to synthesize
as meta-knowledge, or depict its tacit knowledge as
external representation to other practitioners.
Consequently, living lab method is hardly scaling up
beyond local practice.
2.2. Knowledge Representation
Many externalization methods have been applied
for knowledge transfer. Boose [8] showed cognitive
maps, decision tables, rules, scripts, etc. as a method of
presenting expertise, and as knowledge types,
explained causal knowledge, terminology, constraints,
example cases, procedures relations, facts, uncertain
ties, etc. Among all, the most used method for
knowledge representation is probably the declarative
method, which is the method of transmitting the rules
of thumb by being embedded in documents, expert
dictionaries, manuals, demonstrations, simulations,
decision support system, etc. Unlike programming or
crafts, which is externalized and visualized as tangible
knowledge outputs, highly abstract knowledge such as
living lab practice including co-creation, process
management, and team building, are difficult to
externalize. It is obvious that the choice of knowledge
representation of such abstract knowledge is a key
component of knowledge transfer.
Among various externalization methods, three
particular methods have been selected to externalize
knowledge with a high affinity for abstraction. They
are multimedia, pattern language and a question-based
representation. Three methods are chosen as the
authors have in-depth experiences in multimedia and
pattern languages in different contexts, and the
question-based method is recommended by experts.
2.2.1. Multimedia. With the progress of engineering
and information technology, the use of multimedia is
experimented as ways of declaration. Multimedia as a
declarative method, can hold not only written text, but
also captured images and videos that activate the
human five senses such as sight, tactile and hearing.
Multimedia representation is particularly effective in
externalizing tacit embodied knowledge which
typically craftsmen and athletes obtain in a course of
experience accumulation. For craft technique
externalization [9], researchers have demonstrated
transferability of the tacit technique by digital means
[16]–[19], For example, Wood and her colleagues [17]
recorded and transmitted the skilled knowledge of
wood carpenters using multimedia, based on the
concept of bridges. Aytekin [19] investigated a digital
platform as means of facilitating knowledge transfer of
Turkish local crafts. For capturing and passing on
distributed tacit knowledge of experts, Yasuoka [20]
utilized a combined forms of multimedia, consisting of
documents, pictures, movies and situ-practice. Crafts
techniques and tacit knowledge behind the techniques
were classified into concepts, element and keywords,
then condensed and externalized with the four media
types [20]. Knowledge management of craft technique
with multimedia format has a huge potential to support
endangered craft skills and knowledge.
2.2.2. Pattern language. Pattern language is a kind of
declarative method and is used as intermediary
between original to potential knowledge users. Pattern
language is a description of common patterns
embedded in good design and practice in a specific
domain. Pattern language was originally proposed by
Christopher Alexander [21] as a tacit knowledge
externalization technique to support non-specialists in
architectural and urban design domain [22], [23].
Pattern language by Alexander contains many common
patterns that good design in architecture and urban
development should have. Each pattern is described by
an element such as a pattern ID, name, photographs or
illustration showing the core messages. The details of
the pattern are described together with use contexts and
solutions, showing "In which context, which method
should be taken". Pattern language advocated in the
field of architecture and urban design has been widely
deployed and utilized in various engineering and
information system fields such as software
development [24], scrum process [25], interaction
design [26]. Although similar knowledge can be
described simply in texts, the recipient of pattern
language benefits from the enriched descriptive
representation combined with various forms and
visuals that helps approximating the relationship of
receivers’ existing knowledge.
2.2.3. Questions. The power of questions to transfer
knowledge has been recognized and practiced since the
days of Socrates and Confucius. Questions have
typically been used to represent and transfer
knowledge in the context of learning, but a scope of
application can be easily expanded also in the
complexed conceptual and abstract knowledge field.
Cooper [27] considered questions as "unusual
representation in knowledge resources", and discussed
the reuse of knowledge, by exemplified with two
system incorporating question-based representation.
Cooper evaluated questions as a way to engage people

without intimidating, and to support practitioners’
reflection. Hmelo-Silver and Barrows [28] pointed out
the value of questioning based dialogues in
collaborative knowledge acquisition such as
interpretation and synthesis. Anzai and Shiose [29]
also discusses question-based dialogues as a tool for
facilitating knowledge sharing in inclusive design
workshops. Together with dialogues, questions can
provide richer pragmatic constraints on knowledge
transfer than either component can achieve alone [30].
This question-based representation in this case should
be considered separately from the general and routine
questions that are often found in the FAQs. The
questions discussed here are for knowledge transfer in
a form of representation that brings reflection in action
[27], [31], identifies the nature of the problems,
disrupts stereotypes and turn the objective problems to
subjective challenge [29].
3. Research design
Aiming at transferring tacit knowledge for living
lab practice to wider practitioners, the authors
conducted a series of experiments, in which
practitioners’ experiences were collectively extracted
and externalized [32], [33](Step 1). The process was
conducted through two workshops for collecting tacit
knowledge for living lab practice, and an analysis
session. In analysis session, collected items were
analyzed and categorized. The depicted knowledge was
finally externalized in a form of three different
representations and evaluated (Step 2). The Step 1, the
knowledge extraction process in workshops and
analysis were reported previously in a conference [32],
[33], so that in this article, only the basic frame and
result will be presented.
3.1. Step 1: Workshops and knowledge
extraction
In order to collect empirical knowledge of living
lab from experienced practitioners, the authors
conducted a data collection workshop, which were
called the Wisdom of Practice workshops. Two
workshops were held in Copenhagen, Denmark and
Tokyo, Japan in 2018, and total 51 people from
Europe, North America and Asia with different
nationalities, professions and domains participated.
The workshops consisted of ten presentations and
group discussions. The presentations were made by
selected ten practitioners and researchers for sharing
personal experience and thoughts on the success or
failure of living lab projects, which were mainly used
to trigger participants’ reflections. After the
presentations, participants in the workshops were
divided into small groups with 4-6 members and
analyzed the key knowledge of success and failure
experience together for 1.5 hours. In order to
encourage discussions and reflections, and support the
externalization of tacit knowledge for living lab
practice, the authors prepared a few templates which
were distributed to the groups. Participants extracted
key message and know-how from the presentations and
discussions by utilizing the templates while sharing the
reflections of the presentations and their personal
experiences in other projects. In the end of the two
workshops, a total 526 pieces of externalized tacit
knowledge for living lab practice were collected.
No
Key words
Category 1: Co-creation
1
Start with questions
2
Stop hypnotize
3
Share personal stories
4
Redefine “us”
5
Externalize & get feedbacks
6
Create stranger’s perspective
7
Search what can be changed
8
Not only democracy
9
Try fast & fail fast
10
Start with hard-core MVP
Section 2: Process management
11
Visualize for discussion
12
Participation affordance
13
Our place, our space
14
Special guest
15
Not satisfied with workshop
16
Accumulate small successes
17
Timely feedback to participants
18
Time for reflection
19
Multi-speed
20
Be-flexible
21
Weakness disclosure
Section 3: Team building
22
Gather multi specialties
23
Limit core stakeholders
24
Capture charisma
25
Community collaboration
26
Start with the passionate
27
CSV: Creating shared “vision”
28
Attractive objective
29
Strategical wording and space
30
Achieve trust from stakeholders
Figure 1. The 30 knowledge for living lab
The primary 526 knowledge about living lab were
analyzed, classified, categorized and integrated, by

using the affinity diagram, the KJ method [34]. As a
result of analysis, three categories and the 30 types of
“tacit knowledge for living lab practice” were yielded.
Three categories are Co-creation, Process management
and Team building. The result is shown in the Figure 1.
The 30 tacit knowledge for living lab practice
presented in Figure 1 were extracted and analyzed
through a bottom-up two-stage process by practitioners
and the authors. Through the two step process, the final
collection of the 30 tacit knowledge for living lab
practice achieved further could be interpreted as valid
representations with various elements and
considerations extracted by living lab practitioners.
3.2. Step 2: Knowledge representations
Next, the 30 tacit knowledge for living lab practice
were externalized in three knowledge representation
forms. The utilized externalization formats are
multimedia, pattern language and questions.
3.2.1. Externalization with multimedia. All 30 tacit
knowledge for living lab practice were externalized in
multimedia format, using a mix of texts, photographs
and videos. The texts explain when the knowledge can
be used, what kind aspects of the knowledge can be
considered as tacit knowledge, and what kind of results
can be expected by using the knowledge. Photos and
videos visually show how the particular tacit
knowledge can be observed in different cases. As an
example, let us explain with “09: Try fast & fail fast
(refer Figure 1)” which explains the importance of
testing and verifying ideas as fast as possible and
learning from failures. For 09, one document, nine
pictures and two videos were prepared. As shown in
Figure 2, photos show a few different examples from
the real living lab cases.
3.2.2. Presentation of pattern language. All 30 tacit
knowledge for living lab practice were externalized
using patter language. Referring to the description of
Alexander [21], the living lab pattern consists of five
elements. They are (1) a pattern name, (2) an
illustration, (3) summary, (4) detail (explaining context
and expected results) and (5) related cases. Figure 3. is
an example with tacit knowledge 09. A pattern name
(1) is described a simple short sentence. Illustration (2)
is abstract but concrete enough to describe conceptual
tacit knowledge. Both name and illustration are
carefully chosen so that users can motorize and recall
pattern easily. Summary (3) describes with further
details still with short and concise expression. In
addition, further details (4) are described such as
context, problem, barriers, solution, actions, and
possible consequence, which make the users imagine
further with details. Finally, one or two related cases
(5) are described in a short story format.
Figure 2. Examples of photo representations
Top: the signals for bikes are under the test,
Bottom: plastic and paper mock ups are
tested
Figure 3. Examples of pattern language
3.2.3. The use of questions. All 30 tacit knowledge for
living lab practice were externalized using a form of
questions. Total 33 questions were created including
questions related to the three categories. The questions
are designed so that the three categories and the 30
tacit knowledge for living lab practice could be derived
as an answer. The key in this question-based
representation is in its composition. Each question
consists of a 5W1H question, and three to five choices,
28
Section 1: CoCreation Implement ideas in society Section 1: CoCreation Implement ideas in society
When?
Know-How
Effects
Episode
We spent a lot of time on testing. Sometimes three months. The
most important thing is to recognize the test as opportunity for
communicate with users, and accept users reaction seriously. In
many projects conducted by a large company, testing is regarded
as a mere process, which produces expected results. Rather utilize
the test for collecting user reactions and review the fundamentals
of our idea.
· Through the tests, users opinions can be reflected.
· Based on the users' opinion, idea becomes more concrete.
· By understanding potential effects of the idea, it becomes easier
to
Make a lot of prototypes, use them and fail fast. Learn from failure.
In the verification, strategically collect quantitative data on effect
and value of ideas.
When to materialize and verify ideas.
Test early and verify ideas,
and learn from failures.
KEYWORD 09
TRY FAST & FAIL FAST
(1)
(2)
(3)
(4)
(5)

an answer, and a dialogue instruction. The dialogue
instruction is used to understand reasons behind the
question and answers. Figure 4. is an example question
with tacit knowledge 09.
Figure 4. Examples of questions
4. Evaluation
Mixed method [35] was applied to evaluate three
external representations of tacit knowledge for living
lab practice. As evaluation, we conducted an interview-
based method with a 7 score scale, using an evaluation
template. The evaluation sessions were conducted
during 2020 January to April, approximately one hour
for each, with six evaluators. Among six evaluators,
two were experienced practitioner of living lab,
another two were intermediate practitioners, who have
some knowledge about living lab but no practical
experience, and plans to conduct a living lab project.
Finally, the other two were beginners who were
interested in but never experienced or involved in
living lab as practice. The interviews were conducted
face to face or online due to the limited social
condition.
The authors conducted interview with six
evaluators who evaluated three external presentations
based on a format the authors prepared. First, the
evaluators browsed three types of external
representations for 30 minutes. Next, the authors
interviewed for 30 minutes, based on the pre-defined
evaluation process and criteria shown in Table 2. The
evaluators evaluated about 1) the 30 extracted tacit
knowledge for living lab practice in general and 2)
three external representations on a scale of 1 (Strongly
disagree), 1 (Strongly disagree), 2 (Disagree), 3(More
or less disagree), 4 (Neutral), 5 (More or less agree), 6
(Agree), and 7 (Strongly agree). The evaluators could
also freely express their thoughts and opinions during
the evaluation process.
Table 1. Characteristics of six evaluators
ID
Expertise
Experience
E
1
Beginner
Interested in Living lab
and have studied for 3
months.
FtoF
E
2
Beginner
Has just started to know
the term Living labs and
got interested in
Online
E
3
Intermediate
Researched on living labs
for 7 years and planed
several living labs but
never practice before.
FtoF
E
4
Intermediate
Plan to run a living lab
project in a few months.
Online
E
5
Experienced
Conducted 7 living labs
on social technology
projects
FtoF
E
6
Experienced
Conducted three living
labs for last 5 years as
regional development
projects.
Online
Table 2. 6 Questions and evaluation criteria
Q
Criteria
Scale
1
Generally speaking, the described 30
knowledge is reflected important tacit
knowledge of experienced practitioners.
1-7
2
Each externalization describes tacit
knowledge of experienced practitioners?
1-7
3
Did you learn anything from
externalization?
Y/N
4
If so, what did you learn?
Open
5
Will externalization trigger your next
action?
Y/N
6
If so, what is your next step?
Open
Table 3. Three discussed topics
ID
Key word
Summary
09
Try Fast &
Fail Fast
It is important to test early and
verify ideas, and learn from
failures
11
Visualize
for
discussion
It is important to visualize in
pictures and diagrams & establish
common ground among
stakeholders.
19
Multi-
speed
Be aware of different and interests
of participants. It is important to
adjust their involvement.
With regard to the three external representations,
we chose three tacit knowledge as discussion items
among 30 as it was practically difficult to get
comments on all 30 items. The three selected items are

shown in Table 3, which are ID-09:Try Fast & Fail
Fast, ID-11: Visualize for discussion, and ID-19:Multi-
speed. Each evaluator evaluated the three items with
the three external representations (multimedia
externalizations, patter languages and questions), and
provided scores and comments about their evaluations.
5. Results
5.1. Tacit knowledge representation
The overall evaluation on the 30 sets of
externalized knowledge was generally well as shown in
the second left “General” column in Table 4. Many
evaluators commented together with positive
evaluations; “I think I got a better perspective (E1)”, "I
felt strong empathy that many other practitioners faced
similar challenges that I am in trouble now. I got a lot
of concrete advice by reviewing the 30 sets (E3) ", "I
wasn't aware before browsing, but there were many
useful externalized knowledge, which I recognize as
my hidden mindset (E5)", "While browsing the 30 sets,
I recalled my experience again and felt empowered that
my experience was valuable (E6)” On the other hand,
there were some negative comments or critics as well.
One of the beginners (E2) mentioned that "I think good
information is collected. However, I wish they showed
more concrete examples in each externalization. I
probably have too little experience, though.”
Table 4. Evaluation results in the 7 scale
General
Multimedia
Pattern
Question
E1
7
7
5
7
E2
7
6
5
7
E3
6
7
6
3
E4
7
6
6
7
E5
6
7
7
5
E6
7
5
7
5
5.2. Tacit knowledge externalization
Next, let us look at evaluations of each external
representation. E1 evaluated highly question and
multimedia while she put lower score regarding the
pattern language. E1 didn’t provide any particular
reasons except “It is easy to understand”. Although the
comment sounded rather positive, the evaluation score
was rather low. Concerning about multimedia, E1
mentioned that the ID-09 knowledge is difficult to
understand only with document, but photos and videos
helped a lot in understanding. He mentioned, “when
combining the document with the photos and videos,
the understanding of the document was increased.” E1
also commented that ID-19 was difficult to understand
even with photographs and videos. Interestingly, E1
mentioned that “it was easier to understand ID-19 with
the document.”
E2 evaluated highly in the order of question and
multimedia. E2 thought answering the questions made
her to consider a right aspect to a problem. Regarding
the externalization utilizing multimedia, she mentioned
that "photos and videos with strong impacts can deepen
understandings than the documents". Particularly “it
was a good opportunity to see knowledge from
different angles”. Because she could imagine in her
head easily even when the message behind the
representation was initially incomprehensible.
However, this doesn’t apply to all 30 knowledge, some
of which was hard to understand even with varied
external representations offered by multimedia. Similar
to E1, E2 also mentioned Pattern language as "hard to
understand".
E3, the intermediate practitioner, evaluated highest
with the Multimedia representation and the evaluation
on pattern language was also high. E3 commented that
“I feel that I could narrow down what I have to pay
particular attentions just by browsing the 30 sets of
externalization as I could see a way of thinking of the
experienced practitioners.
E4 scored the most in question while evaluations on
multimedia and pattern were also high at the score 6.
She particularly interested in combining different
external representations for example questions and
pattern. For example, she suggested to provide a few
questions after each section so that she could review
her understanding after browsing the pattern.
E5 put a low rating on questions because “the
process of answering the questions one by one is
tedious”, and “It lacks overview.” It was also pointed
out that the information provided in the question-based
representation was quite limited, so that the
experienced practitioners could not help considering
beyond the description. To think “there must be more
to say than the short selections of answers.” was rather
stressful.
E6 evaluated highly with pattern language with
strong enthusiasm. He mentioned that the short
description of the pattern language had decent
granularity so that he could freely expand his thought
to his own wider experience and reflected upon it.
From time to time, he also recalled some key aspects to
pay attention, which played a valuable reminder of
things he had forgotten but important.
5.3. Trigger to action
The last two question in the interview to the six
evaluators was whether they learn from external
representations and whether the learning would lead to

action. All evaluators commented “yes” to the
questions. This is probably because the evaluators were
originally interested in living lab and have actively
been seeking for good advice for their practice.
However, depending on evaluators, there was a big
difference in which external representation was most
influential to them. E2 strongly supported the form of
question. By checking the questions, browsing the list
of answers, he started to think about something he had
never thought about before. The reflection by browsing
the answer choices, answering question, having
interactive dialogues gave him various ideas for
practice. On the contrary, E5 and E6 evaluated pattern
language highest because “pattern language was
explicitly shown what I did not think clearly as a key
for the living lab practice, but I totally agreed its
importance (E5)”, and “Pattern language conveyed
right amount of information with right granularity
(E6).”
Both E5 and E6 mentioned ID-19 as something
they would consider next time they were in their fields.
Similarly, E2 mentioned about ID-09, as something
they want to try out when they involved in a living lab
project.
6. Discussion and reflection
In this article, based on the extracted tacit
knowledge, the three external representations,
multimedia, pattern language and questions, were
tested and evaluated. The evaluation suggested that the
three external representations have their own unique
characteristics, and there is no best method for
everyone under any conditions. In this section, we are
going to discuss some findings through externalization
and evaluation, and discuss future issues.
6.1. Characteristics of the representations
In the evaluation sessions, the evaluators
commented different aspects of the three tacit
knowledge external representations. Representations
using the question formats were most popular among
two beginners. Although experienced practitioners
criticized that questions could not provide a
comprehensive perspective, two evaluators who had no
experience in living lab were clearly fond of the style.
It can be interpreted that the beginners often do not
know what to pay attention in conducting a living lab,
and questions and answers could narrow down the
scope of consideration. In other words, the beginners
can be overwhelmed easily with rich information as
they do not to know how to. For beginners, it is a
challenging task to grasp or comprehend a whole view,
so probably there is no urgent need of overview. As
indicated in a comment E1 about pattern language,
“Rather than the manuals like pattern language or
methods, I would like to understand living lab
concept”. The questions provide, in a sense, a right
amount of information and they are guaranteed all the
necessary information are presented in front of them.
The question-based representations created an easy
context for what is generally considered important.
Representation using multimedia was also highly
evaluated by almost all evaluators. Many of the
comments from E1 and E2, namely beginners, showed
visual representations such as photos and videos
appealed them, which can be interpreted that the
emphasis on visual expression is suitable for beginners.
The documents are also useful for embodying
knowledge, while visual expressions can transmit
information necessary for each subject with a decent
granularity than texts and keywords. Multimedia
became more effective also due to its combination of
multiple representations prepared in one platform.
Pattern language was most highly evaluated by the
experienced practitioners. They expressed their
opinions vigorously, and talked about their related
experiences energetically. The comments indicate that
the value of pattern language was in its ability to
initiate appropriate reflections on the practitioner’s
own experiences, which has high proximity with the
exemplified knowledge. Furthermore, the knowledge
that the practitioners have been recognized but not
externalized by themselves, are expressed verbally or
visually in a form of pattern language. By browsing the
patterns, it seems that the practitioners’ experience was
amplified and empowered. The practitioners expressed
their view more freely on a particular knowledge as if
they were the one who described. However, not
everyone has the same view. Although the beginners
found pattern language was also interesting, they were
hardly impressed by, and sometimes confused to depict
essence from.
6.2. External representations and maturity
It is a well-known learning behavior that acquiring
new knowledge often lead some action [36]. The fact
that all evaluators answered that they could imagine
their next step, indicates that the 30 tacit knowledge for
living lab practice have been appropriately described
and extracted some essence for the knowledge about
living labs. Interestingly, all evaluators performed their
reflection in action [31] at their own level, in spite that
they favored different externalization and expressed
differently. The triggered representations varied greatly
depending on the person. Some found that the question
could help them to identify the next important action,

while others found pattern language most important as
they recognized the value of their practice and decided
to conduct the action more consciously.
The analysis and evaluation of three external
representations of tacit knowledge for living lab
practice, showed that the question-based representation
was suitable to convey knowledge as learning
experience, while pattern language tends to be
accepted by the experienced practitioners. Since the
evaluation was conducted just by interviews with six
practitioners, it is not durable to make a general
evaluation. However, our results indicate that there is a
certain correlation between experience levels on living
lab and externalization characteristics. In other words,
a way to convey knowledge may be related to
knowledge receivers’ maturity level. In practice fields
where gather practitioners with various maturity levels,
externalization approach should be well considered
depending on who want to utilize, what kind of
knowledge to utilize and how to utilize knowledge.
7. Conclusion
In this article, in order to promote living lab
practice, the tacit knowledge extraction for living lab
practice, and three types of external representations are
examined. In living lab practice, there are a lot of tacit
knowledge about how to design, organize and operate
living labs, which are not externalized. The evaluation
of tacit knowledge externalization itself were highly
accepted, while the evaluation of the three types of
external representations varied greatly depending on
the evaluators. Our analysis suggests that the maturity
level of the practitioners who utilize the described
knowledge in the living lab practice and the format of
the external representation are highly correlated. The
tacit knowledge of living lab practice is difficult to
convey if it is only in writing or document. In the field
of knowledge transfer such as living lab practice, it is
still a major issue to consider how to externalize
knowledge in an appropriate manner, and transmit the
knowledge to practitioners. In the field where there are
practitioners with many maturity levels, the format of
external representation should be carefully considered.
With no doubt, this paper exposed some challenges.
The proposed various knowledge externalization
methods were in a sense, arbitrarily selected as
methods considered appropriate for abstract
knowledge. It would be valuable to deploy different
externalization approaches to understand relations
between external representation formats and ability of
knowledge transfer. In addition, the limited number of
evaluators and the limitation caused by online
interviews due to social conditions certainly influenced
the analysis. It is possible to improve the evaluation
and analysis quality through refining better framework
with more appropriate evaluation protocol.
Along with the development of digital and
communication technology, human knowledge became
possible to externalize in various ways. Some
externalization approach, which were considered
impossible to record and transfer became possible.
Nowadays knowledge externalization utilizes not only
visual, but also other five senses such as tactile, smell
and hearing. The target of knowledge externalization is
not only abstract knowledge such as tacit procedural
knowledge used for the implementation of living labs,
but also skills accumulated by long-term technical
training such as traditional crafts [20]. This indicates
that the studies on tacit knowledge externalization open
up new ways of knowledge transfer, and also created
new possibilities in wider domains.
8. References
[1] G. Fischer, “Symmetry of ignorance, social creativity,
and meta-design,” Knowledge-Based Syst., 2000, doi:
10.1016/S0950-7051(00)00065-4.
[2] D. Schuurman, L. De Marez, and P. Ballon, “Living
Labs: a systematic literature review,” Open Living Lab
Days 2015, 2015.
[3] B. Dutilleul, F. A. J. Birrer, and W. Mensink,
“Unpacking European Living Labs: Analysing
Innovation’s Social Dimensions,” Cent. Eur. J. Public
Policy, 2010.
[4] EC ENoLL, “European Network of Living Labs
(ENoLL),” Openlivinglabs.eu, 2015.
http://www.openlivinglabs.eu/.
[5] M. Yasuoka, F. Akasaka, A. Kimura, and M. Ihara,
“Living labs as a methodology for service design - An
analysis based on cases and discussions from a systems
approach viewpoint,” Proceedings of International
Design Conference, DESIGN, vol. 1, no. 2012. pp. 127–
136, 2018, doi: 10.21278/idc.2018.0350.
[6] D. Schuurman, A.-L. Herregodts, A. Georges, and O.
Rits, “Innovation Management in Living Lab Projects:
The Innovatrix Framework,” Technol. Innov. Manag.
Rev., 2019, doi: 10.22215/timreview/1225.
[7] C. Dell’Era and P. Landoni, “Living lab: A
methodology between user-centred design and
participatory design,” Creat. Innov. Manag., 2014, doi:
10.1111/caim.12061.
[8] J. H. Boose, “A survey of knowledge acquisition
techniques and tools,” Knowl. Acquis., 1989, doi:
10.1016/S1042-8143(89)80003-2.
[9] I. Nonaka and H. Takeuchi, The Knowledge-creating
Company. Oxford University Press, 1995.
[10] M. Hossain, S. Leminen, and M. Westerlund, “A
systematic review of living lab literature,” Journal of
Cleaner Production. 2019, doi:
10.1016/j.jclepro.2018.12.257.
[11] B. B. Kareborn and A. Stahlbrost, “Living Lab: an open

and citizen-centric approach for innovation,” Int. J.
Innov. Reg. Dev., vol. 1, no. 4, p. 356, 2009, doi:
10.1504/IJIRD.2009.022727.
[12] A. Følstad, “Living Labs for innovation and
development of information and communication
technology: a literature review,” Electron. J. Virtual
Organ. Networks, vol. 10, no. August, pp. 99–131,
2008.
[13] S. Leminen and M. Westerlund, “Towards innovation in
Living Labs networks,” Int. J. Prod. Dev., vol. 17, no.
1/2, p. 43, 2012, doi: 10.1504/IJPD.2012.051161.
[14] B. Bergvall-Kåreborn, M. Holst, and A. Ståhlbröst,
“Concept design with a living lab approach,” in
Proceedings of the 42nd Annual Hawaii International
Conference on System Sciences, HICSS, 2009, doi:
10.1109/HICSS.2009.123.
[15] A. G. Nystrom and S. Leminen, “Living lab. A new
form of business network,” Concurrent Enterprising
(ICE), 2011 17th International Conference on. pp. 1–10,
2011.
[16] N. Wood, “Transmitting craft knowledge : designing
interactive media to support tacit skills learning,” 2006.
[17] U. H. Lassen and N. Wood, “‘Plumb line scribe’: using
multimedia to preserve traditional craft skills,” Cr. Res.
J., vol. 4, no. 1, pp. 31–52, 2013.
[18] N. Wood, “Silent witness , using video to record and
transmit tacit knowledge in creative practices,” in
Hantverkare emellan / Between craftspersons, G.
Almevik, Ed. Craft Laboratory, Mariestad, University of
Gothenburg, 2014, pp. 56-69.
[19] B. A. Aytekin and K. Rızvanoğlu, “Creating learning
bridges through participatory design and technology to
achieve sustainability in local crafts: a participatory
model to enable the transfer of tacit knowledge and
experience between the traditional craftsmanship and
academic education In,” Technol. Des. Educ., vol. 29,
no. 3, pp. 603–632, 2019.
[20] M. Yasuoka, “Designing Knowledge Management
System for Supporting Craftsmen’s Collaboration
beyond Temporal Boundaries,” Proc. 53rd Hawaii Int.
Conf. Syst. Sci., vol. 3, pp. 4880–4889, 2020, doi:
10.24251/hicss.2020.599.
[21] C. Alexander, “The Timeless Way of Building,” New
York Oxford University Press. 1979, doi:
10.1080/00918360802623131.
[22] A. Dearden, J. Finlay, L. Allgar, T. Grange, B. Park, and
B. Mcmanus, “Using Pattern languages in Participatory
Design,” in Participatory Design Conference, 2002, pp.
104–113.
[23] A. Dearden, J. Finlay, L. Allgar, and B. McManus,
“Evaluating pattern languages in participatory design,”
in Conference on Human Factors in Computing Systems
- Proceedings, 2002, doi: 10.1145/506486.506535.
[24] P. Avgeriou and U. Zdun, “Architectural patterns
revisited : A pattern language,” in 10th European
Conference on Pattern Languages of Programs,
EuroPLoP 2005 - Preliminary Conference Proceedings,
2005.
[25] M. Beedle, M. Devos, Y. Sharon, K. Schwaber, and J.
Sutherland, “SCRUM : An extension pattern language
for hyperproductive software development,” Pattern
Lang. Progr. Des., 1999, doi: 10.1.1.15.3987.
[26] J. O. Horchers, “A pattern approach to interaction
design,” AI Soc., 2001, doi: 10.1007/bf01206115.
[27] L. P. Cooper, “The power of a question: A case study of
two knowledge capture systems,” Proc. 36th Annu.
Hawaii Int. Conf. Syst. Sci. HICSS 2003, 2003, doi:
10.1109/HICSS.2003.1174258.
[28] C. E. Hmelo-Silver and H. S. Barrows, “Facilitating
collaborative knowledge building,” Cogn. Instr., 2008,
doi: 10.1080/07370000701798495.
[29] T. Shiose, “Inclusive Design,” in Field Informatics,
Springer, Berlin, Heidelberg, 2012, pp. 107–121.
[30] N. Asher and A. Lascarides, “Questions in Dialogue.,”
Linguist. Philos., vol. 21, pp. 237–309, 1998, doi:
https://doi.org/10.1023/A:1005364332007.
[31] A. D. Schön, The Reflective Practioner: How
Professionals Think In Action. New York, USA: Ingram
Publisher Services, 1983.
[32] F. Akasaka, A. Kimura, M. Ihara, and M. Yasuoka,
“How to make successful Living Labs? : Extraction and
description of key know-how for Living Lab projects,”
in International Conference on Serviceology (ICServ),
2018.
[33] F. Akasaka, M. Yasuoka, M. Nakatani, A. Kimura, and
M. Ihara, “Patterns for living lab practice: describing
key know-how to promote service co-creation with
users,” Int. J. Autom. Technol., vol. 14, no. 5, pp. 769–
778, 2020, doi: 10.20965/ijat.2020.p0769.
[34] R. Scupin, “The KJ method: A technique for analyzing
data derived from Japanese ethnology,” Hum. Organ.,
1997, doi: 10.17730/humo.56.2.x335923511444655.
[35] J. C. Greene, V. J. Caracelli, and W. F. Graham,
“Toward a Conceptual Framework for Mixed-Method
Evaluation Designs,” Educ. Eval. Policy Anal., 1989,
doi: 10.3102/01623737011003255.
[36] D. A. Kolb, “Experiential Learning: Experience as The
Source of Learning and Development,” Prentice Hall,
Inc., 1984, doi: 10.1016/B978-0-7506-7223-8.50017-4.