Conference PaperPDF Available

Is Task Board Customization Beneficial? - An Eye Tracking Study

Authors:

Abstract and Figures

The task board is an essential artifact in many agile development approaches. It provides a good overview of the project status. Teams often customize their task boards according to the team members' needs. They modify the structure of boards, define colored codings for different purposes, and introduce different card sizes. Although the customizations are intended to improve the task board's usability and effectiveness, they may also complicate its comprehension and use. The increased effort impedes the work of both the team and team externals. Hence, task board customization is in conflict with the agile practice of fast and easy overview for everyone. In an eye tracking study with 30 participants, we compared an original task board design with three customized ones to investigate which design shortened the required time to identify a particular story card. Our findings yield that only the customized task board design with modified structures reduces the required time. The original task board design is more beneficial than individual colored codings and changed card sizes. According to our findings, agile teams should rethink their current task board design. They may be better served by focusing on the original task board design and by applying only carefully selected adjustments. In case of customization, a task board's structure should be adjusted since this is the only beneficial kind of customization, that additionally complies more precisely with the concept of fast and easy project overview.
Content may be subject to copyright.
Is Task Board Customization Beneficial?
An Eye Tracking Study
Oliver Karras B, Jil Kl¨under, and Kurt Schneider
Software Engineering Group
Leibniz Universit¨at Hannover
30167 Hannover, Germany
Email:{oliver.karras, jil.kluender, kurt.schneider}@inf.uni-hannover.de
Abstract. The task board is an essential artifact in many agile devel-
opment approaches. It provides a good overview of the project status.
Teams often customize their task boards according to the team mem-
bers’ needs. They modify the structure of boards, define colored codings
for different purposes, and introduce different card sizes. Although the
customizations are intended to improve the task board’s usability and
effectiveness, they may also complicate its comprehension and use. The
increased effort impedes the work of both the team and team externals.
Hence, task board customization is in conflict with the agile practice of
fast and easy overview for everyone.
In an eye tracking study with 30 participants, we compared an original
task board design with three customized ones to investigate which de-
sign shortened the required time to identify a particular story card. Our
findings yield that only the customized task board design with modified
structures reduces the required time. The original task board design is
more beneficial than individual colored codings and changed card sizes.
According to our findings, agile teams should rethink their current task
board design. They may be better served by focusing on the original task
board design and by applying only carefully selected adjustments. In case
of customization, a task board’s structure should be adjusted since this
is the only beneficial kind of customization, that additionally complies
more precisely with the concept of fast and easy project overview.
Keywords: Agile development, task board, customization, eye tracking
1 Introduction
Agile software development is a general term for a set of development approaches
which focus on social aspects. These approaches aim at increasing the developers’
productivity, delivering working software in time and minimizing the risk of
failure within software projects [27]. The core concept of agile development is
based on fundamental values which are concretized by defined principles that
are in turn fulfilled by certain practices [6].
eXtreme programming (XP) [5] and Scrum [24] are the most commonly used
and combined agile approaches [3, 20]. One practice of XP is the informative
arXiv:1708.00275v1 [cs.SE] 1 Aug 2017
workspace. According to Beck and Andres, this practice is about how to “make
your workspace about your work. An interested observer should be able to walk
into the team space and get a general idea of how the project is going within
15 seconds” [5, p. 39f.]. Cockburn [8] provides a similar concept of the so-called
information radiator. “An information radiator displays information in a place
where passerby can see it” [8, p. 114]. An information radiator has to fulfill two
features – representing information that changes over time, and requiring very
little effort to view the display. In total, an implementation of these two concepts
must be easy-to-use and offer a fast overview with minimal effort [22].
One implementation of both concepts is the task board [25]. It is one key
artifact of agile development [3,14] which serves a dual purpose of supporting
a team’s work organization and constituting at a glance how much work is left
[10,18]. Additionally, a task board allows communication and collaboration since
it tracks and visualizes the software development process and thus simplifies its
accessibility for everyone [17, 20].
Although the original task board design of Cohn [10] provides a clear overview,
teams tend to customize their own [26]. Sharp et al. [27] analyzed six different
mature XP teams and their task boards. They identified that the teams’ task
boards were consistent in terms of usage, but not regarding a particular de-
sign. The different task board designs resulted from combinations of various
customizations like modified structures, individual colored codings and changed
card sizes. Customization itself is not serious since a task board can be eas-
ily and flexibly adjusted due to its physical nature [13, 26]. Additionally, agile
approaches involve customization by offering corresponding degrees of freedom
[20,28]. Furthermore, any adjustment of a task board by an agile team according
to its needs is plausible since the team members work with it every day [26].
However, multiple combined customizations complicated the maintenance
and comprehensibility of a task board. In particular, the increased effort impedes
the work of a team as well as team externals with a customized task board
[13, 15, 27]. Thus, the underlying practice of a task board as an informative
workspace for fast and easy project overview for everyone gets lost.
While the tight social and technical cohesion found in mature agile teams are
not disputed, the effect of single practices like the informative workspace is little
understood [27]. Berczuk emphasizes that “any team is best served by following
the rules of the agile method with as few adjustments as possible” [7, p. 6].
Corresponding to Pikkarainen et al. [20], adoption and change of agile practices
are aspects of future studies. Therefore, we investigated whether specific single
task board customizations contribute to a task board’s usage in comparison with
an original task board design. As an example, we focused on the identification
of a particular story card as one main task of using a task board.
We conducted an eye tracking study to compare an original task board de-
sign corresponding to literature [10, 17] with three customized ones. Each cus-
tomized task board differs exactly in one single aspect from the original one, such
as modified structures, individual colored codings, or changed card sizes. Each
modification could contribute to achieving a better overview of a task board in
order to identify a particular story card faster. In our study, we observe whether
a particular task board customization improves the work with a task board.
These results identify whether specific kinds of customization are beneficial or
not for a task board’s usage. Our findings can help agile teams to rethink their
current task board design in order to improve it.
The contribution of this paper is the insight that modified structures are
the only kind of customization that shortens time to identify a particular story
card. Individual colored codings and changed card sizes even have detrimental
effects on the performance. Agile teams should reconsider their current task
board designs. They may be better served by focusing on the original task board
design and applying carefully selected adjustments. A task board’s structure
should be adjusted since this kind of customization is beneficial and complies
with the agile practice informative workspace.
This paper is structured as follows: Section 2 discusses related work. We
describe the task board and its major kinds of customization in section 3. In
section 4, we report our eye tracking study and document its findings, which we
discuss in section 5. Section 6 concludes the paper.
2 Related Work
2.1 Task Board: Key Artifact of Agile Software Development
Several researchers investigated the task board’s usage and role in the agile
software development process.
Sharp et al. [25] systematically consider the use and role of story cards and
a task board in one mature XP team. Based on story cards and the task board,
the authors analyze the team’s collaborative work by using the distributed cog-
nition framework. Thus, the information flows in, around and within the XP
team can be substantiated to answer “what if” questions regarding changes to
the story cards’ and task board’s form to illustrate consequences for the team-
work. Sharp and Robinson [26] extend the previously mentioned study on three
mature XP teams. Their results show significant similarities between the teams’
usage of story cards and task board, but not in their particular designs. After dis-
cussing the importance of a physical representation of both artifacts, the authors
highlight important aspects that need to be taken into account for technological
tool-support of agile development. In a further study, Sharp et al. [27] investigate
the role of story cards and a task board from two complementary perspectives: a
notational and a social one. Based on both perspectives, they explain that these
two physical artifacts are important key properties of successful teams. Any at-
tempt to replace these artifacts with technological support needs to take into
account the complex relationships between both perspectives and the artifacts.
Petre et al. [18] consider the use of public visualizations, i.e. story cards and
task boards, in different software development teams. In a number of empiri-
cal studies, the authors observe differences in the use of paper and whiteboards
between traditional and agile teams. The findings are used to identify possible
implications of these differences for software development in general. Liskin et
al. [15] explore the use and role of story cards and task board within a Kan-
ban project. Their findings reveal that despite a task board for requirements
visualization and communication some requirements are still too implicit and
caused misunderstandings. Katsma et al. [14] investigate the usage of software-
and paper-based task boards in globally distributed agile development teams.
They conclude that paper-based task boards currently offer many advantages
compared to its software-based solutions. By applying the media synchronicity
theory, Katsma et al. [14] explain the current use and future development of
software tools to support globally distributed agile development teams. Perry
[17] reports his experiences about transparency problems in agile teams due to
difficulties in the transition from a physical to electronic task board. He dis-
cusses the advantages and disadvantages of physical and electronic task boards.
Based on his observation, he concludes that both task board types have their
place in team collaboration. However, the simple power and utility of physical
task boards should not be neglected. Hajratwala [13] observes the creation and
evolution of various task boards over time in different projects. He explains the
reasons why the task boards evolved, and recommends key attributes that a task
board should have.
The previous investigations focus on both the usage and role of story cards
and task boards in agile software development. The main focus is on the general
work with a task board and its importance for agile development. Addition-
ally, different task board designs and their evolution over time are presented.
Although differences in the designs were recognized, none of the researchers con-
sidered its possible impact on work with this artifact. Our paper addresses this
topic by investigating whether task board customization is beneficial or not.
2.2 Viewers’ Consideration of Software Development Artifacts
There are already several researchers who used eye tracking to investigate a
viewer’s consideration of a respective software development artifact.
Ahrens et al. [1] conducted an eye tracking study to analyze how software
specifications are read. They identified similar patterns between paper- and
screen-based reading. The results contribute awareness by considering readers’
interests based on how they use a specification. Gross and Doerr [11] performed
an explorative eye tracking study to investigate software architects’ information
needs and expectations from a requirements specification. The results allow first
insights into the relevance of certain artifact types and their notational represen-
tations. Gross et al. [12] extended their previously mentioned eye tracking study
by analyzing information needs and expectations of usability experts. Based
on the findings, the authors introduced the idea of a view-based requirements
specification to fulfill needs of different roles in software development. Santos
et al. [23] evaluated the effect of layout guidelines for igoal models on novice
stakeholders’ ability to understand and review such models. They identified no
statistically significant differences in success, time taken or perceived complex-
ity between tasks conducted with well and badly designed model layouts. Ali et
al. [2] applied eye tracking to the verification of requirements traceability links.
Their data analysis allowed the identification and ranking of developers’ pre-
ferred source code entities. Thus, the authors defined two weighting schemes to
recover traceability links combined with information retrieval techniques.
All previous studies apply eye tracking to analyze how specific software de-
velopment artifacts are read by persons with different functions. We follow this
approach by using eye tracking to investigate the work with a task board. Our
study specifically focuses on the impact of different task board customizations
on a task board’s usage by team externals respectively new team members.
3 Task Board: Structure and Content
The task board’s origins are the informative workspace practice of Beck and
Andres [5] as well as the concept of information radiator by Cockburn [8]. They
present first ideas of story cards pinned on a wall or whiteboard. In their books,
they offer possible implementations of these concepts.
Cohn [10] describes a first concrete task board design in his book “Agile
Estimating and Planning”. According to his definition, a task board consists of
up to seven columns to track and visualize a team’s progress in development.
The seven columns are:
1. Stories: A backlog of all story cards
2. To Do : All task cards to implement particular story cards
3. Tests Ready: Status of a story cards’ acceptance tests
4. In Process: Task cards developers have signed up for
5. To Verify: Implemented task cards that need to be verified
6. Hours: Total working hours remaining for particular story cards
7. Done: All implemented and verified task cards
Furthermore, Cohn [10] defines that a task board includes one row for each story
card. Each row contains all task cards that are related to the corresponding story
card. According to Cohn [10], the columns Tests Ready,To Verify,Hours and
Done are optional.
3.1 Task Board Customizations
Based on the previously mentioned findings in literature, we considered further
research papers about the design and content of task boards. Additionally, we
analyzed different task boards with respect to their design in online galleries of
team spaces [4, 19, 29]. Thus, we identified three major kinds of customization:
modified structures,individual colored codings, and changed card sizes.
Modified structures are changes regarding the amount and usage of a task
board’s rows and columns. Petre et al. [18] describe a task board as a vertical
surface for story cards. This task board has a codified structure to indicate a
story card’s status. Other researchers [17, 21, 22] report in greater detail about
this codified structure. Pries-Heje and Pries-Heje [21] focus on a task board for
Scrum, which consists of the four columns Backlog,Task in Progress,Done, and
Done Done corresponding to their description. A similar task board structure
is mentioned by Rubart and Freykamp [22]. The columns of this task board are
named Selected Product Backlog,Tasks To Do,Work In Progress, and Done.
Perry [17] also reports that a simple task board has four columns called Story,
To Do,In Progress, and Complete.
All descriptions have in common that the task board structure consists of
the same four columns with only slightly different labels. However, none of these
researchers mentions the use of rows on a task board. We could identify two
variants for the use of rows based on our consideration of team spaces in online
galleries. The first variant uses one row for each story card, which corresponds to
Cohn’s definition [10]. The second one uses rows in specific columns like To Do
and Work In Progress to visualize the assignment of developers to story cards.
The comparison of these insights with Cohn’s original task board structure [10]
shows clear differences regarding the amount and use of a task board’s rows and
columns between theory and practice.
Individual colored codings are colored cards and markers with arbitrary
meaning which need to be memorized. Several researchers report the widespread
individual use of colored codings on task boards. Katsma et al. [14] describe
the use of different colored cards to indicate various card types, e.g. red for
bugs cards. Liskin et al. [15] mention colored markers on cards to represent
assigned developers. Sharp et al. [25–27] observe the use of colored markers
and cards as status indicators and card types in four mature XP teams. These
findings correspond to our observations of the task boards presented in the online
galleries. Even though we cannot clarify the exact meanings of the used colored
codings, we observe that their use is widely scattered.
Changed card sizes consider the size of story cards which are used to write
down user stories and display them on the task board. The size of story cards
has a wide range. Azizyan et al. [3] as well as Katsma et al. [14] report about
story cards the size of sticky notes or post-its. In contrast, Perry [17] and Sharp
et al. [27] state that a story card’s size can be up to an index card of 5 ×7
inches. These insights coincide with our observations of the online galleries. We
identified the same range of card sizes from post-its up to index cards.
3.2 Task Board Designs
In consideration of the previously described findings, we developed four task
board designs for our eye tracking study. These designs are based on a dataset
of real story cards from a completed software project. While one task board
design is similar to Cohn’s initial definition of a task board design [10], each of
the other designs takes one of the three major customizations into account.
During the design development, we took into account that all task boards
represent the same content, except for exactly one specific difference accord-
ing to the customizations. Fig. 1 represents an overview of our four task board
designs. All task boards have four columns, labeled with Stories,Task To Do,
W.I.P (Work In Progress), and Done. These labels are adopted from the original
task board of the completed software project whose story cards were used. We
decided to change as little as possible from the original dataset. Therefore, we
retain the labels of the task board since they are similar to the previously men-
tioned ones. Furthermore, these four columns cover all three obligatory columns
corresponding to Cohn’s definition [10].
Fig. 1a presents the task board with an original design which is similar to
Cohn’s definition [10]. This task board does not have Cohn’s row structure [10]
since the used dataset of real story cards did not consider this aspect. Therefore,
the story cards could not be grouped to achieve a reasonable row structure.
Fig. 1b shows the task board with modified structures. We decided to use
the second variant of additional rows over specific columns since Cohn’s row
structure [10] was not applicable to the used dataset. We did not add additional
columns to change only one structural aspect. Thus, we added rows over the
columns Task To Do and W.I.P. to visualize the assignment of developers to
story cards. Each row starts with a letter that represents one developer.
Fig. 1c represents the task board with individual colored codings. In accor-
dance with literature [15,25], we added colored markers on the right lower corner
(a) Task board: Original design (b) Task board: Modified structures
(c) Task board: Individual colored codings (d) Task board: Changed card sizes
Fig. 1: Task board designs
of the story cards. Each of the three colors (green, orange and blue) represents
one developer and his assignment to the corresponding story card.
Fig. 1d illustrates the task board with changed card sizes. We decided to
minimize the story cards to sticky note size (ca. 4 ×6 inches), since story cards
have originally index card size (ca. 5 ×7 inches).
All task boards have the same amount of handwritten story cards whose
content is based on the real dataset. The first three task boards (see Fig. 1a,
Fig. 1b, and Fig. 1c) contain 40 story cards of index card size. The last task board
(see Fig. 1d) contains 40 story cards of post-it note size. While the amount and
general position of the story cards are the same in each column and task board,
we shuffled the story cards before placing them on the task boards. Thus, we
achieved a random placement regarding the story cards’ content and no task
board equals exactly any other.
4 Eye Tracking Study
The aim of our eye tracking study was to understand whether task board cus-
tomization facilitates identifying a particular story card faster compared to an
original task board design. We proceeded to achieve this aim by comparing the
original task board design with each of the three task board customizations. Such
an investigation enables us to judge whether the original task board design or
the respective task board customization should be preferred. We were interested
in answering the following research question:
RQ: Does the respective task board customization facilitate identifying a par-
ticular story card faster compared to the original task board design?
To answer the research question, we tested the following hypotheses for each
of the three task board customizations:
H0: There is no speed difference in identifying a particular story card between
the original task board design and the respective task board customization.
H1: There is a speed difference in identifying a particular story card between
the original task board design and the respective task board customization.
4.1 Study Design
In this study, we performed three separate within-subjects experiments with
counterbalancing. The dependent variable was the task completion time for iden-
tifying a particular story card. The independent variable was the task board
design with two levels: the original task board design and one of the three task
board customizations. We measured the task completion time by observing the
participants with the SMI Eye Tracking Glasses1. Each experiment represents a
scenario in which the participant joins an ongoing development project as a new
1https://www.smivision.com/eye-tracking/product/eye-tracking-glasses/
team member who has to work with the existing task board. We decided to focus
on the perspective of a new team member since a task board should support a
fast and easy project overview for everyone, i.e. the team and team externals
respectively new team members. If a new team member already benefits from a
customization, a whole team should also benefit from it.
We analyzed task completion times with a two-tailed paired samples t-test at
a significance level of p= 0.05. This allows us to determine whether the respec-
tive task board customization leads to a statistically significant speed difference
in identifying a particular story card compared to the original task board design.
Thus, we can identify whether a particular task board customization is beneficial
for a task board’s usage. An existing speed difference would allow us to reject
H0, while a missing one would not allow such a rejection.
4.2 Study Procedure
The eye tracking study was carried out with 30 participants consisting of 10
undergraduate and 20 graduate students of computer science. All participants
had basic knowledge about agile software development and were close to the
next step in graduation. Thus, they represent potential new team members in a
software development team, which corresponds to our target population.
All in all, the whole eye tracking study with all three experiments was car-
ried out within three months. Each experiment compared the original task board
design with one of the three major task board customizations. We randomly as-
signed each participant to one of the three experiments. In each experiment, we
conducted 10 separate sessions each with one of the 10 assigned participants.
Each session included an introduction to the experiment with its task of con-
sidering two task boards. In this context, we explained the basic concept of a
task board. Depending on the experiment, we assigned the letter “J” (see Fig.
1b) respectively the color “green” (see Fig. 1c) to the participant since the task
boards with modified structures respectively individual colored codings required
the assignment of a row or color to the participant. After the calibration of the
SMI Eye Tracking Glasses for the participant, we captured their examination of
the task board. We repeated the same process for the second task board design.
4.3 Analysis and Results
Table 1 shows the measured task completion times of each participant for the
particular experiment and respective task board design. The first five subjects of
each experiment (see Table 1, Group 1) received the original task board design
first and then the customized one. The other five subjects of each experiment (see
Table 1, Group 2) received the designs in reversed order. For each experiment, we
verified that the data is normally distributed by applying the Shapiro-Wilk test.
Subsequently, we performed the two-tailed paired samples t-tests at a significance
level of p= 0.05. Thus, we can determine whether an observed difference exists
due to the test conditions or by chance. Additionally, we calculate Cohen’s d
which is the most common type of effect size for t-tests that indicates whether
or not the difference between two groups’ mean is large enough to have practical
relevance independently from statistical significance.
In Table 2, we present the results of our conducted two-tailed paired samples
t-tests and their effect size d.
The analysis of the first experiment yields a significant difference in the task
completion times for the original task board design (M= 15.0s, SD = 3.6s) and
the modified structures (M= 9.8s, SD = 3.9s); t(9) = 2.39, p = 0.04. Hence,
H0can be rejected for the first experiment. Modified structures shorten time to
identify a particular story card compared to the original task board design. The
value of Cohen’s dis 0.76, which is close to the threshold of 0.8 for a large effect
[9]. Hence, the identified difference has almost large practical relevance.
The t-test of the second experiment shows a significant difference between the
task completion times for the original task board design (M= 11.7s, SD = 3.2s)
and the individual colored codings (M= 14.3s, SD = 4.7s); t(9) = 2.86, p =
0.02. The null hypothesis H0can be rejected for the second experiment. Conse-
quently, the original task board design allows to identify a particular story card
faster compared to the individual colored codings. Cohen’s dis 0.90 and thus
Table 1: Experiment results – Task completion time [s]
Subj. Experiment 1 Subj. Experiment 2 Sub j. Experiment 3
Original Structures Original Codings Original Cards
Group 1
P1 16 10 P3 16 15 P4 10 11
P2 18 4 P14 16 22 P5 13 27
P11 16 11 P15 11 12 P24 30 36
P12 12 9 P16 10 9 P25 4 19
P13 19 4 P17 13 16 P26 19 18
Group 2
P6 18 10 P18 12 20 P23 22 12
P7 8 16 P19 13 16 P27 9 19
P8 12 9 P20 11 13 P28 19 28
P9 13 15 P21 5 6 P29 17 27
P10 18 10 P22 10 14 P30 12 16
Mean 15.0 9.8 Mean 11.7 14.3 Mean 15.5 21.3
SD 3.6 3.9 SD 3.2 4.7 SD 7.5 7.9
Table 2: Two-tailed paired samples t-test
Experiment 1 2 3
Calculated t-value -2.39 2.86 2.41
t-value from table
(df = 9, α = 0.05) 2.26
Calculated p-value 0.04 0.02 0.04
Result Significant Significant Significant
(p-value 0.05?)
Cohen’s d0.76 0.90 0.76
greater than the threshold of 0.8 for a large effect [9]. The determined difference
between the individual colored codings and the original task board design has
large practical relevance.
The results of the third experiment also show a significant difference in the
task completion time for the original task board design (M= 15.5s, SD = 7.5s)
and the changed card sizes (M= 21.3s, SD = 7.9s); t(9) = 2.41, p = 0.04.
Consequently, we can reject H0. This leads to the insight that changed card
sizes increase the required time for identifying a particular story card compared
to the original task board. The calculated effect size dis 0.76 and thus close to
the threshold of 0.8. We identified a difference between changed card sizes and
the original task board design that has almost large practical relevance.
4.4 Interpretation
Our findings provide insights with respect to the influence of task board cus-
tomizations in comparison with an original task board design. Whereas modi-
fied structures shorten time to identify a particular story card, individual colored
codings and changed card sizes increase the required time.
The performed t-tests substantiate that there is a statistically significant dif-
ference between the respective task board customization and the original task
board design. Our results indicate that customizing a task board’s structure sup-
ports its usage. In case of customization, agile teams should focus on adjusting
the structure of a task board according to their needs. Since this customization
supports the work of new team members who are unfamiliar with the task board,
we assume that a whole team will also benefit from it. Such a customized task
board provides a fast and easy project overview for everyone, i.e. the team and
team externals respectively new team members. Thus, the task board complies
more precisely with the agile practice informative workspace.
However, according to our results, not every customization is beneficial for
a task board’s usage. Adjustments on story cards such as individual colored
codings or changed card sizes lead to an increased amount of time to identify
a particular story card. Even though these two kinds of customization do not
necessarily support a task board’s usage, they are extensively applied in practice
by agile teams [15,25–27]. Therefore, our findings are in conflict with the observed
widely distributed use of these customizations.
In total, we identified a statistically significant difference in each of the three
experiments. Each difference indicates that one of the two compared task board
designs (customized vs. original) allows identifying a particular story card faster.
All findings have an almost large effect size dthat emphasizes their practical rel-
evance. According to our results, modified structures should be preferred com-
pared to the original task board design, which is, in turn, preferable to individual
colored codings and changed card sizes. Hence, the original task board design is
a good solution. In case of customization, however, agile teams may be better
served by adjusting their task board’s structure instead of its story cards. As an
answer to our research question, we can summarize:
A: We identified that only the modified structures allow identifying a particu-
lar story card faster compared to the original task board design. Both of the
other customizations result in an increased amount of time. Hence, adjust-
ing a task board’s structure is the only beneficial option of all investigated
customizations.
4.5 Threats to Validity
In the presented eye tracking study, we considered threats to construct, external,
internal and conclusion validity corresponding to Wohlin et al. [30].
Construct validity: We selected the content for the story cards from a com-
pleted software project. All task boards (see Fig. 1) were based on this content.
Thus, we have a mono-operation bias since we only use one dataset for the task
boards’ content. As a consequence, the constructed task boards do not convey
a comprehensive overview of the task boards’ complexity in practice. However,
we expected that the amount of 40 handwritten story cards and their different
arrangement on each task board result in sufficient realistic complexity for the
participants. Another threat to validity was the participants falsely reporting
having finished. Our experiments required the exact measuring of the task com-
pletion time. However, people are afraid of being evaluated and they are inclined
to convey the impression of being better than they really are. Therefore, this hu-
man tendency endangered the outcome of our experiment. We counteracted this
threat by using eye tracking combined with an additional acoustical statement of
the participants when they identified the particular story card. Thus, we could
determine the exact task completion time of each participant beyond doubt.
The single use of eye tracking is a further threat to validity. This mono-method
bias is problematic since it only allows a restricted explanation of our findings.
However, we focused on an objective measure instead of a subjective one since
objective measures can be reproduced more easily and are thus more reliable.
The given task of identifying a particular story card caused an interaction of
testing and treatment. The comparison of task boards with the given task could
imply to find the story card as fast as possible. Even though we did not mention
to measure task completion time, the participants could be aware of the time as
a factor. Instead of understanding the task board designs, they could only have
tried to be as fast as possible. We mitigated this threat to validity by using eye
tracking. Thus, we could observe how the participants examine the task boards
and make sure that all of them took the respective design into account.
External validity: The choice of involving almost graduated students as
participants, and the use of data from a completed software project produced a
good level of realism. At the same time, the experimental setting endangered the
external validity since the environment was different from the real world. None
of the task boards had true pragmatic value for the participants since none of
them had a genuine working task with the task board. Future evaluation should
be done on real industry projects with team members that truly work with the
task board.
Internal validity: In our eye tracking study, the three experiments were
distributed over three months altogether. This large period of time could have
an effect on the participants’ motivation to contribute to our study. However, we
could not compare all task board designs within one experiment due to the use
of eye tracking, which is time-consuming as well as exhausting for the partici-
pants. A single session with one participant required as much as 25 minutes for
the comparison of two task board designs. Additionally, we could mitigate pos-
sible learning effects since all task board designs equaled one another except for
exactly one specific difference with respect to the corresponding customization.
Conclusion validity: We decided to use eye tracking to improve the re-
liability of our results since an objective measuring is easier to reproduce and
it is more reliable than a subjective one. Additionally, we only selected stu-
dents as participants who were close to their graduation. Hence, they form a
more homogeneous group which counteracts the threat of erroneous conclusions.
Therefore, we mitigated the risk that the variation due to the subjects’ random
heterogeneity is larger than due to the investigated task board designs.
5 Discussion
This presented work investigates the task board as one implementation of the
agile practice informative workspace and the benefit of task board customization.
Although agile teams use task boards in a similar manner, they tend to cus-
tomize their task boards according to their needs. Combined customizations such
as modified structures, individual colored codings, and changed card sizes lead
to complexity, which impedes a task board’s maintenance and comprehensibility.
The increased effort is in conflict with a task board’s underlying agile practice
of fast and easy project overview for everyone, i.e. the team and team externals
respectively new team members. We performed an eye tracking study to analyze
whether there is a significant speed difference in the time required to identify a
particular story card between the original task board design and the respective
task board customization.
We contribute the insight that only modified structures improve a task board’s
usage. In contrast, individual colored codings and changed card sizes did not im-
prove performance beyond the original design.
The modified structures are the only beneficial customization. We assume
that the additional rows improve the arrangement of the story cards. Spatially
close object seems to be grouped since they are perceived as belonging to each
other. This effect is called law of proximity, which is part the Gestalt Principles
[16]. The additional rows influence the story card’s visual appearance by posi-
tion without further support. The story cards’ improved proximity simplifies a
viewer’s consideration of the task board. This finding can help agile teams to
rethink their task board in order to improve it. They may be better served by
focusing on the original task board design and by only adjusting its structure
according to their needs. Thus, they can create a task board which complies
more precisely with the concept of fast and easy project overview for everyone.
In contrast, individual colored codings and changed card sizes are not ben-
eficial. The missing benefit of individual colored codings is caused by a coun-
teracting effect of combined laws of the Gestalt Principles. According to the
law of similarity, using colors for similar objects supports the visual appearance
of belonging together. At the same time, the story cards’ spatial arrangement
complies with the law of proximity. The colored markers are more difficult to
perceive since the law of proximity dominates the law of similarity. Therefore,
individual colored codings do not provide a benefit for customizing a task board.
The changed card sizes are not an improving task board customization, either.
According to our results, a viewer’s effort increases by considering and recogniz-
ing smaller story cards to identify a particular one. Smaller story cards are more
difficult to perceive and read, which complicates a task board’s clarity. Thus,
changed card sizes provide no benefit, either.
The impact of the different task board designs on the performance of a single
team is low. Even if a team member identifies 120 times a day a story card with
an average saving of 5 seconds per identification, his total saving would only be
10 minutes per workday. The benefit of our results is the finding that the original
design of a task board by Cohn [10] with its underlying agile practices consti-
tutes already a good solution for a single team to be productive. Even though
agile approaches offer corresponding degrees of freedom for customization, in the
worst case each team of a company has its own specific task board design. Due to
the wide variety of customization options, the individual task boards complicate
the collaboration across teams and the work of team externals. Thus, the col-
laboration performance of multiple teams, as well as the work of team externals,
can be improved by focusing on one consistent and beneficial task board design.
All in all, we can conclude that not each kind of task board customization is
beneficial. Based on our findings, we agree with Berczuk [7]: Teams are better
served by adjusting their task boards as little as possible. As a consequence, agile
teams should rethink their current task board design with respect to the applied
customizations. The original task board design (see Fig. 1a) is already a good
solution. However, if customization is desired, teams should focus on adjusting
a task board’s structure since only this kind of customization improves the use
of a task board according to our results.
6 Conclusion
This work contributes the insight that not every kind of task board customiza-
tion is beneficial. Agile teams tend to extensively customize their task boards
according to their needs [27]. However, the use of modified structures, individual
colored codings, and changed card sizes impede work with a task board. Thus,
task board customization is in conflict with the agile practice of fast and easy
project overview for everyone, i.e. the team and team externals.
We performed an eye tracking study consisting of three separate experiments
comparing an original task board design with each of three identified major
task board customizations. Based on these results, we identified statistically
significant differences in all three experiments. These findings show that modified
structures such as additional rows support a task board’s usage with respect to
the used exemplary main task of identifying a particular story card. In contrast,
individual colored codings and changed card sizes do not improve performance
beyond the original design.
Our work points to the conclusion that agile teams should rethink their cur-
rent task board design. They may be better served by focusing on the original
task board design and applying carefully selected adjustments. In case of cus-
tomization, teams should adjust the task board’s structure since this is the only
beneficial kind of customization. Additionally, such a customized task board
design complies more precisely with its implemented agile practice.
Acknowledgment
This work was supported by the German Research Foundation (DFG) under
ViViReq (2017 – 2019). We follow ethical guidelines of the Central Ethics Com-
mission of our university. They regulate subject information and rights. Since rec-
ognizable persons should not be visible on distributed video, our data is archived
internally for future reference.
References
1. Ahrens, M., Schneider, K., Kiesling, S.: How Do We Read Specifications? Experi-
ences from an Eye Tracking Study. In: Requirements Engineering: Foundation for
Software Quality. Springer International Publishing (2016)
2. Ali, N., Sharafl, Z., Gueheneuc, Y.G., Antoniol, G.: An Empirical Study on Re-
quirements Traceability Using Eye-tracking. In: 28th IEEE International Confer-
ence on Software Maintenance. IEEE, Piscataway, NJ (2012)
3. Azizyan, G., Magarian, M.K., Kajko-Matsson, M.: Survey of Agile Tool Usage and
Needs. In: Agile Conference. IEEE, Piscataway, NJ (2011)
4. Babik, L., Sheridan, R.: Breaking Down Walls, Building Bridges, and Takin’ Out
the Trash, https://www.infoq.com/articles/agile-team-spaces
5. Beck, K., Andres, C.: Extreme Programming Explained: Embrace Change.
Addison-Wesley, Boston, 2. edn. (2007)
6. Beck, K., Beedle, M., Van Bennekum, A., Cockburn, A., Cunningham, W., Fowler,
M., Grenning, J., Highsmith, J., Hunt, A., Jeffries, R., et al.: Manifesto for Agile
Software Development (2001)
7. Berczuk, S.: Back to Basics: The Role of Agile Principles in Success with an Dis-
tributed Scrum Team. In: Agile Conference. IEEE, Los Alamitos, Calif. (2007)
8. Cockburn, A.: Agile Software Development: The Cooperative Game. Addison-
Wesley, Upper Saddle River, NJ, 2. edn. (2009)
9. Cohen, J.: A Power Primer. Psychological Bulletin 112(1) (1992)
10. Cohn, M.: Agile Estimating and Planning. Prentice Hall PTR, Upper Saddle River,
NJ, 12. edn. (2012)
11. Gross, A., Doerr, J.: What Do Software Architects Expect from Requirements
Specifications? Results of Initial Explorative Studies. In: 1st IEEE International
Workshop on the Twin Peaks of Requirements and Architecture. IEEE, Piscataway,
NJ (2012)
12. Gross, A., Doerr, J.: What You Need is What You Get!: The Vision of View-Based
Requirements Specifications. In: 20th IEEE International Requirements Engineer-
ing Conference. IEEE, Piscataway, NJ (2012)
13. Hajratwala, N.: Task Board Evolution. In: Agile Conference. IEEE, Piscataway,
NJ (2012)
14. Katsma, C., Amrit, C., van Hillegersberg, J., Sikkel, K.: Can Agile Software Tools
Bring the Benefits of a Task Board to Globally Distributed Teams? In: Advances
in Global Sourcing. Models, Governance, and Relationships. Springer (2013)
15. Liskin, O., Schneider, K., Fagerholm, F., M¨unch, J.: Understanding the Role of
Requirements Artifacts in Kanban. In: 7th International Workshop on Cooper-
ative and Human Aspects of Software Engineering. Association for Computing
Machinery, Inc, New York, NY (2014)
16. Palmer, S.E.: Vision science: Photons to phenomenology. MIT Press (1999)
17. Perry, T.: Drifting Toward Invisibility: The Transition to the Electronic Task
Board. In: Agile Conference. IEEE, Los Alamitos, Calif. (2008)
18. Petre, M., Sharp, H., Freudenberg, S.: The Mystery of the Writing that isn’t on
the Wall: Differences in Public Representations in Traditional and Agile Software
Development. In: 5th International Workshop on Cooperative and Human Aspects
of Software Engineering. IEEE, Piscataway, NJ (2012)
19. Pietri, W.: An XP Team Room, http://scissor.com/resources/teamroom/
20. Pikkarainen, M., Haikara, J., Salo, O., Abrahamsson, P., Still, J.: The Impact of
Agile Practices on Communication in Software Development. Empirical Software
Engineering 13(3) (2008)
21. Pries-Heje, L., Pries-Heje, J.: Why Scrum Works: A Case Study from an Agile
Distributed Project in Denmark and India. In: Agile Conference. IEEE, Piscataway,
NJ (2011)
22. Rubart, J., Freykamp, F.: Supporting Daily Scrum Meetings with Change Struc-
ture. In: Proceedings of the 20th ACM Conference on Hypertext and Hypermedia.
ACM, New York, NY (2009)
23. Santos, M., Gralha, C., Goul˜ao, M., Ara´ujo, J., Moreira, A., Cambeiro, J.: What
is the Impact of Bad Layout in the Understandability of Social Goal Models? In:
IEEE 24th International Requirements Engineering Conference (2016)
24. Schwaber, K., Beedle, M.: Agile Software Development with Scrum. Prentice Hall,
Upper Saddle River, NJ (2002)
25. Sharp, H., Robinson, H., Segal, J., Furniss, D.: The Role of Story Cards and the
Wall in XP Teams: A Distributed Cognition Perspective. In: Agile Conference.
IEEE, Los Alamitos, Calif. (2006)
26. Sharp, H., Robinson, H.: Collaboration and Co-ordination in Mature eXtreme Pro-
gramming Teams. International Journal of Human-Computer Studies 66(7) (2008)
27. Sharp, H., Robinson, H., Petre, M.: The Role of Physical Artefacts in Agile Soft-
ware Development: Two Complementary Perspectives. Interacting with Computers
21(1-2) (2009)
28. Sutherland, J., Downey, S., Granvik, B.: Shock Therapy: A Bootstrap for Hyper-
Productive Scrum. In: Agile Conference. IEEE, Piscataway, NJ (2009)
29. Wake, B.: A Gallery of Team Rooms and Charts, http://xp123.com/articles/
a-gallery-of-team- rooms-and-charts/
30. Wohlin, C., Runeson, P., H¨ost, M., Ohlsson, M.C., Regnell, B., Wessl´en, A.: Ex-
perimentation in software engineering. Springer, Berlin (2012)
... Santos et al. [42] evaluated the effect of layout guidelines for i* goal models on novice stakeholders' ability to understand and review such models. Karras et al. [27] compared the original task board design with three customized ones. They investigated the impact of the different design alternatives on developers' work with and comprehension of a task board. ...
Conference Paper
[Context:] The descriptions of interactions and system functions are two of the most important artifact types in requirements specifications. Their common notations are use cases and requirements which are related to each other. There are different variants to link a use case with its associated requirements due to a wide variety of use case templates. The main purpose of all linking variants is to highlight the interrelationships between use cases and requirements. Besides considering both artifacts for themselves, a reader needs to interrelate them to achieve a high understanding of the overall content. [Objective / Method:] Due to the effort to create and maintain links, we investigated the impact of different linking variants on the reading behavior in an eye tracking study with $15$ subjects. [Results:] Our findings indicate that all investigated variants cause comparable visual effort and share the most frequent sequential reading pattern. In all cases, the use case was read first and then the requirements. Nevertheless, the different variants result in divergent reading behaviors. Especially, links embedded in the table of a use case description significantly increase the number of attention switches from the use case to the requirements. [Conclusion:] These attention switches represent the reading behavior of interrelating the use case and the associated requirements which only occurred in case of the most detailed linking variant.
Preprint
Full-text available
A wide variety of use case templates supports different variants to link a use case with its associated requirements. Regardless of the linking, a reader must process the related information simultaneously to understand them. Linking variants are intended to cause a specific reading behavior in which a reader interrelates a use case and its associated requirements. Due to the effort to create and maintain links, we investigated the impact of different linking variants on the reading behavior in terms of visual effort and the intended way of interrelating both artifacts. We designed an eye tracking study about reading a use case and requirements. We conducted the study twice each with 15 subjects as a baseline experiment and as a repetition. The results of the baseline experiment, its repetition, and their joint analysis are consistent. All investigated linking variants cause comparable visual effort. In all cases, reading the single artifacts one after the other is the most frequently occurring behavior. Only links embedded in the fields of a use case description significantly increase the readers' efforts to interrelate both artifacts. None of the investigated linking variants impedes reading a use case and requirements. However, only the most detailed linking variant causes readers to process related information simultaneously.
Conference Paper
Full-text available
Requirements traceability (RT) links help developers to understand programs and ensure that their source code is consistent with its documentation. Creating RT links is a laborious and resource-consuming task. Information Retrieval (IR) techniques are useful to automatically recover traceability links. However, IR-based approaches typically have low accuracy (precision and recall) and, thus, creating RT links remains a human intensive process. We conjecture that understanding how developers verify RT links could help improve the accuracy of IR-based approaches to recover RT links. Consequently, we perform an empirical study consisting of two controlled experiments. First, we use an eye-tracking system to capture developers' eye movements while they verify RT links. We analyse the obtained data to identify and rank developers' preferred source code entities (SCEs), e.g., class names, method names. Second, we use the ranked SCEs to propose two new weighting schemes called SE/IDF (source code entity/inverse document frequency) and DOI/IDF (domain or implementation/inverse document frequency) to recover RT links combined with an IR technique. SE/IDF is based on the developers preferred SCEs to verify RT links. DOI/IDF is an extension of SE/IDF distinguishing domain and implementation concepts. We use LSI combined with SE/IDF, DOI/IDF, and TF/IDF to show, using two systems, iTrust and Pooka, that LSIDOI/IDF statistically improves the accuracy of the recovered RT links over LSITF/IDF.
Conference Paper
[Context and motivation] Writing good specifications is difficult and takes time. There are several guidelines such as the Volere template to assist writing a good specification. They provide a table of contents which can be used like a checklist to consider all relevant aspects. Voluminous specifications take more time to write, and also more time to read. A larger specification is not always a better one. [Question/Problem] A requirements engineer should be aware of how readers make use of a specification and consider their interests in writing it. In addition, some people prefer reading on a screen while others hold a preference for paper printouts. Some parts or aspects may be read differently in both representations. [Principal ideas/results]: We have conducted an Eye Tracking study investigating how specifications are read. We compared paper-based with on-screen presentation, and different reading perspectives such as UI designers, tester, software architects etc. We derived study goals by using GQM down to the level of quantitative and statistical eye tracking analyses. [Contribution]: There is a two-fold contribution: (a) Observations and findings about the way specifications are read; e.g., we had expected paper-based reading to be faster. Instead, we found similar reading patterns on paper versus on screen. (b) Insights with respect to eye tracking as a research method for requirements engineering. We discuss strengths and shortcomings, and provide lessons learned.
Conference Paper
Much of the knowledge used within an XP team is tacit, i.e. it is hidden and intangible. Two tangible artefacts that carry information about the team's work are the index cards which capture stories and tasks to be implemented and the wall where they are displayed (which we refer to as the 'Wall). It is widely acknowledged that these are key elements supporting the work of the XP team, but no systematic investigation of their role has been reported to date. In this paper, we focus on the use of these artefacts within one XP team. We use distributed. cognition, a framework for analysing collaborative work, to explicate the information flows in, around and within the team that are supported by the index cards and the Wall. We then interrogate the models produced using this analysis to answer 'what if' questions.
Chapter
The experiment data from the operation is input to the analysis and interpretation. After collecting experimental data in the operation phase, we want to be able to draw conclusions based on this data. To be able to draw valid conclusions, we must interpret the experiment data.
Conference Paper
Software requirements specifications play a crucial role in software development projects. Especially in large projects, these specifications serve as a source of communication and information for a variety of roles involved in downstream activities like architecture, design, and testing. This vision paper argues that in order to create high-quality requirements specifications that fit the specific demands of successive document stakeholders, our research community needs to better understand the particular information needs of downstream development roles. In this paper, the authors introduce the idea of view-based requirements specifications. Two scenarios illustrate (1) current problems and challenges related to the research underlying the envisioned idea and (2) how these problems could be solved in the future. Based on these scenarios, challenges and research questions are outlined and supplemented with current results of exemplary user studies. Furthermore, potential future research is suggested, which the community should perform to answer the research questions as part of a research agenda.
Conference Paper
This paper describes the creation and evolution of various styles of task boards. It contains examples of how to create a board for both a greenfield project and an established legacy project. Additionally, it describes the forces which caused these boards to evolve over time.
Conference Paper
Software requirements specifications (SRS) serve as an important source of information for software architects with regard to deriving suitable architectural design decisions. However, from a software architect's viewpoint, using these documents efficiently and effectively is often difficult. One could attribute these observations to the fact that SRS also have to address and satisfy the information needs of other document consumers involved in downstream activities like interaction design, user interface design or testing - which is, indeed, very challenging for requirements engineers. In this position paper, we present goals and initial results of explorative studies aimed at investigating information needs that should be fulfilled in SRS from the viewpoint of software architects. In these studies, we gained first insights into the relevance of certain artifact types (like descriptions of interactions or system functionalities) and their suitable representation in terms of notations. Furthermore, the analysis of these initial results revealed variances within the group of software architects regarding information needs. This has motivated the planning and conduction of further studies in the near future that will investigate factors such as expertise or individual motivation, which might influence the information needs from software architects' viewpoints.