Conference PaperPDF Available

From DigiQuilt to DigiTile: Adapting educational technology to a multi-touch table

Authors:

Abstract and Figures

To realize the potential of multi-touch tables, interaction designers need to create meaningful applications for them in real-world contexts. One convenient shortcut towards that end is adapting a meaningful application from another interface paradigm. In this paper, we detail the process of adapting DigiQuilt, a single-user desktop educational technology, to DigiTile, a collaborative multi-touch application. With this case study, we concretely demonstrate the utility of adapting and how previous research and theory can inform that process. In particular, we show how learning theory (1) motivated the transition from the desktop to the multi-touch table, (2) guided the design process, and (3) informed the evaluation.
Content may be subject to copyright.
From DigiQuilt to DigiTile:
Adapting Educational Technology to a Multi-Touch Table
Jochen Rick and Yvonne Rogers
The Open University
Department of Maths, Computing, and Technology
Walton Hall, Milton Keynes, MK7 6AA, UK
{j.rick, y.rogers}@open.ac.uk
Abstract
To realize the potential of multi-touch tables, interaction
designers need to create meaningful applications for them
in real-world contexts. One convenient shortcut towards
that end is adapting a meaningful application from another
interface paradigm. In this paper, we detail the process of
adapting DigiQuilt, a single-user desktop educational tech-
nology, to DigiTile, a collaborative multi-touch application.
With this case study, we concretely demonstrate the utility of
adapting and how previous research and theory can inform
that process. In particular, we show how learning theory (1)
motivated the transition from the desktop to the multi-touch
table, (2) guided the design process, and (3) informed the
evaluation.
1 Introduction
Each new interface requires studies of how the technol-
ogy can best be appropriated. Early work tends to focus
on technical innovations and basic usability. “Demo apps”
that demonstrate the features and potential of the new UI
are created. Increasingly, more sophisticated applications
are developed to solve real world problems in ecologically-
valid settings.
Interaction designers are tasked with creating these ap-
plications. Do they start from scratch? Or, can they adapt
existing applications, developed for other user interfaces, to
the new hardware? Here, we make the case for the latter.
We detail our own success in adapting DigiQuilt, a single-
user desktop application, into DigiTile, a multi-user multi-
touch tabletop application.1
1We chose to change the name for the multi-touch version for two rea-
sons. First, it makes it easier to distinguish between the two in writing.
Second, quilting is seen differently in the U.S., where DigiQuilt was con-
ceived, and the U.K., where DigiTile is used.
2 Adapting Applications
Adapting is one of the best and most natural ways we
have to understand novel interfaces. A novel interface is a
new medium. As with any new medium, it is difficult to
predict its affordances. One of the best practices for un-
derstanding a new medium is remediation—the practice of
taking techniques and practices that work in one domain
and applying them to the new domain [1]. Many new me-
dia are created and explored in this matter. For instance,
early sound recording was based on a parallel to written
text; Edison envisioned his phonograph as writing sounds
[4]. By applying the practices of the better understood
medium, a better understanding of the new medium can be
achieved. With experience and innovation, new uses par-
ticularly suited to the new medium emerge. So, while Edi-
son envisioned a machine that could both read and write
sounds, recorded sound technology came into prominence
with the application of pre-recorded musical records. Based
on these principles, a particularly fruitful way to investigate
new user interfaces is to apply ideas from already existing
applications to the new interface.
Adapting an application also has several practical advan-
tages. First, the plethora of existing applications provide
a good source of inspiration for design. Even if only one
percent of all desktop applications would make for viable
multi-touch table applications, there would still be plenty
of worthy candidates, given the large amount of desktop ap-
plications. Much of good design is appropriating previous
good ideas [11]. A successful application embodies several
working ideas. Adaptation allows the designer to build on
these.
Second, adapting can shorten development time by lever-
aging on the previous design work. A speedy development
cycle can be critical to research on novel interfaces. It can
be difficult for application-based research to keep up with
new hardware innovations; by the time an application has
been built for specific hardware, it is out of date. There
is only a limited amount of time when research results can
have an important impact. If the novel interface becomes
commercially viable, work on the novel interface can move
out of the research domain before significant results can be
achieved. Adapting an existing application can cut that de-
velopment time down significantly, because the designer of
the new application can appropriate the design work put into
the existing application.
Third, the existing application and the adapted applica-
tion can inform each other. For novel user interfaces, a com-
parison to a similar application for an established interface
can provide researchers a point to compare and contrast.
Thus, adapting can also aide in evaluating the new interface.
While adapting has its advantages, there are also chal-
lenges that come with adapting to a new interface. Depend-
ing on the nature of the application, overcoming these chal-
lenges can range from easy to difficult. For the easy case,
consider using a tabletop for baggage screening. In baggage
screening, the user is trying to identify dangerous objects,
such as guns, from an x-ray image. Having multiple users
view the image from different angles can help increase ac-
curacy [3]. To adapt this task to a multi-touch table is trivial:
The display just needs to be placed horizontally. This works
well, because the interaction (clearing the bag or signaling
for a hand search) does not change much (if at all). For ap-
plications where the graphical user interface does not need
to change, middle-ware that maps touch input into desktop
commands, such as mouse movements, can speed up the
adaption process [28].
Where some applications are easy to adapt, others re-
quire serious design and development work. While reme-
diation works to a certain extent, the remediated practice
is usually different to the original practice [1]. Changing
the interfaces changes the activity. For these adaptations,
guiding theory of the application domain and previous re-
search results can inform the process. First, these sources
can provide motivation for adapting a specific application.
Second, they can inform the concrete design. Third, they
can help develop criteria for evaluation. In this paper, we
concretely demonstrate how learning theory and previous
research informed these three phases of adapting DigiQuilt
to a multi-touch table.
3 From DigiQuilt to DigiTile: A Case Study
DigiTile was created for the ShareIT project,2which
aims to further the understanding of novel interfaces for
supporting collaboration in real-world situations. One path
for supporting collaboration with computers is single dis-
play groupware [26], where multiple people can simulta-
neously interact with a shared display. This user paradigm
2http://shareitproject.org
is being supported by a number of new technologies. One
such technology is multi-touch tabletop computers. Table-
top computers have a wide design space, especially when
part of a larger ubiquitous computing infrastructure [19].
Collaboration tables [23], tabletop computers that allow
multiple people to interact, are still a fairly novel user in-
terface. Hardware innovations are still being published on a
regular basis [6, 7, for example]. Yet, a reasonable hardware
basis exists for conducting HCI research. For example,
MERL’s DiamondTouch [2] has been actively researched
since 2001. Software toolkits, such as DiamondSpin [24]
and reacTIVision [9], help developers overcome some of
the more vexing problems of developing software for these
platforms. With this hardware and software basis, research
on tabletop computing can focus on creating applications.
Yet, designing applications from scratch can be a difficult
and time-consuming task.
To shortcut this process, we adapted an existing applica-
tion, DigiQuilt, to study the potential of multi-touch table-
tops to support collaboration in an authentic setting. In par-
ticular, DigiTile was designed to support learning, a task
where collaboration can often be beneficial. Designing
computer-based learning environments is an established re-
search field with established criteria [25] for success. While
multi-touch table technology is relatively new, there have
also been published research on users using multi-touch
tabletops for learning [14, 16, for example]. Finally, there
is the previous research on DigiQuilt. DigiTile drew on all
of these areas.
This case study first introduces DigiQuilt as an interest-
ing educational technology. Next, it motivates adapting Di-
giQuilt to a multi-touch table. Then, it details the signifi-
cant design decision in creating DigiTile and the rationale
behind those decisions. Finally, the plans for evaluating Di-
giTile are discussed.
3.1 DigiQuilt
DigiQuilt is a construction kit [17] for learning about
math and art by designing patchwork quilt blocks [13]. It
is based on the the instructional theory of constructionism,
which holds that people learn particularly well when de-
signing personally-meaningfully public artifacts [15]. In
constructionism, learners are motivated to construct the ar-
tifact, because it is meaningful to themselves and because
it can be shared with others. In DigiQuilt, third and fourth
graders assemble colored pieces into a square quilt block
(Figure 1).
Clicking a color button in the palette (1), turns the pieces
at (2) that color. Clicking on a piece creates a copy of
that piece, which can be dragged into the quilt block (3)
or into the work area (4). In the work area, pieces can
be rotated and assembled into more complex pieces. The
1
2
3
4
5 6
7
Figure 1. Using DigiQuilt to create a quilt
block design (actual 3rd grader’s design)
complex pieces can then be copied and transferred into the
quilt block. The software keeps a complete history of the
quilt block, which learners can navigate (6) to revert un-
wanted changes. Once finished, the design can be saved (5).
Saved designs can be opened and edited. Designs can be
printed out to be displayed in the classroom, used as wrap-
ping paper, etc. In user studies, children enjoyed creating
designs with DigiQuilt and proudly showed them to their
classmates, teacher, and family [12].
In addition to being aesthetically pleasing, the patch
work quilts also lend themselves to mathematical analysis.
The designs embody fraction concepts and are often sym-
metric. For instance, the design in Figure 1 is half red and
half yellow; it is also diagonally symmetric. To highlight
the learning goals, learners are given increasingly difficult
challenges to accomplish, such as creating a design that is
half red or creating a design that is horizontally symmet-
ric. To help learners reflect on fractions, the fraction of the
whole that a certain color covers is displayed next to that
color (Figure 1, Point 1). To help learners reflect on sym-
metry, different grids (Figure 1, Point 7) can be overlaid on
the quilt. These tools help learners reflect on the targeted
concepts.
3.2 Motivation
The previous section introduced DigiQuilt as a success-
ful and compelling single-user desktop application; how-
ever, that by itself is not enough to justify adapting it to a
multiple-user tabletop application. Adapting to a new in-
terface requires examining both the application and at the
implementation level. Could this application benefit signif-
icantly from the move to the new interface? If not, why
adapt. How hard is it to implement the transition to the new
interface? Even if the application would be compelling, im-
plementation issues might make it impractical.
3.2.1 Application
The primary reason to adapt DigiQuilt to a multi-touch ta-
ble is the potential to support collaborative learning. Col-
laboration is an established method for enhancing learning.
Single display groupware has been shown to better sup-
port collocated collaborative learning than taking turns on
single-user application [8].
One relevant theory of collaborative learning is
Roschelle’s [21] theory of convergent conceptual change
when two learners work together with a reflective tool (a
tool that responds to user input to reflect the embedded do-
main concepts), they tend to converge on an understand-
ing that is better than either would achieve independently.
When learners work together on a challenge, they naturally
articulate how they would solve the challenge, based on
their understanding. If their strategies clash, it triggers them
to justify their understanding. The reflective tool allows the
learners to demonstrate or test their understanding. Thus, an
understanding can be confirmed or rejected. As they work
on the challenge together, learners’ conceptual understand-
ings do not just converge with each other, but also with the
domain concepts embodied in the tool.
DigiQuilt is a reflective tool, giving feedback when
changes are made (e.g., the fraction updates when a new
piece is placed). So, we expect that two learners working
with DigiQuilt to engage in convergent conceptual change.
There is also practical evidence that collaboration might be
warranted: DigiQuilt users often make comments on oth-
ers’ designs and sometimes want to take over the controls
to show something [12].
Roschelle’s model of collaboration is particularly com-
pelling, because it shows how collaboration between equals
can benefit both partners. On the flip side, if collaborators
are not evenly skilled, as is often the case in a classroom
setting, a more appropriate model of collaboration may be
that of Vygotsky’s [29], where a more skilled person helps
a less skilled person gain competence. No matter which
model of collaboration (even or uneven) is appropriate to
a specific pair, we would still expect the collaboration to
benefit learning.
3.2.2 Implementation
While implementation is a challenge when actually adapt-
ing, it is worth considering beforehand. If there is not a
good fit between the source application and the new inter-
face, it may be difficult to successfully adapt. The interface
may prove awkward or the application may lose its com-
pelling nature in the process.
DigiQuilt’s existing interface is a good match for a multi-
touch table. It is a fairly simple application with a small
number of user actions. The primary action is to move
pieces to the work area and quilt block. While this is easy
enough with a mouse, simply dragging the shapes with a
finger on a multi-touch table could be an even better phys-
ical mapping. Early prototypes of DigiQuilt were carried
out with physical shapes on paper. While this physical ap-
proach had significant drawbacks,3moving pieces by hand
was an intuitive interface [12]. Besides dragging pieces, the
main interactions in DigiQuilt are pushing buttons, some-
thing that translates directly to a multi-touch table. Digi-
Quilt does make use of the keyboard to name designs. This
small amount of typing can be achieved with a virtual on-
screen keyboard (see Figure 4).
3.3 Design: To DigiTile
As outlined above, the basic interactions of DigiQuilt are
well supported by a multi-touch table, but there are still im-
portant design decisions to be made in adapting. In this sec-
tion, we detail how learning theory and previous research
helped us negotiate these design challenges. We organize
the subsections around design features well documented in
the literature on multi-touch tables—group / table size, ori-
entation, and collaboration.
3.3.1 Group / Table Size
A first problem to consider is group size and table size [22].
Based on the theory of convergent conceptual change, two
is the ideal number of users: Two can challenge and help
each other without spending undue amount of time coordi-
nating.
We wanted the learners to collaborate on one design, so
the table needed to be small enough to allow intended users
(10–12 year-olds) to both reach every part of a central tile.
The smaller DiamondTouch table (32 inch diagonal) eas-
ily allowed for these young users to reach almost all of the
table.
3While a physical interface was intuitive, it failed in other ways. First,
it was hard to properly constrain the physical world: Users could easily
overlap tangible pieces in ways that did not allow for fractional analysis.
Second, physical pieces do not allow for several useful operations easily
implemented in software: assembling simple pieces into a complex piece
(such as an L shape), creating copies of these complex pieces, swapping
colors, reverting to an earlier design point, etc. Third, with a large number
of colors and shapes, a physical implementation would require an over-
whelming amount of different pieces.
3.3.2 Orientation
Unlike a desktop display, different users need not have the
same perspective on a tabletop display. They could sit
across from each other, around the corner, or next to each
other. This makes it difficult to maintain the desktop orien-
tation, where eyes up equals the top of the screen and eyes
down means the bottom of the screen [27]. Developing soft-
ware without a dominant orientation can be tricky [24].
For DigiTile, different positions would change how
learners view the tile. If seated around a corner, horizontal
symmetry for one partner would be vertical symmetry for
the other. If seated across from each other, the tile would ap-
pear upside down. For more abstract designs, this would not
be a problem; however, many DigiQuilt designs are based
on real-life objects which have a preferred orientation (e.g.,
a stick figure). While adults tend to favor working across
from each other, children often prefer working next to each
other [23]. So, it made sense to position the learners next
to each other to share the same orientation. This configura-
tion (dyads next to each other) allows DigiTile to maintain
a desktop-style orientation (Figure 2).
3.3.3 Software Implementation
DigiTile is implemented in Squeak [5], a multimedia-
enabled cross-platform open-source Smalltalk. A small
Java application reads DiamondTouch events and broad-
casts them to Squeak via OSC (Open Sound Control).
Squeak then handles the events. While it was not designed
for creating applications for a multi-touch table, it was rel-
atively easy to adapt Squeak for this purpose. For instance,
all Squeak interfaces can be rotated.
More pressing for DigiTile, Squeak allows for multiple
concurrent mouse pointers, which can be a challenge for
some UI toolkits [23]. Mapping touch input to mouse in-
put is not entirely straightforward, since hovering and click-
ing do not work quite the same as for a mouse. In our
mapping, any touch was treated as a mouse down condi-
tion; further movements were treated as mouse drag move-
ments. This simple mapping matched well with DigiQuilt’s
interface, since its main interaction techniques were drag-
ging tile pieces and pressing buttons. Other common mouse
functions, such as pressing secondary buttons and scrolling,
were not necessary.
DigiQuilt itself was implemented in Squeak. While the
source code for DigiQuilt was available, we choose to im-
plement DigiTile from the ground up. Like most software
which has been slowly evolved over an extended time by
several developers, DigiQuilt’s code base had serious prob-
lems. The size of the pieces was hard coded. The model
and view were tightly coupled. The file format was opti-
mized for size, not flexibility. We took this opportunity to
rewrite the application with a cleaner design, being espe-
1
2 3
4
5
6
Figure 2. Using DigiTile to create a tile
cially careful to decouple model and view; thus, different
views for multi-touch application and the desktop applica-
tion can share the same underlying model.
3.3.4 Collaboration
As with DigiQuilt, the main focus of DigiTile (Figure 2) is
the quilt block (1). Since the learners are positioned next
to each other, we placed a palette area on both the left (2)
and right (3) side. Practically, each learner has their own
palette to use. This positioning allows learners to feel like
they have specific ownership over part of the interface. To
encourage sharing, we created a shared work area (4). We
intentionally used an odd number of work zones, so that one
work zone would be in the middle, implying no left or right
ownership. Like in DigiQuilt, the palette buttons are labeled
with the name of the color. In classroom use of DigiQuilt,
the labelling proved to be useful in allowing children to talk
about their designs [12]. Similarly, we believe that these
labels can help aide dialogue (e.g., students can easily dif-
ferentiate between lime and green).
After some user testing, it was obvious that dropping a
piece accidentally was fairly common in the multi-touch
case. In addition, partners often disapproved of a specific
change. For these reasons, navigating the history was more
important for DigiTile than DigiQuilt. We developed a
graphical history (5) that allowed users to easily back up
to a previous design. In contrast, DigiQuilt only provided
undo and redo buttons (Figure 1, Point 6).
While Squeak supports concurrency at the implementa-
tion level, concurrency at the application level is not always
a good idea. For instance, saving an application that is con-
currently being worked on is problematically ambiguous.
To solve this problem, we implemented a menu bar (6).
Figure 3. Selecting a grid to test for symmetry
Figure 4. Saving a tile design
When a menu is pressed, the rest of the interface becomes
unavailable, signified graphically by darkening the applica-
tion area (Figure 3). Other elements of the interface were
relatively insensitive to concurrent interaction. Even if two
tiles were dropped simultaneously, they could be processed
separately. The only practical difference being that two his-
tory positions would have been created, rather than one.
Using the menu, users launch file functions (new, open,
save, etc.) and can apply different grids. Different inter-
faces appear for saving (Figure 4) and opening (Figure 5)
designs. The grid menu enables feedback on symmetry.
When a line of symmetry is selected, the corresponding icon
appears next to the “Grid” menu label; that icon shows a star
when that line of symmetry is valid (see the diagonal icon
in Figure 3).
Because of collaboration, the learning process in Digi-
Figure 5. Opening an existing tile
Tile is different than the learning process in DigiQuilt. To
effect convergent conceptual change, DigiTile use should
encourage different perspectives for different users. To fur-
ther this, we provide different ways for the two learners to
analyze the quilt. DigiQuilt displays the fraction that is
covered by a color on the respective colored button. Di-
giTile allows the user to change the representation from re-
duced fraction to least-common-divisor fraction to percent-
age to visual pie chart. One learner could display his or her
palette with percentages, while the other displays reduced
fractions. This allows for us to create challenges like “a tile
that is 1
2red and 50% yellow. This challenge allows the
learners to discover that 1
2and 50% are the same. Using
multiple-linked representations to represent a mathematical
concept leads to deeper math understanding [10]. This strat-
egy would not work as well in the single-user DigiQuilt.
3.4 Evaluating DigiTile
Since the focus of this paper is the design process, we do
not have room to detail study results. Instead, we show how
learning theory and previous work informed the design of
the evaluation. Previous research on DigiQuilt has already
engaged many of the questions of whether tiling is a useful
medium to engage with mathematics and art. This allows
us to concentrate on questions that arise from adapting the
interface to a multi-touch table. As HCI researchers, we are
interested in how the multi-touch table interface can (better)
support collaborative authoring.
An initial user study was conducted with 10–12 year
olds to investigate whether and how pairs of learners col-
laborate using DigiTile. In particular, we were interested in
whether the kind of reflective dialogue, that Roschelle based
convergent conceptual change on, occurred in this design
task. After a brief familiarization phase, we assigned pairs
Figure 6. Working together on a difficult chal-
lenge
mathematical challenges. The goal was to see how pairs
would collaborate when faced with a difficult challenge—
one that required significant thought and work. Some chal-
lenges proved trivial and were accomplished in less than
a minute with minimum planning and negotiation between
the participants. We increased the difficulty until the task
became challenging. At that point, the pairs had a harder
time forming and trialing approaches. Because both par-
ticipants could interact simultaneously, it was necessary for
one participant to recruit the other in trialing an approach
(Figure 6).
This initial study confirmed that DigiTile could encour-
age the kind of collaboration we were looking for. It also
gave us feedback on how learners dealt with the different
tools (graphical history, fraction representations, grid menu,
etc.) in the full version of DigiTile. Even before this study,
we believed a tailored version, leaving out features not nec-
essary to the challenge at hand, would suit the learning task
better. Yet, we left all features in as the study was largely
formative, allowing us to see how intended users engaged
the tools and how the interface could be improved.
In a follow-up study, we worked with DigiTile in a class-
room, where the teacher felt that DigiTile’s focus on frac-
tions would be appropriate. We worked with the teacher and
an educational specialist to streamline DigiTile for that set-
ting. To reduce complexity, we reduced the number of col-
ors and shapes (Figure 7). Reducing the shapes meant that
the fractions would no longer be as complex. With all the
shapes available, it was fairly easy to place pieces to create
denominators of 256 or 400 (larger than the learners were
familiar with); in the simplified version, denominators are
unlikely to reach above 64 (with a 4x4 grid) or 100 (with a
Figure 7. Using the simplified version for a
training task
5x5 grid). Reducing the number of colors removed colors,
such as cyan, that children were not that familiar with. It
also gave us more space to represent the fractions. As one
of the learning goals in the curriculum was converting from
fraction to decimal representation, we showed both the frac-
tion and decimal representation simultaneously (underneath
the color name).
As researchers, we were still interested in how different
features of the interface would affect the collaboration. One
way to enforce collaboration is to allocated resources un-
evenly. So, we created another version of DigiTile that split
the colors between the users (Figure 8). We also designed
challenges, such as design a tile that is 1
2red and 1
2yel-
low, that take advantage of this split. In this second study,
we will compare the collaboration of the simplified version
with that of the split version. Is the split effective in pro-
moting collaboration? Is there more reflective dialogue in
one case? As of the writing, this study has been conducted,
but the data has not been analyzed.
4 Conclusions
Creating meaningful applications for novel interfaces is
an important challenge for HCI researchers. There are dif-
ferent approaches to solve this problem. For instance, the
Equator Project applied creative exploration as the basis for
development [20]. Here, we made the argument for adapt-
ing an existing application. With the DigiTile case study,
we concretely demonstrated both the benefits and the chal-
lenges in implementing this approach.
While DigiTile is only one example of adapting to a
multi-touch table, we believe many of the issues presented
here are universal. Implementation issues of group / table
Figure 8. Using the split-palette version for a
training task
size, orientation, and collaboration are inherent in the de-
sign space of multi-touch tables. By giving a concrete ex-
ample, we also demonstrate the specific benefits of adapt-
ing. For example, while code reuse may seem like a tempt-
ing benefit of adapting, we show the difficulties in realizing
this. In contrast, design work and research on the source
application was beneficial in motivation, design, and evalu-
ation. DigiQuilt had an extensive design process, including
paper prototypes and classroom-evaluated design iterations
[18]. It took several years to evolve DigiQuilt; it took less
than half a year to create DigiTile.
Adding collaboration changed how the application was
used. Rather than seeing this as a problem, we see this as
an opportunity to utilize the strengths of the new interface.
Multi-touch tables are better suited to collocated collabora-
tion than a desktop setup. As learning can benefit from col-
laboration, DigiTile is more than just a port of DigiQuilt.
The learning scenario and the interface were changed to
take advantage of the multi-touch interface. The represen-
tation of fractions and the role of the challenges were sig-
nificantly changed to suit the new learning paradigm. By
adapting DigiQuilt, we were able to create a novel appli-
cation to support collaborative learning with a multi-touch
table.
5 Acknowledgements
The work is part of the ShareIT project funded by the
EPSRC, grant number EP/F017324/1. We would like to
thank our other collaborators in that project for their help-
ful feedback: (alphabetically) Sheep Dalton, William Farr,
Rowanne Fleck, Amanda Harris, Eva Hornecker, Paul Mar-
shall, Richard Morris, Nadia Pantidi, and Nicola Yuill. Spe-
cial thanks go to K.K. Lamberty for supporting us in adapt-
ing DigiQuilt and Kim Bryant for help in creating the fig-
ures. We thank MERL for loaning us the DiamondTouch.
References
[1] J. D. Bolter and R. Grusin. Remediation: Understanding
New Media. The MIT Press, Cambridge, MA, 1999.
[2] P. Dietz and D. Leigh. DiamondTouch: A multi-user touch
technology. In Proceedings of UIST ’01, pages 219–226,
New York, 2001. ACM Press.
[3] C. Forlines, C. Shen, D. Wigdor, and R. Balakrishnan. Ex-
ploring the effects of group size and display configuration
on visual search. In Proceedings of CSCW ’06, pages 11–
20, New York, 2006. ACM Press.
[4] L. Gitelman. Scripts, Grooves, and Writing Machines: Rep-
resenting Technology in the Edison Era. Stanford University
Press, Stanford, CA, 1999.
[5] M. Guzdial and K. Rose, editors. Squeak: Open Personal
Computing and Multimedia. Prentice Hall, Upper Saddle
River, NJ, 2002.
[6] J. Y. Han. Low-cost multi-touch sensing through frustrated
total internal reflection. In Proceedings of UIST ’05, pages
115–118, New York, 2005. ACM Press.
[7] S. Hodges, S. Izadi, A. Butler, A. Rrustemi, and B. Bux-
ton. ThinSight: Versatile multi-touch sensing for thin form-
factor displays. In Proceedings of UIST ’07, pages 259–268,
New York, 2007. ACM Press.
[8] K. M. Inkpen, W. ling Ho-Ching, O. Kuederle, S. D. Scott,
and G. B. D. Shoemaker. "This is fun! We’re all best friends
and we’re all playing": Supporting children’s synchronous
collaboration. In Proceedings of CSCL ’99, pages 252–259,
Mahwah, NJ, 1999. Lawrence Erlbaum Associates.
[9] M. Kaltenbrunner and R. Bencina. reactivision: A
computer-vision framework for table-based tangible inter-
action. In Proceedings of TEI ’07, pages 69–74, New York,
2007. ACM Press.
[10] J. Kaput. Linking representations in the symbolic systems
of algebra. In S. Wagner and C. Kieran, editors, Research
agenda for mathematics education: Research issues in the
learning and teaching of algebra, pages 167–194. National
Council of Teachers of Mathematics, Reston, VA, 1989.
[11] J. Kolodner. Case-Based Reasoning. Morgan Kaufmann,
1993.
[12] K. K. Lamberty. Getting and Keeping Children Engaged
with a Constructionist Design Tool for Craft and Math. PhD
thesis, Georgia Institute of Technology, Atlanta, GA, May
2007.
[13] K. K. Lamberty and J. L. Kolodner. Exploring digital quilt
design using manipulatives as a math learning tool. In
P. Bell, R. Stevens, and T. Satwicz, editors, Proceedings of
ICLS 2002, pages 552–553, Mahwah, NJ, 2002. Lawrence
Erlbaum Associates.
[14] M. R. Morris, A. M. Piper, A. Cassanego, and T. Wino-
grad. Supporting cooperative language learning: Issues in
interface design for an interactive table. Research Report 8,
Department of Computer Science, Stanford University, Palo
Alto, CA, 2005.
[15] S. Papert. Situating constructionism. In I. Harel and S. Pa-
pert, editors, Constructionism, pages 1–13. Ablex, 1991.
[16] A. M. Piper, E. O’Brien, M. R. Morris, and T. Winograd.
SIDES: A cooperative tabletop computer game for social
skills development. In Proceedings of CSCW ’06, pages 1–
10, New York, 2006. ACM Press.
[17] M. Resnick, A. Bruckman, and F. Martin. Pianos not
stereos: Creating computational construction kits. Interac-
tions, 3(6):41–50, 1996.
[18] J. Rick and K. K. Lamberty. Medium-based design: Extend-
ing a medium to create and exploratory learning environ-
ment. Interactive Learning Environments, 13(3):179–212,
2005.
[19] Y. Rogers, Y.-K. Lim, and W. R. Hazlewood. Extending
tabletops to support flexible collaborative interactions. In
Proceedings of TABLETOP ’06, pages 71–78, Washington,
DC, 2006. IEEE Computer Society.
[20] Y. Rogers, M. Scaife, E. Harris, T. Phelps, S. Price,
H. Smith, H. Muller, C. Randell, A. Moss, I. Taylor, D. Stan-
ton, C. O’Malley, G. Corke, and S. Gabrielli. Things aren’t
what they seem to be: innovation through technology inspi-
ration. In Proceedings of DIS 2002, New York, 2002. ACM
Press.
[21] J. Roschelle. Learning by collaborating: Convergent con-
ceptual change. In T. Koschman, editor, CSCL: Theory
and Practice of an Emerging Paradigm, pages 209–248.
Lawrence Erlbaum Associates, Mahwah, NJ, 1996.
[22] K. Ryall, C. Forlines, C. Shen, and M. R. Morris. Explor-
ing the effects of group size and table size on interactions
with tabletop shared-display groupware. In Proceedings of
CSCW ’04, New York, 2004. ACM Press.
[23] S. D. Scott, K. D. Grant, and R. L. Mandryk. System guide-
lines for co-located, collaborative work on a tabletop dis-
play. In K. Kuutti, E. H. Karsten, G. Fitzpatrick, P. Dour-
ish, and K. Schmidt, editors, Proceedings of ECSCW 2003,
pages 159–178, Netherlands, 2003. Kluwer.
[24] C. Shen, F. D. Vernier, C. Forlines, and M. Ringel. Dia-
mondSpin: an extensible toolkit for around-the-table inter-
action. In Proceedings of CHI ’04, pages 167–174, New
York, 2004. ACM Press.
[25] E. Soloway, M. Guzdial, and K. E. Hay. Learner-centered
design: The challenge for HCI in the 21st century. Interac-
tions, 1(2):36–48, April 1994.
[26] J. Stewart, B. B. Bederson, and A. Druin. Single display
groupware: A model for co-present collaboration. In Pro-
ceedings of CHI ’99, pages 286–293, New York, 1999. ACM
Press.
[27] A. Tang, M. Tory, B. Po, P. Neumann, and S. Carpendale.
Collaborative coupling over tabletop displays. In Proceed-
ings of CHI 2006, New York, 2006. ACM Press.
[28] E. Tse, S. Greenberg, and C. Shen. GSI Demo: Multiuser
gesture / speech interaction over digital tables by wrapping
single user applications. In Proceedings of ICMI ’06, pages
76–83, New York, 2006. ACM Press.
[29] L. S. Vygotsky. Mind in Society: The Development of
Higher Psychological Processes. Harvard University Press,
Cambridge, MA, 1978.
... Works like (Falloon and Khoo, 2014;Mercier et al., 2015;Rick and Rogers, 2008) are examples of studies where single multi-touch displays were used to enhance CPS skills, among others. Rick and Rogers (2008) present a game to learn relationships between mathematics and art on a multi-touch tabletop, and report on it being successful at promoting reflective dialogue in children aged 10-12. ...
... Works like (Falloon and Khoo, 2014;Mercier et al., 2015;Rick and Rogers, 2008) are examples of studies where single multi-touch displays were used to enhance CPS skills, among others. Rick and Rogers (2008) present a game to learn relationships between mathematics and art on a multi-touch tabletop, and report on it being successful at promoting reflective dialogue in children aged 10-12. Mercier et al. (2015) test the effectiveness of the multi-touch display with respect to the usage of paper by comparing the problem solving process of children aged 10-11 in both platforms. ...
Article
Collaborative problem solving (CPS) is an essential soft skill that should be fostered from a young age. Research shows that a good way of teaching such skills is through video games; however, the success and viability of this method may be affected by the technological platform used. In this work we propose a gameful approach to train CPS skills in the form of the CPSbot framework and describe a study involving 80 primary school children on user experience and acceptance of a game, Quizbot, using three different technological platforms: two purely digital (tabletop and handheld tablets) and another based on tangible interfaces and physical spaces. The results show that physical spaces proved to be more effective than the screen-based platforms in several ways, as well as being considered more fun and easier to use by the children. Finally, we propose a set of design considerations for future gameful CPS systems based on the observations made during this study. RESEARCH HIGHLIGHTS • Collaborative problem solving (CPS) is a valuable skill that should be fostered from a young age. • Games are a successful way of training CPS, but the platform used may affect its effectiveness. • We present a framework and a game implemented to foster CPS, and compare its acceptance and user experience with 80 primary school students in three different implementations: tabletops, tablets and physical spaces. • Physical spaces are perceived as easier as and more fun than screen-based sedentary activities, and they are reported as the most desirable to use again both inside and outside the educational. They may also provide additional benefits for CPS enhancement in comparison with purely digital platforms, especially where planning and organization are concerned.
... While Daryanto [17] added that interactive multimedia is media that is equipped with a controller operated by learners to choose the action that they want. Through multimedia tools (visual graphics, animations, audio, and video) and feedback (interactive) provided, learning will become more interesting because the learners feel the multisensory experience thus can increase the students' motivation [18] Some previous studies about interactive learning multimedia showed that it has the potential to engage students in meaningful collaborative learning, thus it significantly affects the correct answer of the students [19], [20], [21], and [22]. Advance interactive display technology has been a growing interest in exploring its use within the educational context [20]. ...
... Informal learning environments, like museums, science centers and aquariums have received attention from the education research community, as supporting social interaction and collaboration is highly related with learning and engagement in informal environments [27] and interactive spaces and surfaces migrate from research lab to learning settings. Already, supporting learning with these technologies has become a vital research area in educational technology [28] and several studies have established that interactive displays have the potential to engage students in meaningful collaborative learning, especially during an informal learning visit [29,30]. ...
Article
Full-text available
Interactive displays are becoming increasingly popular in informal learning environments as an educational technology for improving students’ learning and enhancing their engagement. Interactive displays have the potential to reinforce and maintain collaboration and rich-interaction with the content in a natural and engaging manner. Despite the increased prevalence of interactive displays for learning, there is limited knowledge about how students collaborate in informal settings and how their collaboration around the interactive surfaces influences their learning and engagement. We present a dual eye-tracking study, involving 36 participants, a two-staged within-group experiment was conducted following single-group time series design, involving repeated measurement of participants’ gaze, voice, game-logs and learning gain tests. Various correlation, regression and covariance analyses employed to investigate students’ collaboration, engagement and learning gains during the activity. The results show that collaboratively, pairs who have high gaze similarity have high learning outcomes. Individually, participants spending high proportions of time in acquiring the complementary information from images and textual parts of the learning material attain high learning outcomes. Moreover, the results show that the speech could be an interesting covariate while analyzing the relation between the gaze variables and the learning gains (and task-based performance). We also show that the gaze is an effective proxy to cognitive mechanisms underlying collaboration not only in formal settings but also in informal learning scenarios.
... Students can put objects over the tabletop and control/transform the provided representations. Evans et al. [39] used an interactive tabletop system with both off-the-shelf and custom-built applications for mathematics learning, while Rick and Rogers [40] used a tabletop, named DigiTile Project, in order to help students identify and create relationships between mathematics and art. Two of the three tasks that students had to take, implemented fraction exercises. ...
... children's transition in understanding amounts using their fingers, to dealing with symbolic digits [27]); linking concrete and symbolic representations (e.g. work on virtual quilts and symbolic representations through fractions [29] and DigiTile [48], which associates geometric shapes and fractions); and developing mathematical thinking in more general terms (e.g. investigation of children's ability to conjecture, justify and/or interpret using a virtual scale on a multitouch surface [30]). ...
Conference Paper
Basic numerical competencies developed in kindergarten form the foundations of math achievement. This indicates the importance of early interventions in the case of numerical difficulties. Building on research on math manipulatives and tangible interfaces, we developed a training of basic numerical competencies using an interactive tabletop in combination with physical LEGO-like blocks. In an experiment, we evaluated the effectiveness of the training on children's learning of the partner number concept, basic numerical competencies and number line estimation, compared to a content-wise similar training with physical manipulatives and a human tutor. We observed significant increases in children's understanding of the partner number concept and basic numerical competencies in both training conditions, but no differential training effects. As children can play on the interactive surface with reasonable autonomy, it seems to provide a low threshold possibility to enrich kindergarten education on numerical concepts.
... Tabletop computers have affordances for collaborative learning because of the large, shared interface that multiple people can see and interact with at once (Figure 1) [9,17,18,32,33]. However, the process of collaborative learning is complex and subject to a variety of factors external to the technology, such as students' collaboration skills, group dynamics, and the nature of the learning activity [34]. ...
Conference Paper
Interaction logs generated by educational software can provide valuable insights into the collaborative learning process and identify opportunities for technology to provide adaptive assistance. Modeling collaborative learning processes at tabletop computers is challenging, as the computer is only able to log a portion of the collaboration, namely the touch events on the table. Our previous lab study with adults showed that patterns in a group’s touch interactions with a tabletop computer can reveal the quality of aspects of their collaborative process. We extend this understanding of the relationship between touch interactions and the collaborative process to adolescent learners in a field setting and demonstrate that the touch patterns reflect the quality of collaboration more broadly than previously thought, with accuracies up to 84.2%. We also present an approach to using the touch patterns to model the quality of collaboration in real-time.
Article
We present findings from an empirical study of how groups of eight users collaborate on a decision-making task around an interactive tabletop. To our knowledge, this is the first study to examine co-located collaboration in larger groups (of 8-12 users) seated around a large-scale high-resolution multi-touch horizontal display. Our findings shed light on: 1) the effect of collaboration patterns of larger groups on equity of participation; 2) the role of participants' position around the tabletop in forming collaborations; and 3) the mechanisms, which facilitate coordination and collaboration in larger group interacting around large-scale tabletops; We also contribute computational methods that leverage image processing to analyze interaction around large-scale tabletops. Finally, we discuss implications for the design of large-scale tabletop systems for supporting co-located collaboration in larger groups.
Article
Full-text available
Interactive displays (IDs) are increasingly employed in informal learning environments, where they are seen as a medium for enhancing students’ creativity, and engagement. Due to the larger space they provide and thus the larger interaction area, they allow for group-work, working in parallel, co-creating artifacts or co-experiencing the interaction in a playful manner. In particular, gaming activities in IDs enhance students’ mental exercise and fantasy and promote students engagement through rewards and collaboration. However, despite the increased prevalence of interactive displays and gamification, we know very little about how designers and instructors can gamify their learning activities by taking advantage of the IDs. In this paper, a framework for developing gamified activities for interactive displays is presented. For the empirical evaluation, pre-post attitudinal surveys and cognitive tests along with photos and observations were recorded and used. The contribution of this article is twofold: 1) an adaptable framework for developing gamified activities on interactive displays (GAID), and 2) the results of a field study where students have been engaged with an interactive display application during an extracurricular activity. By incorporating GAID to a traditional informal learning activity, it is found that students’ knowledge acquisition, satisfaction, enjoyment and intention to participate on similar events in the future are significantly improved.
Article
The position of a ball was measured by using the touchscreen of a mobile phone during its rolling motion. The translational speed of the ball was determined using the recorded position and time data. The speed was also calculated by a conventional method. The speed values determined by the two methods were consistent, thus it was proven that a touchscreen could be used to detect position in physics experiments. Touchscreens of other smart mobile devices and touch tables can also be used for the same purpose.
Article
Full-text available
The abstract for this document is available on CSA Illumina.To view the Abstract, click the Abstract button above the document title.
Article
Full-text available
Introduction Learning about fractions – how to compare, reduce, represent, write, or draw them – can be a difficult undertaking. Manipulatives enable children to grasp difficult math concepts both literally and figuratively. Pattern tiles, fraction sticks, and fraction pies are common manipulatives used in elementary math to help with these difficult concepts. Many children use pattern tiles to make interesting designs. Through play, they learn about the relationships between some shapes and how to fit them together. Manipulatives can work well as a learning tool for math because they offer a tangible, movable, physical representation that can be used to help learners visualize math concepts in a concrete way. But their affordances for learning are not always well-realized, either because something goes by unnoticed, the play children do with them isn't part of what the teacher supports, or students do not have the manipulatives available when they want to play. Books available on the market (e.g., Sehi-Smith, 1997) make some attempts to enhance learning with pattern tiles by providing structured activities for learners, and some manipulatives have been programmed for use on computers (MATTI, 2001; Mankus, 2001). While they are mathematically interesting and fun to use, they do not focus on integration into the curriculum in formal ways that could reach all students and motivate productive play. How can we incorporate math manipulatives into the curriculum in such a way that they promote understanding, foster creativity, and perhaps support other curricular areas for more children? This is the question we wish to answer in the DigiQuilt project. Constructionist (Papert, 1991) and Learning by Design (Kolodner et. al., 1998, 2002) approaches both suggest taking a design approach (i.e., asking students to design patterns using manipulatives). Both suggest, as well, that a design approach affords learning not only the math concepts needed for designing but design skills and those skills involved in completing a project. We propose that patchwork and mosaic design projects are a plausible method to promote understanding of math concepts (such as fractions, area, and perimeter) and art concepts (such as symmetry and balance). Further, we propose that virtual manipulatives 1 provide some extra affordances for learning that physical manipulatives lack. With this foundation in mind, we are designing a software environment that integrates math and art for learners to design patchwork quilt blocks and a set of challenges and activities to orchestrate the learning of fractions and simple art concepts. This paper will begin by telling about the software and continue by describing results of a preliminary study exploring the plausibility of the medium and integrating the software in the context of a greater learning environment. DigiQuilt: Describing the system and supporting design decisions DigiQuilt (Figure 1) is a digital construction kit for designing quilt blocks. Users create quilt blocks by selecting colored shapes from a palette and placing them into a grid. Shapes can fill a whole grid square, or they can fill some fraction of the square (1/4 or 1/2). Users can rotate shapes to be able to fit them together and place them to make patterns. The selection of shapes and their sizes is based on the ratios and shapes used in Froebel's gifts (Froebel Web, 2001). For ease of 1 By "virtual" manipulatives, we mean software representations viewed on the screen and manipulated using computer input (e.g. the mouse) rather than in the traditional manner of physical manipulatives.
Conference Paper
How does designing for novel experiences with largely untried technologies get its inspiration? Here we report on a project whose goal was to promote learning through novel, playful visions of technologies. To this end, we experimented with a diversity of ambient and pervasive technologies to inspire and drive our design. Working as a large multi-disciplinary group of researchers and designers we developed novel and imaginative experiences for children. To crystallise our ideas we designed, implemented and experimented with a mixed reality adventure game, where children had to hunt an elusive, virtual creature called the Snark, in a large interactive environment. We describe our experiences, reflecting on the process of design inspiration in an area where so much remains unknown.
Article
Technology and Culture 43.1 (2002) 191-193 Scripts, Grooves, and Writing Machines: Representing Technology in the Edison Era. By Lisa Gitelman. Stanford, Calif.: Stanford University Press, 2000. Pp. viii+282. 49.50/49.50/19.95. Lisa Gitelman, who teaches English and media studies at the Catholic University of America, has written another of those books that attempt to "locate" technology amid a dynamic welter of cultural relationships and meanings. In this instance, the technologies in question all deal with the spoken or written word. Gitelman wants to show that late Victorians' conceptions of the written word influenced their technological innovations and that, in turn, their technologies changed the way they thought about texts. No simplistic "inventor/producer/consumer" mentality here. Scripts, Grooves, and Writing Machines examines five "experiments and innovations in the area of inscription" (p. 2) between 1877 and 1914 that influenced (and were influenced by) the 1877 introduction of a phonographic device by Thomas Edison. These include the practice of shorthand, letters to Edison expressing public expectations of technological change, patent and copyright laws, the labeling of musical recordings and silent films as consumable performances, and the typewriter. Gitelman's anticipated audience is "scholars of literature, linguistics and communication" (p. 8), and her goal is to "explore writing and reading as culturally and historically contingent experiences and, at the same time, to broaden the current widely held view of technology in its relation to textuality" (p. 1). It is this latter goal that will most interest historians of technology, for Gitelman makes the intriguing assertion that technologies take the form they do because of preexisting cultural assumptions, that is, that "technologies of inscription are materialized theories of language" (dust jacket). Chapter 1 examines different methods of shorthand proposed in the last half of the nineteenth century. Like the phonograph, shorthand stores up words spoken in the past so that they can be reproduced in the present. Like the phonograph, it received much public attention because it offered much the same thing as a phonograph recording, a verbatim reproduction of a text. Thus, shorthand contributed to the evolution of a modern understanding of public and bureaucratic life, an understanding that emphasized the importance of text and the potential of objectivity. Chapter 2 depicts different visions of the phonograph's future embedded in the culture's attitudes toward technology in general. Gitelman juxtaposes Edison's own imaginative anticipations for the invention with letters sent to him by ordinary people encouraging him to follow their technological visions. These visions turn out to be wide-ranging, and Gitelman's brief exegesis of some twenty thousand letters provides some of the more cogent and entertaining material in the book. Chapter 3 examines the phonograph's accomplishments -- storing and replaying sound -- in the context of the patent system and copyright laws. Here Gitelman focuses attention on the myriad ways in which technical literature and commercial products, particularly early recordings, "exhibit a rhetoric of exclusion on the bases of class, race, and gender" (p. 121). Chapter 4 investigates the phonograph's transition from invention to consumer product by an analysis of the label texts found on recordings and silent film reels. In particular, Gitelman focuses our attention on producers' efforts to define for their audiences the qualities that consumers should desire in these new technologies. This is an interesting case study in the turn-of-the-century development of market characteristics and definitions of product "quality." Gitelman succinctly describes her goals for the fifth chapter: "I will demonstrate how the connections between psychology, spiritualism, and typing in the 1890s find resolution within the term automatic writing" (p. 186). What follows ranges from Edison and Christopher Sholes to Gertrude Stein and Bram Stoker's Dracula, with an emphasis on the notion that typing introduced modern conceptions of gendered labor and human/ machine interfaces. Mirroring her approach to technology, Gitelman's own presentation is "plural, decentered, indeterminate." None of the chapters ends in a satisfactory conclusion that sums up the argument at hand. Nor does the book as a whole offer conclusions on the relationship of late-nineteenth-century technology and textuality. Rather, Gitelman...
Article
Hako-ne is an augmented reality hybrid art/technology project. Users will be able to see characters of 3D musical notes moving on each side of the augmented musical dollhouse by using a handheld Optical See-Through Information Viewer (OSTV), which consists ...
Article
The recent introduction of computationally-enhanced tables that support simultaneous, multi-user input has important implications for co-located, face-to-face activity. Educational applications particularly stand to benefit from this new technology, which can combine the benefits of small group work with the enhancements offered by digital media. In this paper, we explore how the unique affordances of interactive tables provide a match for the needs of foreign language education, and how the design of tabletop software can be subtly altered to encourage desired educational outcomes. We present three prototype applications, and explore four design variations (feedback modality, feedback privacy, spatial configuration, and interaction visualizations) to assess their impact on student participation and self-assessment. We present observations of the use of our prototypes in two settings: (1) a controlled laboratory study and (2) authentic use by students as part of a language course at our university, and discuss our preliminary findings and avenues for future exploration.