ArticlePDF Available

Abstract and Figures

End-user software engineering (EUSE) is a research area that aims to invent new kinds of technologies that collaborate with end users to improve the quality of their software. The practice that EUSE research aims to support is end users using new tools and methods to improve the quality of the software that they and other end users have created. There is a need for this outcome because research shows both that the number of end users creating their own software greatly exceeds the number of professional software developers, and that the software they create is riddled with errors. In this paper, we describe the present state of EUSE, and challenges in moving forward toward a bright future. We show how the future of EUSE may become over-siloed, restricting future researchers' vision of what can be achieved. We then show that focusing on the in-the-moment intents of end-user developers can be used to derive a number of promising directions forward for EUSE researchers, and how theories can help us further desilo future EUSE research. Finally, we discuss how overcoming challenges for the future of end-user software engineering may also bring direct benefits to the future of "classic" software engineering.
Content may be subject to copyright.
Future of End-User Software Engineering:
Beyond the Silos
Margaret M. Burnett
School of Electrical Engineering and Computer Science
Oregon State University
Corvallis, OR 97331 USA
Brad A. Myers
Human Computer Interaction Institute
Carnegie Mellon University
Pittsburgh, PA 15213 USA
End-user software engineering (EUSE) is a research area that
aims to invent new kinds of technologies that collaborate with end
users to improve the quality of their software. The practice that
EUSE research aims to support is end users using new tools and
methods to improve the quality of the software that they and other
end users have created. There is a need for this outcome because
research shows both that the number of end users creating their
own software greatly exceeds the number of professional software
developers, and that the software they create is riddled with errors.
In this paper, we describe the present state of EUSE, and
challenges in moving forward toward a bright future. We show
how the future of EUSE may become over-siloed, restricting
future researchersvision of what can be achieved. We then show
that focusing on the in-the-moment intents of end-user developers
can be used to derive a number of promising directions forward
for EUSE researchers, and how theories can help us further de-
silo future EUSE research. Finally, we discuss how overcoming
challenges for the future of end-user software engineering may
also bring direct benefits to the future of classic software
Categories and Subject Descriptors
D.2 [Software Engineering]. H.1.2 [Models and Principles]:
User/Machine Systems.
General Terms
Reliability, Human Factors, Theory
End-user software engineering (EUSE), end-user development
Software engineering is no longer solely applicable to
professional software engineers. Anyone who develops software,
whether as a professional developer or not, may well need to
employ beyond-coding processes to help ensure that the software
they develop is of sufficiently high quality for the purpose for
which they intended it.
Who, besides professional developers, actually develops
software? According to data from the U.S. Bureau of Census and
Bureau of Labor, more than 12 million people in the U.S. say they
do programming at work, and more than 50 million use
spreadsheets and databasescompared to only about 3 million
professional programmers in the United States [54]. Thus,
depending on how conservatively one differentiates usefrom
program in, the number of end-user programmers in the United
States falls somewhere between 12 million and 50 million
peopleseveral times the number of professional programmers.
Clearly then, end-user programming empowersit has already
empowered millions of end users to create their own software.
Nardi defined end-user programmers as being distinct from
professional developers in that end usersprograms are not the
end in itself, but rather a means to accomplish their own tasks
[39]. There are a variety of research systems and tools aimed at
this audience; examples include an accountant creating
spreadsheet formulas (which are computational instructions) to
keep track of a budget in Excel or a web-literate end user building
a quick mashup to help sort real estate based on custom criteria.
Note that, although Nardis definition does not define all end-user
programmers as lacking formal computer science training, many
subpopulations of end-user programmers do lack such training.
In this paper, we consider the present and future of an area of
software engineering that addresses populations like these. The
future of this area that we envision breaks down the silos that have
begun to emerge. We focus on where these silos are, and the
benefits that could ensue to end-user software engineering
research in the absence of these silos.
1.1 End-User Programming versus End-User
Software Engineering
Some examples of the programming environmentsused by end
users are shown in Figure 1. For example, CoScripter [38] (Figure
1(a)) allows web users to automate web-based tasks by
demonstrating the desired sequence of actions on the web pages.
AgentSheets [46] (Figure 1(b)) is aimed mainly at children and
teachers. These AgentSheets users demonstrate rules of
behaviorfor different characters in a game or simulation by
placing the characters onto the game board and demonstrating
both the circumstance that requires the new behavior and the
behavior itself. LabVIEW [25] (Figure 1(c)) aims at a different
audience: scientists and engineers doing sensor-based
computation. LabVIEW users wire together nodes representing
various sensors and measurement instruments to construct
dataflow diagrams defining the desired computations. Finally,
TouchDevelop [36] (Figure 1(d)) enables end users to create
mobile apps of their own by choosing from applicable actions
from a continually updating smartkeyboard.
Using systems like these, end users create a variety of programs to
use on their desktops, phones, and the web. (Following Ko et al.,
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
FOSE'14, May 31 June 7, 2014, Hyderabad, India.
Copyright 2014 ACM 978-1-4503-2865-4/14/05…$15.00.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specific permission and/or a
fee. Request permissions from
FOSE’14, May 31 – June 7, 2014, Hyderabad, India
Copyright 2014 ACM 978-1-4503-2865-4/14/05...$15.00
we define a program as a collection of specifications that may
take variable inputs, and that can be executed (or interpreted) by a
device with computational capabilities” [29]. The variability of
input values requires that the program be able to execute on future
inputs, which is one way it is different from simply doing a
computation once manually.) Some ways in which end users
create these programs include writing and editing mathematical
formulas, dragging and dropping objects onto a logical
workspace, typing and editing textual code, sketching user
interface states and indicating hot spots and transitions,
connecting objects in a diagram, or demonstrating intended logic
to the system.
Unfortunately however, the software that end users are creating
with these techniques is riddled with errors. Evidence abounds of
the pervasiveness of errors in the software that end users create
(e.g., [19]). These errors can have significant impact. For
example, after Hurricane Katrina, dozens of end users created web
sites to allow updates about survivorslast known locations [52].
Confusion reigned with these sites. Data on one site were often
missing or inconsistent with data on the next site. Leveraging data
across sites was not possible, due to differing data formats and
uncertain quality [52]. Turning to a 2011 example from the
European Spreadsheet Risks Interest Group’s huge collection of
spreadsheet errors, a formula error caused the village of West
(a) CoScripter [38], used for web automations. (Left): script. (Right):
Web context for the script. (Original screen shot.)
(b) AgentSheets [46], used for agent-based games and
simulations. (Top): Program state that includes some truck
objects. (Bottom): A visual rule that controls truck behavior.
(Excerpted from
under Creative Commons License
(c) A dataflow diagram in LabVIEW [25], which is used primarily for
lab applications. (Excerpted from
D.png, under Wikipedia Commons License
(d) The mobile version of TouchDevelop [36] for developing
mobile apps.
Figure 1: Several end-user programming environments.
Baraboo, Wisconsin to underestimate the cost of a borrowing
decision by about $400,000 [19, case FH1218). Even when errors
in end-user-created software are non-catastrophic, however, their
effects can matter. For example, web applications created by
small-business owners to promote their businesses do just the
opposite if they contain broken links or pages that display
incorrectly, resulting in client frustration, loss of revenue, and/or
loss of credibility [1].
Given that so much of end-user-created software suffers from
quality too low for the purposes for which it was created, there is
a clear need for researchers and tool makers to address this. From
much prior research in classic software engineering, we know that
to improve quality, the software lifecycle beyond just the create
stage must be supported. That is, end-user developers, like
professional developers, must be supported beyond the coding
stage of their software development.
To help end-user developers do just that, a new research area
emerged about a decade ago to address this problem. The area is
known as end-user software engineering (EUSE). As Ko et al.
summarize, end-user software engineering technologies are
technologies that collaborate with end users to improve software
quality [29]. Thus, the differences between end-user
programming and end-user software engineering are that end-user
programming focuses mainly on how to allow end users to create
their own programs, whereas end-user software engineering
considers how to support the entire software lifecycle and its
attendant quality issues. Differences between end-user software
engineering and traditional software engineering are several, but
they all stem from a difference in population.
When taking this lifecycle view of end userssoftware, the term
end-user developer becomes more appropriate than end-user
programmer, drawing upon Lieberman et al.s definition of end-
user development: a set of methods, techniques, and tools that
allow users of software systems, who are acting as non-
professional software developers, at some point to create, modify
or extend a software artifact[37]. This terminology points out
that end users involved in software development may or may not
actually create new code as part of what they do in the software
development process. For example, they might participate in
activities like configuring their software, co-designing their
software side-by-side with professional programmers, and
tailoring their software [15, 17, 20, 26, 61]. Because the term
end-user developer implies participation in any software
development lifecycle stagenot restricted to just creating
codefor the remainder of this paper we will use the term end-
user developers .
1.2 A Future of End-User Software
Engineering: Challenges in De-siloing
EUSE research is at a crossroads. Following too closely in the
footsteps of its classic software engineering predecessors raises a
risk of over-siloing the area, to the detriment of the end-user
developers it aims to serve. When we refer to siloes, we mean a
system, process, department, etc. that operates in isolation from
others” [40] (italics added).
In one sense, end-user software engineering de-silos already: it
removes the silo that fenced off software engineering benefits
from everyone except professional software developers. However,
end-user software engineering has not gone much further in de-
siloing, and we believe the area could benefit from a great deal
more. For example, it primarily considers end-user developers
only as (atomic) people, rarely anything smaller or larger;
considers software development tasks only one lifecycle phase at
a time; and suffers from siloing individual tools and findings
rather than building connective tissue between them. Such silos
raise risks that an area can become overly narrow, building upon
research in one silo or another, but losing the potential for big
steps forward that can enable a research area to transform.
Thus, this paper sets out a number of de-siloing challenges that we
hope will influence the future of end-user software engineering.
We consider them from the perspectives of who, when, why and
A key difference between end-user software engineering and
traditional software engineering is the kind of developers served.
A premise of classical software engineering research is that
professional developers have some training and/or experience in
software engineering methods, and also at least a little interest in
following good software engineering practices. In contrast, end-
user software engineering research cannot assume the presence of
any of these.
From this difference, two requirements become clear for end-user
software engineering tools and techniques. First, EUSE tools and
techniques must be designed for audiences that are usually
unfamiliar with professional software engineering practices (even
though they may have programmed before). Second, these tools
and techniques must fit well with their audiences non-software-
engineering-oriented interests, motivations, and practices.
2.1 Today’s End-User Developer Target
Fulfilling these requirements requires considering more deeply
who the target audiences are. The term end-user programmers
was once used synonymously with novice programmers to
mean anyone with a lack of experience programming regardless
of their motivations, but that trend largely disappeared after
Nardis clarifying book on end-user programming [39]. Still, in
todays end-user software engineering literature, assumptions
about the population of end-user developers to be served are
sometimes left implicit, and differ from paper to paper. For
example, some researchers use the term to mean people who have
neither experience nor interest in software development per se
(e.g., [50]), others use it to mean anyone using a particular
programming environment that targets end users (e.g., [6, 36]),
and others use it to mean people who can program in one
particular application area only (such as end-user webmasters
Such assumptions are true of some end-user developer
populations, but not all of them. For example, counterexamples to
many of the above assumptions can be seen in Scaffidi et al.s
survey responses from 831 information workers (subscribers to
Information Week) [51]. As Table 1 shows, these users77% of
whom were in non-technical jobsdid a surprising amount of
programming, often using only ordinary office productivity tools.
One reasonable way to establish what assumptions to make about
a target end-user developer population is to empirically study the
target audience as part of designing EUSE tools for that audience,
and some EUSE research has moved in this direction (e.g.,
accountants using spreadsheets [59] and web designers [41]). An
advantage of informing EUSE tool designs by empirically
studying a particular population is that it enables EUSE
researchers to fulfill the two requirements enumerated at the
beginning of Section 2: namely, not relying upon familiarity with
software engineering, and fitting with the target audiences
motivations and practices. However, a disadvantage is that it can
encourage siloing EUSE research by population. Further, each silo
(population) must be pre-identified and researched on its own,
without including other end-user developer populations.
2.2 Challenge: Targeting a More Nuanced
A de-siloing alternative can be derived from Nardis original
definition. Recall that, according to Nardi [39], the distinguishing
factor of professional programmers vs. end-user developers is a
difference in intent, not a difference in experience. Indeed, Nardi
goes on to explain that end users are not necessarily casual,
novice,or naiveusers. Ko et al. emphasized this notion by
defining end-user programming (and end-user development) as a
role (a close cousin to high-level intent) rather than a persons
semi-permanent identity [29].
Defining end-user developers based on intent allows a more
nuanced version of who, thus widening the population of those
who could be served by end-user software engineering research.
Roles can include not only traditional populations of end-user
developers like teachers and spreadsheet users engaging in
software development, but also populations who, although they
program in various ways fairly regularly, still have little interest in
software engineering in at least some of their software
development tasks.
As an extreme example, a professor of software engineering might
be systematic and rigorous in validating software created for
distribution to other researchers, but ad hoc about validating a
grant budget spreadsheet. Indeed, an ad hoc view of beyond-
coding software engineering tasks has been shown to arise often
across a wide span of end-user developers1, from scientists and
mathematicians who program often in traditional languages [56]
to spreadsheet users [59] to teachers [63] to museum exhibit
designers [7].
The decoupling of intent from personal identity helps to de-silo
end-user software engineering because it removes the need to
separate end-user developers as people into groups: instead,
researchers can focus on intent, which slices across many
different classes of end-user developers. Intent as a central
concept ties end-user software engineering research directly to the
end-user developers motivations and values in that particular
software development task, which in turn can affect their software
engineering processes.
2.2.1 Intent as a basis for addressing the challenge
Moving to intent as the key definer of when someone is acting as
an end-user developer suggests the possibility of raising intent to a
first-class factor in defining future EUSE tools and techniques.
One way focusing on intent can help advance EUSE research is
by revealing flawed assumptions in EUSE work. One common
assumption is if we build it they will come.That is, if a new
tool or feature becomes available that can improve users
correctness or efficiency, then surely conscientious users will use
it. However, a focus on intent clarifies that this assumption may
not hold for end-user developers.
For example, what might a laboratory scientists intent be, when
working on a laboratory application to collect and measure data
from a variety of sensors? Some possibilities might be build my
lab applicationor fix my broken lab application or add new
capabilities to last years lab application.” Note that none of these
intents sounds much like find a new tool that might give me
some useful new ways to guard the quality of my application.”
Since intent resides in peoples heads, understanding its
implications on EUSE is facilitated by turning to applicable
theories of how people think. Blackwells Model of Attention
Investment [5] is a model applicable to this situation; it helps to
explain how end-user developers make choices about what
problem-solving tools and features to use in computer systems.
Blackwells model rests on three constructs: the usersperceptions
of the costs, benefits, and risks they will face in completing their
task (accomplishing their intent) one way versus some other way.
According to Blackwell, users expect that exploring an unknown
tool or feature will have a cost, some benefits, and some risks. For
example, in the lab example above, suppose the scientists usual
way of accomplishing such tasks is to use the environments
editor. If something goes wrong, the scientist can fix it using the
editor. This has no learning cost, and if the scientist believes s/he
is succeeding, it seems to have the benefit of accomplishing the
task with no risk. Only if the scientist realizes that there are
problems too great to fix that way (i.e., there is high risk that the
benefits will not be realized) is s/he likely to consider switching to
a new unknown tool.
Now suppose the scientist decides that a more powerful problem-
solving tool is needed and, in looking around the environment,
sees a tool that might help. The scientist ventures to open it up,
and is now faced with a first impression of the cost of learning the
tool and of the cost of using it for the particular problem at hand.
1 From now on, we will use this phrase as an abbreviation
meaning “developers operating in an end-user development
Table 1. Programming-like features, in a variety of
common software applications, used by at least 33% of the
information workers [52].
Used by
(% of
(% of
Slide Editors
& Word
Create document templates
Make inter-document hyperlinks
Record desktop editor macros
Use functions (e.g., “sum”)
Create charts in spreadsheets
Inter-spreadsheet references
Create spreadsheet templates
Record spreadsheet macros
Create/edit spreadsheet macros
Reference records by key
Create tables
Create database views
Create stored procedures
Web Pages
Create hyperlinks
Create web forms
Create web page scripts
Create JavaScript functions
This first impression is the perceived cost the scientist is
imagining. S/he is also considering the perceived risk
(probability) that going down this path will be a waste of time or,
worse, will leave the application in a mess from which it would be
difficult to recover. If users perceive an EUSE tool’s costs or risks
to be greater than the perceived benefits, they are unlikely to use
the tool [5, 50].
Viewing intent in light of Blackwells Model of Attention
Investment points out that EUSE tools and techniques can target
an intent-nuanced whoby (1) connecting to a target end-user
developer’s intent whenever it may arise, and (2) using that
moment to entice them to take appropriate software engineering
steps when the need arises by making clear from the outset the
upfront and ongoing costs, benefits, and risks that are likely.
One approach to accomplishing these two steps is to explicitly use
inferences of users intentions as starting points of EUSE support
tools. Another approach is to use curiosity-based enticement to
draw end users into engaging at an appropriate moment with
software engineering features. A third is to base tools on past
intents of the target audience as predictors of commonintents.
Although possible intent-focused approaches to EUSE techniques
are relatively unexplored, the next section describes a special case
with promising beginnings in that direction.
2.2.2 A special-case who: EUSE & learning
An interesting case study in de-siloing by moving from whoas
people to whoas roles has occurred in the realm of learning.
Originally, tools for helping people learn software development
skills aimed at teaching programming (e.g., syntax, semantics, and
basic programming skills) to people who wanted to learn
programming. However, a relatively recent trend is to reach out to
people who may not be interested in programming to try to
convince them to want to programi.e., to manipulate intent, or
in the terminology of the Blackwell model, by manipulating users
perceptions of benefits. A system can manipulate intent in (at
least) three ways: (1) it can build new intent, (2) it can serve
existing intent (if intent is known), and/or (3) it can attempt to
infer intent, often as a precursor to (1) or (2).
The aim of a system building new intent is to reach out to users
who are motivated by some benefit other than a program of my
own.” Such systems present programming as a means to that other
benefit, with the hoped-for result that the users then decide to use
the programming system in order to attain that benefit. One
example is to gamifythe process of learning programming (e.g.,
Pex4Fun [60] (Figure 2(a)) and Gidget [35]), leveraging the
drawing power of a game to engage users in learning
programming as a side effect of playing the game.
A system can build intent to learn other software development
skills too, not just coding. For example, a few researchers have
worked to draw users toward higher success at design. Besides
offering a non-software-development benefit (like telling stories,
or designing their own web pages), these works feature
scaffolding to reduce usersperceptions of the costs of achieving
the benefit they seek. For example, Kerr et al. provided story
templates to help users with story design ideas in the setting of
StoryTelling Alices successor [27], and Lee et al. provided
manipulable examples to help end users explore web page design
ideas [34]. Approaches like these can help improve end-user
developerssoftware development skills.
Approaches like the ones discussed so far have been shown to be
effective for users who are at least open to expanding their
software development skills. However, many end users are
reluctant to do soresistance to various software development
tasks has been reported for a variety of end-user developers (e.g.,
[14, 42, 56, 63]). As these reports show, to end-user developers
like these the benefits seem low, and/or the costs and/or risks
seem high.
Fortunately, the keys to attracting even reluctant learners to learn
software development skills become clear from the ideas of intent
and the cost/benefit/risk factors of the Attention Investment
model. That is, a system can introduce appropriate software
development practices to end-user developers by serving their
actual intents (if known) to provide these users with the benefits
they seek or to reduce their risks or costs in a way that also shapes
their software development behaviors.
For example, the Whyline research began by first researching
common debugging intents empirically. End user developers
most common debugging intents were harvested by investigating
their most common questions. The investigation showed that their
(a) Pex4Fun’s gamification [60] can build intent to program in
people who like to play games. (Original screen shot.)
(b) Whyline (Alice version) [31] directly serves an existing intent
that has been reported empirically. Here, the user asks “why
didn’t Pac resize 0.5?” (Original screen shot)
(c) Idea Garden [10] first infers intent and cost, and then offers
help to reduce the cost of achieving that intent.
Figure 2: Three ways systems can build upon intent.
most common questions were whyand why not, so the
Whyline was developed to allow users to ask those questions
directly [31], and to answer them through scaffolded views of the
kinds of problem-solving approaches professional developers use,
such as tracing through calling sequences with their parameter
values. See Figure 2(b) for a Whyline view of a calling sequence
with parameter values.
Serving existing intents can also be combined with building new
intents. For example, Surprise-Explain-Reward [64] is a strategy
to entice users toward desirable software engineering behaviors.
The strategy attempts to arouse the end-user developers curiosity
about some aspect of their software that seems to be erroneous
(such as a faulty cell formula), by decorating the cell if the tool
has something to say about it. If the softwares accuracy is
important to that developer, they may become curious and follow
up. The decoration (notification that the software has something to
say) is the surprise, the information they get by following up (e.g.,
by hovering over the decoration) is a brief explanation, and the
reward is (ideally) a found error and a sense that there is less risk
of the software producing incorrect answers. Surprise-explain-
reward has been used successfully in several EUSE settings; one
example was to entice end users into incrementally entering
assertions to guard spreadsheet correctness [8].
Another intent-oriented strategy is to target the end-user
developers intent by inferring it or asking about it in context [10,
12]. For example, the Idea Garden (Figure 2(c)) infers intent from
context, and also infers the costs of behaviors the end-user
developer may be exhibiting, and then provides help that will
reduce the user’s difficulties (cost) of accomplishing that intent
The approaches discussed here to de-siloing the learning of EUSE
practices share a set of common attributes: (1) targeting end-user
developer intent, often as a right nowmoment, (2) drawing end-
user developers into doing just a little of the desired behavior at a
time when it can benefit them, and (3) giving them the supports
they need, just in time and in their own environment, so that they
can succeed at the software engineering behaviors they are
engaging in.
These approaches generalize beyond learning settings, as some of
the above examples have already demonstrated. That is, building
new intent by serving an existing intent does not have to be
leveraged only in learning systems or learning modes.”
Approaches like these can be included in any system that end-user
developers use, to encourage incremental, life-long improvement
of their software development skills, in the contexts of doing their
ordinary work.
From a lifecycle perspective, end-user software engineering
begins, by definition, by moving beyond the coding phase silo to
connect it with other phases of the software lifecycle.
3.1 Today’s EUSE: Beyond Coding
Researchers in end-user software engineering have devised a
variety of techniques to support software engineering activities
beyond coding. Perhaps the first such example of explicit support
beyond coding was the What You See Is What You Test
(WYSIWYT) research to help end-user developers systematically
test their spreadsheets [49, 21]. WYSIWYT incrementally tracks
test coverage of the spreadsheet formulas (via definition-use
association coverage of the formula subexpressions), and
colorizes the cell borders on a continuum from red (no test
coverage yet) to blue (fully covered). At any time, if the user
notices that a value (like the Max of... cell in Figure 3(a)) is
correct, she can check it off, and the coverage is automatically
recalculated so that corresponding cell border colors update
immediately in the spreadsheet.
There has also been research to support other stages of the end
users software development lifecycle, such as design [48],
assertions [8, 45], reverse-engineering [16], refactoring [3, 58],
and explicitly guarding against or detecting errors and software
smells” [2, 18, 24, 53]. Figure 3(b) shows an example of the
Topes approach [53], which guards against errors.
These approaches are steps forward in bringing to end-user
developers some of the benefits of software engineering
methodsbut they do not go far enough. Most of these
approaches address each lifecycle stage in isolation (e.g., just
refactoring, or just designing) so, while they break out of the
codingsilo, they then erect new silos for each lifecycle phase.
3.2 Challenge: Beyond “Beyond Coding”
Such silos for each lifecycle phase do not match the opportunistic
workstyle of many end-user developers [7, 56]. In developing
their software, exactly when are end-user developers doing
design, when are they testing, and so on?
For example, consider a teacher who wants to create a grades
spreadsheet, and suppose she begins by entering a few column
headings across the top (say, ID and name), and then typing
in all her students IDs and names into the corresponding
columns. So far she has not entered a single line of code
(formula)so is she in the design stage? In one way she is,
because she has begun to specify the structure of the spreadsheet,
(a) The WYSIWYT testing approach for end-user developers [49,
21]. (Original screenshot.)
(b) Topes allow end-user developers to define and error-check
data inputs [53]. (Original screen shot )
Figure 3: Examples of beyond-coding approaches.
and indeed, according to design theorist Schön, she is doing
design according to the reflection-in-action model [11, 55].
However, from a traditional software engineering standpoint, she
is not, because she has not specified any relationships or logic, yet
has already skipped ahead to data entry.” Suppose she next
implementsa little of the program (enters several formulas), and
simultaneously testsa couple of the formulas (glances at their
values based on the inputs she put in), and decides things are
going well.
This opportunistic mix of several elements from the software
lifecycle is the way many of todays end-user developers with no
prior software engineering training work [7, 56]. Without support
for navigating seamlessly among lifecycle phase momentsin
their own environments, these end-user developers may never
choose to engage in tools that could help them. (Recall our
emphasis on developersintents in Section 2.) Yet, as the software
they create grows in complexity or criticality (e.g., [19, 57]),
sometimes they need these supports to attain the quality their
software needs.
At a minimum, then, EUSE tools need to be integrated into the
work styles and environments these developers use. To do so
requires tool builders to ensure that the tools they build for
various lifecycle stages are interactive and incremental, so that
end-user developers can easily do a little using one lifecycle
stages tool, then switch to another to do a little, then switch back.
This may require entirely different reasoning algorithms than have
used in the lifecycle phase counterparts in classic software
engineering. For example, some EUSE efforts have been to
develop new EUSE static and dynamic reasoning algorithms that
are highly incremental and responsive so as to support exactly this
kind of incremental phase switching (e.g., [23, 32, 49]).
Still, from a de-siloing perspective, this approach is somewhat
incremental. It is supportive of the fact that end-user developers
will need to rapidly and intermittently switch in and out of their
tools, but still silos the EUSE tools and research efforts by
lifecycle phase.
An alternative approach would be a more dramatic departure from
classic software engineering approaches. We believe it has more
potential to transform but also brings more challenges. In
essence, it is to promote end-user developers work styles and
intents as a way to define our vision, giving EUSE researchers the
potential to transcend one-lifecycle-phase-at-a-time research silos
in favor of bigger visions.
One way to go about such an approach would be to frame the
EUSE problem more along the lines of the problem-solving
activity rather than the software engineering lifecycle. To see
how this can be done, we describe an emerging EUSE research
project (in which we are involved) that characterizes its target as
supporting exploratory programming by end-user developers
The project targets opportunistic end-user developers who are
exploring possible ways to achieve their development goals,
working to understand as much as needed of unknown code
fragments, and exploring ways to fit some of the pieces together.
Some code fragments may come from the users own files, and
others discovered on the web. Some may have been written by
this user, some by other end-user developers, and some by
professional developers. As things go wrong, the end-user
developer may need to revisit some variants tried before that were
For example, consider Frieda, an office manager in charge of
her departments budget tracking. (Frieda was a participant in a
set of interviews with spreadsheet users that the first author
conducted. Frieda is not her real name.) Every year, the company
she works for produces an updated budget tracking spreadsheet
with the newest reporting requirements embedded in its structure
and formulas. But this spreadsheet is not a perfect fit to the kinds
of projects and sub-budgets she manages, so every year Frieda
needs to change it. She does this by working with four variants of
the spreadsheet at once: the one the company sent out last year
(we will call it Official-lastYear), the one she derived from that
one to fit her departments needs (Dept-lastYear), the one the
company sent out this year (Official-thisYear), and the one she is
trying to put together for this year (Dept-thisYear).
Using these four variants, Frieda exploratively mixes reverse
engineering, reuse, programming, testing, and debugging, mostly
by trial-and-error. She begins this process by reminding herself of
ways she changed last years by reverse engineering a few of the
differences between Official-lastYear and Dept-lastYear. She then
looks at the same portions of Official-thisYear to see if those
same changes can easily be made, given her departments current
She can reuse some of these same changes this year, but copying
them into Dept-thisYear is troublesome, with some of the
formulas automatically adjusting themselves to refer to Dept-
lastYear. She patches these up (if she notices them), then tries out
some new columns or sections of Dept-thisYear to reflect her new
projects. She mixes in testing along the way by entering some
of the budget values for this year and eyeballing the values that
come out, then debugs if she notices something amiss. At some
point, she moves on to another set of related columns, repeating
the cycle for these. Frieda has learned over the years to save some
of her spreadsheet variants along the way (using a different
filename for each), because she might decide that the way she did
some of her changes was a bad idea, and she wants to revert to try
a different way she had started before.
To support this style of development, the exploratory
programming project is creating a variety of visualizations, search
facilities, and interaction techniques to provide effective ways for
users to find, understand, assess, reuse and create variants, and be
able to ask whyquestions to understand differences between
variations of a system. It will also include supports for helping
end-user developers enhance their problem-solving strategies
when they encounter difficulties.
A lifecycle view of the project would not capture everything here:
exploratory programming includes design, implementation,
testing, debugging, and reusebut the human activity of
exploration is more holistic than these five different lifecycle
phases. For example, the lifecycle view would not capture the user
backing out of part of a solution, and would not capture a tool
helping the user find a useful problem-solving strategy. Thus, the
exploratory programming example suggests that first-class
consideration of developer intents (such as explore) may help
EUSE researchers devise future EUSE approaches that transcend
todays lifecycle-phase silos.
Devising new EUSE ideas for tools and techniques that actually
help end-user developers is difficult if we do not understand why
end-user developers might use them or how the new ideas could
work. Understanding these whys and hows has a de-siloing theme
in its ability to connect one type of knowledge with another.
One way to connect our tools and techniques with an
understanding of the developers they target is to ground our EUSE
research efforts in formative empirical studies, so as to understand
the target group of end-user developers. The research paradigm
of grounding research ideas with formative empirical work is
common in the EUSE area, and many of the research efforts we
have summarized here have proceeded in this way.
However, this research paradigm still leaves an important gap: a
missing or weak connection among similar EUSE research
projects. To close this gap, we argue for the importance of more
theoretical grounding in future EUSE work. Theory can help
yield a more holistic understanding of how EUSE results fit
together. Although a few EUSE projects have been human-theory-
based, such projects are still relatively rare.
As an example of the benefits such theories can bring to EUSE,
recall from Section 2 how the Model of Attention Investment [5]
provides a generalized way to gain insights into how and whether
an end-user developer might go about deciding whether to even
try a new EUSE tool or feature. Indeed, this model has already
been used to shape and predict acceptance of a variety of EUSE
approaches (e.g., [8, 10, 50, 59, 64]). Informing EUSE
approaches from the outset using a common model (here,
Attention Investment) connects seemingly disparate tools and
approaches through their shared set of fundamentals.
A second way theory can benefit EUSE research is to guide the
pre-study design and post-study interpretation of empirical
studies. For example, Ko and Myers developed a theoretical
framework and methodology for investigating causes of software
errors in programming systems [30]. This framework draws from
a cognitive theory of human errors and situates it with results
from many of the pioneering empirical studies of programming.
The resulting framework and related empirical methodology has
been used to iteratively evaluate and design the end-user
programming system Alice [13], and to then inform the work on
Whyline for Alice [31].
Theory can also connect tools after the facteven tools that, on
the surface, seem to have little to do with one another. For
example, Information Foraging Theory, although a relative
newcomer to software engineering, has already been used to show
commonalities and design patterns among more than 20 different
software engineering tools spanning three domains: debugging,
refactoring, and reuse [22]. Information Foraging Theory [43],
which is rooted in earlier foraging theories from biology, is a
predator/prey theory that can both explain and predict how
software developers (information predators) go about gathering
the information they need (the prey) when performing
information-intensive software tasks like debugging. With the
exception of emerging work by Kuttal et al. [33], Information
Foraging Theory has not yet been applied to the design of EUSE
Another predator/prey model, which centers on notions of
ecosystem diversity, shows a fourth way theories can benefit
future EUSE work: using theory-derived measures to understand
why and how empirical measures of developer behaviors connect
with empirical measures of artifacts that might seem to be
surrogates for developer measures [44]. Posnett et al. used this
theory to derive complementary measures of focusfor software
development (by professional developers): developer attention
focus and module activity focus. By using symmetric measures
and concepts for both human diversity of attention among models
and module diversity of exposure to humans, the theory provides
a new way to unify and to contrast results from data about humans
and data about artifacts. Demonstrating its usefulness, the results
of their empirical case study on the concept of focus captured
nuanced advantages and disadvantages of focusing developer
attention on a small number of modules that prior atheoretic
measures have not been able to expose.
EUSE research began with the following overarching research
question: Is it possible to bring the benefits of rigorous
software engineering methodologies to end users?
To investigate this goal, EUSE researchers have built upon a
blend of foundations from software engineering and human-
computer interaction. The results may have implications not only
for future EUSE research, but also for classic software
engineering research into how to support professional software
The challenges of supporting end-user developers to do software
engineering activities lend a magnifying lens to problems that
may also affect professional software developers. A case in point
is the Whyline debugging technique, which was developed
originally for end-user developers in the Alice setting (Figure
2(b)) [31], and was later used as the basis for a Whyline for
professional developers working in Java [32]. Empirical
evaluations of the tool showed that novice Java developers with
the Whyline were twice as fast as expert Java developers without
In addition to the Whyline case, the theory research on software
errors, born in the EUSE setting as discussed in Section 4, has
influenced several mainstream software engineering efforts (e.g.,
[28, 62, 65]). Similarly, research into how gender differences tie
with software tool usage, originally investigated in EUSE [4], was
later shown to be applicable to professional software development
tools as well [9].
At its core, the EUSE vision is of a different kind of partnership
between developers and tools than is sometimes the case in classic
software engineering. In the EUSE partnership vision, a
developers intents and work style take priority over what a tool
might want the developer to dobecause if an end-user
developer does not appreciate the tools behavior, the developer is
likely to simply abandon it.
Because of this priority, and also because of its dual heritage from
software engineering and human-computer interaction, EUSE
research is arguably ahead of classic software engineering in
research that focuses squarely on the developers and their intents.
As the examples above demonstrate, bringing these focuses into
classic software engineering research can bring benefits to
professional developers, just as they do to end-user developers.
Indeed, going beyond the examples above, the use of more
advanced intent-oriented approaches such as building intent and
inferring intent (Section 2.2) could be used to entice professional
developers into more productive or effective software engineering
behaviors, or to deliver to them the information they need at the
very moment they need it. Implications for classic software
engineering research then could be to draw upon ideas we have
presented for end-user software engineering, so as to bring
benefits of end-user software engineering research back to
professional developers.
In this paper, we have argued for the benefits of de-siloing to
achieve the kinds of transformative changes that could come
about in EUSEs future. Specifically, we advocate de-siloing
EUSE research from the perspectives of who, when, why and
EUSEs who: EUSE research is beginning to move beyond
assumptions of a fairly homogenous population of end-user
developers. Instead, we argue for the more modern notion that
end-user development is a role, or intent, at some particular time,
not a personal identity. An immediate benefit is that de-siloing
who EUSE targets in this way can open the door to new
populations of end-user developers, working alone or together.
More fundamentally, intent-focused approaches can promote
intent to a first-class concept in EUSE, allowing a rich set of
intent-oriented operations, such as attempting to manipulate it,
infer it, target it, or reward it.
EUSEs when: EUSE already de-silos end-user programming by
moving research beyond the coding phase silo, but it does not go
far enough. We argue that it is not wise, once beyond the coding
silo, to then build a new silo for each phase of the software
lifecycle. Instead, we advocate moving toward research projects
that are not lifecycle-phase-centric, instead focusing on supporting
end-user developers as they actually workopportunistically,
incrementally, and rarely one lifecycle phase at a time. One
possible approach is to start with the activity as the developer sees
it (e.g., explore my options in getting the functionality I want),
rather than starting from a lifecycle phase perspective (e.g.,
EUSEs why and how: We advocate a greater incorporation of
theory into EUSE research. Theories enable us to connect multiple
research efforts, even if on the surface they have little in common.
For example, creating multiple new EUSE approaches from a
common set of foundations connects all the approaches based on
those particular foundations. Finding ways to pattern-match
existing tools to a common theory can also connect those tools.
Theory can even be used to connect disparate data sets, if those
data sets revolve around a concept that can be intellectually
unified and symmetrically measured. In short, theories can enable
EUSE researchers to see through separate silos by generalizing
across them.
Realizing these goals will be challenging, and it would not be
realistic to expect these kinds of changes in the short term. Still,
we argue that end-user software engineering, with its dual heritage
in software engineering and human-computer interaction, is in a
good position to eventually achieve them. By doing so, we can
not only change the reliability that end users can expect from the
software they develop for themselves, but at the same time
contribute to their problem-solving skills, to their enjoyment of
their software development activities, and to our own ability as
researchers to have bigger visions, beyond just one silo at a time.
Wed like to acknowledge the many researchers whose pioneering
works have helped establish the area of end-user software
engineering. We also thank the students and collaborators who
contributed to our own work in end-user software engineering.
Our work in developing this paper was supported in part by the
National Science Foundation under grants IIS-1116724, CNS-
1240957, IIS-1314356, IIS-1314384, and CCF-1302113. Any
opinions, findings and conclusions or recommendations expressed
in this material are those of the author(s) and do not necessarily
reflect those of the National Science Foundation.
[1] Anonymous. 2013. User-unfriendly PR: Nearly 9 of 10 of
Americans have negative feelings about brands with poorly
performing websites and mobile apps, new SOASTA study
reveals, Bulldog Reporter Daily Dog (June 20, 2013),
feelings-about-brands- (Accessed Dec. 8, 2013).
[2] Abraham, R. and Erwig, M. 2009. Mutation operators for
spreadsheets, IEEE Transactions on Software Engineering,
35, 1, 94-108.
[3] Badame, S. and Dig, D. 2012. Refactoring meets spreadsheet
formulas, IEEE International Conference on Software
Maintenance, 399-409.
[4] Beckwith, L., Burnett, M., Wiedenbeck, S., Cook, C., Sorte,
S., Hastings, M. 2005. Effectiveness of end-user debugging
software features: Are there gender issues? ACM Conference
on Human Factors in Computing Systems (Portland, Oregon,
April 2005), 869-878.
[5] Blackwell, A. 2002. First steps in programming: A rationale
for attention investment models, IEEE Human-Centric
Computing Languages and Environments (Arlington, VA,
Sept. 2002), 2-10.
[6] Bogart, C., Burnett, M., Cypher, A., Scaffidi, C. 2008. End-
user programming in the wild: A field study of CoScripter
scripts, IEEE Symposium on Visual Languages and Human-
Centric Computing (Herrshing am Ammersee, Germany,
Sept. 2008), 39-46.
[7] Brandt, J., Guo, P., Lewenstein, J., and Klemmer, S. 2008.
Opportunistic programming: How rapid ideation and
prototyping occur in practice. ICSE08 Workshop on End-
User Software Engineering. (Leipzig, Germany, May 2008).
[8] Burnett, M., Cook, C., Pendse, O., Rothermel, G., Summet,
J. and Wallace, C. 2003. End-user software engineering with
assertions in the spreadsheet paradigm, International
Conference on Software Engineering, (Portland, OR, May
2003), 93-103.
[9] Burnett, M., Fleming, S., Iqbal, S., Venolia, G., Rajaram, V.,
Farooq, U., Grigoreanu, V., Czerwinski, M. Gender
differences and programming environments: Across
programming populations, ACM-IEEE International
Symposium on Empirical Software Engineering and
Measurement, (Bolzano-Bozen, Italy, Sept. 2010), Article
28, 10 pages.
[10] Cao, J., Kwan, I., Bahmani, F., Burnett, M., Fleming, S.,
Jordahl, J., Horvath, A., Yang, S. 2013. End-user
programmers in trouble: Can the Idea Garden help them to
help themselves? IEEE Symposium on Visual Languages and
Human-Centric Computing (San Jose, CA, Sept. 2013), pp.
[11] Cao, J., Riche, Y., Wiedenbeck, S., Burnett, M., Grigoreanu,
V. 2010. End-user mashup programming: Through the
design lens, ACM Conference on Human Factors in
Computing Systems, 1009-1018.
[12] Cao, X. and Iverson, L. 2006. Intentional access
management: Making access control usable for end-users,
Symposium on Usable Privacy and Security (SOUPS), (July
2006, Pittsburgh, PA), 20-31.
[13] Conway, M., Audia, S., Burnette, T., Cosgrove, D.,
Christiansen, K., Deline, R., Durbin, K., Gossweiler, R.,
Koga, S., Long, C., Mallory, B., Miale, S., Monkaitis, J.,
Patten, J., Pierce, J., Shochet, J., Staack, D., Stearns, B.,
Stoakley, R., Sturgill, S., Viega, J., White, J., Williams, G.,
Pausch, R. 2000. Alice: Lessons learned from building a 3D
system for novices, ACM Conference on Human Factors in
Computing Systems, 486493.
[14] Costabile, M., Mussio, P., Provenza, L., Piccinno, A. 2008.
End users as unwitting software developers. ICSE08
Workshop on End-User Software Engineering. (Leipzig,
Germany, May 2008). 6-10.
[15] Costabile, M., Mussio P., Provenza L., and Piccinno A. 2009.
Supporting end users to be co-designers of their tools. 2nd
International Symposium on End-User Development (LNCS
5435), (Siegen, Germany, Springer-Verlag, Berlin
Heidelberg, March 2009), 70-85.
[16] Cunha, J., Fernandes, J., Mendes, J., Pacheco, H., Saraiva, J.
2012. Bidirectional transformation of model-driven
spreadsheets, 5th International Conference on Model
Transformation (ICMT 12), 105-120.
[17] Eagan, J. and Stasko, J. 2008. The Buzz: Supporting user
tailorability in awareness applications. ACM Conference on
Human Factors in Computing Systems, (Florence, Italy,
April 2008), 1729-1738.
[18] Erwig M., Abraham R., Cooperstein I., Kollmansberger S.
2005. Automatic generation and maintenance of correct
spreadsheets. International Conference on Software
Engineering (St. Louis, Missouri, May 2005), 136-145.
[19] European Spreadsheet Risk Interest Group, EuSpRIG horror
stories: Spreadsheet mistakes news stories, (accessed Dec. 8,
[20] Fischer, G. and Giaccardi, E. 2006. Meta-design: A
framework for the future of end user development. In H.
Lieberman, F. Paternò, & V. Wulf (Eds.), End User
Development (Kluwer Academic Publishers, Dordrecht, The
Netherlands), 427-457.
[21] Fisher, M., Rothermel, G., Creelan T. and Burnett, M. 2006.
Scaling a dataflow testing methodology to the multiparadigm
world of commercial spreadsheets, IEEE International
Symposium on Software Reliability Engineering, (Raleigh,
NC, November 2006), 13-22.
[22] Fleming, S., Scaffidi, C., Piorkowski, D., Burnett, M.,
Bellamy, R., Lawrance, J., Kwan, I. 2013. An information
foraging theory perspective on tools for debugging,
refactoring, and reuse tasks, ACM Transactions on Software
Engineering and Methodology, 22, 2, (March 2013), Article
14, 41 pages.
[23] Groce, A., Kulesza, T., Zhang, C., Shamasunder, S., Burnett,
M., Wong, W., Stumpf, S., Das, S., Shinsel, A., Bice, F.,
McIntosh, K. (to appear). You are the only possible oracle:
Effective test selection for end users of interactive machine
learning systems, IEEE Transactions on Software
Engineering (to appear).
[24] Hermans, F., Sedee, B., Pinzger, M., Van Deursen, A. 2013.
Data clone detection and visualization in spreadsheets,
International Conference on Software Engineering, 292-301.
[25] Johnson, G. 1997. Labview Graphical Programming:
Practical Applications in Instrumentation and Control, 2nd
Edition. (McGraw-Hill, New York).
[26] Kahler, H. 2001. More than words - Collaborative tailoring
of a word processor. Journal of Universal Computer Science,
7, 9, 826-847.
[27] Kerr, J., Chou, M., Ellis, R., Kelleher, C. 2013. Setting the
scene: Scaffolding stories to benefit middle school
students learning to program, IEEE Symposium on Visual
Languages and Human-Centric Computing, (San Jose, CA,
Sept. 2013), 95-98.
[28] Kim, S., Zimmermann, T., Whitehead, J., and Zeller, A.
2007. Predicting faults from cached history, International
Conference on Software Engineering, 489-498
[29] Ko, A., Abraham, R., Beckwith, L., Blackwell, A., Burnett,
M., Erwig, M., Scaffidi, C., Lawrance, J., Lieberman, H.,
Myers, B., Rosson, M., Rothermel, G., Shaw, M.,
Wiedenbeck, S. 2011. The state of the art in end-user
software engineering, ACM Computing Surveys, 43, 3 (April
2011), Article 21, 44 pages.
[30] Ko, A. and Myers, B. 2005. A framework and methodology
for studying the causes of software errors in programming
systems, Journal of Visual Languages and Computing 16, 1
(Feb. 2005), 41-84.
[31] Ko, A. and Myers, B. 2004. Designing the Whyline: A
debugging interface for asking questions about program
behavior, ACM Conference on Human Factors in Computing
Systems (Vienna, Austria, April 2004), 151-158.
[32] Ko A. and Myers, B. 2008. Debugging reinvented: Asking
and answering why and why not questions about program
behavior, International Conference on Software Engineering
(Leipzig, Germany, May 2008), 301-310.
[33] Kuttal, S., Sarma, A., Rothermel, G. 2013. Predator behavior
in the wild web world of bugs: An information foraging
theory perspective, IEEE Symposium on Visual Languages
and Human-Centric Computing (San Jose, CA, Sept. 2013),
[34] Lee, B., Kumar, S., Brafman, R., Klemmer, S. 2010.
Designing with interactive example galleries, ACM
Conference on Human Factors in Computing Systems
(Atlanta, GA, Apr. 2010), 2257-2268.
[35] Lee, M. and Ko, A. 2012. Investigating the role of purposeful
goals on novicesengagement in a programming game. IEEE
Symposium on Visual Languages and Human-Centric
Computing, 163-166.
[36] Li, S., Xie, T., Tillmann, N. 2013. A comprehensive field
study of end-user programming on mobile devices, IEEE
Symposium on Visual Languages and Human-Centric
Computing (San Jose, CA, Sept. 2013), 43-50.
[37] Lieberman H., Paterno F. and Wulf V. (Eds). 2006. End-
User Development. (Kluwer / Springer. 2006).
[38] Lin, J., Wong, J., Nichols, J., Cypher, A., Lau, T. 2009. End-
user programming of mashups with Vegemite, ACM
International Conference on Intelligent User Interfaces
(Sanibel Island, Florida, Feb. 2009), 97-106.
[39] Nardi B. A Small Matter of Programming: Perspectives on
End User Computing. 1993. (Cambridge, MA: The MIT
[40] Oxford Dictionary,
nglish/silo (accessed Dec. 9, 2013).
[41] Ozenc, F., Kim, M., Zimmerman, J., Oney, S., Myers B.
2010. How to support designers in getting hold of the
immaterial material of software, ACM Conference on Human
Factors in Computing Systems (Atlanta, GA., April 2010),
[42] Petre M. and Blackwell, A. 2007. Children as unwitting end-
user programmers, IEEE Symposium on Visual Languages
and Human-Centric Computing (Couer d’Alene, Idaho, Sept.
2007), 239-242.
[43] Pirolli, P. and Card, S. 1999. Information foraging. Psych.
Rev. 106, 4, 643675.
[44] Posnett, D., D’Souza, R., Devanbu, P., Filkov, V. 2013. Dual
ecological measures of focus in software development,
International Conference on Software Engineering, 452-461.
[45] Raz O., Koopman P., Shaw M. 2002. Semantic anomaly
detection in online data sources. International Conference on
Software Engineering (Orlando, Florida, May 2002), 302-
[46] Repenning, A. 2011. Programming goes back to school,
Communications of the ACM, 55, 5 (May 2012), 38-40.
[47] Rode, J., Rosson, M., Perez-Quinones, M. 2006. End user
development of web applications, End User Development
(Springer), 161-182.
[48] Rosson, M., Sinha H., Bhattacharya, M., Zhao, D. 2007.
Design planning in end-user web development. IEEE
Symposium on Visual Languages and Human-Centric
Computing, 189-196.
[49] Rothermel, G., Burnett, M., Li, L., Dupuis, C. and Sheretov,
A. 2001. A methodology for testing spreadsheets, ACM
Trans. Software Engineering and Methodology 10, 1 (Jan.
2001), 110-147.
[50] Ruthruff, J., Prabhakararao, S., Reichwein, J., Cook, C.,
Creswick, E. and Burnett, M. 2005. Interactive, visual fault
localization support for end-user programmers, Journal of
Visual Languages and Computing, 16, 1-2 (February/April
2005), 3-40.
[51] Scaffidi, C., Ko, A., Myers, B., and Shaw, M. 2006.
Dimensions characterizing programming feature usage by
information workers, IEEE Symposium on Visual Languages
and Human-Centric Computing, 59-64.
[52] Scaffidi, C., Myers, B., and Shaw, M. 2006. Challenges,
motivations, and success factors in the creation of Hurricane
Katrina person locatorweb sites, Psych. Programming
Interest Group Workshop (Sept. 2006), 9 pages.
[53] Scaffidi, C., Myers, B., and Shaw, M. 2008. Topes: Reusable
abstractions for validating data. International Conference on
Software Engineering, 1-10.
[54] Scaffidi, C., Shaw, M., and Myers, B. 2005. Estimating the
numbers of end users and end user programmers. IEEE
Symposium on Visual Languages and Human-Centric
Computing, 207-214.
[55] Schön, D. 1983. The Reflective Practitioner: How
Professionals Think In Action. (Basic Books, New York,
[56] Segal, J. 2007. Some problems of professional end user
developers, IEEE Symposium on Visual Languages and
Human-Centric Computing, (Couer dAlene, Idaho, Sept.
2007), 111-118.
[57] Shaw, M. 2004. Avoiding costly errors in your spreadsheets.
Contractors Management Report 11, 2-4.
[58] Stolee, K. and Elbaum, S. 2013. Identification, impact, and
refactoring of smells in pipe-like web mashups, IEEE
Transactions on Software Engineering 39, 12 (Dec. 2013),
[59] Subrahmaniyan, N., Burnett, M. and Bogart, C. 2008.
Software visualization for end-user programmers: Trial
period obstacles, ACM Symposium on Software Visualization
(Herrsching am Ammersee, Germany, September 2008),
[60] Tillmann, N., de Halleux, J., Xie, T., Gulwani, S., Bishop, J.
2013. Teaching and learning programming and software
engineering via interactive gaming, International Conference
on Software Engineering (San Francisco, CA, May 2013),
[61] Trigg, R. and Bødker, S. 1994. From implementation to
design: Tailoring and the emergence of systematization.
ACM Conference on Computer Supported Cooperative
Work, (Chapel Hill, North Carolina, October 1994), 45-54.
[62] Walia G. and Carver, J. 2009. A systematic literature review
to identify and classify software requirement errors,
Information and Software Technology 51. 10871109.
[63] Wiedenbeck, S. 2005. Facilitators and inhibitors of end-user
development by teachers in a school environment. IEEE
Symposium on Visual Languages and Human-Centric
Computing, 215222.
[64] Wilson, A., Burnett, M., Beckwith, L., Granatir, O., Casburn,
L., Cook, C., Durham, M. and Rothermel, G. 2003.
Harnessing curiosity to increase correctness in end-user
programming, ACM Conference on Human Factors in
Computing Systems, (Ft. Lauderdale, FL, April 2003), 305-
[65] Xie, J., Chu, V., Lipford, H. and Melton, J. 2011. ASIDE:
IDE support for web application security, ACM Annual
Computer Security Applications Conference (Dec. 2011).
... The term end-user was initially introduced to distinguish those who purely use software systems from professional developers. Now, it refers to people (such as secretaries, accountants, teachers) who develop software as part of their professional practice, without having either a degree in CS/SE or extensive experience in software development (Burnett & Myers, 2014;Costabile et al., 2008;Ko et al., 2011;Ye & Fischer, 2007). In 2005, there were about three million professional programmers in the U.S. (Scaffidi et al., 2005), and over 12 million more people said they were programming at work even though it was not their job description. ...
... • they have a specific idea that they want to develop during the activity Agile methods accommodate end-users' working style (Fronza, et al., 2017;Kastl et al., 2016;Meerbaum-Salant & Hazzan, 2010), which is preferably collaborative (Costabile et al., 2008), opportunistic, incremental, and by trial-and-error phases (Burnett & Myers, 2014). Thus, we select the following XP practices by considering participants' characteristics and activities focus (i.e., programming, process, team) : ...
... Participants are generally open to expanding their software development skills, but they do not expect a programming course. For this reason, we help them explore their design ideas by using manipulatable examples (Burnett & Myers, 2014), from the perspective of learning-by-doing. Moreover, we let them create new configurations and designs by tailoring software components in their software environments. ...
Full-text available
In this work, we present an instructional strategy to expose end-users to Agile-based Software Engineering (SE) practices and enhance their ability to developing high-quality software. Moreover, we introduce a SE approach for the collection of metrics to assess the effectiveness of the instructional strategy. We conducted two case studies to validate the effectiveness of our strategy; the comprehensive analysis of the outcome products evaluates the strategy and demonstrates how to interpret the collected metrics.
... SOA facilitates isolation practice of software development. Earlier SILO [15] based approach had been inefficient where same functionalities were being developed, deployed and maintained multiple times by other organizations refer to Figure 1 Every actor writes their messages to other actor's mailbox for sending their request to be processed. The response sent by another actor is written back to callback queue of the actors. ...
Full-text available
Fascinated with robotics open source integrated component MARIE [1], ROS [5] and open source component PLAYER/STAGE [2], CARMEN [3] available in the eld of robotics application. We aim at developing an asynchronous concurrent robotics application based on subsumption [6] architecture using PLAYER, STAGE and already available services freely available on the internet. We used wanderer and wall follower services embedded inside an actor [10][11] and integrated these services to two dierent robots of same congurations on our promise based frame-work that provides actor as service. The two robots switch their services based on color detected (red,green,blue) during their movement in an environment asynchronously. Using actor as a service lls the gap of SOA [4] & EDA [7] by providing synchronous and asynchronous support for communication. We measured the performance of time taken in completion of services in promise [8] based implementation ,synchronous and asynchronous callback [9] based implementation. We developed a model to prove deadlock freeness in our integrated architecture using petrinet [15] interface composition. We have also been able to justify that component integrated on promise based framework takes less time in service completion than synchronous and asynchronous callback based services.
... In order to bring the benefits of a Software Engineering (SE) approach to end-users, Burnett and Myers recommend to consider and respect the student's (i.e., end-users in EUSE terminology) goals and working style [8]. The latter can often be opportunistic and incremental, collaborative, and organised into trial-and-error phases. ...
Conference Paper
Full-text available
Software engineering principles are relevant to students at school, irrespective of whether they will become professional developers. We discuss two types of situations for teaching software engineering principles that we covered in a number of publications. Firstly, we discuss app development and robotics examples as more traditional settings for software development. Secondly, we show how the creation of infographics and videos represents an opportunity to promote agile approaches. In schools, computer science subjects are often taught in the context of other subjects, which we address through the second situation above. We focus on a range of activities that aim at fostering software engineering principles, while also considering the achievement of the existing curricular learning objectives.
Modern software does not stand alone; it is embedded in complex physical and sociotechnical systems. It relies on computational support from interdependent subsystems as well as non-code resources such as data, communications, sensors, and interactions with humans. Both general-purpose programming languages and mainstream programming language research focus on symbolic notations with well-defined abstractions that are intended for use by professionals to write programs that solve precisely specified problems. There is a strong emphasis on correctness of the resulting programs, preferably by formal reasoning. However, these languages, despite their careful design and formal foundations, address only a modest portion of modern software and only a minority of software developers. Several persistent myths reinforce this focus. These myths express an idealized model of software and software development. They provide a lens for examining modern software and software development practice: highly trained professionals are outnumbered by vernacular developers. Writing new code is dominated by composition of ill-specified software and non-software components. General-purpose languages may be less appropriate for a task than domain-specific languages, and functional correctness is often a less appropriate goal than overall fitness for task. Support for programming to meet a specification is of little help to people who are programming in order to understand their problems. Reasoning about software is challenged by uncertainty and nondeterminism in the execution environment and by the increasingly dominant role of data, especially with the advent of systems that rely on machine learning. The lens of our persistent myths illuminates the dissonance between our idealized view of software development and common practice, which enables us to identify emerging opportunities and challenges for programming language research.
Full-text available
The current generation of connected devices and the Internet of Things augment people’s capabilities through ambient intelligence. Ambient Intelligence (AmI) support systems contain applications consuming available services in the environment to serve users. A well-known design of these applications follows a service architecture style and implement artificial intelligence mechanisms to maintain an awareness of the context: The service architecture style enables the distribution of capabilities and facilitates interoperability. Intelligence and context-awareness provide an adaptation of the environment to improve the interaction. Smart objects in distributed deployments and the increasing machine awareness of devices and people context also lead us to architectures, including self-governed policies providing self-service. We have systematically reviewed and analyzed ambient system governance considering service-oriented architecture (SOA) as a reference model. We applied a systematic mapping process obtaining 198 papers for screening (out of 712 obtained after conducting searches in research databases). We then reviewed and categorized 68 papers related to 48 research projects selected by fulfilling ambient intelligence and SOA principles and concepts. This paper presents the result of our analysis, including the existing governance designs, the distribution of adopted characteristics, and the trend to incorporate service in the context-aware process. We also discuss the identified challenges and analyze research directions.
Full-text available
This paper presents how the TAREME (Trigger-Action Rule Editing, Monitoring, Executing) platform provides support for executing and analysing personalized automations in Internet of Things scenarios. The platform allows the creation and execution of trigger-action personalization rules that can change the state of connected smart objects and devices, send alarms or reminders, and modify applications’ state depending on contextual events. This paper focuses on how the platform supports analytics about the actual use of the rules and provides associated information, which can be useful to better understand users’ personalization needs. Such features have been deployed in a first round of six trials, which have shown the feasibility of the approach and reported fruitful feedback.
End-user (non-professional) programmers often opportunistically create programs, they evaluate various alternatives and reuse existing code by merging components from it or modifying it to suit the context or problems of their programs. Finding and evaluating which program variants to reuse code from is challenging because the searching mechanisms within online repositories are not optimal. To understand the reuse behavior of end-user programmers and to provide implications on how to further support them, we conducted an empirical study in which eight end-user programmers foraged in online repositories, specifically App Inventor Gallery and File Exchange. Using Information Foraging Theory, we qualitatively analyzed the end-user programmers’ behavior and focused on not only program variants from a single source, but also on similar variants from various sources developed over time and by different authors. This analysis revealed new cue types and strategies specific to novice and experienced end-user programmers as they foraged between- and within-variants.
An increasing number of activities is proposed to students at all school levels to foster programming and computational thinking skills. However, limited effort has been spent so far to facilitate the acquisition of Software Engineering (SE) principles, even if this would foster a set of crucial skills. In particular, Agile methods are a good candidate for the educational context. The endeavor to bring SE principles to non-vocational high schools sets several challenges. For the existing wide range of schools, customized approaches are needed, which leverage different types of activities (e.g., programming or non-programming activities). Moreover, SE should be taught while pursuing the existing curricular objectives during activities that can tap into students’ interests. This work describes and compares different approaches to foster SE principles in diverse types of non-vocational schools. The presented approaches take into account the final curricular objectives; moreover, Agile methods are practically applied and repeated until they become a habit to students. We evaluated the effectiveness of the proposed approaches through classroom experiences, which show that the participants learned how to organize their activities by applying a SE approach.
Conference Paper
The facilitation of coding camps is complicated by the fact that these events usually attract large groups of participants, having different backgrounds, ages, needs, and cultural characteristics. Moreover, facilitators typically do not know each participant's learning style and needs. This work describes the RoboCards, a set of cards to solve an introductory robotics exercise that we have designed to support the facilitation of robotics camps for beginners (aged 6 to 12). Moreover, this work reports the first observations that we have collected while facilitating our robotics camps using this tool and sheds light on future work.
Conference Paper
Full-text available
TouchDevelop represents a new programming environment that enables users to develop mobile applications directly on mobile devices. TouchDevelop has successfully drawn a huge number of end users, who have published thousands of TouchDevelop scripts online. To enhance end-user programming on mobile devices, we conduct a comprehensive field study of 17322 TouchDevelop scripts and 4275 users. Our study consists of an overall study on the characteristics of scripts (e.g., structural features, code reuse) and users (e.g., expertise), and a longitudinal study on how they evolve over time. Our study results show important characteristics of scripts such as dense external method calls, high code-reuse ratio, and also reveal interesting evolution patterns of users. The findings and implications in our study provide valuable guidelines for improving tool support or services for end users and increasing the popularity of end-user programming on mobile devices.
Conference Paper
Full-text available
End-user programmers often get stuck because they do not know how to overcome their barriers. We have previously presented an approach called the Idea Garden, which makes minimalist, on-demand problem-solving support available to end-user programmers in trouble. Its goal is to encourage end users to help themselves learn how to overcome programming difficulties as they encounter them. In this paper, we investigate whether the Idea Garden approach helps end-user programmers problem-solve their programs on their own. We ran a statistical experiment with 123 end-user programmers. The experiment's results showed that, even when the Idea Garden was no longer available, participants with little knowledge of programming who previously used the Idea Garden were able to produce higher-quality programs than those who had not used the Idea Garden.
Web active end users often coalesce web information using web mashups. Web contents, however, tend to evolve frequently, and along with the black box nature of visual languages this complicates the process of debugging mashups. While debugging, end users need to locate faults within the code and then find a way to correct them; this process requires them to seek information related to web page content and behavior. In this paper, using an information foraging theory lens, we qualitatively study the debugging behaviors of 16 web-active end users. Our results show that the stronger scents available within mashup programming environments can improve users’ foraging success. Our results lead to a new model for debugging activities framed in terms of information foraging theory, and to a better understanding of ways in which end-user programming environments can be enhanced to better support debugging.
Conference Paper
Web active end users often coalesce web information using web mashups. Web contents, however, tend to evolve frequently, and along with the black box nature of visual languages this complicates the process of debugging mashups. While debugging, end users need to locate faults within the code and then find a way to correct them; this process requires them to seek information related to web page content and behavior. In this paper, using an information foraging theory lens, we qualitatively study the debugging behaviors of 16 web-active end users. Our results show that the stronger scents available within mashup programming environments can improve users' foraging success. Our results lead to a new model for debugging activities framed in terms of information foraging theory, and to a better understanding of ways in which end-user programming environments can be enhanced to better support debugging.
Conference Paper
Research suggests that storytelling can motivate middle school students to explore computer programming. However, difficulties finding and realizing story ideas can decrease time actually spent on programming. In this paper, we present guidelines for constructing story scenes that reliably inspire ideas for novice programmers creating stories. To evaluate the impact of pre-built scenes with strategic design constraints on early programming behavior and attitudes, we conducted a between-subjects study comparing participants who used pre-built scenes and participants who crafted their own scenes. The results suggest that story starter scenes enable novice users to explore programming in the environment sooner, allow users to add and modify significantly more novel programming constructs during the length of the study, and maintain motivation for learning to program via storytelling.
Engagement is a necessary condition for learning, and previous studies have shown that engagement can be significantly affected by changing the presentation of game elements within an educational game. In a three condition controlled experiment, we examined how changing the presentation of the data elements referred to in a game's goals would influence the purposefulness of the goals and thereby affect players' motivation to achieve them. A total of 121 self-described programming novices were recruited online to play the game. We found that 1) those using vertebrate elements completed twice the number of levels compared to those using inanimate elements, 2) those using vertebrate and invertebrate elements spent significantly more time playing the game overall compared to those using inanimate elements, and 3) those using inanimate elements were more likely to quit the game, especially on difficult levels. These findings suggest that the presentation of game elements that influence the purposefulness of goals can play a significant role in keeping self-guided learners engaged in learning tasks.
Theories of human behavior are an important but largely untapped resource for software engineering research. They facilitate understanding of human developers’ needs and activities, and thus can serve as a valuable resource to researchers designing software engineering tools. Furthermore, theories abstract beyond specific methods and tools to fundamental principles that can be applied to new situations. Toward filling this gap, we investigate the applicability and utility of Information Foraging Theory (IFT) for understanding information-intensive software engineering tasks, drawing upon literature in three areas: debugging, refactoring, and reuse. In particular, we focus on software engineering tools that aim to support information-intensive activities, that is, activities in which developers spend time seeking information. Regarding applicability, we consider whether and how the mathematical equations within IFT can be used to explain why certain existing tools have proven empirically successful at helping software engineers. Regarding utility, we applied an IFT perspective to identify recurring design patterns in these successful tools, and consider what opportunities for future research are revealed by our IFT perspective.
Conference Paper
The number of end-users who write spreadsheet programs is at least an order of magnitude larger than the number of trained programmers who write professional software. We studied a corpus of 3691 spreadsheets and we found that their formulas are riddled with the same smells that plague professional software: hardcoded constants, duplicated expressions, unnecessary complexity, and unsanitized input. These make spreadsheets difficult to read and expensive to maintain. Like automated refactoring in the object-oriented domain, spreadsheet refactoring can be transformative. In this paper we present seven refactorings for spreadsheet formulas implemented in RefBook, a plugin for Microsoft Excel. To evaluate the usefulness of RefBook, we employed three kinds of empirical methods. First, we conducted a User Survey with 28 Excel users to find out whether they preferred the refactored formulas. Second, we conducted a Controlled Experiment with the same 28 participants to measure their productivity when doing manual refactorings. Third, we performed a Retrospective Case Study on the EUSES Spreadsheet Corpus with 3691 spreadsheets to determine how often we could apply the refactorings supported by RefBook. The results show: (i) users prefer the improved quality of refactored formulas, (ii) RefBook is faster and more reliable than manual refactoring, and (iii) the refactorings are widely applicable. On average RefBook is able to apply the refactorings in less than half the time that users performed the refactorings manually. 92.54% of users introduced errors or new smells into the spreadsheet or were unable to complete the task.
Conference Paper
Massive Open Online Courses (MOOCs) have recently gained high popularity among various universities and even in global societies. A critical factor for their success in teaching and learning effectiveness is assignment grading. Traditional ways of assignment grading are not scalable and do not give timely or interactive feedback to students. To address these issues, we present an interactive-gaming-based teaching and learning platform called Pex4Fun. Pex4Fun is a browser-based teaching and learning environment targeting teachers and students for introductory to advanced programming or software engineering courses. At the core of the platform is an automated grading engine based on symbolic execution. In Pex4Fun, teachers can create virtual classrooms, customize existing courses, and publish new learning material including learning games. Pex4Fun was released to the public in June 2010 and since then the number of attempts made by users to solve games has reached over one million. Our work on Pex4Fun illustrates that a sophisticated software engineering technique-automated test generation-can be successfully used to underpin automatic grading in an online programming system that can scale to hundreds of thousands of users.