Content uploaded by Jeff Sutherland
Author content
All content in this area was uploaded by Jeff Sutherland on Mar 14, 2016
Content may be subject to copyright.
Get the Cutter Edge free: www.cutter.com/consortium/
Agile Can Scale: Inventing and Reinventing
SCRUM in Five Companies
agile can scale
by Jeff Sutherland
In recent months, a wide range
of publications Software
Development, IEEE Software,
Cutter IT Journal, Software Testing
and Quality Engineering, and even
The Economist have published
articles on agile software develop-
ment methodologies, reflecting a
growing interest in these new
approaches to software develop-
ment (Extreme Programming [XP],
Crystal Methodologies, SCRUM,
Adaptive Software Development,
Feature-Driven Development, and
Dynamic Systems Development
Methodology among them). In
addition to these named
methodologies, scores of organiza-
tions have developed their own
lighter approaches to building
software. The formation of the
Agile Alliance by a group of
expert consultants and authors
on development process has
fueled increasing interest in
ways to deliver quality software
in short, fixed delivery schedules,
under severe time-to-market
pressures [8].
The goal of SCRUM is to deliver as
much quality software as possible
within a series of short time-boxes
called sprints, which last about a
month. SCRUM is characterized by
short, intensive, daily meetings of
every person on a software
delivery team, usually including
product marketing, software
analysts, designers, and coders,
and even deployment and support
staff. SCRUM project planning uses
lightweight techniques such as
Burndown Charts, as opposed to
Gantt charts. A Gantt chart is only
as good as the assumptions
inherent in the critical path repre-
sented on the chart. In agile devel-
opment, the critical path usually
changes daily, rendering any given
Gantt chart obsolete within 24
hours. The solution is using a tech-
nique to calculate the velocity of
development. The neural networks
in the brains of team members are
used on a daily basis to recalculate
the critical path. This allows the
plan to be recalculated and the
velocity of burndown of work to
be computed. Team efforts to
accelerate or decelerate the
velocity of burndown allow a team
to fly the project into a fixed
delivery date.
A typical Burndown Chart is illus-
trated in Figure 1. It consists of the
cumulative time it takes to
complete outstanding tasks for
deliverable software for a SCRUM
sprint. Each developer breaks
down tasks into small pieces and
enters into an automated backlog
system two variables on each
active task every day. The auto-
mated system then estimates daily
the remaining work for each task
and sums the work remaining for
each task to generate the cumula-
tive backlog. Requiring only one
minute of each developers time
each day to update two data items
for active tasks, the automated
system produces the Burndown
Chart. It shows how fast the
140
120
100
80
60
40
20
0
26-Sep 6-Oct 16-Oct 26-Oct 5-Nov 15-Nov
PatientKeeper
Backlog Days
Figure 1 — Burndown chart. (Source: Advanced Development Methodologies)
Vol. 14, No. 12 5
outstanding backlog is decreasing
each day. In the daily SCRUM
meetings, the team members
determine what actions taken that
day will maximize the downward
movement of the cumulative
backlog. This is equivalent to the
team manually recalculating the
critical path of the project during
a 15-minute daily meeting. Exper-
ience has shown that SCRUM
project planning will consistently
produce a faster path to the end
goal than any other form of project
planning reported to date, with
less administrative overhead than
any previously reported approach.
Details of the SCRUM approach
have been carefully documented
elsewhere. SCRUM is the only
agile methodology that has been
formalized and published as an
organizational pattern for software
development [2]. The process
assumes that requirements will
change during the period between
initial specification and delivery of
a product. It supports Humphreys
Requirements Uncertainty Prin-
ciple [9], which states that for a
new software system, the require-
ments will not be completely
known until after the users have
used it. SCRUM allows for Zivs
Uncertainty Principle in software
engineering, which observes
that uncertainty is inherent and
inevitable in software development
processes and products [15]. And
it accounts for Wegners mathe-
matical proof (lemma) that it is not
possible to completely specify an
interactive system [14]. Most soft-
ware systems built today are
object-oriented implementations,
and most of those object-oriented
systems depend on environmental
inputs to determine process
outputs (i.e., they are interactive
systems).
Traditional, heavyweight software
methodologies assume that
requirements can be specified in
advance, that they will not change
during development, that the users
know what they want before they
see it, and that software develop-
ment is a predictable, repeatable
process. These assumptions are
fundamentally flawed and incon-
sistent with the mathematical
lemmas and principles cited
above. As a result, 31% of software
projects, usually driven by a variant
of the waterfall methodology, are
terminated before completion [3].
This article serves as a short retro-
spective on the origins of SCRUM,
its evolution in five companies,
and a few key learnings along the
way. It will provide a reference
point for further investigation and
implementation of SCRUM for
those interested in using a proven,
scalable, lightweight development
process that supports the princi-
ples of the Agile Alliance as
outlined in the Manifesto for Agile
Software Development (see
www.agile alliance.org).
EASEL CORPORATION:
THE FIRST SCRUM
SCRUM was started in 1993
for software teams at Easel
Corporation, where I was VP of
object technology. In the initial
SCRUM, we built the first object-
oriented design and analysis tool
that incorporated round-trip
engineering. A second SCRUM
implemented the first product to
completely automate object-
relational mapping in an enterprise
development environment. I was
assisted by two world-class devel-
opers Jeff McKenna, now an
Extreme Programming consultant,
and John Scumniotales, now a
development leader for object-
oriented design tools at Rational
Corporation.
In 1995, Easel was acquired by
VMARK. SCRUM continued there
until I joined Individual in 1996
as VP of engineering to develop
Personal Newspage (now
www.office.com). I asked Ken
Schwaber, CEO of Advanced
Development Methodologies, to
help me incorporate SCRUM into
Individuals development process.
In the same year, I took SCRUM to
IDX Systems when I assumed the
positions of senior VP of engi-
neering and product development
and CTO. IDX, one of the largest
US healthcare software compa-
nies, was the proving ground for
multiple team SCRUM implemen-
tations. At one point, almost 600
developers were working on
dozens of products. In 2000,
SCRUM was introduced to
PatientKeeper, a mobile/wireless
healthcare platform company
where I became CTO. So I have
experienced SCRUM in five comp-
anies that varied widely in size.
They were proving grounds for
SCRUM in all phases of company
growth, from startup, to initial IPO,
to mid-sized, and then to a large
6 December 2001 ©2001 Cutter Information Corp.
company delivering enterprise
systems to the marketplace.
“All-at-Once” Software Development
There were some key factors that
influenced the introduction of
SCRUM at Easel Corporation. The
book Wicked Problems, Righteous
Solutions [5] by Peter DeGrace
and Leslie Hulet Stahl reviewed
the reasons why the waterfall
approach to software development
does not work for software devel-
opment today. Requirements are
not fully understood before the
project begins. The users know
what they want only after they see
an initial version of the software.
Requirements change during the
software construction process.
And new tools and technologies
make implementation strategies
unpredictable. DeGrace and Stahl
reviewed All-at-Once models of
software development, which
uniquely fit object-oriented imple-
mentation of software and help to
resolve these challenges.
All-at-Once models of software
development assume that the
creation of software is done by
simultaneously working on
requirements, analysis, design,
coding, and testing and then
delivering the entire system all
at once. The simplest All-at-
Once model is a single super-
programmer creating and
delivering an application from
beginning to end. All aspects of the
development process reside in a
single persons head. This is the
fastest way to deliver a product
that has good internal architectural
consistency, and it is the hacker
model of implementation. The
next level of approach to All-at-
Once development is handcuffing
two programmers together, as in
the XP practice of pair program-
ming [1]. Two developers deliver
the entire system together. This
has been shown to deliver better
code (in terms of usability, main-
tainability, flexibility, and extend-
ibility) faster than work delivered
by larger teams. The challenge is
to achieve a similar productivity
effect in the large with an entire
team and then with teams
of teams.
Our team-based All-at-Once model
was based on both the Japanese
approach to new product develop-
ment, Sashimi, and SCRUM. We
were already using production
prototyping to build software.
It was implemented in slices
(Sashimi) where an entire piece
of fully integrated functionality
worked at the end of an iteration.
What intrigued us was Hirotaka
Takeuchi and Ikujiro Nonakas
description of the team-building
process in setting up and manag-
ing a SCRUM [13]. The idea of
building a self-empowered team
in which everyone had the global
view of the product on a daily
basis seemed like the right idea.
This approach to managing
the team, which had been so
successful at Honda, Canon, and
Fujitsu, resonated with the systems
thinking approach being promoted
by Peter Senge at MIT [12].
We were also impacted by recent
publications in computer science.
As I alluded above, Peter Wegner
at Brown University demonstrated
Building a self-empowered
team in which everyone had
the global view of the
product on a daily basis
seemed like the right idea.
that it was impossible to fully
specify or test an interactive
system, which is designed to
respond to external inputs
(Wegners Lemma) [14]. Here
was mathematical proof that any
process that assumed known
inputs, as does the waterfall
method, was doomed to failure
when building an object-oriented
system.
We were prodded into setting up
the first SCRUM meeting after
reading James Copliens paper on
Borlands development of Quattro
Pro for Windows [4]. The Quattro
team delivered one million lines of
C++ code in 31 months, with a
four-person staff growing to eight
people later in the project. This
was about 1,000 lines of deliver-
able code per person per week,
probably the most productive
project ever documented. The
team attained this level of produc-
tivity by intensive interaction in
daily meetings with project
management, product manage-
ment, developers, documenters,
and quality assurance staff.
Software Evolution and
“Punctuated Equilibrium”
Our daily meetings at Easel were
disciplined in the way we that we
now understand as the SCRUM
pattern [2]. The most interesting
Get the Cutter Edge free: www.cutter.com/consortium/ Vol. 14, No. 12 7
8
effect of SCRUM on Easels devel-
opment environment was an
observed punctuated equilib-
rium effect. A fully integrated
component design environment
leads to rapid evolution of a soft-
ware system with emergent, adap-
tive properties, resembling the
process of punctuated equilibrium
observed in biological species.
It is well understood in biological
evolution that change occurs
sharply at intervals separated by
long periods of apparent stagna-
tion, leading to the concept of
punctuated equilibrium [6].
Computer simulations of this
phenomenon suggest that periods
of equilibrium are actually periods
of ongoing genetic change of an
organism. The effects of that
change are not apparent until
several subsystems evolve in
parallel to the point where they
can work together to produce a
dramatic external effect [10]. This
punctuated equilibrium effect has
been observed by teams working
in a component-based environ-
ment with adequate business
process engineering tools, and the
SCRUM development process
accentuates the effect.
By having every member of the
team see every day what every
other team member was doing,
we began to see how we could
accelerate each others work.
For instance, one developer
commented that if he changed a
few lines of code, he could elimi-
nate days of work for another
developer. This effect was so
dramatic that the project acceler-
December 2001
ated to the point where it had to
be slowed down. This hyperpro-
ductive state was seen in several
subsequent SCRUMs, but never
one so dramatic as the one at
Easel. It was a combination of (1)
the skill of the team, (2) the flexi-
bility of a Smalltalk development
environment, and (3) the way we
approached production prototypes
that rapidly evolved into a deliver-
able product.
The first SCRUM worked from a
unique view of a software system.
A project domain can be viewed
as a set of packages that will form
a release. Packages are what the
user perceives as pieces of func-
tionality, and they evolve out of
work on topic areas (see Figure 2).
Topic areas are business object
components. Changes are intro-
duced into the system by intro-
ducing a unit of work that alters a
component. The unit of work in
the initial SCRUM was called a
Synchstep.
System evolution proceeds in
Synchsteps (see Figure 3). After
one or more Synchsteps have gone
to completion and forced some
Packages
Topics
Figure 2 — Initial SCRUM view
of a software system.
By having every member of
the team see every day what
every other team member
was doing, we began to see
how we could accelerate
each other’s work.
refactoring throughout the system,
a new package of functionality
emerges that is observable to the
user. These Synchsteps are similar
to genetic mutations. Typically,
several interrelated components
must mutate in concert to produce
a significant new piece of function-
ality. This new functionality
appears as a punctuated equilib-
rium effect to builders of the
system. For a period of time, the
system is stable with no new
behavior. Then when a certain
(somewhat unpredictable)
Synchstep completes, the whole
system pops up to a new level of
functionality, often surprising the
development team.
The key to entering a hyperproduc-
tive state was not just the SCRUM
organizational pattern. We did
Figure 3 — Firing a Synchstep.
Packages
Topics
©2001 Cutter Information Corp.
constant component testing of
topic areas, integration of pack-
ages, and refactoring of selected
parts of the system. These activi-
ties have become key features
of XP [7].
Furthermore, in the hyperproduc-
tive state, the initial SCRUM
entered what professional athletes
and martial artists call the zone.
No matter what happened or what
problems arose, the response of
the team always was far better
than the response of any indi-
vidual. It reminded me of the
stories about the Celtics basketball
team at their peak, when they
could do no wrong. The impact of
entering the zone was not just
hyperproductivity. Peoples
personal lives were changed.
Team members said they would
never forget working on such a
project, and they would always be
looking for another experience like
it. It induced open, team-oriented,
fun-loving behavior in unexpected
persons. Those individuals who
could not function well in an open,
hyperproductive environment self-
selected themselves out of the
team by finding other jobs. This
actually reinforced positive team
behavior similar to biological
systems, which select for fitness
to the environment, resulting in
improved performance of indi-
vidual organisms.
VMARK: THE FIRST SENIOR
MANAGEMENT SCRUM
When Easel Corporation was
acquired by VMARK (now
Informix), the original SCRUM
team continued its work on the
same product. The VMARK senior
management team was intrigued
by SCRUM and asked me to run a
weekly senior management team
SCRUM to drive all the companys
products to the Internet. These
meetings started in 1995, and
within a few months, the team had
caused the introduction of two
new Internet products and reposi-
tioned current products as Internet
applications. Some members of
this team left VMARK to become
innovators in emerging Internet
companies, so SCRUM had an
early impact on the Internet.
INDIVIDUAL: THE FIRST INTERNET
SCRUM
In the spring of 1996, I returned
to Individual, Inc., a company I
cofounded as VP of engineering.
Much of the SCRUM experience at
Individual has been documented
by Ken Schwaber [11]. The most
impressive thing to me about
SCRUM at Individual was not that
the team delivered two new
Internet products and multiple
releases of one of the products
in a single quarter. It was the fact
that SCRUM eliminated about sev-
eral hours a day of senior manage-
ment meeting time starting the day
the SCRUM began. Because the
company had just gone public at
the beginning of the Internet
explosion, there were multiple
competing priorities and constant
revision of market strategy. As a
result, the development team was
It was incredibly productive
to force all decisions to
occur in the daily SCRUM
meeting.
constantly changing priorities and
unable to deliver product. The
management team was meeting
daily to determine status of priori-
ties that were viewed differently by
every manager. These meetings
were eliminated immediately, and
the SCRUM served as the focus for
decisionmaking.
It was incredibly productive to
force all decisions to occur in the
daily SCRUM meeting. If anyone
wanted to know the status of
specific project deliverables or
wanted to influence any priority,
he or she could only do it in the
SCRUM. I remember the senior VP
of marketing sat in on every
meeting for a couple of weeks
sharing her desperate concern
about meeting Internet deliver-
ables and timetables. The effect on
the team was not to immediately
respond to her despair. Over a
period of two weeks, the team
self-organized around a plan to
meet her priorities with achievable
technical delivery dates. When she
agreed to the plan, she no longer
had to attend any SCRUM meet-
ings. The SCRUM reported status
on the Web with green lights,
yellow lights, and red lights for all
pieces of functionality. In this way,
the entire company knew status in
real time, all the time.
Get the Cutter Edge free: www.cutter.com/consortium/ Vol. 14, No. 12 9
IDX SYSTEMS: THE FIRST SCRUM
IN THE LARGE
During the summer of 1996, IDX
Systems hired me to be its senior
VP of engineering and product
development. I replaced the
technical founder of the company,
who had led development for
almost 30 years. IDX had over
4,000 customers and was one of
the largest US healthcare software
companies, with hundreds of
developers working on dozens of
products. Here was an opportunity
to extend SCRUM to large-scale
development.
The key learning at IDX
was that SCRUM scales
to any size.
The approach at IDX was to turn
the entire development organiza-
tion into an interlocking set of
SCRUMs. Every part of the organi-
zation was team based, including
the management team, which
included two vice presidents, a
senior architect, and several direc-
tors. Front-line SCRUMs met daily.
A SCRUM of SCRUMs, which
included the team leaders of each
SCRUM in a product line, met
weekly. The management SCRUM
met monthly.
The key learning at IDX was that
SCRUM scales to any size. With
dozens of teams in operation,
the most difficult problem was
ensuring the quality of the SCRUM
process in each team, particularly
when the entire organization had
to learn SCRUM all at once. IDX
was large enough to bring in
productivity experts to monitor
throughput on every project. While
most teams were only able to
meet the industry average in func-
tion points per month delivered,
several teams moved into the
hyperproductive state, producing
deliverable functionality at four to
five times the industry average.
These teams became shining stars
in the organization and examples
for the rest of the organization
to follow.
PATIENTKEEPER SCRUM:
INTEGRATION WITH EXTREME
PROGRAMMING
In early 2000, I joined Patient-
Keeper, Inc. as chief technology
officer and began introducing
SCRUM into a startup company.
I was the 21st employee, and we
grew the development team from
a dozen people to 45 people in six
months. PatientKeeper deploys
mobile devices in healthcare insti-
tutions to capture and process
financial and clinical data. Server
technology synchronizes the
mobile devices and moves data to
and from multiple back-end legacy
systems. A robust technical archi-
tecture provides enterprise appli-
cation integration to hospital and
clinical systems. Data is forward-
deployed from these systems in a
PatientKeeper clinical repository.
Server technologies migrate
changes from our clinical reposi-
tory to a cache and then to data
storage on the mobile device.
PatientKeeper proves that SCRUM
works equally well across tech-
nology implementations.
The key learning at PatientKeeper
has involved the introduction of
Extreme Programming techniques
as a way to implement code deliv-
ered by a SCRUM organization.
While all teams seem to find it
easy to implement a SCRUM orga-
nizational process, they do not
always find it easy to introduce XP.
We have been able to do some
team programming and constant
testing and refactoring, particularly
as we have migrated all develop-
ment to Java and XML. It has been
more difficult to introduce these
ideas when developers are
working in C and C++. After a
year of SCRUM meetings in all
areas of development, our
processes are maturing enough
to capitalize on SCRUM project
management techniques, which
are now being automated.
CONCLUSIONS
After introducing SCRUM into five
different companies of different
sizes and with different technolo-
gies, I can confidently say that
SCRUM works in any environment
and can scale into programming in
the large. In all cases, it will radi-
cally improve communication and
delivery of working code. The next
challenge for SCRUM, in my view,
is to provide a tight integration of
the SCRUM organization pattern
and XP programming techniques.
I believe this integration can
generate a hyperproductive
10 December 2001 ©2001 Cutter Information Corp.
SCRUM on a predictable basis.
The first SCRUM did this intuitively
before XP was born, and that was
its key to extreme performance
and a life-changing experience.
In addition, the participation of
SCRUM leaders in the Agile
Alliance [8], a group which has
absorbed all leaders of well-
known lightweight development
processes, will facilitate wider
use of SCRUM and its adoption
as an enterprise standard
development process.
REFERENCES
1. Beck, K. Extreme Programming
Explained: Embrace Change.
Addison-Wesley, 1999.
2. Beedle, M., M. Devos, et al.
SCRUM: A Pattern Language
for Hyperproductive Software
Development. In Pattern
Languages of Program Design 4,
edited by N. Harrison, B. Foote,
and H. Rohnert. Addison-Wesley,
1999.
3. Boehm, B. Project Termination
Doesnt Mean Project Failure.
IEEE Computer, Vol. 33, No. 9
(September 2000), pp. 94-96.
4. Coplien, J.O. Borland Software
Craftsmanship: A New Look at
Process, Quality, and Productivity.
In Proceedings of the 5th Annual
Borland International Conference.
Borland International, 1994.
5. DeGrace, P., and L.H. Stahl.
Wicked Problems, Righteous
Solutions. Prentice Hall, Yourdon
Press, 1990.
6. Dennett, D.C. Darwins
Dangerous Idea: Evolution and
the Meanings of Life. Simon &
Schuster, 1995.
7. Fowler, M. Is Design Dead?
Software Development, Vol. 9,
No. 4 (April 2001).
8. Fowler, M., and J. Highsmith.
The Agile Manifesto. Software
Development, Vol. 9, No. 8 (August
2001), pp. 28-32.
9. Humphrey, W.S. Introduction to
the Personal Software Process.
Addison-Wesley, 1996.
10. Levy, S. Artificial Life: The Quest
for a New Creation. Pantheon
Books, 1992.
11. Schwaber, K., and M. Beedle.
Agile Software Development with
SCRUM. Prentice Hall, 2001.
12. Senge, P. M. The Fifth
Discipline: The Art and Practice
of the Learning Organization.
Doubleday/Currency, 1990.
13. Takeuchi, H., and I. Nonaka.
The New New Product
Development Game. Harvard
Business Review (January-
February 1986).
14. Wegner, P. Why Interaction Is
More Powerful Than Algorithms.
Communications of the ACM, Vol.
40, No. 5 (May 1997), pp. 80-91.
15. Ziv, H., and D. Richardson. The
Uncertainty Principle in Software
Engineering. In Proceedings of the
19th International Conference on
Software Engineering (ICSE97).
IEEE, 1997.
Jeff Sutherland is CTO of PatientKeeper,
where he directs PatientKeepers
(formerly Virtmeds) team of engineers
in developing additional services and
products. Prior to joining PatientKeeper,
Dr. Sutherland served as CTO at IDX, the
nations third-largest hospital information
systems company, where he was respon-
sible for setting the architectural direction
for products across all business units.
While at IDX, Sutherland adapted the
SCRUM process to large organizations,
and SCRUM has become accepted as an
industry standard for rapid application
development. He also launched
Outreach, the first Web-enabled patient
information system, and IDXSite, the first
Web-based physicians practice manage-
ment system.
Dr. Sutherland is also the founder of two
companies, Object Databases (now
Matisse Software) and Individual, Inc.
While at Object Databases, he developed
and supported multimedia object data-
base systems. As founder and vice presi-
dent of engineering at Individual, Inc.,
he developed the technology for imple-
menting the first automated personal
newsletter.
Dr. Sutherland received his masters of
science degree in statistics and mathe-
matics from Stanford University and his
doctorate in biometrics, medical imaging,
and radiation physics from the University
of Colorado School of Medicine.
Dr. Sutherland can be reached at
PatientKeeper, Inc., 20 Guest Street, Suite
500, Brighton, MA 02135, USA. Tel: +1
617 987 0394; E-mail: jsutherland@
patientkeeper.com.
Get the Cutter Edge free: www.cutter.com/consortium/ Vol. 14, No. 12 11