Content uploaded by Gary Pan
Author content
All content in this area was uploaded by Gary Pan
Content may be subject to copyright.
Why can’t we bet on ISD Outcomes?:
ISD “Form” as a Predictor of Success
Mike Newman1, Shan L Pan2, & Gary Pan3
1Manchester Business School, The University of Manchester & Copenhagen Business School,
Denmark. mike.newman@manchester.ac.uk
2Department of Information Systems, National University of Singapore, Singapore.
pansl@comp.nus.edu.sg
3Singapore Management University. garypan@smu.edu.sg
Abstract: The record of failure to deliver large-scale Information Systems (IS) in a
timely fashion that offer value to major commercial and public organizations is
legendary. Just looking to critical success factors such as top management support and
user involvement in order to understand how to deliver better systems can at best be a
partial solution. We seem to overlook an obvious area in our organizations: what can
we learn from our information system development (ISD) historical patterns? In order
to develop this idea we draw on parallels in sport where current performance and
behaviour are believed to be closely linked to historical precedents, or “form”. In that
domain, historical patterns are a fallible but valuable predictor of success. Our thesis is
that past negative patterns in ISD will tend to repeat themselves without radical
intervention. Put another way, failure begets failure. After examining the game of
football as an allegory for ISD, we look briefly at two organizations that have
experienced a pattern of failure in the IS area in the past but have transformed the way
they build IS, moving from negative patterns to successful ones. The article ends with
suggestions for managers charged with developing new IS as to how they might use
their understanding of ISD “form” to improve their chances of success.
1. Introduction
“Those who cannot remember the past are condemned to repeat it”.
George Santayana
Mike Newman, San Pan & Gary Pan
2
If you browse some of the multitude of gambling websites, you will find a plethora of
opportunities to bet on sporting outcomes as well as other non-sporting activities (e.g.
political outcomes). What you will not find is odds offered the outcome of major
Information Systems (IS) projects such as the NHS’ Connecting for Health or specific
components of it such as the “Choose and Book” system. Yet there are many parallels
between sporting outcomes and the outcomes of major IS projects. It is strongly
believed that a history of success/ failure (“form”) is a strong predictor of sporting
outcomes. Our thesis is that understanding ISD “form” can help IS developers to
understand and predict success and failure and where necessary, to punctuate the
process with more radical strategies.
Much is made in the popular press as well as academic writing of the enigma of
building IS that offer value to commercial as well as public organizations. But the
specter of failure haunts organizations as they pile resources on top of resources but
produce a failing process of IS Development (ISD) akin to a monetary black-hole,
escalating for years until someone “pulls the plug” [2, 6, 12, 16]. Along with a
plethora of IS failures is a multitude of solutions offered. We often look to critical
success factors such as top management support and user involvement in order to
understand how to deliver better systems but this is clearly subject to the critique of
partiality. In looking at critical success factors and concentrating on the present we
seem to be ignoring a source of evidence that has much to teach us: an organization’s
ISD history. If we want to avoid repeating the past failures then organizations must
make a realistic assessment of their history and attempt to transcend them if they
discover weaknesses. Simple tinkering with the success factors will probably not work
(unless you are very lucky). Organizations would do better if they learn from the past
in order not to repeat their mistakes [3, 5, 12] and avoid the trap of “learning to fail”
[8].
Some simple examples from sport may help to illuminate the point. In competitive
sport, pundits try to assess the chances of a team or an individual win by looking to
recent patterns of success and/or failure. In horse racing, a horse (and jockey) is
assessed for its form. Form is a view of the horse’s performance over the last few
races: its recent history. The parallels with ISD are obvious: a horse’s form becomes
the organization’s ISD history; the current conditions and the ability of the jockey can
be compared with the critical success factors in the ISD process. Interestingly, in the
betting industry form is probably more important than all other factors in forming
odds. The major difference with ISD is the time horizon: ISD projects last months or
years rather than a few minutes. Nonetheless, past patterns are believed to have a
strong relationship to current behavior. Without radical change, past negative patterns
of ISD tend to be reproduced in current developments [8].
Why can’t we bet on ISD Outcomes?: ISD “Form” as a Predictor of Success
3
In this article, we begin by using football as an allegory for ISD [13]. We deepen
our analysis by looking at how two large organizations have experienced a pattern of
failure in the ISD in the past but have transformed the way they build IS, moving from
negative patterns to successful ones [11, 18]. The article ends with suggestions for
managers charged with developing new IS as to how they might use ISD form to
improve their chances of success.
1.1 Football as an allegory for ISD
“Allegory: a work in which the characters and events are to be understood as
representing other things and symbolically expressing a deeper, often spiritual, moral,
or political meaning” (Encarta UK Dictionary). In our paper the allegory is a game of
football which is used to represent the way we develop and adopt Information Systems
in Organizations. Figure 1 represents a picture of a field of play in football with the
goals at either end.
Why do we use football as our allegory? Football is a world-wide game and
consequently the basic rules and parameters are widely known. But it is also a
complex game involving many stakeholders (owners, managers, players, reserves,
coaches, scouts, referees, regulatory bodies, media etc.). In ISD there are also many
stakeholders with competing demands. In football, no matter how strong the side is
there is always a risk of losing a particular match. Failure is common in ISD projects
even in previously successful project teams. The 90 minutes game time in football
consists of 11 players per side so it is essentially a team game employing players with
different specialisms (goalkeeper, defenders, midfield and attackers). IS project teams
also have many specialists: from those who interface with users to back-room
technical personnel to telecommunication experts. Also the context(s) of the game is
complex. Much can depend on the physical conditions of the pitch, the weather, the
referee, and in a wider sense the changing regulations agreed by the football governing
bodies (e.g. recent changes to the so-called offside rule; the discussion over the use of
goal-line technology). Context in ISD can provide many unexpected perturbations
(entry and exit of key persons; new technology emerging, budget crises, etc.). Each
football team has “form” or a history of wins, losses and draws and individual players
who are constrained or prevented to play by injuries and the totting up process of
yellow (a caution) and red (sending off) cards awarded in previous matches for foul or
even dangerous play. These injuries and cards can have a significance beyond the
Mike Newman, San Pan & Gary Pan
4
current game, of course. Organizations also have “form” when it comes to their history
of successful and failed projects [14]. In a general sense, we carry our histories into
the future, constraining some events and enabling others.
While there are obvious parallels with ISD, and we shall be elaborating on these
parallels later, there are limitations to this allegory. First is the time dimension: a
football game is 90 minutes of continuous action whereas an ISD project can take
months or years to complete. Second, in a football game we would normally be
watching the game live or mediated through the media. In research we normally
interview third party witnesses for their accounts of events. By careful interpretations
of their accounts and with additional evidence from documents and observations we
build up a story of the IS project [10]. A football game has three possible outcomes for
a club: win, loss or draw. In ISD we talk of success, failure or something less well
defined but, in contrast to football where outcome is never in doubt, ISD outcomes are
often ambiguous and relative1. Finally, a football game is essentially competitive and
can be conflictual. While we acknowledge this, others have drawn parallels with the
“battles” and conflicts that can occur between designers and users in ISD and
sometimes even referring to ISD as a “game” [4]. In summary, every allegory has
limitations and the author and readers alike should not try to read too much into the
story that is told. Used carefully, however, an allegory can reveal deeper insights into
the phenomena we are studying, namely ISD.
1.2 Historical context – (“form” or antecedent conditions)
In summary: in addition to the critical events, a process model [17] looks at the history
or form of the game (figure 1). In the case of a football game this might include among
other issues a side’s recent form (wins, losses, draws), injuries to players, the number
of games played, cards (yellows and reds), and the entry or exit of players or manager
from the club. These issues are believed to be strongly associated with the current
game and the side’s chance of winning. Indeed, some pundits talk of teams having
slumps and winning or losing streaks2. We know that the record of failure to deliver
1 Owing to space considerations, we will not be addressing the complex question as to
what constitutes success and failure in ISD in this paper. Where the matter is not
obvious, we will adopt a stakeholder view which may involve multiple and conflicting
opinions on the subject.
2 In the case of football there is often a large amount of money wagered world-wide on
a particular game with separate odds for a home win, away win or a draw. Spread
Why can’t we bet on ISD Outcomes?: ISD “Form” as a Predictor of Success
5
large-scale (IS) in a timely fashion that offer value to major commercial and public
organizations is legendary. Our thesis is that past negative patterns (i.e. negative form)
in ISD will tend to repeat themselves [14]. In other words, organizations, just like
football clubs, can experience slumps in ISD performance, producing and reproducing
failure after failure until the organization is mired in a culture of failure [9, 8].
Figure 1 - A Football Allegory: History or “Form”3
In looking back and learning from our historical ISD form, we can look ahead to
more favorable outcomes in the IS we build. Successful processes can then be
institutionalized so as to create new, positive histories to build upon [14]. When
football clubs appear to be developing losing streaks, the owners often move quickly
to change the manager (and consequently the playing system) and/ or to bring in new
players. In ISD projects the timescale often precludes such speedy reactions even
though radical solutions may be required to break the cycle of failure (new project
managers and other staff, new IT partners, new methodologies, etc. [18].
betting is a more sophisticated version of this with real-time odds offered on possible
events during the game (e.g. next goal scorer).
3 The above figure also shows the major, critical events En at time tn. In a previous
paper we developed a model that attempted to explain ISD outcomes in terms of
history, context and outcomes [13].
E1, t1
E2, t2
E3, t3
E4, t4
E5, t5
History or “Form”
(Recent form (wins, losses, draws), Injuries,
# games, cards, entry/ exit of players or
manager etc.)
E6, t6
E7, t7
Mike Newman, San Pan & Gary Pan
6
The next few pages illustrate the ideas we have discussed above. We show two
major organizations that developed patterns (a “culture”) of failure and show how they
turned this around by “shocking” the status quo4.
2. Case 1: Telecoms Corp
Telecoms Corp (or just Telecoms) is a subsidiary of a US-based conglomerate, herein
called GenComm. They employ about 14,000 people in several geographically
dispersed locations in North America. Telecoms’ annual revenue was about $1.6
billion, on which it earned $118 million in profits. The Supply Division was
responsible for centralized purchasing, inventory control, warehousing, and
distribution from two large central warehouses. Telecoms operated within an industry
that was undergoing de-regulation. The pressures to contain costs confirmed the need
for more sophisticated IS management, especially in the materials management area.
Historically, materials and inventory management at Telecoms suffered from gross
inefficiencies and waste. Previously, no attempt to build IS to control inventory costs
had been successful. IS at Telecoms had traditionally been developed for Supply by
the head office group, Management Information Services (MIS). Responding to ad
hoc requests, MIS had built several isolated systems spread over three hardware
platforms. It was clear that Supply had a “mish-mash” of eight or nine systems that
had been implemented with no overall plan in mind. Several of these systems were
quite large and represented high costs of development and operation. At Telecoms
there were deep misgivings concerning the ability of MIS to deliver successful IS.
Telecoms Corp attempted to introduce a state-of-the-art Materials Management
System (MMS) in their Supply Division. This attempt lasted well over 10 years with
three repeated cycles of failure creating a negative history and low expectations (“cast
of thousands, cost of millions, delivering nothing”). But at Telecoms there was a high
tolerance of failure. For example, the financial penalties of failure were not so great
because of the company’s semi-regulated state: greater costs were compensated for by
higher call charges. Staff had secure jobs and many were in posts whose tenure was
measured in decades rather than years. However, growing competition highlighted the
financial waste in the Supply area and the ongoing failure of MMS was focussed upon.
The transition point came when the long-standing Vice President (VP) of Supply was
asked to step aside and was subsequently retired early. The new VP removed several
4 For details of the interpretive research methods employed, please refer to the original
papers [18, 11].
Why can’t we bet on ISD Outcomes?: ISD “Form” as a Predictor of Success
7
key people including the project manager at Supply and made it clear that he was not
going to work with the senior managers who had served under the old VP. He had
learned of the trouble with MMS and wanted to establish control over the project
immediately. Besides the removal of several key project staff, the new VP also
terminated the contract with the existing software partner by paying a negotiated
percentage for the modules delivered and paying the remaining portion of the software
license fee upon acceptance. Under pressure, the software contractor agreed on terms
of severance of the contract, getting most of their contracted money out of the deal but
losing the lucrative, five-year maintenance contract. A new project manager was
introduced and he formed an alliance with MIS who together successfully re-justified
the project and delivered the MMS successfully within time and budget.
3. Case 2: US Insurance Corporation
The US Insurance Corporation is a large insurance corporation with assets of many
billions of dollars. The Corporation offers a full range of insurance products both
personal and commercial through a nation-wide branch network of 22 offices and
several sub-offices. The case focuses on the introduction of new state-of-the-art claims
system called Claims Automation Information System or CAIS. Up to the time of
introducing CAIS, computer support for the major insurance functions (underwriting
and claims) was non-existent. Competitors were developing sophisticated IS thereby
reducing the cost of claims and improving customer service. The history of ISD at US
Insurance could be characterized by a large gulf between the branch personnel -
traditionally the users and the main data suppliers, and the head office function - the
originator of most IS. Needless-to-say, the resulting IS at US Insurance enjoyed a poor
acceptance rate among users either because the quality was unacceptable or because
they had become irrelevant after the months or often years required to build them: the
reputation of the IS group at Headquarters (HQ) was poor. For the proposed CAIS, the
radical, user-centered approach to project management adopted was both an
acknowledgement of past problems and an attempt to create a discontinuity with the
old way of developing ISD projects. The project was predicted to save $10m costs per
year for an investment of $16m.
The project proceeded for a year when it became apparent that the user project
leader was unable to continue for various reasons. The project group quickly
intervened to solve this problem successfully by replacing the User with a key IS
person and one of the visionaries on the project. The project was able to proceed
rapidly after this intervention but it did not prevent a major problem on the near
horizon: the summer pilot tests at two branches produced crucial technical problems.
Mike Newman, San Pan & Gary Pan
8
The Management Information Services (MIS) group reasserted its leadership and
provided internal programmers to re-write the system “from the screens backwards”.
The team was successful and these interventions established the viability of the project
system. After this the project enjoyed a period of stability for the next 18 months as
the software was developed. Following this a new pilot was re-commissioned at a test
branch using the new software. Although three hundred errors were detected in the
system these flaws were removed incrementally over the next few months until CAIS
was ready to be rolled out to the branch network and implemented thereafter at the rate
of one installation per month. This challenge was overcome successfully and the
system stabilized. As a result the project was completed. After this success the project
team then moved on to further development work (i.e. underwriting) using the new
methodology, now institutionalized, which was codified and labeled Business Systems
Engineering.
4. Lessons and Practical Implications
Telecoms Corp exhibited three cycles of failure to deliver an MM system in the
Supply Division each exhibiting similar patterns leading to failure. Only after a radical
re-organization of the project team and a rapprochement between the project team and
the central MIS group was it possible to create the conditions leading to success. There
were similarities in the case of US Insurance. The history of developing IS by the HQ
was very poor as judged by the users in Claims. Previous IS were judged to be either
irrelevant or of poor quality if indeed they were completed at all and as a consequence
user expectations for the CAIS project were very low. The new claims system
represented a radical break from history in the structure of the project (user-led as
opposed to MIS-led); technology (e.g., state-of-the-art IS technology); and people
(user support staff). Returning to the original statement, “Those who cannot remember
the past are condemned to repeat it” we can begin to see some lessons for IS
practitioners and their managers. Ignorance of the past is often accompanied by a
strong attachment to old ways of developing systems (old technologies, old methods,
old structures) rather like ancients clinging to their idols. If these old ways worked
before, well and good, but if they failed before repeatedly, why should they work
now? Our thesis is that failing patterns in ISD tend to be reproduced unless
discontinuities are introduced [9]. And this has been confirmed in the two cases we
have presented.
The cases teach us that we need a realistic and critical assessment of past patterns
(ISD form) and if there is a history of failure of ISD outcomes we need to design new
and often radical approaches to developing systems. One promising part of the
Why can’t we bet on ISD Outcomes?: ISD “Form” as a Predictor of Success
9
assessment is to examine the socio-technical (ST) factors in building the system: task,
technology, actors and structure [7]5. For example, in the case of Telecoms Corp, in
response to a crisis unrelated to the project, senior management changed the actors, the
technology of ISD and the structure: the project manager was removed and replaced
by a senior person with a new approach (actors); the failing development approach
was replaced by getting MIS into the process to build the IS (technology); the software
partner was bought out and MIS was grafted in as a partner (structure). The task of
building a MMS remained the same throughout. Similarly, at US Insurance we saw a
new and radical approach to building and delivering a sophisticated claims system in
order to avoid the expected failures as evidenced from the past. Again there were
changes to traditional ST elements such as actors, the technology of ISD and structure:
there were to be new IS staff and one of the visionaries on the project became the
project leader (actors); the use of state-of-the-art systems to build systems such as the
use of a model office (technology); MIS group leading the project (structure).
Interestingly, as with Telecoms Corp, the task (delivering a new claims system)
remained a constant need throughout. If we put these elements together and develop an
ISD form it suggests a new, radical approach to IS planning, risk assessment and
budgeting. This amounts to a procedure that organizations might consider conducting
before embarking on major systems projects: a pre-methodology audit. The steps
illustrated in Table 1 demonstrate how this might be done in practice.
Individually and collectively in organizations, we carry our histories into the future
with us but as we have tried to show is that does not mean we cannot overcome the
weaknesses of our past ISD efforts. Our thesis has been that historically failing
approaches to ISD, without any radical changes, will tend to be reproduced in current
practices. This has been borne out by the two case examples we have presented. It was
only when radical steps were taken to change the way they built systems that they
could break the cycle of failure and curtail the escalation of resources [8]. A negative
ISD form will give you a guide to show you what not to do or what you cannot do,
but the project team still has to manage the current process and overcome the many
vicissitudes on the path to delivering a successful IS just as was experienced in the
cases presented. A football club may have good form but you still have to win your
current game. In looking back and learning from our historical ISD form, we can look
ahead to more favorable outcomes in the IS we build. Successful processes can then be
institutionalized so as to create new, positive histories to build upon. We may not be
able to bet on successful IS outcomes but we can improve our chances of delivering
systems that offer value and benefits to stakeholders.
5 We have begun this process and have applied our socio-technical process model to
detailed case evidence from a variety of countries and contexts [15, 1].
Mike Newman, San Pan & Gary Pan
10
Identifying and Institutionalizing ISD form
The current Project Manager (PM) or Champion needs to assess the recent
history of delivering major IS in your organization or division (using up to,
say, five years of history):
Step 1: Identify Building Approaches
Specifically, the PM should examine the building approaches that were
previously used in terms of Social-Technical elements and ask some critical
questions of IS staff and users (a questionnaire could also be used as a
supplement) namely:
Actors (e.g. Were the right people in charge of the project? In both cases
described above, key people had to be replaced or retired. They were seen as
part of the problem and associated with failing patterns).
Technology (e.g. Was the technology workable or too complicated? Was
there a package solution available and was it used? Was the development
methodology appropriate?).
Structure (e.g. Were all major stakeholders involved and did they buy in to
the project?).
Task (e.g. What problems were we solving and were they relevant and
important and did they remain so?).
Were there any significant interactions between the above Social-Technical
elements?
Assess IS that were completed, delivered and accepted by users and those
that were not.
Have any major commitments been made in the recent past (technology
platforms, software packages (e.g. ERP systems etc.)? These may constrain
your current degrees of freedom to make changes.
Step 2: Identify Project Development Patterns
Look for patterns in previous projects and decide where the weaknesses are
in those Social-Technical elements detailed above.
These patterns and weaknesses constitute your assessment of ISD form in
your division or organization (say, on a 5 point scale from excellent to very
Why can’t we bet on ISD Outcomes?: ISD “Form” as a Predictor of Success
11
poor). If there are major failings in the ISD form (e.g. rated as poor or very
poor), propose radical changes for your current project that are seen as a
discontinuity with the past and get the buy-in for these. You could use the
same Social-Technical factors for this stage (e.g. new methodologies; new
leadership; new technologies etc.). For example, in our two cases we would
rate their ISD form as very poor. A further refinement would be to assess
form for each socio-technical element.
Step 3: Institutionalize the Assessment of ISD Form
Develop a project proposal and business plan that reflect this new, radical
approach and present it to senior management.
As appropriate, employ staff and co-opt users who are motivated and
sympathetic to the new approach.
Be flexible. ISD is a process and radical changes and large IS take time to
build and will produce unplanned side-effects which may affect your
ability to deliver the project as specified. You may have to make adjustments
and learn from what works and from what does not.
If you cannot get the level of support you need for the new approach,
consider a smaller project or abandoning the project altogether.
If the new approach works, consider codifying it and persuading senior
management to use it on future projects.
Table 1: A Summary of the Steps Involved in Developing an ISD form
References
1. Bob-Jones, B. Newman, M., and Lyytinen, K. (2008) Picking Up the Pieces after a
“Successful” Implementation: Networks, Coalitions and ERP Systems, Forthcoming
in the Proceedings of AMCIS 2008, Toronto, Canada.
2. Drummond, H. (1996) Escalation in Decision-Making: The Tragedy of Taurus.
Oxford University Press, Oxford, U.K.
3. Ewusi-Mensah, K. (1997) Critical issues in abandoned information systems
development projects. Communications of the ACM 40, 9, 74-80.
4. Hirschheim, R., and Newman, M. (1991), "Information Systems Development as
Symbolism: Myth, Metaphor, and Magic". Information Systems Research, 2 (1), 29-
62.
Mike Newman, San Pan & Gary Pan
12
5. Iacovou, C. and Dexter, A. (2005) Surviving IT project cancellations.
Communications of the ACM 48, 4, 83-86.
6. Keil, M. (1995) Pulling the plug: software project management and the problem of
project escalation. MIS Quarterly 19, 4, 421-447.
7. Leavitt, H. (1964) Applied organization change in industry: structural, technical,
and human approaches in: new perspectives in organizational research. Chichester:
Wiley, 55–71.
8. Lyytinen, K. and Robey, D. (1999) Learning failure in information systems
development. Information Systems Journal 9, 2, 85-101.
9. Lyytinen, K. and Newman, M. (2005) “Information Systems Development as
Punctuated Socio-Technical Change” Working paper, The University of Manchester.
10. Myers, M. and Newman, M. (2007) “The Qualitative Interview in IS Research:
Examining the Craft”. Information and Organization. 17 (1), 2-26.
11. Newman, M. and Robey, D., (1992) "A Social Process Model of User-Analyst
Relationships", MIS Quarterly, June, pp. 249-266.
12. Newman, M. and Sabherwal, R. (1996) Determinants of commitment to
information system development: A longitudinal investigation. MIS Quarterly 20, 1,
23-54.
13. Newman, M., (2007). Context, Process and Outcomes of ISD: An Allegorical
Tale. Proceedings of the Americas Conference on Information Systems, Keystone,
Colorado, August 10-12.
14. Newman, M., Pan, S., Pan, G. (2006) “Can Information Systems Development
“Form” Teach Us Anything?” Working paper, The University of Manchester.
15. Newman, M. and Zhao, Y. (2008) “The Process of ERP Implementation and BPR:
A Tale from two Chinese SMEs”. Information Systems Journal. Published
electronically April 15th.
16. Pan, S., Pan, G., Newman, M., and Flynn, D. (2006) Escalation and de-escalation
of commitment to information systems projects: insights from a project evaluation
model. European Journal of Operational Research 173, 3, 1139-1160.
17. Pentland, B. T. (1999) Building Process Theory with Narrative: From Description
to Explanation. Academy of Management Review, 24(4), 711-724.
18. Robey, D., and Newman, M., (1996) “Sequential Patterns in Information Systems
Development: An Application of a Social Process Model.” ACM Transactions of
Information Systems, 14, January, 30-63.
