Working PaperPDF Available

Implications of digital transformation, Agile, and DevOps for IT curricula, with recommendations


Abstract and Figures

Author’s note Direct efforts on this working draft are discontinued and it will not be submitted for publication in this form. It was a working vehicle, helpful for a time in assembling citations and thinking. For published work, see the report “Renewing the IT Curriculum: Responding to Agile, DevOps, and Digital Transformation” available at, or for download on ResearchGate ( Abstract The use and importance of information technology is accelerating throughout the economy, a movement known as “digital transformation.” New delivery techniques are emerging, oriented around a core of Agile software development. Agile as a movement has extended its reach into product management, operations, IT infrastructure, and many other aspects of management information systems,notably in the recent movement termed “DevOps.”Established practices such as waterfall software development, project management, and the use of IT process frameworks are being questioned. Workforce requirements are shifting, and there is a need for an updated educational response more consistent with digital trends. Existing curricula are analyzed for fit with the new trends. A new learning progression based on a “scaling” narrative is proposed as a basis for reference curricula.
Content may be subject to copyright.
Implications of digital transformation, Agile, and DevOps for
IT curricula and pedagogy
Working Paper
DRAFT (v0.53)
25 June 2016
Charles Betz
University of St. Thomas
Author’s note
Direct efforts on this working draft are discontinued and it will not be submitted for
publication in this form. It was a working vehicle, helpful for a time in assembling
citations and thinking.
For related work, see the report “Renewing the IT Curriculum: Responding to Agile,
DevOps, and Digital Transformation” available at
The use and importance of information technology is accelerating throughout the
economy, a movement known as “digital transformation.” New delivery techniques are
emerging, oriented around a core of Agile software development. Agile as a movement
has extended its reach into product management, operations, IT infrastructure, and many
other aspects of management information systems, notably in the recent movement
termed “DevOps. Established practices such as waterfall software development, project
management, and the use of IT process frameworks are being questioned. Workforce
requirements are shifting, and there is a need for an updated educational response more
consistent with digital trends. Existing curricula are analyzed for fit with the new trends.
A new learning progression based on a “scaling” narrative is proposed as a basis for
reference curricula.
© 2016, Charles T. Betz
Let us start this discussion with two industry reports:
In September 2015, Minneapolis-based Target Corporation laid off 275 workers with IT skillsets
such as business analysis and project management, while simultaneously hiring workers with
newer Agile skills. As quoted by a local news site, Target stated:
As a part of our transition to an Agile technology development and support
model, we conducted a comprehensive review of our current structure and
capabilities we are eliminating approximately 275 positions and closing an
additional 35 open positions. The majority of the impact was across our
technology teams and was primarily focused on areas such business analysis
and project management.(KARE 11 Staff, 2015)
Consider also the following from Jim Fowler, Chief Information Officer at General Electric:
“When I am in business meetings, I hear people talk about digital as a function
or a role. It is not. Digital is a capability that needs to exist in every job.
Twenty years ago, we broke ecommerce out into its own organization, and
today ecommerce is just a part of the way we work. That's where digital and IT
are headed; IT will be no longer be a distinct function, it will just be the way
we work. …
[W]e've moved to a flatter organizational model with "teams of teams" who
are focused on outcomes. These are colocated groups of people who own a
small, minimal viable product deliverable that they can produce in 90 days.
The team focuses on one piece of work that they will own through its complete
lifecycle…in [the “back office”] model, the CIO controls infrastructure, the
network, storage, and makes the PCs run. The CIOs who choose to play that
role will not be relevant for long…” (Heller, 2016)
Fowler echoes the strategic concerns and directions of leading CIOs around the world. And
beneath his words lie fundamentally new approaches to delivering IT and digital
value. These
approaches carry a number of names, but the term “Agile” is prominent, with a number of
See the appendix for this paper’s definition of the terms “Information Technology,” “Agile,” “DevOps,” and
variants (DevOps, Lean IT, Lean Product Development, Web-scale IT, Lean UX, and so forth).
These trends, decades in the making, cannot be dismissed as "flavor of the month" or a fad.
Technologies come and go, and it is appropriate that education professionals focus on underlying
principles and practices. Curricula are expensive to change, and commercial training exists for
good reason. As stated in the joint ACM/IEEE/AIS publication Computing Curricula 2005: The
Overview Report:
Institutions tend to be cautious and conservative, and the complex nature of
academic degree programs means that it is difficult to implement significant
changes rapidly… the pace of change in computing is quite rapid, while
the pace of institutional change generally can be quite slow . . .
(Shackelford et al., 2005)
However, we are not faced today with merely another ephemeral technology cycle. Rather, we
are experiencing a generational change in foundational models of IT-related delivery and
execution, in response to accelerating digital transformation.
The new IT-based digital economy is informed top to bottom by an ecosystem of ideas founded
on Agile software development. This is accompanied by rethinking or outright rejection of
certain practices, including waterfall software development, lengthy “analysis and design”
phases, stage-gated project management, and the use of process improvement frameworks. Many
are introduced to such practices via formal education, where they are still often presented
Startups; large, established firms; and now mid-size companies are embracing the insights and
practices of digital transformation, and local workforce requirements are shifting. We believe
that academic systems and curriculum owners need up to date understanding and relevant,
mainstream offerings appropriate to these trends.
Academic curricula and industrial frameworks
At the broadest level, our complex digital economy relies on two primary classes of knowledge
formalization and transmission:
academic curricula
industrial IT training, including frameworks and product collateral
Academic curricular guidance (summary view)
There are a number of professional organizations concerned with formal academic curricula in
the digital and computing space:
The Association for Computing Machinery (ACM)
The Institute for Electrical and Electronic Engineers, in particular the Computer Society
The Association for Information Systems (AIS)
The ACM serves the broadest role; it is the common player in a wide variety of guidance for
educators from kindergarten through post-secondary. As the ACM notes, “In the decades since
the 1960s, ACM, along with leading professional and scientific computing societies, has
endeavored to tailor curriculum recommendations to the rapidly changing landscape of computer
technology. (Association for Computing Machinery, 2016).
In 2005, a Joint Task Force of the ACM, AIS, and IEEE-CS established the boundaries between
various technological fields in an authoritative report titled: Computing Curricula 2005: The
Overview Report (Shackelford et al., 2005). In this document (which features extensive
discussion of the history of computing as an academic topic), the authors note that computing can
be defined as:
any goal-oriented activity requiring, benefiting from, or creating computers.
Thus, computing includes designing and building hardware and software
systems for a wide range of purposes; processing, structuring, and managing
various kinds of information; doing scientific studies using computers; making
computer systems behave intelligently; creating and using communications and
entertainment media; finding and gathering information relevant to any
particular purpose, and so on. The list is virtually endless, and the possibilities
are vast.
The disciplines identified in the 2005 report have been used as a basis for organizing academic
curricula at least in the U.S.:
1. Computer Science
2. Computer Engineering
3. Information Systems
4. Information Technology
5. Software Engineering
1-3 have been recognized for decades, while 4 and 5 are of more recent vintage. The 2005 report
did not claim these were exhaustive; it specifically provided for “the emergence of new
computing disciplines.” (p. 7).
The disciplines are described in detail in that document, and excerpted here:
Computer engineering “is concerned with the design and construction of computers and
computer-based systems. It involves the study of hardware, software, communications, and the
interaction among them… CE study may emphasize hardware more than software or there may
be a balanced emphasis. CE has a strong engineering flavor.” (p. 13).
Computer science “spans a wide range, from its theoretical and algorithmic foundations to
cutting-edge developments in robotics, computer vision, intelligent systems, bioinformatics …
While other disciplines may produce graduates with more immediately relevant job-related skills,
computer science offers a comprehensive foundation that permits graduates to adapt to new
technologies and new ideas” (pp. 13-14).
Software engineering is the discipline of developing and maintaining software systems that
behave reliably and efficiently, are affordable to develop and maintain, and satisfy all the
requirements that customers have defined for them . . . it has evolved in response to factors such
as the growing impact of large and expensive software systems in a wide range of situations and
the increased importance of software in safety-critical applications. Software engineering is
different in character from other engineering disciplines due to both the intangible nature of
software and the discontinuous nature of software operation . . .” (p.15)
Information systems “focuses on integrating information technology solutions and business
processes to meet the information needs of businesses and other enterprises … A majority of
Information Systems (IS) programs are located in business schools. All IS degrees combine
business and computing coursework. A variety of IS programs exist . . . programs in Computer
Information Systems usually have the strongest technology focus, while programs in
Management Information Systems emphasize the organizational and behavioral aspects of IS.
Degree program names are not always consistent.” (p.14).
Information technology is viewed as the converse of information systems: “its emphasis is on
the technology itself more than on the information it conveys . . . organizations of every kind are
dependent on information technology. They need to have appropriate systems in place. These
systems must work properly, be secure, and upgraded, maintained, and replaced as appropriate.
Employees throughout an organization require support from IT staff who understand computer
systems and their software and are committed to solving whatever computer-related problems
they might have. Graduates of information technology programs address these needs.” (p.14).
As the most recent disciplinary addition, the 2005 report notes that some IT programs are of low
quality and fail to serve either their students or their communities in a responsible way. The latter
group of programs seeks to increase enrollments and/or provide the image of being responsive to
local needs by creating an IT program that is little more than the repackaging of existing courses
offered in other disciplines.” (pp. 37-38)
More detailed analysis (including useful graphics) of the scoping of these frameworks is
incorporated into the 2005 report, and will be reviewed below in the context of Agile & digital
Industrial frameworks
There has always been some distance between the academic disciplines and their application.
One consequence of this gap is that the IT industry has, largely independently of academic and
research guidance, created a number of higher-order management frameworks such as:
ITIL (originally the Information Technology Infrastructure Library)
COBIT (aka Control Objectives for Information Technology)
TOGAF (The Open Group Architecture Framework)
PMBOK (Project Management Body of Knowledge)
See (CCTA, 1996; ISACA, 2012; ISO/IEC, 2005; Office of Government Commerce, 2000; Open
Group, 2015b; Project Management Institute., 2013; The Stationery Office, 2011d)).
These are not technical standards such as TCP/IP; they operate at a higher level of abstraction. In
general, an industry framework is structured artifact that seeks to articulate a professional
consensus regarding a domain of practice. The intent is usually that the guidance be mutually
exclusive and collectively exhaustive within the domain, so that persons knowledgeable in the
framework have a broad understanding of domain concerns.
The influence and economic impact of such guidance should not be underestimated. These
frameworks provide the basis for organizing and managing massive industrial resources and
governing the performance of those charged with delivering value.
Significant training, consulting, and certification ecosystems monetize them. They inspire reams
of ancillary and derivative literature. Auditors test against their “best practice” recommendations.
And academic institutions and curricula task forces uncritically accept them as a suitable basis
for curricula: for examples in the current curricula, see (Lunt et al., 2008), p. 115, 117; (Topi et
al., 2010), p. vii, 7, 18, 44, 46.
Summary definitions follow:
ITIL, originally the Information Technology Infrastructure Library, is a publication of the UK
government, now via the Axelos commercial joint venture with Capita Group. It is in its 4th
major edition (2011). While often considered an “operational” framework, ITIL spans the
lifecycle of IT services and systems, from Strategy to Design to Transition to Operations to
Continual Service Improvement. However, due to the particular structure of UK government
publications, ITIL does not cover project management or the SDLC per se, or analysis,
architecture and design.
ITIL is a lengthy and expensive to obtain publication, containing about 2000 pages of narrative
guidance covering 26 processes and additional IS/IT functions and activities, from IT Finance to
Event Management.
COBIT, originally the Control Objectives for Information Technology, is a set of guidance from
the IS Audit and Control Association. It has a broader scope than ITIL, as it includes architecture
and project management. Where ITIL contains narrative of considerable length, COBIT is terser
and more structured, providing a useful framework for the construction and testing of suitable
controls for IT governance.
PMBOK, or the Project Management Body of Knowledge, is a publication of the Project
Management Institute. It represents the codification of formal project management knowledge.
There is a comparable Axelos publication, Prince2.
TOGAF, or The Open Group Architecture Framework, is a framework and method for IT
and enterprise architecture practices.
Many other frameworks exist, under varying governance models from open to proprietary. An up
to date list is maintained by Van Haren Publishing in their publication Global Standards and
Publications (Van Haren Publishing, 2016). A variety of Agile frameworks also have formed,
which will be discussed separately. Finally, there is a broad ecosystem of vendor-specific
certifications as well, to educate practitioners in the specifics of various commercial products.
The academic response to the commercial frameworks, especially in terms of their certification
agenda, can be cautious. As noted in (Shackelford et al., 2005):
In some of its forms, certification competes with academic programs. It is clear
that certification is a major trend. As with anything else, some certifications
are respected, while others are controversial. When degree granting
institutions partner with vendor-specific certification programs, academic
integrity becomes an issue. The reader should be aware that such partnerships
invite controversy about academic integrity and ethics. (p.44)
Beyond the concern over academic integrity, certifications tend to become loci of professional
identity and allegiance, and in this manner can interfere with rational, specific discussion of
computing’s many facets, especially in terms of its management.
The attitude of the Agile community to frameworks and certification is skeptical at best. Stage-
gated project management (i.e. as proposed by PMBOK) is losing support, for reasons articulated
by Reinertsen and others. ITIL, COBIT, and TOGAF all come in for criticism as well.
Overview of the Agile ecosystem and digital practices
Both academic curricula and frameworks can be seen as metacognitive responses
to complex,
emergent issues of sociotechnical systems and human-computer interaction at the most abstract
level. As such, they are always partial and evolving.
This paper argues that we are faced with discontinuity in this evolutionary process. A new
functional industry consensus,
based in a solidifying theoretical foundation, drawing on insights beyond traditional
computer science,
validated in industrial practice, and
challenging both current academic curricula and industry frameworks,
is emerging from the combination of Agile software development, DevOps practices, and
web/cloud-scale infrastructure management.
The intellectual antecedents of Agile extend back decades, as well documented in (Larman &
Basili, 2003). Agile methods (e.g. (Beck, 2000)) were enthusiastically adopted by the new
generation of Internet startups, and converged with Cloud computing into a new style of IT
delivery (see (Limoncelli, Chalup, & Hogan, 2014), appendix B for an authoritative account).
Influenced by management thinkers such as Nonaka, (Takeuchi & Nonaka, 1986), Lean product
management is the dominant execution paradigm (Cantor, 2014; Poppendieck & Poppendieck,
2003; Reinertsen, 2009; Schwaber & Sutherland, 2012), replacing or at least reducing the role of
project management (Apke, 2015; Cobb, 2011; Koskela Gregory, 2002; Memon, 2014). Scrum
co-founder Ken Schwaber cites the influence of classical control theory in his rejection of 1990s
waterfall methods (Schwaber, 2002), specifically the work of Ogunnaike and Ray (Ogunnaike &
Ray, 1994).
Philosophies of design thinking and Lean UX drive the industry desire for an hypothesis-driven,
empirical, customer-focused experimental approach to product design (Jeff Gothelf & Josh
Seiden, 2013; Lockwood, 2010; Sussna, 2015). Fast product feedback is recognized as an
essential success factor for value realization (Reinertsen, 1997; Ries, 2011), and thus the tempo
of digital product deployment has increased dramatically (Allspaw & Hammond, 2009; Humble
& Farley, 2011).
In the sense of the “Revised Bloom’s Taxonomy,”
This new style relies on incremental and iterative test driven development (Beck, 2003), and
ongoing refactoring to prevent technical debt (Fowler & Beck, 1999). It calls for continuous
integration of code (Duvall, Matyas, & Glover, 2007), infrastructure automation and automated
build management (Bass, Weber, & Zhu, 2015; Duvall et al., 2007; Humble & Farley, 2011;
Shortland & Lei, 2012), dynamic capacity on demand (Allspaw & Robbins, 2010; Limoncelli et
al., 2014), and loosely-coupled “microservice” systems architected in keeping with the CAP
theorem (Brewer & Fox, 1999; Newman, 2015). Strict adherence to common API standards
(REST, JSON) ensures service interoperability (Hernandez 2012).
Specialized functional (“silo”) segregation is replaced by cross-functional collaboration, enabled
by (at companies like Amazon and Spotify) “two-pizza team” product-centric cells of no more
than 8 participants (Kniberg and Ivarsson 2012; Choi 2014).
Employers now seek “T-shaped” professionals who can be more flexible in terms of their roles
on fast-moving digital product teams (see (Reinertsen, 2009, p. 150), “The Principle of T-Shaped
Resources: Develop people who are deep in one area and broad in many”). The importance of
high-trust, psychologically safe teams is gaining increasing attention due to the quantifiable
impact culture is observed to have on product delivery (Rozovsky, 2015).
Web- and Cloud-scale engineers point to the influence of complex systems research (Allspaw &
Robbins, 2010, Chapter 7). Systems thinking is a guiding influence (e.g. as discussed in (Larman
& Vodde, 2009, Chapter 2, “Systems Thinking”). The Cynefin framework by Dr. Dave Snowden
has emerged in the Agile world as a foundational sense-making artifact explaining the overall
relationships between simple, complicated, complex, and chaotic modes of operation (Snowden
& Boone, 2007).
A recent watershed moment was the emergency drafting of Google site reliability engineers to
intervene in the failing initiative, which was being run according to the accepted
“best practice” wisdom. This remarkably successful turnaround raised the visibility of the new IT
practices worldwide (Brill, 2014) and has resulted in their advocacy even in the US federal
government (Schwartz, 2014), long a bastion of the legacy practices. Two new agencies have
been created to assist in their adoption (Balter, 2015; Lee, 2016).
Research firm Gartner Group notes that “DevOps has been by far the most popular search term
on related to IT operations in 2015(Williams & Holub, 2016). The annual
State of DevOps report presents extraordinary, order of magnitude claims in terms of IT delivery
and stability for organizations with mature Agile and DevOps practices (Puppet Labs, 2015):
60x fewer failures
168x better MTTR
30x more frequent delivery
200x shorter lead times
This same research also indicated that some traditional IT “best practices” are questionable. For
example, the State of DevOps research found that having Change Advisory Boards (a staple of
the IT Infrastructure Library process framework) approve production changes does not result in
the intended benefit of stability, while delaying delivery significantly
Conversely, the same research shows that “practices that make up continuous delivery —
deployment automation and automated testing, continuous integration, and version control
for all production artifacts have a significant predictive relationship to deployment pain, IT
performance and change fail rate. In turn, IT performance predicts organizational performance, as
measured by productivity, market share and profitability.” (Puppet Labs, 2015), p.13, emphasis
Finally, as noted in the Harvard Business Review, Now agile methodologieswhich involve
new values, principles, practices, and benefits and are a radical alternative to command-and-
control-style managementare spreading across a broad range of industries and functions and
even into the C-suite (Rigby, Sutherland, & Takeuchi, 2016). The proven effectiveness of these
methods in managing many forms of complexity, including problems not usually seen as “IT” or
even “digital,” leads to broader and broader appetite for appropriate instruction and curricula.
Analyzing current curricula and frameworks in light of digital,
DevOps, and Agile
Curricula under review
Of the five major disciplines outlined in (Shackelford et al., 2005), this paper is most concerned
Software engineering
Information systems
Information technology
and less concerned with for core computer science and computer engineering, which will be out
of scope for the rest of this paper.
The analysis therefore focuses on the following specific curricula guidance:
Information Technology 2008: Curriculum Guidelines for Undergraduate Degree
Programs in Information Technology, by a joint task force of the ACM and IEEE (aka
“IT2008” throughout this paper) (Lunt et al., 2008)
Graduate Software Engineering 2009 (GSwE2009) Curriculum Guidelines for Graduate
Degree Programs in Software Engineering (iSSEc Project, 2009)
IS 2010 Curriculum Guidelines for Undergraduate Degree Programs in Information
Systems (Topi et al., 2010)
This is only one data point, and this paper does not suggest wholesale abandonment of practices such as ITIL-based
Change management. It does call for more empirical evidence as to their claimed benefits, and their grounding in
disciplines such as control theory.
Information Technology Competency Model of Core Learning Outcomes and Assessment
for Associate-Degree Curriculum (Hawthorne, Campbell, Tang, Tucker, & Nichols,
Software Engineering 2014: Curriculum Guidelines for Undergraduate Degree
Programs in Software Engineering (Sobel, Lethbridge, Díaz-Herrera, & Barrie
Thompson, 2015)
Needs in general
In general, the current SE/IS/IT curricula guidance has the following opportunities and needs
when considered in light of the Agile trends documented above:
Product management
o In part as a result of the creation of perhaps hundreds of thousands of “product
owners” via the popular Scrum methodology, there is a need for more guidance on
management, beyond the concepts of “requirements and “business
o There is a need for increased recognition of the central importance of fast product
feedback in development processes
o The disciplines as scoped seem to encourage a “business/IT” divide that serves as
a barrier to fast feedback.
o More concern in the technical disciplines for the organizational context driving
the development investment is also needed.
o The topic of design thinking may have reached a tipping point for inclusion into
Software lifecycle
o Despite periodic recognition of the existence of Agile methods, learning
objectives still are scoped by classic waterfall project phases. Agile is often
covered as a subset of project management.
o There is a need for increased teaching of the centrality of version control as an
enabling industrial discipline, and the challenges of implementing, managing, and
optimizing it.
This may seem an obscure technical practice to some, but version control
is recognized as the most critical enabling discipline of the Agile
movement. In the words of Pivotal Labs’ Andrew Clay Shafer, “In
software development, version control is the foundation of every other
Agile technical practice. (in (Allspaw & Robbins, 2010), Kindle Location
Including service
6012-6013) The State of DevOps report supports this statistically (Puppet
Labs, 2015), p.13.
When employed comprehensively, it is the structured, unambiguous digital
representation of the complete, complex socio-technical system as it
moves through the state changes of its lifecycle.
o Increased attention is needed for later-stage software lifecycle concerns (Release,
Deploy, Operate). A full-lifecycle perspective is expected by industry, as
computing is increasingly delivered as an operational service upgraded on a
frequent basis. A technology-independent consensus as to best practices is
beginning to stabilize this formerly organization-specific topic.
Next generation infrastructure & operations (aka Site Reliability Engineering/SRE)
o There is much opportunity for further attention to infrastructure engineering,
infrastructure services, and technical operations, to better prepare students for a
world of Cloud-scale computing defined by “infrastructure as code.”
Resource and execution management
o Little attention to questions of demand and work management in the context of
complex, stressed digital organizations (e.g. applied queuing theory)
o A “one-size-fits-all” approach to IS/IT management concerns such as process
management, governance, and resource management.
o A more nuanced and critical presentation of industry frameworks (ITIL, PMBOK,
etc) as the basis for curricula guidance.
o Opportunity for cross-disciplinary partnership with operations research and
control theorists
Culture, organization, and execution
o More attention to questions of collaboration and culture. How are product-centric
teams including business analysts, developers and engineers to be managed?
o Providing students some grounding in the cultural aspects is important so that they
can recognize for their own good and the good of their employers if the operative
culture supports high performance.
o Opportunity for cross-disciplinary partnership with psychologists and human
factors specialists.
Analysis of current disciplines in light of current trends
A series of high-level scoping diagrams accompanied the descriptions of the disciplines in
(Shackelford et al., 2005). These are excerpted here for software engineering (SE), information
systems (IS), and information technology (IT).
A 2-part weighting matrix was also derived, and is shown below.
Figure 1. Computing topic weighting from (Shackelford et al., 2005), pp. 24-25
The list of knowledge areas is comprehensive and still largely relevant (some exceptions will be
identified below). The shortcomings listed above can be mapped onto the KAs reasonably
Knowledge Area
Product mgmt
Culture, org, process
Programming Fundamentals x x
Integrative Programming x x
Algorithms and Complexity
Computer Architecture and Organization x x
Operating Systems Configuration & Use x x
Net Centric Principles and Design x
Net Centric Use and configuration x x
Platform technologies x x x
Human-Computer Interaction x x
Information Management (DB) Practice x x
Legal / Professional / Ethics / Society x x
Information Systems Development x x x x
Analysis of Business Requirements x x x
E-business x
Analysis of Technical Requirements x x
Engineering Foundations for SW x x
Engineering Economics for SW x x
Software Modeling and Analysis x
Software Design x x
Software Verification and Validation x x
Software Evolution (maintenance) x
Software Process x x
Software Quality x
Distributed Systems x x
Security: issues and principles x x
Security: implementation and mgt x x
Systems administration x x x
Management of Info Systems Org. x x
Systems integration x x x
Technical support x x x
Organizational Theory x
Organizational Behavior x x
Organizational Change Management x
General Systems Theory x x
Risk Management (Proje ct, safety risk) x x x
Project Management x x x
Business Models x x* x
Functional Business Areas x x
Mathematical foundations x x x
Interpersonal communication x x
Evaluation of Business Performance x x
If we analyze these shortcomings in terms of their KAs, we see that no existing discipline covers
the required range.
Software engineering
(graphic from p.21)
The Software Engineering discipline is the most technical of the 3 under consideration (see
above for summary description). The following observations have been made by (Iivari,
Hirschheim, & Klein, 2004):
the issue of IS alignment is ignored in the SE tradition The problem of organizational
implementation is totally neglected in SWEBOK despite the fact that organizational
implementation is often problematic …”
Their observations are supported by the choice of knowledge area weightings in (Shackelford et
al., 2005):
Knowle dge Area
SE min
SE max
Management of Info Systems Org. 0 0
Organizational Theory 0 0
Organizational Behavior 0 0
Organizational Change Management 0 0
General Systems Theory 0 0
Business Models 0 0
Functional Business Areas 0 0
Evaluation of Business Performance 0 0
These knowledge areas are critical to the full product lifecycle implied by digital. Engineers with
no grounding in these domains may struggle to understand key elements of organizational
mission, product success, and even team behavior, to the detriment of their careers and their
employers’ interests.
Current draft is based only on 2005 Core document. To do: review (iSSEc Project, 2009; Sobel
et al., 2015)
Information Systems
(graphic from p.19)
Information systems has the converse issue from software engineering: it is non-technical in
important ways:
“When most colleges created their computing degree programs, computer science was the only
choice that had strong ties to mathematics, science, and/or engineering. (IS programs developed
around the same time but their primary ties were to business schools.)” (Shackelford et al., 2005)
p 31. As noted by (Iivari et al., 2004), “the distinctive feature of ISD methods and approaches,
when compared with SE methods, is that they handle or at least recognize the organizational
alignment process.”
The idea that “the business” determines requirements and IT organizations then serve as “order
takers” is contrary to current digital thinking; instead, product hypotheses are formed and a series
of rapid, feedback-based iterations are then executed, requiring much greater integration across
“business” and “IT.”
The non-technical topics absent in SE are clearly handled by MIS:
Knowle dge Area
IS min
IS max
SE min
SE max
IT min
Intelligent Systems (AI) 1 1 0 0 0
Management of Info Systems Org. 3 5 0 0 0
Organizational Theory 1 4 0 0 1
Organizational Behavi or 3 5 0 0 1
Organizational Change Management 2 2 0 0 1
General Systems Theory 2 2 0 0 1
Business Models 4 5 0 0 0
Functional Business Areas 4 5 0 0 0
Evaluation of Business Performance 4 5 0 0 0
And when we look at selected technical knowledge area weightings for the IS domain, the
overview is promising: the IS graduate is expected to have at least passing familiarity with a
variety of topics here.
Knowle dge Area
IS min
IS max
Programming Fundamentals 2 4
Integrative Programming 2 4
Algorithms and Complexity 1 2
Operating Systems Configuration & Use 2 3
Net Centric Use and configuration 2 4
Analysis of Technical Requirements 2 4
Software Design 1 3
Software Process 1 2
Systems administration 1 3
However, it is not clear that MIS programs are indeed covering the indicated technical
fundamentals, either in their own coursework or through requiring course offerings in CSci or
Software Engineering. It is possible that the lack of technical capacity in the typical business
school is likely to cause increasing issues. It is also arguable that current MIS survey texts
(Bidgoli, 2014; Kroenke & Boyle, 2016; Laudon & Laudon, 2015; Marakas & O’Brien, 2011;
Rainer, Prince, & Watson, 2015; Sousa & Oz, 2015) show little or no awareness of the trends
discussed in this paper.
Current draft is based only on 2005 Core document. To do: review (Topi et al., 2010)
As we turn to IS2010:
“Broad use of IT control and infrastructure frameworks, such as ITIL and COBIT” (vii)
Information Technology
(graphic from p.20)
Information technology is a challenging curricula area. As (Shackelford et al., 2005) states:
“Because IT is a very new discipline, its focus has been on developing educational programs that
give students a foundation in existing concepts and skills. Many in the community of IT faculty
assert that research in their field will grow to create and develop new knowledge in relevant
areas. When that happens, an appropriate snapshot would feature a shaded area that extends
significantly further to the left. However, this is an ambition and not yet an achievement. This
figure reflects the current status of IT . . . Some will argue that IT offers an agenda that is too
If we compare the 1990s industrial environment that gave rise to “IT” curricula, to current day
practices, we see critical differences. In general, 1990s computing was more fragile, less well
understood, less pervasive, less automated, more specialized, and needed higher degrees of
system engineering and support.
Consider the following key value proposition for information technology as a discipline:
“Employees throughout an organization require support from IT staff who understand computer
systems and their software and are committed to solving whatever computer-related problems
they might have” (Shackelford et al., 2005), p. 14. The IT profession in terms of this statement is
now under pressure on two fronts.
First, in terms of end user computing, the movement towards consumerization continues.
Elaborate hardware and software provisioning processes requiring extensive corporate operations
have been replaced by simple self-service provisioning portals and drop-shipped commodity
devices (laptops and cell phones). Robust UX design practices coupled with readily searchable
online repositories and community support replace the need for specialized training and service
desk operators. HTML5-delivered Software as a Service reduces the need for field operations to
touch the local devices, which are often serviced through replacement and re-provisioning.
Second, in the data center, distributed computing and networking is moving from the era of
physical servers and more specialized devices and hand-tuned platforms to extensively
virtualized infrastructures defined as code (“infrastructure as code”). Even telecommunications
carriers at the most massive scale anticipate a transition from single-purpose network functional
hardware to virtualized networking functions running on commodity silicon. While engineers
will still be required to understand networking fundamentals such as queuing, routing, and
TCP/IP, the future of commercial certifications is less clear.
As networking becomes just another concern to be represented as code, the barriers for generally
educated computer scientists and software engineers are reduced, and functional barriers between
“applications” and “network” teams will come under pressure. This dynamic applies to other
technical specialties such as storage, database, middleware, and so forth. Conversely, the demand
for core computer science skills continues to increase.
Some of the other initial justifications for IT are not holding up well. Collaboration technology
(email, calendaring, etc) has been unbundled into a variety of fast-moving SaaS offerings,
increasingly self-service and with diminishing need for its engineering and operations in small
and mid-size organizations.
In terms of the knowledge areas, IT shares deficits similar to Software Engineering, notably a
lack of engagement with business models and performance. Perhaps more surprising is the fact
that it also has no weighting for “Management of Information Systems Organizations,” as that is
often a practical career responsibility for IT professionals.
Knowledge Area
IT min
IT max
Engineering Foundations for SW 0 0
Management of Info Systems Org. 0 0
Business Models 0 0
Functional Business Areas 0 0
Evaluation of Business Performance 0 0
The implications for the IT educational programs are significant. Where is the space for
technologists who do not necessarily understand the drivers of their business, who are educated
to be order takers?
Current draft is based only on 2005 Core document. To do: review (Lunt et al., 2008).
A proposed new discipline: digital management
In summary, as we have analyzed the existing disciplines, we see that:
Software engineering and information technology programs lack coverage of business
models and performance, which limits their responsiveness on the product management
Information systems programs call for technical knowledge, but do not seem to be
delivering it to the level needed for the new economy
Traditional knowledge areas need further elaboration to support emerging concerns,
practices and techniques.
In response, this paper proposes that a new digital curriculum is needed. This curriculum would
bring in knowledge of of business model, organizational performance, organizational change
management, while also retaining a more practical and technical component as compared to IS
The valid concern would be whether such curricula would be too broad. Initial experimentation
with the numerical weightings suggests not; some relief can be achieved by eliminating KAs
such as Graphics and Visualization, Intelligent Systems (AI), and Decision Theory, which can be
seen as more likely to be covered by emerging programs in Data Science. It is true that in
general, the digital professional would graduate with more breadth than depth.
Four sub-tracks, reflecting the themes of concern in this section, might be possible, with the
candidate choosing one as the vertical bar of their “T” shaped career strategy:
Digital Product Management
Digital Lifecycle Management
Digital Reliability Engineering
Digital Delivery (focusing on digital organization, culture, and process)
The following matrix is an initial attempt at defining the new digital curricula and its sub-tracks
in terms of the traditional KAs:
Knowledge Area Type
IS min
IS max
SE min
SE max
IT min
IT max
Digital min
Digital max
Product mgmt
Digital RE
Culture, org, process
Programming Fundamentals C 2 4 5 5 2 4 2 4 x x
Integrative Programming C 2 4 1 3 3 5 2 4 x x
Algorithms and Complexity C 1 2 3 4 1 2 1 2
Computer Architecture and Organization C 1 2 2 4 1 2 1 2 x x
Operating Systems Principles & Design C 1 1 3 4 1 2 1 1
Operating Systems Configuration & Use C 2 3 2 4 3 5 2 3 x x
Net Centric Principles and Design C 1 3 2 4 3 4 0 2 x
Net Centric Use and configuration C 2 4 2 3 4 5 2 4 x x
Platform technologies C 1 3 0 3 2 4 2 3 x x x
Theory of Programming Languages C 0 1 2 4 0 1 0 0
Human-Computer Interaction C 2 5 3 5 4 5 2 3 x x
Graphics and Visualization C 1 1 1 3 0 1 0 0
Intelligent Systems (AI) C 1 1 0 0 0 0 0 0
Information Management (DB) Practice C 4 5 1 4 3 4 2 3 x x
Legal / Professi onal / Ethics / Society C 2 5 2 5 2 4 2 3 x x
Information Systems Development C 5 5 2 4 1 3 3 5 x x x x
Analysis of Business Requirements C 5 5 1 3 1 2 3 5 x x x
E-business C 4 5 0 3 1 2 3 5 x
Analysis of Technical Requirements C 2 4 3 5 3 5 3 4 x x
Engineering Foundations for SW C 1 1 2 5 0 0 1 3 x x
Engineering Economics for SW C 1 2 2 3 0 1 1 3 x x
Software Modeling and Analysis C 3 3 4 5 1 3 2 3 x
Software Design C 1 3 5 5 1 2 2 3 x x
Software Verification and Validation C 1 2 4 5 1 2 2 4 x x
Software Evolution (maintenance) C 1 2 2 4 1 2 2 4 x
Software Process C 1 2 2 5 1 1 2 3 x x
Software Quality C 1 2 2 4 1 2 2 3 x
Distributed Systems C 2 4 2 4 1 3 1 3 x x
Security: issue s and principles C 2 3 1 3 1 3 1 3 x x
Security: implementation and mgt C 1 3 1 3 3 5 3 5 x x
Systems admini stration C 1 3 1 2 3 5 2 4 x x x
Management of Info Systems Org. C 3 5 0 0 0 0 2 4 x x
Systems inte gration C 1 4 1 4 4 5 2 4 x x x
Digital media development C 1 2 0 1 3 5 0 0
Technical support C 1 3 0 1 5 5 3 5 x x x
Organizational Theory NC 1 4 0 0 1 2 2 3 x
Decision Theory NC 3 3 0 0 0 1 0 0
Organizational Behavi or NC 3 5 0 0 1 2 2 4 x x
Organizational Change Management NC 2 2 0 0 1 2 2 3 x
General Systems Theory NC 2 2 0 0 1 2 1 2 x x
Risk Management (Project, safety risk) NC 2 3 2 4 1 4 1 3 x x x
Project Management NC 3 5 4 5 2 3 1 3 x x x
Business Models NC 4 5 0 0 0 0 2 4 x x* x
Functional Business Areas NC 4 5 0 0 0 0 1 3 x x
Mathematical foundations NC 2 4 3 5 2 4 1 2 x x x
Interpersonal communication NC 3 5 3 4 3 4 3 5 x x
Evaluation of Busine ss Performance NC 4 5 0 0 0 0 2 4 x x
90 150 76 142 73 128 75 139
* Infrastructure services require understanding of business models, arguably.
Associate degrees in digital management
Review (Hawthorne et al., 2014) and propose approach
Towards a new pedagogical approach
Considering whether this should be 2nd paper
This paper proposes a new reference curriculum and pedagogy, both in terms of the content of its
learning objectives (what is to be taught), as well as learning progression (how it is to be
taught). The discussion now turns to the learning progression, as it can inform the learning
A new learning progression
The term learning progression refers to the purposeful sequencing of teaching
and learning expectations across multiple developmental stages, ages, or
grade levels. The term is most commonly used in reference to learning
standardsconcise, clearly articulated descriptions of what students should
know and be able to do at a specific stage of their education. (Great Schools
Partnership, 2013)
The learning progression is based on a scaling model.” The intent is to develop a reasonable
learning progression for teaching the digital curriculum, consistent with influences such as Agile
and systems thinking, and expeditionary learning (see ).
Traditionally, pedagogy is considered distinct from curricula. However, the learning progression
suggested here actually informs the curricula, as it highlights certain organizational state changes
that have been unaddressed previously, and are the source of much debate in industry currently. It
also helps establish a logical order for the new curriculum.
In introducing the emergence model, we consider two alternative narratives. For the past several
decades, these two narratives have been used as the basis for learning progressions in IT
management education and training:
The Technical Stack narrative
The Software Development Lifecycle (SDLC) narrative and its derivations
This paper analyzes and critique these narratives from a pedagogical standpoint, and derives a
new Scaling narrative based on current digital industry practice.
The Technical Stack narrative
The Technical Stack narrative emerges from core computing practices, which are built as a series
of layers proceeding from detailed, particular concerns to higher-order matters. For example,
simple conceptual stacks might be represented thus:
Operating System
OSI networking “stack” Common conceptual
IT “stack”
Materials Science
Electrical Engineering
Computer Science
Applied Information
IT in scientific context
Figure 2. Various "stacks"
In the first two examples, we see at the bottom what are sometimes called “bit-level” or “close to
the metal” concerns – the encoded signals passing over networks or through the processors and
registers of computing machines. As we move “up,” the concerns become abstracted; that is, they
are represented by more general constructs that mask the lower level complexity and in turn
become complex. This is of course a common ontological construct and can be seen in science in
general, per the third example.
The stack narrative is indisputably powerful. Abstraction is a fundamental tool of human
understanding. However, when used as a basis for a learning progression, it encounters some
First, how do we know when a given underlying layer has been covered “thoroughly enough”?
Sometimes the dependencies are clear; sometimes they are more philosophical. In either case, the
temptation is to cover a large “batch” of the lower stack level, which can delay the all-important
practical experience of seeing the full stack in action.
Second, venture capitalist Anshu Sharma warns of the so-called stack fallacy, which he defines
as the “mistaken belief that it is trivial to build the layer above yours.” (Sharma, 2015) While he
makes this observation in the context of companies’ business strategies, we believe it has
applicability for stack-based learning progression. (We will call this the “trivial application”
fallacy, below.)
In particular, if any problem is simply a trivial application of the lower layer, then by implication
a large proportion of students’ time should be spent at the lowest level of the stack. This may not
be an optimal workforce or educational strategy. Again, the practical full stack experience is
delayed or obscured by excessive focus on component pieces or lower-order concerns.
This tendency is sometimes called “reductionism.” The “trivial application” fallacy is
humorously expressed in this XKCD comic
Figure 3. Fields by Purity, as illustration of Stack "trivial application" fallacy
Various current Management Information Systems texts are based on a simplified Stack narrative
such as:
Infrastructure (computing fundamentals,
hardware, IT assets, databases, networking)
IT applications (ERP systems, vertical
IT management concerns (projects and
Figure 4. Management Information Systems instructional "stack"
Typically prefaced with an initial discussion of business context, this bottom-to-top progression
can be seen (to varying degrees) in the tables of contents for current MIS texts such as (Laudon &
Laudon, 2015; Marakas & O’Brien, 2011; Sousa & Oz, 2015). The business context is usually
assumed to be large, bureaucratic organizations. Because there is an assumption that the
instruction must cover the largest scale application, the discussion of lower technical levels can
be overly deep and tedious for the student. Little attention is paid to smaller scale organizations,
including team dynamics within the enterprise context.
Third, sometimes the Stack narrative is inverted, such that the progression is from the top
business problem and then “decomposed” into progressively more technical aspects. This is
From, reprinted in conformance w/ web site statement “licensed under a Creative Commons
Attribution-NonCommercial 2.5 License…This means you're free to copy and share these comics (but not to sell
commonly seen in applications of architecture techniques (e.g. the Zachman framework
(Zachman, 1987)).
Decomposition is also the approach advocated by older structured systems analysis and design
techniques (e.g. (Martin, 1989b)). When executed as part of a large, phase-gated waterfall
“batch,” (see next section) it too easily generates unrealistic planning assumptions that do not
survive contact with implementation reality also known as the “ivory tower architect
The SDLC narrative
The Stack Narrative is rigorous. But it does not well represent the operational needs of applied
information technology. The ongoing delivery of digital value occurs in terms of service
moments of truth
, and the last thing the digital consumer wants is to be reminded of the
underlying layers.
The other problem with the Stack Narrative is that it is time-independent. This is not how value
is delivered. Applications of digital technology require some conceiving vision, coupled with
resources and some program of action. The term “lifecycle” is often used, e.g. the
systems/software development lifecycle or SDLC.
The “waterfall model” (Royce, 1970) is one of the best-known representations of the SDLC:
Figure 5. “Waterfall” lifecycle - similar to Royce
A term popularized in the marketing context by (Carlzon, 1987).
Royce did not actually advocate for a strict, phase-gated model. See (Lenfle & Loch, 2010) for a
well-documented account of the McNamara-era Pentagon role in the establishment of phase-
gated project management as accepted practice, and the generational consequences.
Phase gating software development in this manner has long been viewed as ineffective (see
previously cited Larman history on iterative and incremental methods (Larman & Basili, 2003)).
Reinertsen identifies the root problem as attempting to treat the generation of information
similarly to the repeatable production of goods, with a resulting and ill-advised “hostility to
variability.” (Reinertsen, 2009, p. 7). In short, the issue is one of confusing production with
product development.
Elaborate phase gating has been in general abandoned by digital-centric companies; phase gating
in the context of product development is losing favor in practice (Apke, 2015; Cobb, 2015;
Narayam, 2015; Takeuchi & Nonaka, 1986). But the economic and educational damage has been
done. Phase gating implies that different delivery phases can be isolated and handled by
specialists. This belief in specialization has defined much IT practice and can be seen reflected in
decades-old training, education, and staffing approaches.
Note for example, in the initial quote regarding the Target Corporation layoffs, that “business
analysts” were impacted. How is it that “business analysis” came to be a specialty, identifiable in
job requisitions that in turn drove the behavior of job seekers, certification bodies and training
providers? Understanding the historical roots is beyond the scope of this paper, but we see the
effects of such specialization throughout the IT workforce and throughout IT education.
In addition to the courses derived from the previously-discussed Stack perspective, colleges offer
courses directly traceable to the Royce lifecycle. The IEEE Software Engineering Body of
Knowledge (Bourque & Fairley, 2014) reflects it, and in turn influences IT curricula note the
first 5 Knowledge Areas in Table 1.
Software Requirements
Software Design
Software Construction
Software Testing
Software Maintenance
Software Configuration Management
Software Engineering Management
Software Engineering Process
Software Engineering Models and Methods
Software Quality
Software Requirements
Software Engineering Professional Practice
Software Engineering Economics
Mathematical Foundations
Engineering Foundations
Table 1. IEEE Software Engineering Body of Knowledge v3 Knowledge Areas
In SE/IS/IT education, it is common to see distinct courses in business analysis, business
architecture, systems analysis and design, requirements management, software quality analysis,
and project management itself. Focusing on these topics as specialties is a well-established
approach in IT education.
We also see the SDLC reflected in significant IT management guidance, sometimes simplified to
a heuristic of “plan, build, and run.” The ITIL guidance is based on a lifecycle consisting of
Service Strategy
Service Design
Service Transition
Service Operations
Continual Service Improvement
The COBIT guidance is structured similarly:
Align, Plan and Organize
Build, Acquire and Implement
Deliver, Service and Support
Monitor, Evaluate and Assess
Notice that all of these frameworks imply that design is distinct from construction.
A reasonable question is, why should we retain curricula based on the waterfall model? If we no
longer teach a phase-gated software development lifecycle, why would we retain the specialties
that were originally scoped from strongly phase-gated approaches?
In reality, complex, potentially valuable business problems are not understood except via
feedback from the attempts at meeting them. This is an essential principle of design thinking and
Lean product development. (See (Lockwood, 2010; Reinertsen, 2009, Chapter 8, “Using Fast
Feedback”).) Therefore, the practical, immediate concerns of how to deliver digital value must be
paramount, and cannot be deferred.
This is not to say that Agile systems development has eliminated all activities of business
analysis or product design. There is literature on Agile requirements (Leffingwell, 2011), use
cases (Cockburn, 2001), and user story mapping (Patton, 2014). Design thinking and Lean UX
(Jeff Gothelf & Josh Seiden, 2013; Lockwood, 2010; Sussna, 2015), while calling for fast
feedback, still do present a higher-level, non-technical view on approaches to defining business
None of these works covers software implementation, so it would seem safe to assume that even
in the Agile world, some generic practice of business analysis exists. The major differences
would be that:
One promising departure in understanding the lifecycle is the IT4IT guidance from the Open Group (Open Group,
2015a). It uses four “value streams:”
Strategy to Portfolio
Requirement to Deploy
Request to Fulfill
Detect to Correct
In the “Requirement to Deploy” value stream we see requirement and all the intermediate steps of analysis, design,
and construction consolidated into one value stream, suitable for iteration at speed. (Request to Fulfill is for
commodity service requests, such as routine provisioning.)
1) large “batches” of business analysis are deprecated in favor of minimum viable
hypotheses that are quickly tested, and
2) business “analysts” positioned on a persistent, small product team are called on to be “T-
shaped,” e.g. also filling technical roles such as testing or writing code.
The question is, how do we teach to these new realities of fast feedback and T-shaped
professionals? Is teaching traditional specialties such as business analysis, as large “batches” of
theory, a suitable approach? How does one teach a full semester ONLY on Requirements
Analysis, or Design, without unconsciously reinforcing the idea of large batches of input and
output, with no feedback?
Finally, in many if not most MIS texts, the critical questions of acquisition and implementation
the actual delivery activities are deferred to the later sections of the course. This reflects what
could be called the “order taking” fallacy, the idea that after systems analysis is complete,
acquisition and operation is a relatively straightforward business problem.
The Emergence narrative
The implicit message of both the Stack and SDLC narratives is that providing systems solutions
to business problems is a rational, analytic process. The Stack narrative falls prey to what we
above termed the “decomposition” and “trivial application” fallacies. The SDLC narrative falls
prey to the “waterfall” and “order taking” fallacies.
Neither narrative well reflects how digital value is conceived and delivered. But there is a
narrative that does. It is the iconic narrative of the scaling startup, and in particular, the refined
concept of the Lean Startup as conceived by Eric Ries (Ries, 2011).
The Lean Startup path is:
Identify a problem
Develop a hypothesis
Test it through the creation of a “Minimum Viable Product.”
Evaluate the results and continue or pivot to different problems/priorities
The following widely circulated illustration illustrates the idealized progression:
Figure 6. "Minimum viable product" progression
The creation of an initial Minimum Viable Product is only a start. The logical (although difficult
to achieve) progression is from a founder (individual), to a team, to a team of teams, to an
enterprise. Such progressions are a core topic of the entrepreneurial literature, and more generally
the literature on organizational growth. Examples in the entrepreneurial press include:
Nail It Then Scale It: The Entrepreneur’s Guide to Creating and Managing Breakthrough
Innovation (Furr & Ahlstrom, 2013)
Scaling Up: How a Few Companies Make It, and Why the Rest Don’t (Harnish, 2014)
Startup CEO: A Field Guide to Scaling Up Your Business (Blumberg, 2013)
Scaling Up Excellence: Getting to More Without Settling for Less (Sutton & Rao, 2014)
These popular treatments in turn cite a wide variety of academic sources such as the research of
Robin Dunbar in terms of the ability of human beings to maintain social connections (Dunbar,
2010). The book Scaling Up Excellence by Sutton and Rao is particularly well-cited; the authors
teach a course on the topic at Stanford University.
The Disciplined Agile Delivery methodology (Ambler & Lines, 2012, Chapter 1) similarly
recommends “recognize which scaling factors, if any, are applicable to your project team and
then tailor your adopted strategies to address the range of complexities the team faces.” This
method proposes three levels:
1. Core Agile Development
2. Agile Delivery
Downloaded 2016-04-21 from, provenance unclear. Many
variations are seen on search for “minimum viable product progression.” Fair use asserted for this draft. To be
3. Agility@Scale
The scaling narrative can also be used to organize IT education in a new way. Betz (C. T. Betz,
2016a) proposes a four-level “emergence model” based on the following sequence of
1. Founder
2. Team
3. Team of teams
4. Enterprise
Pedagogically, the major transitions (founder to team, team to team of teams, team of teams to
enterprise) provide additional educational opportunities in helping the learner understand the
dynamics of digital organizations. As noted in (C. T. Betz, 2016a, Chapter Introduction) “The
hypothesis is that in embracing the scaling problem we can develop an effective pedagogy that
can orient even the greenest student.
Figure 7. Emergence model (as depicted in (C. T. Betz, 2016a))
New and traditional IT concerns can be rationally organized in this framework, as well, which
allows the retention and discussion of concerns such as project and process management at the
appropriate scale.
Towards full lifecycle, full stack education
So, we have covered three narratives: Stack, SDLC, and Scale. They are intended as orthogonal.
The following visualization may be helpful:
Idea realization - operation
Idea realization - operation
Figure 8. Three dimensions of IT narratives
Although we have argued that neither the stack nor the lifecycle are a good basis for a learning
sequence, they remain important. Digital value is based on the stack, and its delivery follows at
least a core sequence of ideation, realization and operation perhaps not as elaborated as the full
Royce waterfall, but still a sequence seen in modern DevOps practices.
Because digital value requires both the full stack and the full lifecycle, both must be complete
and present throughout the learning progression. This implies that, just as a startup must
quickly establish a complete platform upon which to deliver value, so should an
instructional sequence intended to teach digital delivery.
This does not invalidate traditional approaches such as SWEBOK’s Knowledge Areas, if they are
considered as learning objectives. Learning objectives can (and possibly should) have an indirect
relationship to curricula and learning progression. They do not need to be taught as large
“batches” of dry, concentrated topic. They could also be taught iteratively, with courses touching
on them multiple times as students are taught at various scales. Such an approach is not unknown
in software engineering itself, in a way similar to Boehm’s “spiral model” (Boehm, 1988).
They should be teachable in the context of a learning progression based on scaling. This of
course would call for pedagogical research, such as controlled studies of alternative approaches.
Fortunately, classroom research is well accepted in software pedagogy: “It has been a long-
standing practice to conduct [software engineering] educational studies in the classroom”
(Lethbridge & Port, 2004, p. 5).
As noted in the 2005 report,
“We confine our attention to the bodies of knowledge and skills defined by each computing
discipline as published in the individual curriculum reports; we do not consider pedagogy or
course definition. We believe that pedagogical issues and the definition of computing courses
that might serve multiple audiences across the computing disciplines are important and timely
concerns. However, we believe we would be ill advised to address such issues in this report. This
decision should not be construed as a precedent for others to follow, and we expect that authors
of subsequent reports will revisit this issue.” (Shackelford et al., 2005) p8
Systems theory and the emergence narrative
The idea of scaling can imply a simple linear progression. However, in a complex sociotechnical
system, scaling up often results in emergent complex and even chaotic behavior. New challenges
appear for example when teams grow so large they no longer can all be co-located.
The foundational principle to basing pedagogy on the scaling narrative can be seen in the
following quote by John Gall:
A complex system that works is invariably found to have evolved from a simple
system that worked. A complex system designed from scratch never works and
cannot be patched up to make it work. You have to start over, beginning with a
working simple system.
John Gall, The Systems Bible (Gall, 2012)
Another version of this same concept can be seen in the idea of “walking skeleton”:
A Walking Skeleton is a tiny implementation of the system that performs a
small end-to-end function. It need not use the final architecture, but it should
link together the main architectural components. The architecture and the
functionality can then evolve in parallel.
-- Alistair Cockburn (Cockburn, 1996)
If we consider the simplest system of IT delivery that “works,” it is the lone individual crafting
their vision (the iconic startup). Even that individual has certain needs, including understanding
the value they seek to deliver, identifying the correct toolset, and construction. They must cover
both full stack and full lifecycle.
If we elaborate the progression from startup to enterprise, one reasonable representation of the
emergent concerns might be:
IT value. What is possible with computing? What is our vision for some digital product value?
IT infrastructure. We want to build something. We have to choose a platform first, in terms of
fitness for purpose, robustness, security, and overall suitability for the digital value we seek to
IT applications. Let’s start building something of use to someone
Product management. What exactly is it we are building? What is the process of discovering our
customer’s needs and quickly testing how to meet them? How do we better define the product
vision, and the way of working towards it, for a bigger team?
Work management. How do we keep track of what we are doing, and communicate our progress
and needs at the simplest level?
Operations management. How do we sustain this surprisingly fragile digital service, in its
ongoing delivery of value?
Team of Teams
The boundary between the "team" and the "team of teams" is a challenging area, and industry
responses remain incomplete and evolving. Hence the notation "THE SCALING PROBLEM" on
the diagram.
Organization and culture. We’re getting big. How do we deal with this? How are we structured?
Why this way and not that? How can we benefit from increasing maturity and specialization,
while still maintaining a responsive product? What are the unwritten values and rules in our
Project and resource management. Work is becoming larger and more complex. How can we
track and execute larger segments of it, and coordinate across multiple teams?
Process management. We have a structure. Work needs to flow across it. Product and project
teams increasingly demand predictable, consistent delivery of internal services, and
synchronization points such as enterprise releases are increasingly critical signals.
Security, and governance, risk, and compliance. We need to cope with external forces
(regulators, vendor partners, security adversaries, auditors) increasingly defining our options.
Enterprise information management. We’ve been concerned with data, information, and
knowledge since the earliest days of our journey. But at this scale, we have to formalize our
approaches and understandings; without that, we will never capture the full value available with
modern analytics and Big Data. Compliance issues are also compelling us to formalize here.
Architecture and portfolio. We need to understand the big picture of interacting lifecycles, reduce
technical debt and redundancy, accelerate development through establishing platforms, and
obtain better economies of scale. We need to define our investment strategy based on a sound
understanding of both business needs and technology limitations.
The emergence model seeks to define a likely order in which concerns are formalized. Any
concern may of course arise at any time: the startup founder certainly is concerned with security!
Formalization means at least one or more of the following:
Dedicated resources
Dedicated organization
Defined policies and processes
Automated tooling
Typically, startups avoid formalized process and project. To the extent the higher-order
concerns exist, they are tacit (understood or implied; suggested; implicit). Certainly, a small
startup does not invest in an enterprise-class service desk tool supporting a full array of IT
management processes, or a full-blown Project Management Office with its own Vice President
and associated portfolio automation. Simple work management, with a manual or automated
Kanban board, is likely their choice for work management.
But by the time they are a team of teams, specialization has emerged and more robust processes
and tools are required. Failure to act on this may result in the “classic symptoms of a group that is
too big coordination snafus and weak social bonds.” (Sutton & Rao, 2014, p. 122) The danger
of course is that the formalization effort may be driven by its own logic, and start to lose track of
the all-critical business context. In a study of 3200 startups, it was seen that “74 percent of these
young organizations suffered from premature scaling” (Sutton & Rao, 2014, p. 269).
By careful examining these stages of maturation, the emergent concerns, and the industry
responses to them, the goal is that the student will have effective tools to critically engage with
the problem of scaling the digital organization.
Updated learning objectives
Todo: map to canonical KAs
Not sure if these should be moved up further separate LOs from pedagogy.
A structured set of learning objectives is included in the appendix, including references
supporting their importance which may be used by educators to further develop curricula.
At a summary level, here are more detailed learning objectives as a basis for the revised
Understand the concept of digital value in terms of applied information technology
Understand principles and current trends in IT infrastructure services, with a primary
focus on Cloud sources.
o Demonstrate capability with Cloud and web-scale IT based on virtualization and
open source platforms.
o Demonstrate capability with configure, build, and deploy automation and
infrastructure as code.
o Demonstrate familiarity with practical version control, including source control,
package management, and branching/merging schemes.
o Demonstrate understanding of security fundamentals for infrastructure services
Understand the fundamentals of managing digital services and applications and their end-
to-end lifecycle pipelines (DevOps).
o Understand history, theory and practice of Agile methods, contrasted with other
delivery approaches. Understand the broader Agile ecosystem in the context of the
digital economy.
o Understand why Agile and related methods are effective in terms of fundamental
underlying principles such as fast feedback.
o Demonstrate capability with automated testing, test-driven development,
continuous integration, and continuous delivery
o Assessment, re-use, and management of open source assets (from code “snippets”
to full platforms)
Understand the fundamentals of product development and management in the digital
context, including design thinking as a philosophy, and practices such as Scrum.
Understand principles of Lean product management and its influence on the Agile
Understand the fundamentals of task and work management in collaborative, team-based
environments, including practices such as Kanban.
Understand basic operations management concepts as applied to digital products,
including current insights from Web- and Cloud-scale operations.
o Demonstrate familiarity and capability with automated deployment and
management strategies at scale, including topics such as feature toggles and
canary environments
Understand organizational structure and culture as it affects digital product delivery
o Understand cultural and organizational factors in collaborative production of
value based on complex software-intensive systems.
Understand the evolving role of project and resource management, including lighter-
weight approaches
o Demonstrate familiarity with principles and techniques for scaling Agile. State
transitions organizations characteristically encounter as they scale.
o Demonstrate understanding of relationship between product and project
Understand process management and continuous improvement in the digital and IT-
centric context. Demonstrate understanding of traditional and next generation process and
management frameworks, and current research and debates in IT process approaches.
Understand the concerns of IT governance, risk, security, and compliance
Understand the challenges of managing data, information and analytics at scale
Understand the evolving role of IT portfolio management and architecture, including
skeptical and challenging viewpoints
Platform support for scaling pedagogy
In order to support the new learning progression, a platform simulation can be effective. Abstract
explanations of complex system interactions (e.g. narratives accompanied by architectural
diagrams) come alive when supported with direct, hands-on experience. Fortunately, modern
computing hardware combined with powerful open source software opens up educational
possibilities unimaginable even ten years ago.
Figure 9. Calavera lab platform
But since the scaling progression starts at the smallest organizational size, the stack and lifecycle
must be minimal. Figure 9 illustrates the Calavera platform (C. T. Betz, 2016b), an open-source
lab platform used to illustrate various aspects of modern IT delivery at the University of St.
Thomas in Minnesota.
Calavera was conceived as a “minimum viable pipeline” including:
a simple Tomcat-based, test-driven Java application, as the subject of development,
testing, compilation, packaging, deployment, and operations
local and remote git source control,
automated build management with Jenkins,
package management with Artifactory,
monitoring with Nagios, and
configuration management with Chef.
Hashicorp Vagrant and Virtualbox are used as the virtualization platform, and all the software is
freely available under various licensing schemes.
All 7 lab nodes can be run on one modern laptop (8gb RAM); larger servers can accommodate
multiple pipeline instances for multiple teams.
Calavera-based laboratories support the following learning topics:
Infrastructure as code
Test-driven development
Continuous integration
Continuous delivery
Digital product management
Team collaboration
IT process management
Calavera is published as a free Github-based open source project (
academy/Calavera). It primarily consists of various “infrastructure as code” resources that, when
instantiated upon a correctly configured platform, generate the interconnected cluster. Interested
parties are invited to download, experiment with, and assist with improving the platform.
Future plans for Calavera include:
More elaborated subject system (e.g. inclusion of Web framework & persistence layer)
Other development stacks (MEAN, .Net, etc)
Container technology
Cloud deployment (e.g. to Amazon EC2)
Cloud/Web-scale systems including auto-scaling, load balancing, caching, etc. Load
The intent of Calavera is not to eliminate attention to particular concerns within either Stack or
SDLC narratives, but rather to enable. Students can learn specialized topics (e.g. database
management, security, usability) within the context of a reasonable simulated system, so they can
apply their learnings (again, perhaps based on SWEBOK Knowledge Areas) in a more
contextualized manner. For example, a database programmer can see the practical impact of
indexing choices within a full stack experience, in which performance and usability questions
become obvious in terms of the user experience.
Flipped capstone: a proposal
Whether the student is pursuing CSci, MIS, or Software Engineering, a common pattern is that
after proceeding through either or both stack-based and SDLC-based courses, the student will be
presented with the opportunity to do a “capstone” project intended to tie together the various
Readers with experience in implementing large scale systems might draw a similarity to
approaches in which component teams develop various parts of a complex system, with
integration deferred until final phases of the project. This is known to be a risky approach and
prone to failure. Mitigating strategies from the Agile community include concepts such as
“integrate early and often” and “build a walking skeleton(Cockburn, 1996).
One proposal is that students should START their IT education by defining their capstone,
perhaps as the outcome of an introductory course or 2-semester sequence. This is similar to the
concept of “expeditionary learning” pioneered by Kurt Hahn, the founder of Outward Bound
( They would learn specialized topics in the context of their capstone
vision. Faculty would take on the role of coaches, assisting in the elaboration of the capstone idea
to something approaching enterprise complexity (if not full enterprise scale).
We are at an exciting time in the industrialization of digital technology and its management. All
industries, including those that have not been viewed as purely “technical” are now being
challenged by the tsunami known as “digital transformation.” Diomidis Spinellis, editor in chief
of IEEE Software, observed last year that
…other industries are also producing what’s in effect software (executable
knowledge) but not treating it as such…Although many industries have
developed their own highly effective processes over the years, software
engineering maintains an essential advantage. It has developed methods and
tools that let even small teams manage extremely high complexity. … This
advantage is important because the complexity in non-software activities is
also increasing inexorably… [T]he time has come to transform our world… by
giving back to science and technology the knowledge software engineering has
produced (Spinellis, 2015).
As Spinellis notes, digital tools and techniques are influencing not just IT, but business practices
worldwide. We need new approaches to instructing the next generation workforce in these topics.
Educational transitions are never easy nor cheap. But meeting the demands of the digital
economy with an appropriately skilled workforce has always been a mission of higher education.
We thank readers for their attention and look forward to continued dialog.
Modern cloud and virtualization techniques enable remarkable possibilities along these lines.
Appendix: Defining “IT,” “Agile,” “DevOps,” and “digital”
As educators and scholars we operate in society and must come to terms with ambiguous
terminology. The following definitions will be used here.
Information Technology (IT)
Information Technology has a problematic history of definition.
As noted in the 2005 core report, “Information technology is a label that has two meanings. In the
broadest sense, the term information technology is often used to refer to all of computing. In
academia, it refers to undergraduate degree programs that prepare students to meet the computer
technology needs of business, government, healthcare, schools, and other kinds of organizations.
In some nations, other names are used for such degree programs.” (Shackelford et al., 2005), p.
Because of the ambiguity of the IT term in particular, we will qualify its use by (e.g.) using terms
such as “the IT 2008 curriculum” or “the CC2005 IT discipline scope.”
Agile and the Agile ecosystem
The term “Agile” is also contentious. Properly and strictly speaking, it refers to the principles
espoused by a group of leading software engineers who met in 2001 and produced the “Agile
Manifesto” (Agile Alliance, 2001).
However, those principles have proven remarkably influential, such that they have been claimed
as inspiration in a variety of domains beyond the original intent of their authors. There are books
on Agile Hiring (Sean Landis, 2011), Agile Contracts (Opelt, Gloger, Pfarl, & Mittermayr,
2013), and Agile Organization Design (Narayam, 2015).
Agile influences are seen “upstream”, e.g. in practices such as Lean UX and Lean Startup (Ries,
2011). Their “downstream” influence is seen in the current “DevOps” movement (Bass et al.,
2015; Humble & Farley, 2011; Kim, Behr, & Spafford, 2013). Other industry themes may or may
not be directly influenced by Agile, but seem at least consistent in spirit, e.g. Lean Accounting
and Beyond Budgeting.
Conversely, various Agile thinkers and practitioners have brought in influences from fields other
than software engineering. Lean thinking has long been seen as compatible with Agile principles
(Poppendieck & Poppendieck, 2003, 2007). The Theory of Constraints (Goldratt & Cox, 2004) is
reflected in works such as (Kim et al., 2013; Larman & Vodde, 2009). Product management
approaches and insights (Reinertsen, 2009; Takeuchi & Nonaka, 1986) have influenced practices
such as Scrum (Schwaber & Sutherland, 2012), Kanban (Anderson, 2010), and the Scaled Agile
Framework (Leffingwell, 2011). Other influences include systems theory (seen e.g. in (Larman &
Vodde, 2009)) and even control theory (as in (Kennaley, 2010)).
Because of this breadth, we strive to use the term “Agile ecosystem” when referring to concepts
or practices not directly represented in the original Agile manifesto. We do capitalize the word
“Agile” as we still recognize the central influence of the Agile Manifesto.
Wikipedia defines DevOps thus:
DevOps (a clipped compound of development and operations) is a culture, movement or
practice that emphasizes the collaboration and communication of both software developers and
other information-technology (IT) professionals while automating the process of software
delivery and infrastructure changes. It aims at establishing a culture and environment where
building, testing, and releasing software, can happen rapidly, frequently, and more reliably.”
(Wikipedia, 2016).
The Agile origins of DevOps are clear; a watershed moment in its evolution was a discussion on
“Agile Infrastructure” between two industry professionals. If we accept that the term “Agile” is
properly applied primarily to software development, then “DevOps” is the broader term as it
encompasses upstream and downstream concerns, including culture. However, much actual
DevOps written material is technical and less concerned with (e.g.) topics such as hiring and
financial management, IT governance, field services, and so forth. It is not a substitute for the
entire spectrum of IS/IT concerns.
Finally, the term “digital” (as in “digital transformation”) has a great deal of currency
(Westerman, Calméjane, Bonnet, Ferraris, & McAfee, 2011; Westerman, Tannou, Bonnet,
Ferraris, & McAfee, 2012). This paper suggests a pragmatic definition of “digital
transformation” as “the increasing IT component of all that we know, use, experience, and
consume.” (“IT” being defined in the broadest sense of applied computing.)
It is this paper’s contention that digital management necessarily implies DevOps and the Agile
ecosystem, but is itself a broader concept. Where the ecosystem’s boundaries become unclear, we
switch to the word “digital.” In particular, the concerns and practices of web-scale IT (Allspaw &
Robbins, 2010; Limoncelli et al., 2014), while compatible, are not usually conflated with Agile,
and seem to imply some distinction even from DevOps. Other frequently mentioned “digital”
concerns include Internet of Things, social media, analytics, mobility, and Cloud.
Appendix: Theory and research in digital management
In proposing new curricula that could alter the priorities of IT education worldwide an
extraordinary suggestion, admittedly a reasonable question would be “what is your basis for
There are various sources emerging, which we cover below. But it is important to realize that
many aspects of IT education and practice currently lack theory. The mathematical
foundations of computing and information are well established. But for example, project
management has origins rooted in practice. An overall theory is elusive (Alleman, 2002; Koskela
Gregory, 2002). One searches in vain for a rigorous basis for ITIL, TOGAF, COBIT, or CMMI.
Relevant theories of organizational development, industrial operations, human factors, and many
other well established domains go unrecognized in far too much IT industry “best practice.”
As digital transformation has accelerated, new theoretical investigations have started to emerge.
The leading thinkers are not necessarily fully engaged in the academic community as yet; many
are still in industry or on the border between industry and academia. Others come from non-IT,
non-CSci/MIS backgrounds.
Examples of the new generation of IT management thought leaders include
theoretical physicist Mark Burgess, Ph.D. (Burgess, 2004, 2014, 2015),
Lean product development specialist and ex-nuclear engineer Donald Reinertsen
(Reinertsen, 1997, 2009; Smith & Reinertsen, 1991),
mathematician Troy Magennis (Magennis, 2011),
mathematician Murray Cantor, Ph.D. (Cantor, 2011, 2014),
statistician Nicole Forsgren, Ph.D., lead investigator on annual State of DevOps reports
(Forsgren Velasquez, Kim, Kersten, & Humble, 2014, 2016; Puppet Labs, 2015)
electrical engineer Mark Kennaley (Kennaley, 2010),
author and educator David Anderson (Anderson, 2010) and
ex-Tripwire CTO and author Gene Kim (Behr, Kim, Spafford, & Information Technology
Process Institute., 2005; Kim et al., 2013).
The following topics need more research attention:
Impact of Cloud, web-scale, DevOps, Lean, and Agile IT on IT management, finances,
operations, portfolios.
Organizational and communications structures in the IT-centric organization. How IT
management structures and capabilities emerge as a function of organizational scale.
Architectural approaches in their representation.
Practical simulations & reference implementations of IT delivery infrastructures; use and
evaluation in classroom and lab settings. Use for validation of IT management
standard abstractions.
Semantic and semiotic analysis of IT management (see e.g.(Somasundaram & Murphy,
2012) applied to broader concerns such as IT portfolios)
History of “IT management” as a professional concern and discipline. History, evolution,
scope, and impact of IT management standards such as TOGAF, ITIL, COBIT, CMMI,
IT4IT. Agile impacts. Internal consistency of standards & inter-standard compatibility
(e.g.(C. Betz, 2011))
Intersection of software development and Lean product development
Human-computer interaction in terms of complex computing system operations.
There are many great practicing software engineering thinkers and methodologists in the Agile community. The
individuals identified here are notable because they are applying statistical and other mathematical techniques
(queuing and Monte Carlo simulations, control theory, and more) to understanding digital management.
Intersection of IT management with classical control theory and systems engineering
Interdisciplinary: Intersection of IT management with industrial operations research,
communications theory
Pedagogical how do we best train people from a wide variety of backgrounds to
participate in the digitally transformed economy? What are the pedagogical implications
of Agile and web-scale approaches?
Efficacy and impact of commercial training modes such as boot camps. Relationship of
these approaches to traditional higher education and placement strategy.
Theory and frameworks
Despite their significant consequences for society, researchers have in general disregarded the
frameworks, and have not (with a few exceptions, e.g. (Bollinger C., 1991; Koskela Gregory,
2002; Lee Young Soon; Kim, Chan Hoon, 2007; Pollard & Cater-Steel, 2009)) subjected them
to the kind of critical, scholarly scrutiny artifacts of such influence would seem to merit
This paper is concerned that these current “best practice” standards are little more than folk
models [insert AIS cite], in some cases based on outdated and discredited philosophies, such as
the incorrect application of Taylorist thinking to problems of product development.
Even where
they may be useful, there is a tendency for their practitioners to over-apply them, as in “if all you
have is a hammer, everything looks like a nail.”
The industry has created the frameworks and approaches (ITIL, COBIT, PMBOK, Scrum,
DevOps, etc) partly in response to the scaling problem. However, such guidance tends to become
a generalized professional identity, that can operate in quasi-religious fashion. (For evidence,
execute a search on “ITIL vs DevOps” or “Agile is dead.”) The challenge for researchers and
instructors is to identify the emergent challenges as specifically and neutrally as possible, without
the overhead of these broader abstractions and their multiple, unmanageable connotations.
For example, instead of being concerned with “project management,” careful analysis focuses
more on constituent aspects such as timing, task dependencies, resource availability, estimation
vs. commitment, and so forth. Instead of “ITIL,” focus on service theory and its relationship to
process management, and within process management, examine factors such as cadence and
synchronization, continuous improvement, measurement, variation, and so forth.
Systems theory and the emergence narrative
New frameworks are emerging as an expression of the new Agile practices (Anderson 2010;
Cohn 2010; Leffingwell 2011; Leffingwell, Yakyma et al. 2014; Burrows 2015). But the
questions again arise. How do we know that they are not simply the new folk models of the
More literature search is called for here.
See the work of Don Reinertsen (Reinertsen, 1997, 2009; Smith & Reinertsen, 1991) for exhaustive analysis and
critique on this topic. Reinertsen is widely cited among Agile authors and is one of that community’s most important
intellectual influences.
moment partial and overly context-bound? These frameworks are already replacing the older
ones, and again serving to organize and manage IT investment. To prevent the missteps of the
past, new research programs and academic involvement are needed so that “best practice”
guidance is defensibly such. This only becomes more critical as we consider the consequences of
universal digital transformation.
Appendix: Reference learning objectives in emergence model
Learning objective
Support (not necessarily to be directly
incorporated in text, but evidence for
objective importance)
Emergence model
(Diana Larsen & James Shore, 2012)
1. Founder’s perspective (minimum viability)
a. Digital value
i. Identify the digitally-enabled
aspects of daily life
(Westerman et al., 2012)
ii. Describe the fundamental
components of digital
(Ries, 2011)
iii. Describe the lifecycle of a
digital product
iv. Define the roles of digital
product consumer, customer,
and sponsor
v. Understand the “what” of
digital value versus the “how”
of digital service systems
IT4IT service definition
b. Digital platforms
i. Understand basic computing
history and principles
ii. Understand the importance of
iii. Describe the primary types of
Cloud computing
(Nist, 2011)
iv. Understand the basic concepts
of “infrastructure as code”
v. Understand the basic
principles of version control
(Chacon, 2009; Loeliger, 2009)
c. Digital application pipelines
i. Understand history of
application development
(Brooks, 1995; DeMarco, 1979;
Martin, 1989a; Page-Jones &
Constantine, 2000; Royce, 1970;
Yourdon, 1975)
ii. Understand causes for and rise
of Agile and DevOps
(Agile Alliance, 2001; Bass et al.,
2015; Beck & Andres, 2005; Beck &
Fowler, 2001; Schwartz, 2014)
iii. Understand fundamentals of
describing system intent
(Cockburn, 2001; Patton, 2014)
iv. Understand importance of and
basic approaches to test-
driven development
(Beck, 2003; Fowler & Beck, 1999)
v. Understand principles of
continuous integration and
(Allspaw & Hammond, 2009; Bass et
al., 2015; Duvall et al., 2007; Humble
& Farley, 2011; Shortland & Lei,
2. Team perspective (collaboration)
a. Product management
i. Understand fundamental goals
and objectives of product
management in a digital
(Kahn & Product Development &
Management Association., 2005)
ii. Understand various
approaches to digital product
design, such as Lean Startup,
Lean UX, Business Model
Canvas, etc.
(Lockwood, 2010; Osterwalder &
Pigneur, 2010; Ries, 2011; Sussna,
iii. Understand origins and
practices of Scrum-based
(Cohn, 2010; Sutherland, 2014;
Takeuchi & Nonaka, 1986)
iv. Understand principles of Lean
Product Development, such as
(Poppendieck & Poppendieck, 2003;
Reinertsen, 1997, 2009)
Cost of Delay
v. Understand the importance of
hypothesis-driven, fast-
feedback-based approaches to
product development
(Allspaw & Hammond, 2009)
b. Work management
i. Understand the fundamental
problem of coordinating tasks
on a small team basis
(Mintzberg, 1983)
ii. Understand the importance
of, and techniques for, limiting
work in process
(Anderson, 2010, 2012; Burrows,
iii. Understand origins and basic
practices of Kanban as applied
to team-based product
(Anderson, 2012; Burrows, 2015)
iv. Understand techniques for
establishing and sustaining
shared mental models of work
(e.g. the Toyota concept of
(Ohno, 1988)
v. Understand the basic
problems of time and space
(Cohn, 2010)
shifting in collaborative
c. Operations management
i. Understand concept of
“operations” and its evolution
in the context of digital
product management
(Allspaw & Robbins, 2010;
Limoncelli et al., 2014)
ii. Understand basic monitoring
(Sturm, Morris, & Jander, 2000)
iii. Understand concepts of state
and configuration
(Burgess, 2004)
iv. Understand concepts of
capacity and performance
management, including
current Web- and Cloud-scale
approaches to highly scaled
(Allspaw & Robbins, 2010; Brill,
2014; Hernandez, 2012; Limoncelli et
al., 2014; Schwartz, 2014)
v. Understand emerging
concerns of complex systems,
automation complacency, and
related topics.
Allspaw. Aeronautics.
3. Team of teams perspective (coordination)
a. Organization and culture
i. Understand the importance of
organization and culture to
successful digital product
(Mintzberg, 1983)
ii. Understand basic matrix
management problem:
product vs. function
(Smith & Reinertsen, 1998)
iii. Understand current models
for product organizations,
such as Amazon and Spotify
(Choi, 2014; Kniberg & Ivarsson,
iv. Understand basic concepts of
organizational culture: how to
describe and various
approaches to influencing
(Rother, 2010)
v. Understand various
alternative scenarios for
digital/business convergence
b. Project and resource management
i. Understand basic history,
practices and challenges of
classic project management
(Glass, 1998; Lenfle & Loch, 2010;
Project Management Institute., 2013)
ii. Understand core issues in the
interaction of project
management with Agile
methods and the current state
of the “scaling Agile”
(Cobb, 2011; Hernandez, 2012)
iii. Understand basic practices of
IT sourcing
(Opelt et al., 2013)
iv. Understand basic practices of
IT human resource
(Glen, 2003)
v. Understand basic practices of
IT financial management
(Quinlan & Quinlan, 2003; Quinlan,
2003; Remenyi, Money, & Bannister,
c. Process management and continuous
i. Understand history and core
concerns of process
management in the digital
(Harmon, 2003; Rummler & Brache,
1995; Sharp & McDermott, 2009)
ii. Understand relationship of
process to product and project
(Hayes & Wheelwright, 1979;
Memon, 2014; West & Wildeman,
iii. Understand role, importance,
(CMMI Product Team, 2001; ISACA,
and challenges of process
frameworks (ITIL, COBIT, SAFe,
2012; Leffingwell et al., 2014;
Schwaber, 2013; The Stationery
Office, 2011a, 2011b, 2011c, 2011d,
iv. Understand basic principles of
continuous improvement
4. Enterprise perspective
a. Governance, risk, security, and
i. Understand basic concerns
related to governance, risk,
security, and compliance
(Conrad, Misenar, & Feldman, 2010)
ii. Define “governance” versus
(ISACA, 2012)
iii. Understand core concepts of
agent, threat, vulnerability,
risk, asset, exposure,
safeguard, and control
(Conrad et al., 2010)
iv. Understand interaction of
security operations and
architectures with digital
Rugged DevOps / SecDevOps
v. Understand concept of anti-
(Izrailevsky & Tseitlin, 2011)
fragility, e.g. via Netflix Simian
b. Enterprise information management
i. Understand history and
concerns of data and
information management,
including data governance,
data quality, and related
(Data Management Association, 2009)
ii. Understand emerging trends
with large scale data and
analytics (Big Data) in the
context of digital product
(Davenport & Harris, 2007)
iii. Understand basic issues of
semantics and domain-driven
(Carlis & Maguire, 2001; Evans,
2004; Hay, 1996)
iv. Understand the CAP theorem
and its impacts on digital
architectures and operations
(Brewer & Fox, 1999)
v. Understand the convergence
of traditional records
management with digital, and
related concerns
c. Portfolio and architecture
i. Understand basic IT/digital
portfolio management
problem and approaches
ii. Understand the importance
and fundamentals of systems
(Snowden & Boone, 2007)
iii. Understand the core digital
lifecycles (application service,
infrastructure service, asset,
product) and their interactions
(C. T. Betz, 2011)
iv. Understand the history and
current state of architecture
and its variants (enterprise,
business, data, solutions,
technical) as an IT and digital
(Spewak & Hill, 1993)
v. Understand the intersection
and contradictions of
architecture and Agile
practices in the digital context,
including concepts such as
technical debt and emergent
Agile Alliance. (2001). Agile Manifesto and Principles. Retrieved January 1, 2016, from
Alleman, G. B. (2002, October 1). Is There an Underlying Theory of Software Project
Management? (A critique of the transformational and normative views of project
management). Retrieved April 15, 2016, from
Allspaw, J., & Hammond, P. (2009). 10 deploys per day: Dev & ops cooperation at Flickr.
Velocity 2009. San Jose, CA: O’Relly Publications. Retrieved from
Allspaw, J., & Robbins, J. (2010). Web operations (1st ed.). Beijing China ; Sebastopol, CA:
Ambler, S. W., & Lines, M. (2012). Disciplined Agile Delivery: A Practitioner’s Guide to Agile
Software Delivery in the Enterprise. IBM Press. Retrieved from\n
Anderson, D. J. (2010). Kanban: Successful Evolutionary Change for your Technology Business.
Sequim, WA: Blue Hole Press.
Anderson, D. J. (2012). Lessons in Agile Management: On the Road to Kanban (Kindle Edi).
Blue Hole Press.
Apke, L. (2015). Managers from Hell, The PMO is Dead, and Other Agile Stories. Amazon
Digital Services LLC.
Association for Computing Machinery. (2016). Curricula Recommendations. Retrieved May 28,
2016, from
Balter, B. (2015). The difference between 18F and USDS. Retrieved April 21, 2016, from
Bass, L., Weber, I., & Zhu, L. (2015). DevOps: A Software Architect’s Perspective. New York:
Addison Wesley.
Beck, K. (2000). extreme programming eXplained : embrace change. Reading, MA: Addison-
Beck, K. (2003). Test-Driven Development By Example. Rivers (Vol. 2).
Beck, K., & Andres, C. (2005). Extreme programming explained : embrace change (2nd ed.).
Boston, MA: Addison-Wesley.
Beck, K., & Fowler, M. (2001). Planning extreme programming. The XP series. Boston:
Behr, K., Kim, G., Spafford, G., & Information Technology Process Institute. (2005). The visible
ops handbook : implementing ITIL in 4 practical and auditable steps (Rev. 1st). Eugene,
OR: Information Technology Process Institute.
Betz, C. (2011). ITIL®, COBIT®, and CMMI®: Ongoing Confusion of Process and Function .
BPTrends. Retrieved from
Ongoing Confusion of Process and Function-Betz-Final.pdf
Betz, C. T. (2011). Architecture and Patterns for IT: Service and Portfolio Management and
Governance (Making Shoes for the Cobbler’s Children), 2nd Edition. Amsterdam:
Elsevier/Morgan Kaufman.
Betz, C. T. (2016a). Agile IT Manaagement: From Startup to Enterprise. Retrieved April 21,
2016, from
Betz, C. T. (2016b). Calavera Project. Retrieved January 1, 2016, from
Bidgoli, H. (2014). Management Information Systesm. Boston, MA: Cengage Learning.
Blumberg, M. (2013). Startup CEO: A Field Guide to Scaling Up Your Business, + Website.
Boehm, B. (1988). A Spiral Model of Software Development and Enhancement. IEEE
Computer, 21(5), 6172.
Bollinger C., T. B. . M. (1991). A Critical Look at Software Capability Evaluations. IEEE
Software, 8(4), 2541.
Bourque, P., & Fairley, R. E. (2014). Guide to the Software Engineering Body of Knowledge,
Version 3.0,. IEEE Computer Society.
Brewer, E., & Fox, A. (1999). Harvest, Yield and Scalable Tolerant Systems. 7th Workshop Hot
Topics in Operating Systems (HotOS 99). IEEE CS.
Brill, S. (2014). Obama’s Trauma Team. Time Magazine. Retrieved from
Brooks, F. P. (1995). The mythical man-month : essays on software engineering (Anniversar).
Reading, Mass.: Addison-Wesley Pub. Co.
Burgess, M. (2004). Analytical network and system administration : managing human-computer
networks. Chichester, England: John Wiley & Sons Inc.
Burgess, M. (2014). Promise Theory: Principles and Applications (Volume 1) . CreateSpace
Independent Publishing Platform (February 5, 2014).
Burgess, M. (2015). In Search of Certainty: The Science of Our Information Infrastructure.
O’Reilly Media.
Burrows, M. (2015). Kanban from the Inside: Understand the Kanban Method, connect it to what
you already know, introduce it with impact. Sequim, Washington: Blue Hole Press.
Cantor, M. (2011). Calculating and Improving ROI in Software and System Programs.
Communications of the ACM, 54(9), 121130.
Cantor, M. (2014). Implementing Product Flow Measures for Lean Software and DevOps. Cutter
IT Journal, 15(6).
Carlis, J. V., & Maguire, J. D. (2001). Mastering data modeling: a user-driven approach.
Boston: Addison-Wesley.
Carlzon, J. (1987). Moments of Truth. Ballinger Pub Co.
CCTA. (1996). IT Service Delivery Tools. (O. of G. C. (OGC), Ed.)IT Infrastructure Library.
London: HMSO.
Chacon, S. (2009). Pro Git. The expert’s voice in software development. Berkeley, CANew
York: Apress ;Distributed to the book trade worldwide by Springer-Verlag.
Choi, J. (2014). The Science Behind Why Jeff Bezos’s Two-Pizza Team Rule Works. Retrieved
CMMI Product Team. (2001). Capability Maturity Model Integration (CMMI), Version 1.1.
Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University.
Cobb, C. G. (2011). Making sense of agile project management : balancing control and agility.
Hoboken, N.J.: Wiley. Retrieved from
Cobb, C. G. (2015). The Project MANAGER’S GUIDE TO MASTERING AGILE: Principles and
Practices for an Adaptive Approach. Hoboken, New Jersey: John Wiley & Sons.
Cockburn, A. (1996). Walking skeleton. Retrieved April 28, 2016, from
Cockburn, A. (2001). Writing effective use cases. The Crystal series for software development.
Boston: Addison-Wesley.
Cohn, M. (2010). Succeeding with Agile: Software Development Using Scrum. Upper Saddle
River, New Jersey: Addison-Wesley.
Conrad, E., Misenar, S., & Feldman, J. (2010). CISSP Study Guide. CISSP Study Guide.
Data Management Association, T. (2009). The DAMA Guide to The Data Management Body of
Knowledge (DAMA-DMBOK Guide). Bradley Beach, NJ: Technics Publications, LLC.
Davenport, T. H., & Harris, J. G. (2007). Competing on analytics : the new science of winning.
Boston, Mass.: Harvard Business School ; London : McGraw-Hill [distributor]. Retrieved
from Table of contents only
DeMarco, T. (1979). Structured analysis and system specification. Englewood Cliffs, NJ:
Yourdon Press.
Diana Larsen, & James Shore. (2012). Your Path through Agile Fluency. Retrieved April 6,
2016, from
Dunbar, R. (2010). How Many Friends Does One Person Need? Dunbar’s Number and Other
Evolutionary Quirks. Harvard University Press.
Duvall, P. M., Matyas, S., & Glover, A. (2007). Continuous integration : improving software
quality and reducing risk. Addison-Wesley signature series. Upper Saddle River, NJ:
Addison-Wesley. Retrieved from
Evans, E. (2004). Domain-driven design : tackling complexity in the heart of software. Boston ;
London: Addison-Wesley.
Forsgren Velasquez, N., Kim, G., Kersten, N., & Humble, J. (2014). 2014 State of DevOps
Forsgren Velasquez, N., Kim, G., Kersten, N., & Humble, J. (2016). 2016 State of DevOps
Fowler, M., & Beck, K. (1999). Refactoring : improving the design of existing code. The
Addison-Wesley object technology series. Reading, MA: Addison-Wesley.
Furr, P., & Ahlstrom, N. (2013). Nail It then Scale It: The Entrepreneur’s Guide to Creating and
Managing Breakthrough Innovation. NISI Publishing.
Gall, J. (2012). The Systems Bible: The beginner’s guide to systems large and small. General
Systemantics Pr/Liberty.
Glass, R. L. (1998). Software runaways. Upper Saddle River, NJ: Prentice Hall PTR.
Glen, P. (2003). Leading Geeks: How to Manag and Lead People who Manage Technology. San
Francisco: Jossey-Bass.
Goldratt, E. M., & Cox, J. (2004). The goal : a process of ongoing improvement (3rd rev.). Great
Barrington, MA: North River Press.
Great Schools Partnership. (2013). Learning Progression. Retrieved April 6, 2016, from
Harmon, P. (2003). Business Process Change: A Manager’s Guide to Improving, Redesigning,
and Automating Processes. Amsterdam: Elsevier.
Harnish, V. (2014). Scaling Up: How a Few Companies Make It...and Why the Rest Don’t
(Rockefeller Habits 2.0) (Gazelles, ).
Hawthorne, E. K., Campbell, R. D., Tang, C., Tucker, C. S., & Nichols, J. (2014). Information
Technology Competency Model of Core Learning Outcomes and Assessment for Associate-
Degree Curriculum. Retrieved from
Hay, D. C. (1996). Data model patterns : conventions of thought. New York: Dorset House ;
Chichester : Wiley [distributor].
Hayes, R. H., & Wheelwright, S. C. (1979). Link manufacturing process and product lifecycles.
Harvard Business Review, 133140.
Heller, M. (2016). GE’s Jim Fowler on the CIO role in the digital industrial economy. Retrieved
April 18, 2016, from
Hernandez, J. G. (2012). Jeff Bezos’ Mandate: Amazon and Web Services. Psychology,
Management and Leadership. Retrieved from
Humble, J., & Farley, D. (2011). Continuous delivery. Boston: Addison-Wesley.
Iivari, J., Hirschheim, R., & Klein, H. K. (2004). Towards a distinctive body of knowledge for
information systems experts: Coding ISD process knowledge in two IS journals.
Information Systems Journal.
ISACA. (2012). COBIT 5.
ISO/IEC. (2005). ISO/IEC 20000 - Information Technology - Service Management.
iSSEc Project. (2009). Curriculum Guidelines for Graduate Degree Programs in Software
Engineering. Retrieved from
Izrailevsky, Y., & Tseitlin, A. (2011). The Netflix Simian Army. Retrieved May 4, 2016, from
Jeff Gothelf, & Josh Seiden. (2013). Lean UX: Applying Lean Principles to Improve User
Experience. O’Reilly Media.
Kahn, K. B., & Product Development & Management Association. (2005). The PDMA handbook
of new product development (2nd ed.). Hoboken, N.J.: Wiley. Retrieved from
KARE 11 Staff. (2015). Target cuts 275 positions, most in technology. Retrieved April 17, 2016,
Kennaley, M. (2010). Sdlc 3.0: Beyond a Tacit Understanding of Agile: Towards the Next
Generation of Software Engineering. Fourth Medium Consulting.
Kim, G., Behr, K., & Spafford, G. (2013). The Phoenix Project: A Novel About IT, DevOps, and
Helping Your Business Win. IT Revolution Press.
Kniberg, H., & Ivarsson, A. (2012). Scaling Agile @ Spotify with Tribes, Squads, Chapters &
Guilds. Retrieved from
Koskela Gregory, L. H. (2002). The underlying theory of project management is obsolete.
Retrieved April 17, 2016, from
Kroenke, D. M., & Boyle, R. (2016). Experiencing MIS. Pearson Education, Inc.
Larman, C., & Basili, V. R. (2003). Iterative and incremental development: A brief history.
Larman, C., & Vodde, B. (2009). Scaling lean & agile development : thinking and organizational
tools for large-scale Scrum. Upper Saddle River, NJ: Addison-Wesley.
Laudon, K. C., & Laudon, J. P. (2015). Management Information Systems : Managing the Digital
Firm - 9th edition. Pearson Education.
Lee Young Soon; Kim, Chan Hoon, J. H. H. (2007). IT Service Management Case based
Simulation Analysis & Design: Systems Dynamics Approach. In 2007 International
Conference on Convergence Information Technology (pp. 15591566). IEEE Computer
Lee, T. (2016). Here’s Obama's plan to prevent future IT disasters like the rollout.
Retrieved April 20, 2016, from
Leffingwell, D. (2011). Agile software requirements : lean requirements practices for teams,
programs, and the enterprise. The Agile software development series. Upper Saddle River,
NJ: Addison-Wesley.
Leffingwell, D., Yakyma, A., Jemilo, D., Knaster, R., Shalloway, A., & Oren, I. (2014). Scaled
Agile Framework. Retrieved from
Lenfle, S., & Loch, C. (2010). Lost Roots: How Project Management Came to Emphasize
Control Over Flexibility & Novelty. California Management Review, 53(1), 3256.
Lethbridge, T. C., & Port, D. (2004). A Brief Guide to Researching and Writing for CSEE&T.
Limoncelli, T. A., Chalup, S. R., & Hogan, C. J. (2014). The Practice of Cloud System
Administration: Designing and Operating Large Distributed Systems, Vol. 2. Pearson
Lockwood, T. (2010). Design Thinking: Integrating Innovation, Customer Experience, and
Brand Value. Allworth Press;
Loeliger, J. (2009). Version control with Git (1st ed.). Beijing ; Sebastopol, CA: O’Reilly.
Lunt, B., Ekstrom, J. J., Gorka, S., Hislop, G., Kamali, R., Lawson, E., … Reichgelt, H. (2008).
Information Technology 2008 Curriculum Guidelines for Undergraduate Degree Programs
in Information Technology Association for Computing Machinery (ACM) IEEE Computer
Society. Retrieved from Curriculum.pdf
Magennis, T. (2011). Forecasting and Simulating Software Development Projects: Effective
Modeling of Kanban & Scrum Projects using Monte-carlo Simulation . CreateSpace
Independent Publishing Platform (October 25, 2011). Retrieved from
Marakas, G. M., & O’Brien, J. A. (2011). Management Information Systems (10e ed.). New
York, N.Y: McGraw-Hill/Irwin.
Martin, J. (1989a). Information Engineering Book I: Introduction. Englewood Cliffs, N.J.:
Prentice Hall.
Martin, J. (1989b). Information Engineering Book II: Planning & Analysis. Englewood Cliffs,
N.J.: Prentice Hall.
Memon, T. F. (2014). Project vs. Product. Thoughtworks. Retrieved from
Mintzberg, H. (1983). Structure in fives : designing effective organizations. Englewood Cliffs,
N.J.: Prentice-Hall.
Narayam, S. (2015). Agile IT organization design: for digital transformation and continuous
delivery. Pearson Education Inc. .
Newman, S. (2015). Building Microservices. O’Relly Media.
Nist. (2011). Publication Moved: NIST SP 800-145, The NIST Definition of Cloud Computing.
Retrieved from
Office of Government Commerce. (2000). Service Support: Service Desk and the Process of
Incident Management, Problem Management, Configuration Management, Change
Management and Release Management. (OGC, Ed.)Information Technology Infrastructure
Library, Version 2. London: The Stationery Office.
Ogunnaike, B. A., & Ray, W. H. (1994). Process Dynamics, Modeling, and Control. Oxford:
Oxford University Press.
Ohno, T. (1988). Toyota production system : beyond large-scale production. Cambridge, Mass.:
Productivity Press.
Opelt, A., Gloger, B., Pfarl, W., & Mittermayr, R. (2013). Agile contracts : creating and
managing successful projects with Scrum. Wiley series in systems engineering and
Open Group, T. (2015a). IT4IT Standard. Open Group, The. Retrieved from
Open Group, T. (2015b). The Open Group Architectural Framework (TOGAF), Version 9. Open
Group, The. Retrieved from
Osterwalder, A., & Pigneur, Y. (2010). Business Model Generation - Canvas. Wiley, 280.
Retrieved from
Page-Jones, M., & Constantine, L. L. (2000). Fundamentals of object-oriented design in UML.
New York, N.Y.Reading, Mass.: Dorset House Pub. ;Addison-Wesley.
Patton, J. (2014). User story mapping : discover the whole story, build the right product (First
Pollard, C., & Cater-Steel, A. (2009). Justifications, Strategies, and Critical Success Factors in
Successful ITIL Implementations in U.S. and Australian Companies: An Exploratory Study.
Information Systems Management, 26(2), 164175.
Poppendieck, M., & Poppendieck, T. D. (2003). Lean Software Development: An Agile Toolkit.
(A. Cockburn & J. Highsmith, Eds.)The Agile Software Development Series. Boston:
Addison Wesley.
Poppendieck, M., & Poppendieck, T. D. (2007). Implementing lean software development : from
concept to cash. The Addison-Wesley signature series. Upper Saddle River, NJ: Addison-
Wesley. Retrieved from
Project Management Institute. (2013). A guide to the project management body of knowledge
(PMBOK guide) (Fifth edit).
Puppet Labs. (2015). 2015 State of DevOps Report.
Quinlan, T. A. (2003). Chargeback and IT Cost Accounting. (T. A. Quinlan, Ed.). Santa Barbara,
CA: IT Financial Management Association.
Quinlan, T. A., & Quinlan, S. J. (2003). Readings in IT Financial Management. (T. A. Quinlan,
Ed.). Santa Barbara, CA: IT Financial Management Association.
Rainer, K., Prince, B., & Watson, H. (2015). Management Information Systems. John Wiley &
Sons, Inc.
Reinertsen, D. G. (1997). Managing the design factory: a product developer’s toolkit. New
York ; London: Free Press.
Reinertsen, D. G. (2009). The principles of product development flow: second generation lean
product development. Redondo Beach, Calif.: Celeritas.
Remenyi, D., Money, A. H., & Bannister, F. (2007). The effective measurement and management
of ICT costs and benefits (3rd ed.). Oxford ; Burlington, MA: CIMA. Retrieved from
Ries, E. (2011). The lean startup : how today’s entrepreneurs use continuous innovation to
create radically successful businesses (1st ed.). New York: Crown Business.
Rigby, D. K., Sutherland, J., & Takeuchi, H. (2016). Embracing Agile. Harvard Business
Review, (May). Retrieved from
Rother, M. (2010). Toyota kata : managing people for improvement, adaptiveness, and superior
results. New York: McGraw Hill.
Royce, W. (1970). Managing the Development of Large Software Systems. In Proc. IEEE
WESCON (pp. 19). Los Angeles: IEEE.
Rozovsky, J. (2015). The five keys to a successful Google team. re:Work. Retrieved from
Rummler, G. A., & Brache, A. P. (1995). Improving performance: how to manage the white
space on the organization chart. The Jossey-Bass management series (2nd ed.). San
Francisco, CA: Jossey-Bass. Retrieved from
Schwaber, K. (2002). Agile Software Development with Scrum. Upper Saddle River, N.J.:
Prentice Hall.
Schwaber, K. (2013). unSAFE at any speed. Ken Schwaber’s Blog: Telling It Like It Is.
Retrieved from
Schwaber, K., & Sutherland, J. V. (2012). Software in 30 days : how agile managers beat the
odds, delight their customers, and leave competitors in the dust. Hoboken, N.J.: John Wiley
& Sons, Inc. Retrieved from
Schwartz, M. (2014). How DevOps Can Fix Federal Government IT. DevOps Enterprise. San
Sean Landis. (2011). Agile Hiring. Artima, Inc.
Shackelford, R., Cross, J. H. I., Davies, G., Impagliazzo, J., Kamali, R., LeBlanc, R., … Topi, H.
(2005). Computing Curricula 2005 The Overview Report. Retrieved from
Sharma, A. (2015). Why Big Companies Keep Failing: The Stack Fallacy. Retrieved April 6,
2016, from
Sharp, A., & McDermott, P. (2009). Workflow modeling : tools for process improvement and
applications development (2nd ed.). Boston: Artech House.
Shortland, A., & Lei, M. (2012). Using Rundeck and Chef to build DevOps Toolchains.
Retrieved from
Smith, P. G., & Reinertsen, D. G. (1991). Developing products in half the time. New York, N.Y.:
Van Nostrand Reinhold.
Smith, P. G., & Reinertsen, D. G. (1998). Developing products in half the time : new rules, new
tools ([New ed.]). New York ; London: Van Nostrand Reinhold.
Snowden, D. J., & Boone, M. E. (2007). A Leader’s Framework for Decision Making. Harvard
Business Review.
Sobel, A., Lethbridge, T. C., Díaz-Herrera, J. L., & Barrie Thompson, J. (2015). Software
Engineering 2014 Curriculum Guidelines for Undergraduate Degree Programs in Software
Engineering. Retrieved from
Somasundaram, K., & Murphy, G. C. (2012). Automatic Categorization of Bug Reports Using
Latent Dirichlet Allocation. In Proceedings of ISEC ’12. Kanpur, UP, India.
Sousa, K. J., & Oz, E. (2015). Management Information Systems (7th Ed.). Australia: Cengage
Spewak, S. H., & Hill, S. C. (1993). Enterprise architecture planning : developing a blueprint
for data, applications, and technology. Boston: QED Pub. Group.
Spinellis, D. (2015). Extending Our Field’s Reach. IEEE Software, 46.
Sturm, R., Morris, W., & Jander, M. (2000). Foundations of service level management.
Indianapolis, Ind.: SAMS.
Sussna, J. (2015). Designing Delivery: Rethinking IT in the Digital Service Economy. O’Relly
Sutherland, J. V. (2014). Scrum : the art of doing twice the work in half the time (First Edit).
Crown Business . Retrieved from
Sutton, R. I. ., & Rao, H. (2014). Scaling up excellence : getting to more without settling for less.
Crown Business/Random House.
Takeuchi, H., & Nonaka, I. (1986). The new new product development game. Journal of Product
Innovation Management, 3(3), 205206.
The Stationery Office. (2011a). ITIL Continual Service Improvement: 2011 Edition. Information
Technology Infrastructure Library. Norwich, U.K.: The Stationery Office.
The Stationery Office. (2011b). ITIL Service Design: 2011 Edition. Information Technology
Infrastructure Library. Norwich, U.K.: The Stationery Office.
The Stationery Office. (2011c). ITIL Service Operation: 2011 Edition. Information Technology
Infrastructure Library. Norwich, U.K.: The Stationery Office.
The Stationery Office. (2011d). ITIL Service Strategy: 2011 Edition. Information Technology
Infrastructure Library. Norwich, U.K: The Stationery Office.
The Stationery Office. (2011e). ITIL Service Transition: 2011 Edition. Information Technology
Infrastructure Library. Norwich, U.K.: The Stationery Office.
Topi, H., Valacich, J. S., Wright, R. T., Kaiser, K. M., Nunamaker, J. . J., Sipior, J. C., & de
Vreede, G. J. (2010). IS 2010 Curriculum Guidelines for Undergraduate Degree Programs
in Information Systems. Retrieved from 2010
ACM final.pdf
Van Haren Publishing. (2016). Global Standards and Publications - Edition 2016/2017.
Amsterdam: Van Haren Publishing.
West, D., & Wildeman, R. C. (2009). Product-Centric Development Is A Hot New Trend.
Forrester Research. Retrieved from
Westerman, G., Calméjane, C., Bonnet, D., Ferraris, P., & McAfee, A. (2011). Digital
Transformation: A Road-Map for Billion-Dollar Organizations. MIT Center for Digital
Business and Capgemini Consulting, 168.
Westerman, G., Tannou, M., Bonnet, D., Ferraris, P., & McAfee, A. (2012). The Digital
Advantage: How Digital Leaders Outperform their Peers in Every Industry. MIT Sloan
Management Review, 124. Retrieved from
Wikipedia. (2016). DevOps. Retrieved May 18, 2016, from
Williams, D. P., & Holub, E. (2016). How to Build a DevOps Release Team. Retrieved from
Yourdon, E. (1975). Techniques of program structure and design. Englewood Cliffs, N.J.:
Zachman, J. (1987). Zachman Framework. IBM Systems Journal, 26(3), 276292.
Document history
Pre-0.50 versions initial draft /ideation
0.50 Incorporated initial analysis of primary ACM curricula guidance as visible on this page: Further work needed to integrate &
harmonize later section of paper.
0.51 small corrections. Considering whether to split paper in 2, one section on curricula, the
other on pedagogy.
0.52 Rewrote section on gaps with a more positive framing
... Best practices such as ITIL are in some cases seen as something that is archaic and inflexible being used to manage modern and advanced technology solutions. According to Betz (2016), other standards or frameworks such as Agile, Lean IT and DevOps are being applied more often as the IT industry is becoming more reluctant to apply old-fashioned best practices such as ITIL. It is clear that to manage IT effectively and efficiently, the ITIL approach has to be expanded to deliver hands-on quality IT services (Addy 2007). ...
Full-text available
IT leaders in higher education institutions (HEI) face a challenge to incorporate the continuous transformation of technology and the way it is applied in HEIs to improve the quality of IT service delivery. Managing IT per se, became more than managing IT systems with a fixed set of knowledge and skills. IT leaders needs to manage IT as a value stream not as separate entities. Various best practices, methodologies, standards and frameworks exist which all address some aspect of the IT value stream. A multi-dimensional framework was designed to address the entire IT value stream and to improve the quality of service delivery and thus satisfy stakeholders’ expectations. The framework incorporates various best practices, methodologies and standards. The framework was validated using in-depth interviews. Thirty interviewees from three entities within the HEI, participated in the research. The purpose of the interviews was to determine which elements of the framework contribute to quality IT services. Respondents completed a service quality matrix as part of the interview. The results were analysed to determine the respondents’ understanding and interpretation of the delivery of quality services. The results highlighted a discrepancy between the IT department’s perception of quality service and the recipients’ perception of said services. The results also highlighted that the framework can be used to align the various service quality perceptions.
Technical Report
Full-text available
Information technology supports efficient operations, enterprise integration, and seamless value delivery, yet itself is too often inefficient, un-integrated, and of unclear value. This completely rewritten version of the best selling Architecture and Patterns for IT Service Management, Resource Planning and Governance retains the original (and still unique) approach: apply the discipline of enterprise architecture to the business of large scale IT management itself. Author Charles Betz applies his deep practitioner experience to a critical reading of ITIL version 3, COBIT version 4, the CMMI suite, the IT portfolio management literature, and the Agile/Lean IT convergence, and derives a value stream analysis, IT semantic model, and enabling systems architecture (covering current topics such as CMDB/CMS, Service Catalog, and IT Portfolio Management). The edition pioneers new ground in applying dynamic modeling (systems dynamics) to large scale IT process interactions, and the fundamental discipline of traceable process, data, and system analysis which made the 1st edition so popular remain. This best seller is a must read for anyone charged with enterprise architecture, IT planning, and IT governance and management. Presents a field-tested, detailed conceptual information model with definitions and usage scenarios, mapped to both the process and system architectures Updated to reflect the new set of concepts, practices, and procedures of ITIL v3.1 Synthesizes Enterprise Architecture, IT Service Management, and IT Governance in a practical way. Covers new applications of IT Service Management, moving from one overall IT value stream to several, described in Lean terms.