Conference PaperPDF Available

Impact of Project Management Tools on Project Estimates and Benefits



We have seen a variety of project management (PM) tools available for project managers in recent years. Yet, projects continue to face significant challenges. The use of PM tools is considered a means to counter these challenges. However, the effectiveness of PM tools has not been demonstrated empirically. The results of this study show that the use of PM tools positively impacts overall project benefits. At the same time, only four PM tools were critical for project success. PM tool use was also found not to improve the accuracy of initial project estimates. Instead, project risk assessment improves the accuracy of project estimates, which in turn relates to overall project benefits. The implications of these results are discussed.
International Conference on Internet Studies,
July 22-24, 2016, Osaka, Japan
Makoto Nakayama,
DePaul University, U.S.A.
Charlie C. Chen,
Appalachian State University, U.S.A.
We have seen a variety of project management (PM) tools available for project
managers in recent years. Yet, projects continue to face significant challenges. The
use of PM tools is considered a means to counter these challenges. However, the
effectiveness of PM tools has not been demonstrated empirically. The results of this
study show that the use of PM tools positively impacts overall project benefits. At the
same time, only four PM tools were critical for project success. PM tool use was also
found not to improve the accuracy of initial project estimates. Instead, project risk
assessment improves the accuracy of project estimates, which in turn relates to overall
project benefits. The implications of these results are discussed.
Keywords: project estimate, project management tool, project risk assessment,
project success
Despite the increasing availability and variety of project management (PM) tools in
recent years, projects continue to face challenges. According to the 2016 Pulse of the
Profession [1], 47% of projects exceed the budget, 51% do not complete on time, and
55% experience scope creep. PM tools are used to design and control project plans,
deliverables, resources, budgets, staff assignments, roles and responsibilities,
communications, and quality management [2]. They have been commonly adopted
not only to minimize project risks but also to counter unique challenges of each
project. The 2012 global survey by PricewaterhouseCoopers [3] notes 77% of
respondents use PM methodologies and software. According to [4], software tools that
comply with the Capability Maturity Model Integration (CMMI) or the Project
Management Body of Knowledge (PMBOK) can effectively manage project
management processes and achieve project success. However, the same study
cautions that a large number of PM tools are still not aligned with state-of-the-art PM
practice. Although [5] has emphasized the importance of using right tools and
methods, it is not clear how much their use can contribute to containing project risks
and helping project success. Therefore, the main research goal of this study is to
assess empirically the extent to which PM tool use helps achieve more accurate
project estimates and better overall project outcomes.
2.1 The Impact of PM Tool Use
Raz and Michael [6] find a positive relationship between project tool use and project
management performance (PMP) score. While there are a variety of PM tools, they
should be selected and applied in the right stage of a project cycle. In the initial stage,
project managers can utilize user stories [7]. As project ideas become more mature,
semi-structured qualitative interviews help project managers gain fine details of
requirements [8]. It is imperative for project managers to know what tools to use
under what project circumstances to estimate projects accurately as well as to achieve
project success. Indeed, Fortune and White [9] note the correct choice or past
experience of PM tools as one of the critical success factors (CSFs) for projects.
Zwikael, Shimizu, and Globerson [10] compare the extent of PM tool use in Israel and
Japan. Project managers in Israel use PM tools more than their Japanese counterparts.
They observe that PM tools enable project managers in Israel to focus more on
executing planning processes and to enjoy the higher intensity of organizational
support processes than their Japanese counterparts. Better planning processes are
likely to yield better project estimations. Therefore, we propose:
H1: The extent of project management tool use positively impacts the degree of
overall project benefits.
H2: The extent of project management tool use minimizes the deviation from project
2.2 Project Risk Assessment, Project Estimation, and Project
The success of a project was once viewed primarily as a matter of scheduling, but it
actually depends on many factors [11]. Among the critical success factors are project
risk addressing, assessment, and management [9]. Project risk is defined as “an
uncertain event or condition that, if it occurs, has a positive or negative impact on one
or more project objectives such as scope, schedule, cost, and quality” [12]. Thus, the
estimate of a project inherently depends on how its project managers assess the degree
of project risks. A decent estimate of a project is a prerequisite for successful
outcomes with projects such as information system development [13]. For instance,
inadequate project estimates result in the amoebic growth of project costs [14], which
in turn minimizes project benefits. Thus, we hypothesize:
H3: The degree of project risks positively impacts the deviation from project
H4: The deviation from project estimation negatively impacts the degree of overall
project benefits.
A survey questionnaire was developed and distributed to 200 randomly chosen
registered Project Management Professionals (PMPs) from the list of Project
Management Institute (PMI) members in China. A final sample of 93 valid responses
was obtained (the response rate of 46.5%) and used to test the hypotheses. The
majority of respondents were department managers (31.2%), followed by project
managers (20.4%), line managers (17.2%), general managers (10.8%), and deputy
general managers (7.5%). As for company characteristics, 38.7% of PMPs in our
response are currently working for state-owned companies, versus 61.3% of PMPs for
private companies. Respondents were also asked what types of projects they are
currently responsible for. A large proportion (60.2%) of respondents work on business
projects, whereas 21.5% work on IT projects and 18.3% on both IT and business
The research model was tested using structural equation modeling with partial least
squares (PLS) because PLS enables a small sample size and latent (reflective)
constructs to be modeled as formative constructs [15]. The constructs of the variables
adapted were Project Tool Use (formative, 4 items) [2], Project Risk Assessment
(formative, 2 items) [16], Deviation from Project Estimates based on the four basic
variables of project estimations (reflective, 4 items) [16], and Overall Project Success
(reflective, 7 items) [2]. The results of path significance tests using a PLS model is
given in Figure 1.
Project Management
Tool Use
Project Risk
Deviation from Project
R2 = 0.055
Overall Project Success
R2 = 0.163
*: significant at α = 0.10, **: α = 0. 50, ***: α = 0.01
Figure 1. Results of PLS model
The summary of hypothesis tests is shown in Table 1. H1 is strongly supported, given
the beta estimate of the path from Project Management Tool Use to Overall Project
Success is -0.382 (p < 0.01). The path between Project Management Tool Use and
Deviation from Project Estimate is not significant. This does not support H2. H3 is
supported, given the beta estimate of the path between Project Risk Assessment and
Deviation from Project Estimate is -0.234 (p < 0.05). Finally, Deviation from Project
Estimate marginally impacted Overall Project Success with the beta estimate of
-0.136 at p < 0.10. Then, H4 is weakly supported.
Table 1. Summary of hypothesis testing
H1: project management tool use overall project benefits
H2: project management tool use less estimation deviations
not supported
H3: project risk assessment less estimation deviations
H4: less estimation deviations overall project benefits
Overall results of this study draw several important lessons for both researchers and
project managers. First, PM tool use is effective to assure project success. Among the
26 PM tools entered into the model, about 85% were not significant for the model and
removed from the final hypothesis testing. The PM tool significant for the model are
those that manage (1) overall project planning, (2) resource costs, (3) risk
management plan, and (4) disciplined change control process. That is, not all PM
tools are critical for project success. Project managers should reference this result and
consider which PM tools to adopt given the relative abundance of PM tools. Second,
PM tool use does not impact the accuracy of project estimates. Rather, better project
risk assessment minimizes the deviation from project estimates. Such results should
not be taken by surprise because most PM tools are not meant to be forecasting tools
but management tools to anticipate contingencies and risks and cope with dynamically
changing projects and their operational environment. Finally, the accuracy of project
estimates does play an important role in achieving better project outcomes. The more
accurate project estimates are, the higher overall project benefits.
There are several limitations of this study. The study participants were project
managers in China. PM operational environments in other geographic areas may give
slightly different results. Also, the sample size of 93 is relatively small. However, the
study did draw project managers with diverse profiles at a high response rate. Future
studies should test the external validity of the results based on more project managers
with possibly different profile distributions in different geopolitical areas.
[1] T. Project Management Institute, 2016 Pulse of the Profession, in, 2016.
[2] O. Zwikael, S. Globerson, From critical success factors to critical success
processes, International Journal of Production Research, 44 (2006) 3433-3449.
[3] PricewaterhouseCoopers, Insights and Trends: Current Programme and Project
Management Practices, in, 2012.
[4] A.M. Pereira, R.Q. Gonçalves, C.G. Von Wangenheim, L. Buglione, Comparison
of open source tools for project management, International Journal of Software
Engineering and Knowledge Engineering, 23 (2013) 189-209.
[5] H. Thamhain, Managing risks in complex projects, Project Management Journal,
44 (2013) 20-35.
[6] T. Raz, E. Michael, Use and benefits of tools for project risk management,
International Journal of Project Management, 19 (2001) 9-17.
[7] M. Cohn, User stories applied: For agile software development, Addison-Wesley
Professional, Boston, MA, 2004.
[8] P. Mayring, Qualitative Sozialforschung, Beltz Verlag, 2002.
[9] J. Fortune, D. White, Framing of project critical success factors by a systems
model, International Journal of Project Management, 24 (2006) 53-65.
[10] O. Zwikael, K. Shimizu, S. Globerson, Cultural differences in project
management capabilities: a field study, International Journal of Project Management,
23 (2005) 454-462.
[11] W. Belassi, O.I. Tukel, A new framework for determining critical success/failure
factors in projects, International Journal of Project Management, 14 (1996) 141-151.
[12] T. Project Management Institute, A Guide to the Project Management Body of
Knowledge (PMBOK® Guide), Project Management Institute, Incorporated,
Newtown Square, PA, 2013.
[13] K. Burnett, Project Initiation, in: The Project Management Paradigm,
Springer-Verlag London Limited, 1998, pp. 131-172.
[14] C. Eden, F. Ackermann, T. Williams, The amoebic growth of project costs,
Project Management Journal, 36 (2005) 15-27.
[15] W.W. Chin, P.R. Newsted, Structural equation modeling analysis with small
samples using partial least squares, in: R. Hoyle (Ed.) Statistical strategies for small
sample research, Sage Publications, Thousand Oaks, CA, 1999, pp. 307-342.
[16] O. Zwikael, A. Sadeh, Planning effort as an effective risk management tool,
Journal of Operations Management, 25 (2007) 755-767.
Full-text available
Task management is the process of managing tasks through its life cycle. This involves planning, testing, tracking and reporting. Task management can help achieve individual goals, individual groups collaborate and share knowledge to achieve common goals. [1] Tasks are also distinguished by complexity, from low to high. Task management can form part of project management and process management for efficient workflows within an organization. Project managers who adhere to task-oriented management have a detailed and up-to-date project schedule, and a good team directs members and advances the project. [2] At present, the Task management system is growing along with the rapid advances in technology, so that anyone can access wherever he is. However, the majority of the current Task management system is not able to attract the attention and interest of its users. The concept of gamification will be applied to the ask management system so that work management is more organized, interesting and fun. This gamified system design will use the Marczewski’s Gamification Framework. Framework that has user types with the needs of learning and self-development. Game mechanics in this user types such as levels, challenges, and rewards can support gamified system users in achieving goals. The tests that have been conducted show the success in applying the Marczewski framework to the functional needs of users.
Full-text available
Software projects often fail, because they are not adequately managed. The establishment of effective and efficient project management practices still remains a key challenge to software organizations. Striving to address these needs, "best practice" models, such as, the Capability Maturity Model Integration (CMMI) or the Project Management Body of Knowledge (PMBOK), are being developed to assist organizations in improving project management. Although not required, software tools can help implement the project management process in practice. In order to provide comprehensive, low-cost tool support for project management, specifically, for small and medium enterprises (SMEs), in this paper we compare the most popular free/open-source web-based project management tools with respect to their compliance to PMBOK and CMMI for Development (CMMI-DEV). The results of this research can be used by organizations to make decisions on tool adoptions as well as a basis for evolving software tools in alignment with best practices models.
Full-text available
In the public arena, we often hear about projects that have suffered massive cost overruns. Often they are related to large public construction projects such as airports, bridges, or public buildings. Large overruns also exist in private industry. However, often these do not appear in the newspapers, so the public is not as aware of them. Of course, not all projects go badly wrong, but quite a few do, and frequently we find ourselves uncertain of the causes for such overruns. In this paper, industrial projects that overrun and overrun in a surprising manner are considered. In other words, the paper considers those many projects where the extent of the overrun is well beyond what might ever have been anticipated, even though what was going wrong within the projects was, for the most part, understood. The basis for the content of the paper (that is, the structure and lessons), are drawn from a postmortem analysis of many large projects as part of claims analysis, particularly “delay and disruption” claims for projects whose total expenditure appeared, at first look, inexplicable or surprising. The aim of the paper is to contribute to an understanding of how projects go badly wrong, when they do, and in particular to draw some lessons from this exploration that are likely to help all managers. The reasons for cost escalation are not just the responsibility of project managers.
Full-text available
After myriad studies into the main causes of project failure, almost every project manager can list the main factors that distinguish between project failure and project success. These factors are usually called Critical Success Factors (CSF). However, despite the fact that CSF are well-known, the rate of failed projects still remains very high. This may be due to the fact that current CSF are too general and do not contain specific enough know-how to better support project managers’ decision-making. This paper analyses the impact of 16 specific planning processes on project success and identifies Critical Success Processes (CSP) to which project success is most vulnerable. Results are based on a field study that involved 282 project managers. It was found that the most critical planning processes, which have the greatest impact on project success, are “definition of activities to be performed in the project”, “schedule development”, “organizational planning”, “staff acquisition”, “communications planning” and “developing a project plan”. It was also found that project managers usually do not divide their time effectively among the different processes, following their influence on project success.
Full-text available
In project management, high levels of risk are considered to be a significant obstacle for project success. This paper investigates whether improving the project plan can lead to improved success for high-risk projects. A quality of planning index was designed to explore how the presence of high risk affects the quality of planning and project success. The index includes managerial aspects such as costs, human resources, procurement and quality, as well as organizational support aspects based on organization maturity models. In a field study based on data collected from 202 project managers regarding their most recent projects, it was found that the levels of risk at the beginning of projects has no effect on their final success. Drilling down to find an explanation for this surprising phenomenon, we found that in the presence of high risk, project managers significantly improve their project plans. Hence, in high-risk projects, better project plans improve all four dimensions of project success: schedule overrun, cost overrun, technical performance and customer satisfaction. However, in low-risk projects, better project plans did not contribute to reducing schedule or cost overruns. In other words, while endless risk management tools are developed, we found that improving the project plan is a more effective managerial tool in dealing with high-risk projects. Finally, the paper presents the most common planning tools currently being used in high-risk projects.
In the preceding chapters we have covered the elements of a project management methodology; project management functions; project roles and responsibilities; and quality management. Thus we now have an awareness of what is necessary and involved in managing a project. This chapter takes us through the initiation of a project to get it started.
Dealing effectively with risks in complex projects is difficult and requires management interventions that go beyond simple analytical approaches. This is one finding of a major field study into risk management practices and business processes of 35 major product developments in 17 high-technology companies. Almost one-half of the contingencies that occur are not being detected before they impact project performance. Yet, the risk-impact model presented in this article shows that risk does not affect all projects equally but depends on the effectiveness of collective managerial actions dealing with specific contingencies. The results of this study discuss why some organizations are more successful in detecting risks early in the project life cycle, and in decoupling risk factors from work processes before they impact project performance. The field data suggest that effective project risk management involves an intricately linked set of variables, related to work process, organizational environment, and people. Some of the best success scenarios point to the critical importance of recognizing and dealing with risks early in their development. This requires broad involvement and collaboration across all segments of the project team and its environment, and sophisticated methods for assessing feasibilities and usability early and frequently during the project life cycle. Specific managerial actions, organizational conditions, and work processes are suggested for fostering a project environment most conducive to effective cross-functional communication and collaboration among all stakeholders, a condition important to early risk detection and effective risk management in complex project situations.
Agile requirements: discovering what your users really want. With this book, you will learn to: Flexible, quick and practical requirements that work Save time and develop better software that meets users' needs Gathering user stories -- even when you can't talk to users How user stories work, and how they differ from use cases, scenarios, and traditional requirements Leveraging user stories as part of planning, scheduling, estimating, and testing Ideal for Extreme Programming, Scrum, or any other agile methodology ----------------------------------------------------------------------------------------------------------Thoroughly reviewed and eagerly anticipated by the agile community, User Stories Applied offers a requirements process that saves time, eliminates rework, and leads directly to better software.The best way to build software that meets users' needs is to begin with "user stories": simple, clear, brief descriptions of functionality that will be valuable to real users. In User Stories Applied, Mike Cohn provides you with a front-to-back blueprint for writing these user stories and weaving them into your development lifecycle.You'll learn what makes a great user story, and what makes a bad one. You'll discover practical ways to gather user stories, even when you can't speak with your users. Then, once you've compiled your user stories, Cohn shows how to organize them, prioritize them, and use them for planning, management, and testing. User role modeling: understanding what users have in common, and where they differ Gathering stories: user interviewing, questionnaires, observation, and workshops Working with managers, trainers, salespeople and other "proxies" Writing user stories for acceptance testing Using stories to prioritize, set schedules, and estimate release costs Includes end-of-chapter practice questions and exercisesUser Stories Applied will be invaluable to every software developer, tester, analyst, and manager working with any agile method: XP, Scrum... or even your own home-grown approach.ADDISON-WESLEY PROFESSIONALBoston, MA 02116www.awprofessional.comISBN: 0-321-20568-5
Only a few studies in the project management literature concentrate on the critical factors that affect project success or failure. Whereas many of these studies generate lists of critical success factors, each list varies in its scope and purpose. The success factors are usually listed as either very general factors or very specific factors affecting only a particular project. However, lacking a comprehensive list makes it difficult not only for project managers but also for researchers to evaluate projects based on these factors. In this study, we suggest a new scheme that classifies the critical factors, and describes the impacts of these factors on project performance. Emphasis is given to the grouping of success factors and explaining the interaction between them, rather than the identification of individual factors. An empirical study is conducted to test the practicality of using such a scheme. The statistical analyses of the results demonstrate the differences between the critical success factors identified in a previous study from literature and the factors identified with the use of our scheme. Many critical factors, such as factors related to project managers' performance, factors related to team members and environmental factors, became apparent with this study. The results are encouraging, in that practitioners support the use of this scheme for determining and analysing critical success factors and how systems respond to these factors.
Perhaps the best known approach for tackling the human and organisational aspects of projects is through the use of ‘critical success factors’ but although the approach has very many champions it is not without its critics. This paper sets out the findings of a major review of the sets of factors that are available and outlines the main reservations that have been expressed about the approach. It then shows how a systems model, the Formal Systems Model, can be used as a framing device to deliver the benefits of taking account of ‘critical success factors’ whilst at the same time avoiding the problems associated with ‘critical success factors’ that give rise to the criticisms.Two IS projects are used to demonstrate use of this framing devise. When observation began at the start of the projects they looked very similar and equally likely to succeed. In the event, one of the projects was largely successful across the whole of the range of measures normally used to judge success whilst the other exhibited most of the characteristics of failure. Analysis using the framing device is well able to demonstrate the marked differences in the ways the two projects were managed and to account for stark contrast in the levels of success achieved. The paper concludes that the Formal System Model allows the underlying benefits of ‘critical success factors’ to be secured whilst overcoming most of the problems associated with a checklist approach.