PresentationPDF Available

Reversible Debugging Software "Quantify the time and cost saved using reversible debuggers"

Authors:
Reversible Debugging Software
“Quantify the time and cost saved using
reversible debuggers”
Tom Britton
Lisa Jeng
Graham Carver
Paul Cheak
Tomer Katzenellenbogen
Abstract and Introduction
Abstract
Reversible debuggers (RDBs) allow software developers to run code backwards as well as forwards, making it
easier for them to find the root cause of bugs. However, most software companies continue to use forward
only debugging techniques to review their codes. This research will use interviews and a survey to quantify
the time saved by software developers writing compiled code on Linux when RDBs are used to find and fix
software bugs. The results, expressed in percentage of time spent, found that developers spend half their
programming time debugging, which equates to an estimated $312billion per year. RDBs were found to
reduce this debugging time by 26%, equalling a saving of $81.1bn per year to the global software industry.
Introduction
Reversible debugging has been an application researched for years. Reversible debuggers (RDBs) allow
software developers to run code backwards as well as forwards. This makes it easier for them to find the root
cause of bugs.
Although a limited number of reversible debugging software products are available in the market, few are
commercially viable, particularly for complex codes. Such codes typically have a higher probability of
difficult intrinsic bugs that require significant amounts of time to find. GDB in particular struggles with poor
performance caused by low speed and high memory consumption.
Hence, most software companies continue to use forward debugging techniques to review their codes
because of the scarcity of suitable reversible debugging tools, and a general lack of awareness within the
market regarding the existence of reversible debugging options. Quantifying the time saved by RDBs will
support marketing efforts of these products to the software development industry.
Research Questions, Methodology,
Constraints and Exclusions
Research Questions:
What is the time saved by software developers writing compiled code in the Linux environment when RDBs are used to fix
software bugs?
In particular, the project will:
1. Quantify the time typically taken by software developers to rectify errors in their code without using RDBs; and
2. Quantify the time typically taken by software developers to rectify errors in their code using RDBs.
Methodology:
1. The research phase will predominantly comprise of short interviews with approximately 10 to 12 organisations that
compile code on the LINUX operating environment. These organisations will also fill in the Cambridge Venture
Project (CVP) survey to help quantify how much time they spend debugging with and without RDBs.
2. Broader research in the form of the CVP survey with potential users of reversible debuggers will be conducted to gain
insights on how much time is currently spent debugging.
3. The results will be quantified in terms of percentage of time spent.
Constraints:
It is anticipated that the project will be subjected to the following constraints:
1. Time available will limit the project scope.
2. Product limitations will require the researchers to target people who compile code on Linux.
Due to the above constraints, this project will exclude:
1. Interviewing software developers who do not programme compiled code on Linux;
2. Quantifying the opportunity costs and other related costs of bugs.
RESULTS: The global cost of software development is
US$1.25 trillion
$1,250
$625
$313
$312
Total
Overhead
Programming
Wages
(admin)
Programming
Wages
(development)
Software development cost structure (US$ billion)
$94
$62
Programming Wages
25% - Fixing Bugs
25% - Making Code Work
20% - Designing Code
30% - Writing Code
Debugging
Productive
Source: Evans Data Corporation (2012), Payscale (2012),
RTI (2002), CVP Surveys (2012)
The global cost of software development is
US$1.25 trillion
50.1%
Programmers total work time spent doing admin*
49.9%
Programming time spent debugging**
*According to 54 questionnaire responses to the CVP survey and 11 interviews, based on the question ‘what
percentage of your time is spent programming (e.g. designing, writing, testing, debugging etc.) compared with other
tasks (e.g. admin, meetings, sales, etc)?’ Other tasks were then grouped as ‘admin’.
**According to 54 questionnaire responses to the CVP survey and 11 interviews, based on the question ‘Of
the time spent programming, what percentage of your time is spent on the following: (1)fixing bugs (2)making code
work (3)designing code (4) writing code.’ (1) and (2) then were grouped as debugging.
The global cost of software development is
US$1.25 trillion
US$156 Billion per year
Wages-only estimated cost of debugging*
US$312 Billion per year
TOTAL estimated cost of debugging including wages
and overheads**
*Developers spend 50% of their time debugging, which equals half programming wages (development): 312/2=$156bn
**Total cost of debugging to software development equals ¼ total software development costs = 1250*0.25=$312bn
The global cost of software development is
US$1.25 trillion
25%
25%
18%
19%
Fixing Bugs
Reworking Code
% of programming time spent debugging
Before RDBs
After RDBs
Total saved with RDBs = 13 percentage point drop in total programming time
Total saved with RDBs = 26% decrease in debugging time
Source: CVP Surveys, 2012
Reversible debuggers have the potential to
significantly improve programming productivity
$81.1 Billion
TOTAL software development costs saved
per year*
Potential value of efficiencies gained by using RDBs
*Programming time (development) costs $312bn per year. RDBs can save 26% of this cost: 312*0.26=$81.1bn
Source: Bloomberg Industries, 2012
USA
34%
UK
38% Germany
30%
China
63%
Australia
45%
China and Australia are leading growth in the
software industry
2007-2011 Industry Growth Rates by Country
Summary
Programmers spend 50.1% of their work time not programming. Half of their time spent
programming is spent debugging.
Total estimated cost of debugging is $312bn per year.
RDBs reduce time spent debugging by 26%.
26% reduction in time spent debugging is estimated to equal $81.1bn in total software
development costs saved per year.
The software industry has been growing across the world, which is likely to put upward
pressure on the cost of debugging.
Further Research
This research used only wages and overhead costs to calculate the cost of bugs to the software
industry. These figures were also used to estimate the dollar amount saved by using RDBs. No
attempt was made to quantify the auxiliary costs of bugs, such as remedying issues that occur in the
field, opportunity costs of software products that are delayed and/or cancelled due to bugs,
consequential damage to software companies' reputations due to bugs in released software and delays
and cancellations of software projects. Extending this research to investigate the auxiliary costs of
bugs once software has been released would more accurately represent the real cost of bugs to the
software industry.
If RDBs reduce debugging time by 26%, a further extension to this research could involve more
accurately quantifying the dollar savings that can be created by using RDBs when auxiliary costs are
taken into account. It would also be interesting to discover whether/which auxiliary costs are
substantially reduced by RDBs.
1. Average software developer wages
2. Total software development costs
3. Global software revenues
4. References
Appendix Section Listing
Salary
US
$76,000
India
$6,353
Finland
$45,979
Australia
$62,700
China
$18,640
Mexico
$17,277
UK
$46,736
Average Salary
$39,098
Source: Payscale (2012)
A1. Average software developer wages
2012
2013
2014
2015
a
Number of software developers
16,000,000
17,300,000
18,600,000
20,000,000
b
Average software developer salary
$39,098
$39,098
$39,098
$39,098
c
Salary & Overhead*
(b x 2)
$78,196
$78,196
$78,196
$78,196
d
Total cost of software development
(a x c)
$1,251,134,537,143
$1,352,789,218,286
$1,454,443,899,429
$1,563,918,171,429
e
Total software developer salary
(a x b)
$625,567,268,571
$676,394,609,143
$727,221,949,714
$781,959,085,714
f
cost of programming
(e x 50%)
$312,783,634,286
$338,197,304,571
$363,610,974,857
$390,979,542,857
g
Cost of debugging
(f x 50%)
$156,391,817,143
$169,098,652,286
$181,805,487,429
$195,489,771,429
h
Potential value of Undo
(f x 13%)
$40,661,872,457
$50,729,595,686
$54,541,646,229
$58,646,931,429
Source: (1) Evans Data Corporation (2012), (2)Payscale (2012), (3) RTI (2002), (4) Own research (2012)
* Ratio of salary to overheads is 1:1 based on RTI research
A2. Total software development costs
Description
2011
2010
2009
2008
2007
2006
Total Global Software Revenues ($ in mn):
By Country (Top 10)
United States
152,982
141,515
132,169
133,718
127,801
114,252
Japan
28,027
28,739
27,702
30,945
29,981
27,994
Germany
22,817
21,757
19,936
20,105
19,053
17,511
United Kingdom
19,094
18,450
17,222
17,043
15,303
13,827
France
14,244
13,541
12,548
12,656
12,141
11,193
Canada
10,020
9,453
8,916
8,926
8,406
7,736
Italy
7,236
7,152
6,725
6,852
6,212
5,817
Australia
7,398
6,897
6,642
6,378
5,839
5,105
China
7,077
6,475
5,995
5,665
5,182
4,349
Netherlands
5,997
5,709
5,246
5,346
5,163
4,693
Source: Bloomberg Industries, 2011
A3. Global software revenues
A4. References
Author
Year
Title
Source
Andrew Johnson
2012
IBISWorld Industry report J62.010
Computer programming in the UK
IBISWorld
Bloomberg
2012
Bloomberg industries Application
Software
Bloomberg
Bloomberg
2012
Bloomberg Industries Infrastructure
Software
Bloomberg
Dale Schmidt
2012
IBISWorld Industry report J51121
Software Publishing in the US
IBISWorld
Evans Data Corporation via
Prweb.com
2011
Global Developer Population and
Demographics
http://www.prweb.com/releases/
prweb2011/9/prweb8834228.htm
Google Adwords
2012
Google Adwords keyword search
Google.com
IDC
2012
Android Marks Fourth Anniversary Since
Launch with 75.0% Market Share in
Third Quarter, According to IDC
http://www.idc.com/getdoc.jsp?c
ontainerId=prUS23771812
IDC via Bloomberg
2012
Bloomberg Industries Handheld
Bloomberg
Liau Yun Qing
2010
Report: Linux adoption highest among
APAC SMBs
http://www.zdnet.com/report-
linux-adoption-highest-among-
apac-smbs-2062062875/
Payscale.com
2012
Salaries report
Payscale.com
RTI
2002
The Economic impacts of inaduquate
infrastructure for software testing
National institute of standards &
technology
slambertron
2012
2012 Future of Open Source Survey: Get
your LPIC now!
http://www.lpi.org/blog/2012-
future-open-source-survey-get-
your-lpic-now
Sourceforge.net
2012
Download statistics for app stores (KDBg
and DDD)
Sourceforge.net
Disclaimer
“This work has been undertaken as part of a student educational project and the material should be viewed
in this context. The work does not constitute professional advice and no warranties are made regarding the
information presented. The Authors, Cambridge Judge Business School and its Faculty do not accept any
liability for the consequences of any action taken as a result of the work or any recommendations made or
inferred.”
... Consequently, automatic software defect repair technology has emerged and become a hot topic in the field of software engineering research [1]. Automatic defect repair technology refers to the ability to quickly and accurately fix program failures in software to reduce debugging costs and improve software quality [2]. ...
... Assuming V S and V LBS are the sets of variables (including local variables and fields) used by method S and method LBS respectively. After derivation, the similarity between S and LBS is defined as the Jaccard similarity coefficient between sets V S and V LBS as stated in Formula (2). When collecting the fields used by a statement, it is also necessary to consider the fields accessed through method calls within the current class. ...
... fields) used by method S and method LBS respectively. After derivation, the similarity between S and LBS is defined as the Jaccard similarity coefficient between sets S V and LBS V as stated in Formula (2). When collecting the fields used by a statement, it is also necessary to consider the fields accessed through method calls within the current class. ...
Article
Full-text available
Due to the explosive growth of software quantity and the mixed ability of software developers, a large number of software defects emerge during the later stages of software maintenance. The search method based on genetic programming is one of the most popular in search algorithms, but it also has some issues. The single-objective approach to validate and select offspring patches without considering other constraints can affect the efficiency of patch generation. To address this issue, this paper proposes an automatic software repair method based on Multi-objective Genetic Programming (MGPRepair). Firstly, the method adopts a lightweight context analysis strategy to find suitable repair materials. Secondly, it decouples the replacement statements and insertion statements in the repair materials, using a lower-granularity patch representation method to encode the patches in the search space. Then, the automatic software defect repair is treated as a multi-objective search problem, and the NSGA-II multi-objective optimization algorithm is used to find simpler repair patches. Finally, the test case filtering technique is used to accelerate the patch validation process and generate correct patches. MGPRepair was experimentally evaluated on 395 real Java software defects from the Defects4J dataset. The experimental results show that MGPRepair can generate test case-passing patches for 51 defects, of which 35 defect patches are equivalent to manually generated patches. Its repair the efficiency and success rate are higher to other excellent automatic software defect repair methods such as jGenProg, RSRepair, ARJA, Nopol, Capgen, and SequenceR.
... Recent studies [Viz20,BJCC13] show that reversible debuggers ease the debugging phase, and help programmers to quickly find the causes of a misbehaviour. Reversible debuggers can be built on top of a causal-consistent reversible semantics [GLM14, LNPV18, FLS21, LLS + 22], and this approach is particularly suited to deal with concurrency bugs, which are hard to find using traditional debuggers [Gra86]. ...
Article
Reversible debuggers help programmers to find the causes of misbehaviours in concurrent programs more quickly, by executing a program backwards from the point where a misbehaviour was observed, and looking for the bug(s) that caused it. Reversible debuggers can be founded on the well-studied theory of causal-consistent reversibility, which only allows one to undo an action provided that its consequences, if any, are undone beforehand. Causal-consistent reversibility yields more efficient debugging by reducing the number of states to be explored when looking backwards. Till now, causal-consistent reversibility has never considered time, which is a key aspect in real-world applications. Here, we study the interplay between reversibility and time in concurrent systems via a process algebra. The Temporal Process Language (TPL) by Hennessy and Regan is a well-understood extension of CCS with discrete-time and a timeout operator. We define revTPL, a reversible extension of TPL, and we show that it satisfies the properties expected from a causal-consistent reversible calculus. We show that, alternatively, revTPL can be interpreted as an extension of reversible CCS with time.
Article
Full-text available
Recent research found that fine‐tuning pre‐trained models is superior to training models from scratch in just‐in‐time (JIT) defect prediction. However, existing approaches using pre‐trained models have their limitations. First, the input length is constrained by the pre‐trained models.Secondly, the inputs are change‐agnostic.To address these limitations, we propose JIT‐Block, a JIT defect prediction method that combines multiple input semantics using changed block as the fundamental unit. We restructure the JIT‐Defects4J dataset used in previous research. We then conducted a comprehensive comparison using eleven performance metrics, including both effort‐aware and effort‐agnostic measures, against six state‐of‐the‐art baseline models. The results demonstrate that on the JIT defect prediction task, our approach outperforms the baseline models in all six metrics, showing improvements ranging from 1.5% to 800% in effort‐agnostic metrics and 0.3% to 57% in effort‐aware metrics. For the JIT defect code line localization task, our approach outperforms the baseline models in three out of five metrics, showing improvements of 11% to 140%.
Article
Engineering high-performance query execution engines is a challenging task. Query compilation provides excellent performance, but at the same time introduces significant system complexity, as it makes the engine hard to build, debug, and maintain. To overcome this complexity, we propose Nautilus, a framework that combines the ease of use of query interpretation and the performance of query compilation. On the one hand, Nautilus provides an interpretation-based operator interface that enables engineers to implement operators using imperative C++ code to ensure a familiar developer experience. On the other hand, Nautilus mitigates the performance drawbacks of interpretation by introducing a novel trace-based, multi-backend JIT compiler that translates operators into efficient code. As a result, Nautilus bridges the gap between compilation and interpretation and provides the best of both worlds, achieving high performance without sacrificing the productivity of engineers.
Article
An increasing number of investors are active on Ethereum, resulting in numerous transactions. These historical transactions can be applied to complete contract testing. For example, it can be used for gas optimization or contract repair to verify that improved contracts meet expectations. Most existing methods deploy private chains to use non-real transactions for contract verification instead of actual historical transactions on the Ethereum mainnet. The challenge of using actual historical transactions for verification is that Ethereum only records the latest state of the account and cannot restore the execution of historical transactions. Due to contract code changes in contract gas optimization, contract defect repair, and other scenarios, we need to test the execution of the contract code before and after the change. However, existing tools cannot customize and modify historical transactions for testing purposes. Therefore, we propose an efficient transaction replay platform, EthReplayer , which can not only replay the historical transactions of Ethereum quickly and faithfully but also realize the modification of transactions to achieve the purpose of testing with actual transactions. Experimental results show that our replay speed is 1.5 times the fastest available, and it only takes 29,594 seconds to replay 1,200 million blocks. In addition, it is applied to contract repair verification, gas optimization verification, and gas estimation, and the results prove the effectiveness of EthReplayer .
ResearchGate has not been able to resolve any references for this publication.