Conference PaperPDF Available

Abstract and Figures

Despite many software engineering efforts and programming language support, resource and memory leaks remain a troublesome issue in managed languages such as Java. Understanding the properties of leak-related issues, such as their type distribution, how they are found, and which defects induce them is an essential prerequisite for designing better approaches for avoidance, diagnosis, and repair of leak-related bugs. To answer these questions, we conduct an empirical study on 452 issues found in repositories of 10 mature Apache Java projects.
Content may be subject to copyright.
Poster: Memory and Resource Leak Defects
in Java Projects: An Empirical Study
Mohammadreza Ghanavati
Heidelberg University
mohammadreza.ghanavati@informatik.uni-heidelberg.de
Diego Costa
Heidelberg University
diego.costa@informatik.uni-heidelberg.de
Artur Andrzejak
Heidelberg University
artur.andrzejak@informatik.uni-heidelberg.de
Janos Seboek
Heidelberg University
seboek@cl.uni-heidelberg.de
ABSTRACT
Despite many software engineering eorts and programming lan-
guage support, resource and memory leaks remain a troublesome
issue in managed languages such as Java. Understanding the proper-
ties of leak-related issues, such as their type distribution, how they
are found, and which defects induce them is an essential prerequi-
site for designing better approaches for avoidance, diagnosis, and
repair of leak-related bugs. To answer these questions, we conduct
an empirical study on 452 issues found in repositories of 10 mature
Apache Java projects.
CCS CONCEPTS
General and reference Empirical studies
;
Software and
its engineering Memory management
;
Software defect
analysis;
KEYWORDS
Empirical study, memory leak, resource leak, leak detection, fault
localization
ACM Reference format:
Mohammadreza Ghanavati, Diego Costa, Artur Andrzejak, and Janos Se-
boek. 2018. Poster: Memory and Resource Leak Defects in Java Projects: An
Empirical Study. In Proceedings of 40th International Conference on Software
Engineering, Gothenburg, Sweden, 27 May–3 June, 2018 (ICSE 2018), 2 pages.
https://doi.org/xx.xxx/xxx_x
1 INTRODUCTION
Resource or memory leaks are caused by software defects related
to mismanagement of system resources. Newer programming lan-
guages oer support for avoiding leaks, particularly memory leaks.
In managed languages such as Java, C#, or Go a Garbage Collector
(GC) is responsible for memory management. However, objects
which are not used but are still reachable by a chain of references
cannot be released by a GC. For example, obsolete object references
from collections are examples of frequent defects causing memory
leaks in Java [
1
]. Even in managed languages, the programmer is
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
ICSE 2018, 27 May–3 June, 2018, Gothenburg, Sweden
©2018 Copyright held by the owner/author(s).
ACM ISBN xxx-xxxx-xx-xxx/xx/xx. .. $15.00
https://doi.org/xx.xxx/xxx_x
Table 1: Studied projects with statistics on their size and
number of issues.
Project #Issues #Bugs #Fixed # Leak Issues
AMQ 123 116 88 80
CASSANDRA 77 65 45 35
CXF 62 61 44 37
DERBY 50 36 23 16
HADOOP 236 201 132 119
HBASE 92 65 44 40
HIVE 78 69 47 44
HTTPCOMP. 31 28 24 20
LUCENE 77 65 42 34
SOLR 74 60 33 27
Total 900 766 522 452
responsible to release nite system resources such as le handles,
threads, or database connections after their use. A typical defect pat-
tern observed here is lack of releasing these resources if exceptions
occur.
Diagnosis of leak-related defects is an important problem for
both researchers and practitioners [
2
]. Various approaches have
been introduced for leak detection and diagnosis based on static and
or dynamic analysis [
4
9
]. Other recent eorts target prevention
of leaks via programming language support such as new language
construct introduced in Java 7, the try-with-resources statement or
Smart pointers in C++.
The impact of these research activities, tools and language en-
hancements critically depends on whether they target prevalent or
rare types of leak defects, whether they can handle dicult cases,
and whether their assumptions are realistic enough to apply in
practice. For instance, Xu [
8
] proposes a method for detecting leaks
caused by obsolete references from within object containers, but
provides only a limited evidence that this is a frequent cause of
leak-related bugs. To this end it is important to understand the prop-
erties of leak-related defects, their detection methods, and repair
activities in a quantitatively representative way.
To our knowledge, available studies in this eld (e.g., [
3
]) are still
limited in terms of sample size and depth of analysis. We believe
that researchers and practitioners would benet from a systematic
empirical study of a large sample of leak-related defects found in
real-world applications.
ICSE 2018, 27 May–3 June, 2018, Gothenburg, Sweden Mohammadreza Ghanavati, Diego Costa, Artur Andrzejak, and Janos Seboek
11
13
8
19
11
43
12
29
19
54
16
21
12
25
29
76
4
8
16
26
0% 25% 50% 75% 100%
SOLR
LUCENE
HTTPCOMP
HIVE
HBASE
HADOOP
DERBY
CXF
CASSANDRA
AMQ
Memory Leak Resource Leak
Figure 1: Frequency of leak types per project.
2 METHODOLOGY
Our study uses ten open source large-scale Java projects from
Apache repository (Table 1). The investigated projects use a central-
ized bug tracker called JIRA to track the issue reports. We obtain
the dataset for our study via the four steps described below. Our
methodology yields a dataset with
452
leak-related issues, each
representing a unique leak bug.
Keyword search.
We select only issues that contain the keyword
”leak” in the issue title or issue description.
Issue type.
As we are only interested in leak-related bugs, we sub-
sequently keep only the issues of type Bug.
Issue resolution.
We restrict our analysis to bug issues for which
there is at least one associated repair patch.
Manual investigation.
In the nal step we manually inspect and
remove the false positives from our dataset by ltering out the
following issues: 1) non-leaky bugs retrieved by our keyword search
heuristic, and 2) wrongly reported leaks by developers.
3 PRELIMINARY RESULTS
We provide here preliminary results on the leak types, the detection
types, and the root causes of the leak-inducing defects.
What are the types of leak defects in studied applications?
Leaks can be triggered by unreleased references (memory leak) or
mismanagement of nite system resources (resource leak). Figure 1
shows the distribution of these two leak types across the studied
projects. In projects AMQ,CXF, and DERBY memory leak issues
are more reported, while in projects HADOOP and HBASE more
than 60% of the issues are identied as resource leak. In the other
projects, the proportion of the reported resource and memory leak
issues are about equal.
How are leak-related defects detected?
Understanding how
Table 2: Distribution of defect detection methods.
Detection method # Leak Issues
Code analysis 151 (33.4%)
Runtime analysis 106 (23.5%)
Failed test 55 (12.2%)
OOM error 51 (11.3%)
Warning 15 (3.3%)
Combined 74 (16.4%)
Total 452 (100%)
Table 3: Leak-triggering defect types and their distribution.
Description of a defect type # Leak Issues
Non-closed resource at error-free execution 135
Object not disposed of if exception is thrown 93
Dead objects referenced by a collection 89
Unreleased reference at error-free execution 29
A race condition defect (concurrency bug) 17
Wrong call schedule of dispose of function 15
Strong reference instead of a weak reference 14
Over-sized cache or buer 12
Incorrect API usage (wrong method or parameter) 10
Bugs not belonging to the above categories 38
the leaks manifest themselves as a defect and how developers de-
tect the leak in the application can provide valuable insights on the
leak diagnosis. Table 2 shows the distribution of major detection
approaches used by the developers of the considered projects. In-
terestingly, code analysis (33%) and runtime analysis (23%) are still
the dominant methods.
What are the common root causes of leak-inducing defects?
To uncover any potential common patterns among the leak-triggering
defects we studied in-depth each issue and the code of the corre-
sponding patches. Table 3 shows that there are indeed dominant
types of defects, with top 3 accounting for 70% of all issues. Our
ndings can support development of automated leak detection
approaches by pointing to most common defect types.
4 FUTURE WORK
In our future work we intend to rene and extend the results from
Section 3 by focusing on the following research questions: How
are the leak-related defects repaired? How complex are the repair
patches? To what degree are all these taxonomies related to each
other? We believe that answering these questions can be benecial
for improving and possibly automatizing the process of localizing
leak-related faults, their root cause analysis, and manual or auto-
mated bug repair. Our quantitative ndings will be used as a basis
for drawing implications for practitioners and researchers on topics
related to leak-related issues.
REFERENCES
[1]
Mohammadreza Ghanavati and Artur Andrzejak. 2015. Automated memory leak
diagnosis by regression testing. In IEEE SCAM. 191–200.
[2]
David Lo, Nachiappan Nagappan, and Thomas Zimmermann. 2015. How Practi-
tioners Perceive the Relevance of Software Engineering Research. In ESEC/FSE.
415–425.
[3]
Fumio Machida, Jianwen Xiang, Kumiko Tadano, and Yoshiharu Maeno. 2012.
Aging-Related Bugs in Cloud Computing Software. In ISSREW. 287–292.
[4]
Rivalino Matias, Artur Andrzejak, Fumio Machida, Diego Elias, and Kishor S.
Trivedi. 2014. A Systematic Dierential Analysis for Fast and Robust Detection
of Software Aging. In SRDS. 311–320. https://doi.org/10.1109/SRDS.2014.38
[5]
N. Mitchell and G. Sevitsky. 2003. LeakBot: An Automated and Lightweight Tool
for Diagnosing Memory Leaks in Large Java Applications. LNCS 2743 (2003),
351–377.
[6]
V. Sor, P. OÃij, T. Treier, and S. N. Srirama. 2013. Improving Statistical Approach
for Memory Leak Detection Using Machine Learning. In ICSM. 544–547.
[7]
Guoqing Xu, Michael D. Bond, Feng Qin, and Atanas Rountev. 2011. LeakChaser:
Helping Programmers Narrow Down Causes of Memory Leaks. In PLDI. 270–282.
[8]
Guoqing Xu and Atanas Rountev. 2008. Precise Memory Leak Detection for Java
Software Using Container Proling. In ICSE. 151–160.
[9]
Dacong Yan, Guoqing Xu, Shengqian Yang, and Atanas Rountev. 2014.
LeakChecker: Practical Static Memory Leak Detection for Managed Languages.
In CGO. Article 87, 11 pages.
... In Sec. III-A, we describe an implementation using a keywordbased pre-filtering step followed by a manual inspection, similar to the approach used in [10], [18]. This can be both used for open-source software which maintain public issue trackers and for proprietary products, as the latter also typically maintain private issue trackers. ...
... A large empirical study on aging-related defects in Javabased applications [10], [18] showed that dead objects not being properly disposed of contribute to 20% aging bugs. Object disposal is forgotten or disabled due to a bug (e.g., unreleased reference), or it fails under exceptional conditions. ...
... The aging-related issues have been empirically proven in many commercial software solutions. The effects of software aging were detected in modern operating systems, Linux [1] and Android [2], Unix cluster [27], cloud controllers, both proprietary [3] and open source solutions [4], Apache web server [5]- [7] and SOAP server [36], Air-Traffic-Control middleware (CARDAMOM) [26], Java Virtual Machine [8], [9], as well as several large open source Java applications, such as Hadoop and Cassandra [10], [18]. ...
Article
The recent trend of network softwarization suggests a radical shift in the implementation of traditional network intelligence. In Software Defined Networking (SDN), for instance, the control plane functions of forwarding devices are outsourced to the controller. Softwarized network components are expected to provide uninterrupted service during long periods of time, which makes them prone to the effects of software aging, a phenomena that has been observed in operational software systems where the failure rate increases or the performance of the software degrades with the elapsed time since the last restart. The effects of software aging in operational networks are typically mitigated by software rejuvenation, i.e., planned restarts cleaning the internal system state in order to prevent or postpone aging-related failures. This paper presents ARES, a three-step methodological framework for the management of the effects of software aging in softwarized networks, applied to the case study of open source SDN orchestration platforms. Using ARES, we demonstrate that software aging is a systematic problem that cannot be neglected in network orchestration systems. It stems not only from aging-related bugs and natural aging due to fragmentation, but also from design choices, e.g., when implementing distributed systems. Measurements for Open Network Operating System (ONOS) and OpenDaylight (ODL) demonstrate how “simple” and common networking tasks let network performance degrade rapidly and even lead to crashes: for instance, adding and removing 300 intents per second in ONOS significantly increases the response time by 50% per day and depletes the memory at the rate of 18GB per day. Moreover, we demonstrate a first rejuvenation approach that can mitigate the effects of aging in ONOS.
... The analysis of the source code of the smartphone's applications includes an analysis of the code's architecture [59] and logic of its operation [60] as well as methods for identifying buffer overflows [61], memory leaks [62] and code inserts [63]. In addition, any operation that requires the extraction, storage or transfer of the user's sensitive data is checked for compliance with the privacy policy [64,65]. ...
Article
Full-text available
Nowadays, the whole driver monitoring system can be placed inside the vehicle driver's smartphone, which introduces new security and privacy risks to the system. Because of the nature of the modern transportation systems, the consequences of the security issues in such systems can be crucial, leading to threat to human life and health. Moreover, despite the large number of security and privacy issues discovered in smartphone applications on a daily basis, there is no general approach for their automated analysis that can work in conditions that lack data and take into account specifics of the application area. Thus, this paper describes an original approach for a security and privacy analysis of driver monitoring systems based on smartphone sensors. This analysis uses white-box testing principles and aims to help developers evaluate and improve their products. The novelty of the proposed approach lies in combining various security and privacy analysis algorithms into a single automated approach for a specific area of application. Moreover, the suggested approach is modular and extensible, takes into account specific features of smartphone-based driver monitoring systems and works in conditions of lack or inaccessibility of data. The practical significance of the approach lies in the suggestions that are provided based on the conducted analysis. Those suggestions contain detected security and privacy issues and ways of their mitigation, together with limitations of the analysis due to the absence of data. It is assumed that such an approach would help developers take into account important aspects of security and privacy, thus reducing related issues in the developed products. An experimental evaluation of the approach is conducted on a car driver monitoring use case. In addition, the advantages and disadvantages of the proposed approach as well as future work directions are indicated.
... A memory leak [13] occurs if objects that are no longer needed remain reachable from GC roots due to programming errors. This leads to a continuously growing memory consumption which can cause the application to run out of memory, crashing it in the worst case [19]. ...
Conference Paper
Modern memory monitoring tools do not only offer analyses at a single point in time, but also offer features to analyze the memory evolution over time. These features provide more detailed insights into an application's behavior, yet they also make the tools more complex and harder to use. Analyses over time are typically performed on certain time windows within which the application behaves abnormally. Such suspicious time windows first have to be detected by the users, which is a non-trivial task, especially for novice users that have no experience in memory monitoring. In this paper, we present algorithms to automatically detect suspicious time windows that exhibit (1) continuous memory growth, (2) high GC utilization, or (3) high memory churn. For each of these problems we also discuss its root causes and implications. To show the feasibility of our detection techniques, we integrated them into AntTracks, a memory monitoring tool developed by us. Throughout the paper, we present their usage on various problems and real-world applications.
... The inappropriate usage of collections is also linked to memory-leak issues in Java [64]. Typically, elements never removed from collections are never released back to the JVM, until the collection itself becomes eligible for garbage collection. ...
Thesis
Full-text available
Software systems are an integral part of modern society. As we continue to harness software automation in all aspects of our daily lives, the runtime performance of these systems become increasingly important. When everything seems just a click away, performance issues that compromise the responsiveness of a system can lead to severe financial and reputation losses. Designing efficient code is critical for ensuring good and consistent performance of software systems. It requires performance expertize, and encompasses a set of difficult design decisions that need to be continuously revisited throughout the evolution of the software. Developers must test the performance of their core implementations, select efficient data structures and algorithms, explore parallel processing when it provides performance benefits, among many other aspects. Furthermore, the constant pressure for high-productivity laid on developers, aligned with the increasing complexity of modern software, makes designing efficient code an even more challenging endeavor. This thesis presents a series of novel approaches based on empirical insights that attempt to support developers at the task of designing efficient code. We present contributions in three aspects. First, we investigate the prevalence and impact of bad practices on performance benchmarks of Java-based open-source software. We show that not only these bad practices occur frequently, they often distort the benchmark results substantially. Moreover, we devise a tool that can be used by developers to identify bad practices during benchmark creation automatically. Second, we design an application-level framework that identifies suboptimal implementations and selects optimized variants at runtime, effectively optimizing the execution time and memory usage of the target application. Furthermore, we investigate the performance of data structures from several popular collection libraries. Our findings show that alternative variants can be selected for substantial performance improvement under specific usage scenarios. Third, we investigate the parallelization of object processing via Java streams. We propose a decision-support framework that leverages machine-learning models trained through a series of benchmarks, to identify and report stream pipelines that should be processed in parallel for better performance.
... The existing studies [78,115] provide only limited information about characteristics of detection types, root causes, and repair actions of leak defects. To fill this gap, we conduct a detailed empirical study on 491 real-world memory, and resource leak defects gathered from 15 large, open-source Java applications [42,43]. ...
Thesis
Modern software systems evolve steadily. Software developers change the software codebase every day to add new features, to improve the performance, or to fix bugs. Despite extensive testing and code inspection processes before releasing a new software version, the chance of introducing new bugs is still high. A code that worked yesterday may not work today, or it can show a degraded performance causing software regression. The laws of software evolution state that the complexity increases as software evolves. Such increasing complexity makes software maintenance harder and more costly. In a typical software organization, the cost of debugging, testing, and verification can easily range from 50% to 75% of the total development costs. Given that human resources are the main cost factor in the software maintenance and the software codebase evolves continuously, this dissertation tries to answer the following question: How can we help developers to localize the software defects more effectively during software development? We answer this question in three aspects. First, we propose an approach to localize failure-inducing changes for crashing bugs. Assume the source code of a buggy version, a failing test, the stack trace of the crashing site, and a previous correct version of the application. We leverage program analysis to contrast the behavior of the two software versions under the failing test. The difference set is the code statements which contribute to the failure site with a high probability. Second, we extend the version comparison technique to detect the leak-inducing defects caused by software changes. Assume two versions of a software codebase (one previous non-leaky and the current leaky version) and the existing test suite of the application. First, we compare the memory footprint of the code locations between two versions. Then, we use a confidence score to rank the suspicious code statements, i.e., those statements which can be the potential root causes of memory leaks. The higher the score, the more likely the code statement is a potential leak. Third, our observation on the related work about debugging and fault localization reveals that there is no empirical study which characterizes the properties of the leak- inducing defects and their repairs. Understanding the characteristics of the real defects caused by resource and memory leaks can help both researchers and practitioners to improve the current techniques for leak detection and repair. To fill this gap, we conduct an empirical study on 491 reported resource and memory leak defects from 15 large Java applications. We use our findings to draw implications for leak avoidance, detection, localization, and repair.
... The preliminary idea of this work is presented in a two-page short paper in ICSE 2018 (Ghanavati et al. 2018). This work provides the following contributions: require further improvement to detect different leak types in practice. ...
Article
Full-text available
Despite huge software engineering efforts and programming language support, resource and memory leaks are still a troublesome issue, even in memory-managed languages such as Java. Understanding the properties of leak-inducing defects, how the leaks manifest, and how they are repaired is an essential prerequisite for designing better approaches for avoidance, diagnosis, and repair of leak-related bugs. We conduct a detailed empirical study on 491 issues from 15 large open-source Java projects. The study proposes taxonomies for the leak types, for the defects causing them, and for the repair actions. We investigate, under several aspects, the distributions within each taxonomy and the relationships between them. We find that manual code inspection and manual runtime detection are still the main methods for leak detection. We find that most of the errors manifest on error-free execution paths, and developers repair the leak defects in a shorter time than non-leak defects. We also identify 13 recurring code transformations in the repair patches. Based on our findings, we draw a variety of implications on how developers can avoid, detect, isolate and repair leak-related bugs.
... These objects cannot be reclaimed by the garbage collector and will therefore accumulate over time. Beside excessive dynamic allocations [27][28][29], memory leaks are one of the major memory anomalies [10]. ...
Conference Paper
Memory leaks are a major threat in modern software systems. They occur if objects are unintentionally kept alive longer than necessary and are often indicated by continuously growing data structures. While there are various state-of-the-art memory monitoring tools, most of them share two critical shortcomings: (1) They have no knowledge about the monitored application's data structures and (2) they support no or only rudimentary analysis of the application's data structures over time. This paper encompasses novel techniques to tackle both of these drawbacks. It presents a domain-specific language (DSL) that allows users to describe arbitrary data structures, as well as an algorithm to detect instances of these data structures in reconstructed heaps. In addition, we propose techniques and metrics to analyze and measure the evolution of data structure instances over time. This allows us to identify those instances that are most likely involved in a memory leak. These concepts have been integrated into AntTracks, a trace-based memory monitoring tool. We present our approach to detect memory leaks in several real-world applications, showing its applicability and feasibility.
... Nevertheless, memory problems can still occur even in garbage-collected languages. One of the major types of memory problems are memory leaks [4], i.e., objects may remain reachable from GC roots even though they are no longer needed. For example, if a developer forgets to remove objects from their containing data structures, these objects cannot be reclaimed by the GC and will accumulate over time. ...
Conference Paper
The complexity of modern applications makes it hard to fix memory leaks and other heap-related problems without tool support. Yet, most state-of-the-art tools share problems that still need to be tackled: (1) They group heap objects only based on their types, ignoring other properties such as allocation sites or data structure compositions. (2) Analyses strongly focus on a single point in time and do not show heap evolution over time. (3) Results are displayed in tables, even though more advanced visualization techniques may ease and improve the analysis. In this paper, we present a novel visualization approach that addresses these shortcomings. Heap objects can be arbitrarily classified, enabling users to group objects based on their needs. Instead of inspecting the size of those object groups at a single point in time, our approach tracks the growth of each object group over time. This growth is then visualized using time-series charts, making it easy to identify suspicious object groups. A drill-down feature enables users to investigate these object groups in more detail. Our approach has been integrated into AntTracks, a trace-based memory monitoring tool, to demonstrate its feasibility.
... The preliminary idea of this work is presented in a two-pages short paper in ICSE 2018 (Ghanavati et al. 2018). This work provides the following contributions: Characterization study. ...
Preprint
Full-text available
Despite huge software engineering efforts and programming language support, resource and memory leaks are still a troublesome issue, even in memory-managed languages such as Java. Understanding the properties of leak-inducing defects, how the leaks manifest, and how they are repaired is an essential prerequisite for designing better approaches for avoidance, diagnosis, and repair of leak-related bugs. We conduct a detailed empirical study on 452 issues from 10 large open-source Java projects. The study proposes taxonomies for the leak types, for the defects causing them, and for the repair actions. We investigate, under several aspects, the distributions within each taxonomy and the relationships between them. We find that manual code inspection and manual runtime detection are still the main methods for leak detection. We find that most of the errors manifest on error-free execution paths, and developers repair the leak defects in a shorter time than non-leak defects. We also identify 13 recurring code transformations in the repair patches. Based on our findings, we draw a variety of implications on how developers can avoid, detect, isolate and repair leak-related bugs.
Conference Paper
Full-text available
Memory leaks are tedious to detect and require significant debugging effort to be reproduced and localized. In particular, many of such bugs escape classical testing processes used in software development. One of the reasons is that unit and integration tests run too short for leaks to manifest via memory bloat or degraded performance. Moreover, many of such defects are environment-sensitive and not triggered by a test suite. Consequently, leaks are frequently discovered in the production scenario, causing elevated costs. In this paper we propose an approach for automated diagnosis of memory leaks during the development phase. Our technique is based on regression testing and exploits existing test suites. The key idea is to compare object (de-)allocation statistics (collected during unit/integration test executions) between a previous and the current software version. By grouping these statistics according to object creation sites we can detect anomalies and pinpoint the potential root causes of memory leaks. Such diagnosis can be completed before a visible memory bloat occurs, and in time proportional to the execution of test suite. We evaluate our approach using real leaks found in 7 Java applications. Results show that our approach has sufficient detection accuracy and is effective in isolating the leaky allocation site: true defect locations rank relatively high in the lists of suspicious code locations if the tests trigger the leak pattern. Our prototypical system imposes an acceptable instrumentation and execution overhead for practical memory leak detection even in large software projects.
Data
Full-text available
Software systems running continuously for a long time often confront software aging, which is the phenomenon of progressive degradation of execution environment caused by latent software faults. Removal of such faults in software development process is a crucial issue for system reliability. A known major obstacle is typically the large latency to discover the existence of software aging. We propose a systematic approach to detect software aging which has shorter test time and higher accuracy compared to traditional aging detection via stress testing and trend detection. The approach is based on a differential analysis where a software version under test is compared against a previous version in terms of behavioral changes of resource metrics. A key instrument adopted is a divergence chart, which expresses time-dependent differences between two signals. Our experimental study focuses on memory-leak detection and evaluates divergence charts computed using multiple statistical techniques paired with application-level memory related metrics (RSS and Heap Usage). The results show that the proposed method achieves good performance for memory-leak detection in comparison to techniques widely adopted in previous works (e.g., linear regression, moving average and median).
Conference Paper
Full-text available
Cloud computing is established on advanced software technologies intended to enhance the scalability of computing infrastructure by making full use of commodity servers. The more cloud computing relies on software technologies, the more software bugs have significant impacts on the system availability. Aging-related bugs, which cause the accumulation of errors in long time software execution, often remain even after the release of a stable version of the software. This paper investigates the bug reports of five major open-source software projects related to cloud computing and confirms the issues about the existence of aging-related bugs. From the investigation how the developers work around such aging-related bugs, the importance of the tool assistance for reproducing the aging problem in replicated site is discussed.
Article
In large programs written in managed languages such as Java and C#, holding unnecessary references often results in memory leaks and bloat, degrading significantly their run-time performance and scalability. Despite the existence of many leak detectors for such languages, these detectors often target low-level objects; as a result, their reports contain many false warnings and lack sufficient semantic information to help diagnose problems. This paper introduces a specification-based technique called LeakChaser that can not only capture precisely the unnecessary references leading to leaks, but also explain, with high-level semantics, why these references become unnecessary. At the heart of LeakChaser is a three-tier approach that uses varying levels of abstraction to assist programmers with different skill levels and code familiarity to find leaks. At the highest tier of the approach, the programmer only needs to specify the boundaries of coarse-grained activities, referred to as transactions. The tool automatically infers liveness properties of these transactions, by monitoring the execution, in order to find unnecessary references. Diagnosis at this tier can be performed by any programmer after inspecting the APIs and basic modules of a program, without understanding of the detailed implementation of these APIs. At the middle tier, the programmer can introduce application-specific semantic information by specifying properties for the transactions. At the lowest tier of the approach is a liveness checker that does not rely on higher-level semantic information, but rather allows a programmer to assert lifetime relationships for pairs of objects. This task could only be performed by skillful programmers who have a clear understanding of data structures and algorithms in the program. We have implemented LeakChaser in Jikes RVM and used it to help us diagnose several real-world leaks. The implementation incurs a reasonable overhead for debugging and tuning. Our case studies indicate that the implementation is powerful in guiding programmers with varying code familiarity to find the root causes of several memory leaks---even someone who had not studied a leaking program can quickly find the cause after using LeakChaser's iterative process that infers and checks properties with different levels of semantic information.
Conference Paper
The number of software engineering research papers over the last few years has grown significantly. An important question here is: how relevant is software engineering research to practitioners in the field? To address this question, we conducted a survey at Microsoft where we invited 3,000 industry practitioners to rate the relevance of research ideas contained in 571 ICSE, ESEC/FSE and FSE papers that were published over a five year period. We received 17,913 ratings by 512 practitioners who labelled ideas as essential, worthwhile, unimportant, or unwise. The results from the survey suggest that practitioners are positive towards studies done by the software engineering research community: 71% of all ratings were essential or worthwhile. We found no correlation between the citation counts and the relevance scores of the papers. Through a qualitative analysis of free text responses, we identify several reasons why practitioners considered certain research ideas to be unwise. The survey approach described in this paper is lightweight: on average, a participant spent only 22.5 minutes to respond to the survey. At the same time, the results can provide useful insight to conference organizers, authors, and participating practitioners.
Conference Paper
Static detection of memory leaks in a managed language such as Java is attractive because it does not rely on any leak-triggering inputs, allowing compile-time tools to find leaks before software is released. A long-standing issue that prevents practical static memory leak detection for Java is that it can be very expensive to statically determine object liveness in large applications. We present a novel (and the first practical) static leak detection technique that bypasses this problem by considering a common leak pattern. In many cases severe leaks occur in loops where, in each iteration, some objects created by the iteration are unnecessarily referenced by objects external to the loop. These unnecessary references are never used in later loop iterations. Based on this insight, we shift our focus from computing liveness, which is very difficult to achieve precisely and efficiently for large programs, to the easier goal of identifying objects that flow out of a loop but never flow back in. We formalize this analysis using a type and effect system and present its key properties. The analysis was implemented in a tool called LeakChecker and used to detect leaks in eight real-world programs, such as Eclipse, Derby, and log4j. LeakChecker not only identified known leaks, but also discovered new ones whose causes were unknown beforehand, while exhibiting a false positive rate suitable for practical use.
Conference Paper
Memory leaks are major problems in all kinds of applications, depleting their performance, even if they run on platforms with automatic memory management, such as Java Virtual Machine. In addition, memory leaks contribute to software aging, increasing the complexity of software maintenance. So far memory leak detection was considered to be a part of development process, rather than part of software maintenance. To detect slow memory leaks as a part of quality assurance process or in production environments statistical approach for memory leak detection was implemented and deployed in a commercial tool called Plumbr. It showed promising results in terms of leak detection precision and recall, however, even better detection quality was desired. To achieve this improvement goal, classification algorithms were applied to the statistical data, which was gathered from customer environments where Plumbr was deployed. This paper presents the challenges which had to be solved, method that was used to generate features for supervised learning and the results of the corresponding experiments.
Conference Paper
A memory leak in a Java program occurs when object references that are no longer needed are unnecessarily maintained. Such leaks are difficult to detect because static analysis typically cannot precisely identify these redundant references, and existing dynamic leak detection tools track and report fine-grained information about individual objects, producing results that are usually hard to interpret and lack precision. In this article we introduce a novel container-based heap-tracking technique, based on the fact that many memory leaks in Java programs occur due to incorrect uses of containers, leading to containers that keep references to unused data entries. The novelty of the described work is twofold: (1) instead of tracking arbitrary objects and finding leaks by analyzing references to unused objects, the technique tracks only containers and directly identifies the source of the leak, and (2) the technique computes a confidence value for each container based on a combination of its memory consumption and its elements' staleness (time since last retrieval), while previous approaches do not consider such combined metrics. Our experimental results show that the reports generated by the proposed technique can be very precise: for two bugs reported by Sun, a known bug in SPECjbb 2000, and an example bug from IBM developerWorks, the top containers in the reports include the containers that leak memory.
Conference Paper
Despite Java's automatic reclamation of memory, memory leaks remain an important problem. For example, we frequently en- counter memory leaks that cause production servers to crash. These servers represent an increasingly common class of Java applications: they are large scale and they make heavy use of frameworks. For these appli- cations, existing tools require too much expertise, and, even for experts, require many hours interpreting low-level details. In addition, they are often too expensive to use in practice. We present an automated, adap- tive, and scalable tool for diagnosing memory leaks, called LeakBot. LeakBot incorporates three new techniques. First, it automatically ranks data structures by their likelihood of containing leaks. This process dra- matically prunes the set of candidate structures, using object reference graph properties and knowledge of how leaks occur. Second, it uses Co- evolving Regions to identify suspicious regions within a data structure and characterize their expected evolution. Third, it uses the first two methods to derive a lightweight way to track those regions' actual evolu- tion as the program runs. These techniques are mutually beneficial: we need only monitor what is highly ranked, and, because the tracking is so cheap, a region's rank can be continually updated with information from production machines. Finally, this whole process can be done without user assistance. We demonstrate LeakBot's eectiveness on a number of large-scale appli- cations that we have analyzed as part of the ongoing consulting practice our group maintains. We have found that the ranking analysis scales (e.g. written in Java, it analyzes 106 objects in 30 seconds with a 300M heap), is selective (e.g. it prunes that set to three candidate leak roots), and is accurate (it discounts non-leaking roots). The CER generation completes in tens of seconds. The lightweight tracking refines the rankings, while lowering throughput by less than 5%.