Joe Palma’s research while affiliated with Southern Methodist University and other places

Tree-based reliability models integrate the benefits of software reliability growth models and input domain reliability models, offering a framework to assess reliability and guidance to improve it. The authors studied five large systems to test their approach

This paper presents our strategy, implementation, and experience with tool support for software measurement, analysis, and quality improvement in a commercial software development environment. The activities that need to be supported include: 1) gathering software product and process measurements, 2) analyzing measurement data, and 3) using the results to assess and improve software quality. We employed existing internal tools, commercial off-the-shelf tools, and new tools that we developed to gather and analyze data, and presented results. These tools were tailored to fit the application environments and were integrated through common rules, multiple purpose tools, and utility programs. This approach has been used successfully in supporting software measurement and quality improvement for several large commercial software products developed in the IBM Software Solutions Toronto Laboratory.

This paper characterizes the testing environment for large commercial software systems, matches reliability model assumptions with the application environment, examines alternative test workload measurements that capture software usage information during testing, and uses two such measurements, test runs and transactions, as our usage dependent time measurements in reliability modeling. Our previous research using test runs, execution time, and test input information for reliability analysis and improvement is extended to ensure better test workload measurements for reliability assessment and prediction. This paper also identifies conditions under which different test workload measurements are appropriate, and presents reliability modeling results using these measurements in several products developed in the IBM Software Solutions Toronto Laboratory.

The paper presents an approach to software reliability modeling using data partitions derived from tree based models. We use these data sensitive partitions to group data into clusters with similar failure intensities. The series of data clusters associated with different time segments forms a piecewise linear model for the assessment and short term prediction of reliability. Long term prediction can be provided by the dual model that uses these grouped data as input fitted to some failure count variations of the traditional software reliability growth models. These partition based reliability models can be used effectively to measure and predict the reliability of software systems and can be readily integrated into our strategy of reliability assessment and improvement using tree based modeling

The paper studies practical reliability measurement and modeling for large commercial software systems based on test execution data collected during system testing. The application environment and the goals of reliability assessment were analyzed to identify appropriate measurement data. Various reliability growth models were used on failure data normalized by test case executions to track testing progress and provide reliability assessment. Practical problems in data collection, reliability measurement and modeling, and modeling result analysis were also examined. The results demonstrated the feasibility of reliability measurement in a large commercial software development environment and provided a practical comparison of various reliability measurements and models under such an environment

This paper studies the collection of appropriate data for software reliability analysis and modeling. Our approach satisfies the data requirements for various reliability models under the constraints imposed by our project environment. These data-model relations and data - environment constraints are characterized to provide a tentative roadmap for data collection. In the process of collecting data for a group of projects in the system testing stage, we encountered various problems and devised solutions and improvement initiatives to deal with them. We summarize our experience in this paper so that similar initiatives in quality improvement under similar environments can be implemented more effectively.