J. R. M. Hosking’s research while affiliated with IBM Research - Thomas J. Watson Research Center and other places

As the capacity of hardware systems has grown and workload consolidation has taken place, the volume of performance metrics and diagnostic data streams has outscaled the capability of people to handle these systems using traditional methods. As work of different types (such as database, batch, and Web processing), each in its own monitoring silo, runs concurrently on a single image (operating system instance), both the complexity and the business consequences of a single image failure have increased. This paper presents two techniques for generating actionable information out of the overwhelming amount of performance and diagnostic data available to human analysts. Failure scoring is used to identify high-risk failure events that may be obscured in the myriad system events. This replaces human expertise in scanning tens of thousands of records per day and results in a short, prioritized list for action by systems staff. Adaptive thresholding is used to drive predictive and descriptive machine-learning-based modeling to isolate and identify misbehaving processes and transactions. The attraction of this technique is that it does not require human intervention and can be reapplied continually, resulting in models that are not brittle. Both techniques reduce the quantity and increase the relevance of data available for programmatic and human processes.

Trimmed L-moments, defined by Elamir and Seheult [2003. Trimmed L-moments. Comput. Statist. Data Anal. 43, 299–314], summarize the shape of probability distributions or data samples in a way that remains viable for heavy-tailed distributions, even those for which the mean may not exist. We derive some further theoretical results concerning trimmed L-moments: a relation with the expansion of the quantile function as a weighted sum of Jacobi polynomials; the bounds that must be satisfied by trimmed L-moments; recurrences between trimmed L-moments with different degrees of trimming; and the asymptotic distributions of sample estimators of trimmed L-moments. We also give examples of how trimmed L-moments can be used, analogously to L-moments, in the analysis of heavy-tailed data. Examples include identification of distributions using a trimmed L-moment ratio diagram, shape parameter estimation for the generalized Pareto distribution, and fitting generalized Pareto distributions to a heavy-tailed data sample of computer network traffic.

We find the distribution that has maximum entropy conditional on having specified values of its first rL-moments. This condition is equivalent to specifying the expected values of the order statistics of a sample of size r. The maximum-entropy distribution has a density-quantile function, the reciprocal of the derivative of the quantile function, that is a polynomial of degree r; the quantile function of the distribution can then be found by integration. This class of maximum-entropy distributions includes the uniform, exponential and logistic, and two new generalizations of the logistic distribution. It provides a new method of nonparametric fitting of a distribution to a data sample. We also derive maximum-entropy distributions subject to constraints on expected values of linear combinations of order statistics.

The Palmer Drought Severity Index, which is intended to be of reasonable comparable local significance both in space and time, has been extensively used as a measure of drought for both agricultural and water resource management. This study examines the spatial comparability of Palmer's (1965) definition of severe and extreme drought. Index values have been computed for 1035 sites with at least 60 years of record that are scattered across the contiguous United States, and quantile values corresponding to a specified index value were calculated for given months and then mapped. The analyses show that severe or extreme droughts, as defined by Palmer (1965), are not spatially comparable in terms of identifying rare events. The wide variation across the country in the frequency of occurrence of Palmer's (1965) extreme droughts reflects the differences in the variability of precipitation, as well as the average amount of precipitation. It is recommended first, that a drought index be developed which considers both variability and averages; and second, that water resource managers and planners define a drought in terms of an index value that corresponds to the expected quantile (return period) of the event.

In some classification problems the feature space is heterogeneous in that the best features on which to base the classification are different in different parts of the feature space. In some other problems the classes can be divided into subsets such that distinguishing one subset of classes from another and classifying examples within such subsets require very different decision rules, involving different sets of features. In such heterogeneous problems, many modeling techniques (including decision trees, rules, and neural networks) evaluate the performance of alternative decision rules by averaging over the entire problem space, and axe prone to generating a model that is suboptimal in any of the regions or subproblems. Better overall models can be obtained by splitting the problem appropriately and modeling each subproblem separately. This paper presents a new measure to determine the degree of dissimilarity between the decision surfaces of two given problems, and suggests a way to search for a strategic splitting of the feature space that identifies regions with different characteristics. We illustrate the concept using a multiplexor problem.

A distribution with finite mean is uniquely determined by the set of expectations of the largest (or smallest) order statistics from samples of size 1,2,…1,2,…. However, this characterization contains some redundancy; some of the expectations can be dropped from the set and the remaining elements of the set still suffice to characterize the distribution. The rth LL-moment of a distribution is a linear combination of the expectations of the largest (or smallest) order statistics from samples of size 1,2,…,r1,2,…,r. We show that a wide range of distributions can be characterized by their LL-moments with no redundancy; a set that contains all of the LL-moments except one no longer suffices to characterize the distribution.

We develop a new concept for aggregating items of evidence for class probability estimation. In Naive Bayes, each feature contributes an independent multiplicative factor to the estimated class probability. We modify this model to include an exponent in each factor in order to introduce feature importance. These exponents are chosen to maximize the accuracy of estimated class probabilities on the training data. For Naive Bayes, this modification accomplishes more than what feature selection can. More generally, since the individual features can be the outputs of separate probability models, this yields a new ensemble modeling approach, which we call APM (Adjusted Probability Model), along with a regularized version called APMR.

Data mining can be regarded as a collection of methods for drawing inferences from data. The aims of data mining, and some of its methods, overlap with those of classical statistics. However, there are some philosophical and methodological differences. We examine these differences, and we describe three approaches to machine learning that have developed largely independently: classical statistics, Vapnik's statistical learning theory, and computational learning theory. Comparing these approaches, we conclude that statisticians and data miners can profit by studying each other's methods and using a judiciously chosen combination of them.

In some classification problems the feature space is heterogeneous in that the best features on which to base the classification are different in different parts of the feature space. In some other problems the classes can be divided into subsets such that distinguishing one subset of classes from another and classifying examples within the subsets require very different decision rules, involving different sets of features. In suchheterogeneous problems, many modeling techniques (including decision trees, rules, and neural networks) evaluate the performance of alternative decision rules byaveraging over the entire problem space, and are prone to generating a model that is suboptimal in any of the regions or subproblems. Better overall models can be obtained by splitting the problem appropriately and modeling each subproblem separately. This paper presents a new measure to determine the degree of dissimilaritybetween the decision surfaces of two given problems, and suggests a...

. Feature merits are used for feature selection in classification and regression as well as for decision tree generation. Commonly used merit functions exhibit a bias towards features that take a large varietyofvalues. Wepresentascheme based on randomization for neutralizing this bias by normalizing the merits. The merit of a feature is normalized by division by the expected merit of a feature that is random noise taking the same distribution of values as the given feature. The noise feature is obtained by randomly permuting the values of the given feature. The scheme can be used for any merit function including the Gini and entropy measures. We demonstrate its effectiveness by applying it to the contextual merit defined byHong(IBM Res. Rep. RC19664, 1994). Keywords: Classification, Data mining, Feature merits, Feature selection. Area of interest: Concept formation and classification. Contact: Jonathan R. M. Hosking, hosking@watson.ibm.com, phone +1 914 945 1031, fax +1 914 945 3434. ...