Content uploaded by Alessandro Moschitti

Author content

All content in this area was uploaded by Alessandro Moschitti on Aug 26, 2015

Content may be subject to copyright.

Estimating the Time Between Failures of Electrical Feeders in the

New York Power Grid

Haimonti Dutta, David Waltz, Alessandro Moschitti, Daniele Pighin, Philip Gross,

Claire Monteleoni, Ansaf Salleb-Aouissi, Albert Boulanger, Manoj Pooleery and Roger Anderson

Center for Computational Learning Systems, Columbia University, New York, NY 10115

{haimonti, waltz}@ccls.columbia.edu; moschitti@disi.unitn.it;

daniele.pighin@gmail.com;{phil, cmontel, ansaf, aboulanger, manoj, anderson}@ccls.columbia.edu

1 Introduction

Electricity generated by steam or hydro-turbines at power plants is transmitted at very high voltage from generating

stations to substations, distributed from substations to local transformers via feeder cables and ﬁnally sent to individual

customers [8] (Figure 1). In the New York City Power Grid, a little more than 1000 primary distribution feeders transmit

electricity between the high voltage transmission system and the household-voltage secondary system. These feeders are

susceptible to different kinds of failures such as emergency isolation caused by automatic substation relays (Open Autos),

failing on test, maintainence crew noticing problems and scheduled work on different sections of the feeder.

Over the past few years, researchers at CCLS have collaborated with the Consolidated Edison Company of New York

to develop systems that can rank feeders and their components (cable sections, joints and splices) according to their

susceptibility to failure. The Ranker for Open-Auto Maintainence Scheduling (ROAMS) [8] was the ﬁrst such system

built using Martingale Ranking [14]. Subsequently, the system was improved to boost ranking performance using an

ensemble of ranking experts [2] which, however, came at a cost of interpretability of the machine learning models. A

comparison of three different techniques for ranking electrical feeders - Martingale Ranking, RankBoost and SVM score

ranker can be found in [9]. More recently, we have begun to focus on estimating measures such as Time Between Failures

(TBF) of feeders resulting in regression problems as opposed to ranking. In this paper, we describe the challenges faced,

our approach to modeling the problem and provide empirical results obtained from the models.

This paper is organized as follows: Section 2 presents related work; Section 3 the challenges faced, Section 4 the data

generation process, Section 5 our approaches to modeling Time Between Failures (TBFs) and Section 6 presents a case

study for modeling TBFs of feeder cables in Brooklyn and Queens.

2 Related Work

Modeling failure rates (i.e. frequency with which an engineered system or component fails) has been studied extensively

in reliability theory ([7], [11]). Begovic et al. [3] study parametric statistical models when only partial information (such

as installation date, number of components replaced in a given year, failure and replacement rates) is available. They use

Weibull distributions to model future failures and for formulating replacement strategies. The Cox Proportional Hazards

model [6] is another semi-parametric regression where the features are modeled as scaling the instantaneous failure rate

Figure 1: Electricity Generation and Distribution

Sta$c&and&

&Dynamic&

A.ributes&

Feature&

Extrac$on&

RF

model

Training Random

Forest

Feeder&K"

Feeder&J"

Time

TBF

Estimated TBF

<X, TBF>

<X’, ?>

Figure 2: The Sampling Procedure for collecting data.

of a component. Guo et al [10] propose a model based on Proportional Intensity and explore tools to analyze repairable

systems – their approach can incorporate time trends, proportional failure intensity and cumulative repair effects. A

technique for approximating the mean time between failure of a system with periodic maintenance is described in work

done by Mondro [15] while models for recurrent events are studied by Lawless and Thiagarajah [12]. In this paper, we

present challenges faced and preliminary results in estimating TBFs using machine learning techniques in the New York

power grid.

3 Challenges in Estimating Time Between Failures

Several challenges exist in generating good regression models for estimating Time Between Failures (TBFs):

1. Few components actually failed during the time for which we have data. Many components have never failed or

failed only once during the time for which we have data, and for these cases we need to learn estimates from

“censored” data, i.e., data on time intervals where we only know that TBF is greater than a) the period for which we

have data, or b) the times from the last failure before we started collecting data until the ﬁrst failure, or c) the time

from the last failure until the present. In some cases we may have two or more failures of the same feeder during

the collection period, making it possible to get more precise data to train on.

2. Another key challenge is that there are several failure modes (such as Open Autos, Failed on Test, Out on Emer-

gency), and so task of prediction is highly non-linear. Key failure causes for feeders include aging, power qual-

ity events (e.g. spikes), overloads (that have seasonal variation, with summer heat waves especially problem-

atic), known weak components (e.g. PILC cable and joints connecting PILC to other sections), at-risk topologies

(where cascading failures could occur), workmanship problems and the stress of HiPot testing and deenergiz-

ing/reenergizing of feeders.

3. Since there are many different causes of failures, it is difﬁcult to pin-point an exact cause of failure; furthermore

the same feeder can fail multiple times within a short time span (often called “infant mortality”) or last more than a

few years. Thus there are considerable ﬂuctuations in survival times resulting in a very imbalanced data set.

4. As pointed out by Begovic et al. [3], “An accurate model of power apparatus lifetime should contain a large number

of factors, which are not practical for monitoring - a partial list should contain the initial quality and uniformity

of the materials the equipment is made of (primarily the insulation), the history of exposure to moisture, impulse

stress, mechanical stress, and many other factors. As those are neither available in typical situations (databases

often do not even associate failures with the age), nor is their impact well documented and understood, the model

that captures the essential behavior is, by necessity and for practical reasons, chosen to contain the most salient

features known to be the strong determinants of lifetime.”

5. Sensors on equipment capture long time series data such as current load on a feeder, power quality events and other

composite measurements of stress on the feeder. This results in creation of huge asynchronous time series databases

– aggregation, interpolation and mining of which provides formidable challenges.

4 Data Generation

Snapshots of the state of a feeder are taken at the time of the failure (Figure 2). For each feeder, the attributes comprise of:

(a) physical characteristics (such as number of cable sections, joints, installed shunts); these characteristics may undergo

!

Figure 3: Random Forests for modeling Time Between Failures (TBFs) of Electrical Feeders.

annual changes. (b) electrical characteristics from load ﬂow simulations (c) dynamic data from telemetry attached to

the feeder (such as power quality events, load forecasts, outage counts) and (d) derived attributed suggested by domain

experts. Since the number of feeders vary considerably in different boroughs in New York, the size of the data set on

which machine learning models are built is also different. There are approximately 900 instances in the data set for

Manhattan, 1300 for Brooklyn and Queens and 350 for Bronx. The training and validation data was collected from July,

2005 - December 2006 and blind-testing is done on data from January 2007 – February 2009.

5 Modeling Time Between Failures (TBFs)

We have applied Support Vector Machines (SVM) [16], CART (Classiﬁcation and Regression Trees) [5], ensemble based

techniques (Random Forests [4]) along with statistical methods, e.g. Cox Proportional Hazards [6], to the task of estimat-

ing TBF. In this paper, we present empirical results obtained from Random Forest (RF) based models (illustrated in Figure

3). To effectively model short time survivors1(feeders which have failures within 90 days of the last failure) and long time

survivors (feeders surviving greater than 365 days), we built classiﬁcation and regression trees for each class of survivors

(short survivors, one year survivors and long term survivors). Our hypothesis was that similar kinds of failures should be

modeled using a single regression model. The decision of how to partition the TBFs for building different models was

based on knowledge acquired from domain experts. However, for the infant mortal cases, i.e. whether to make a model

for feeders that failed within 10 days versus 20 days, we relied entirely on empirical analysis.

Once the Random Forest of trees was generated, the next problem was how to combine the regression models such

that when an unseen test example was presented to it, we could accurately predict the time between failures. The process

of combining models was tricky because different models predicted TBFs in different ranges and simply averaging results

was unreasonable; for instance say a test instance was passed through an RF model and regressors for each class were

allowed to come up with predictions; assume the short survivor model predicted a TBF of 2 days, the one year survivor

predicted 265 days and the long term survivor predicted 1089 days; an average yields 452 days which does not indicate

whether the feeder is generally one that is infant mortal or a longer survivor or neither. To avoid this problem, we tried

different mechanisms of combining models: (1) Weighted Averaging, where weights were the proportion of instances in

the training set that belong to a particular class of survivors (2) Build a decision tree on the training data to obtain class

labels – 0 indicating short survivor, 1 representing one year survivors and 2 representing long term survivors. Given a test

instance, the decision tree predicts which class it belongs to and then we use the corresponding tree to come up with a

prediction. (4) Clustering the training data using (a) K-Nearest Neighbor (b) K-Means and (c) KMeans++ each with three

different distance metrics – euclidean distance, L1 Norm and the cosine metric. We also investigated different seeding

mechanisms for initiating the clustering algorithms. “Seeding” a clustering algorithm is the problem of choosing the initial

cluster centers as input to an algorithm. For example, the canonical K-means algorithm, takes as input a set of k “seeds”

which are the ﬁrst set of candidate centers to the iterative algorithm. A recent, signiﬁcant advance in k-means clustering

seeding technology was made by Arthur and Vassilvitskii ([1]). Their algorithm, k-means++, is extremely light-weight and

simple, however has strong formal performance guarantees (the clustering, induced by the seeding alone, approximates

the optimum value of the k-means clustering objective by a factor of O(log k)). We implemented this procedure to seed

the clustering of TBFs, along the time dimension, for the purposes of ﬁnding good 3-clusterings. In the following section

1These are also referred to as instances suffering from “infant mortality” (IM)

RMSE: 163.86

Figure 4: Actual vs Predicted Time Between Failures on the Brooklyn and Queens Data using a Random Forest of

three trees and combination by k-Nearest Neighbors method.

we present empirical results for estimating TBFs in Brooklyn and Queens2.

6 Case Study: Estimating TBFs of Feeders in Brooklyn and Queens

The training data collected between July 2005 - July 2006 contains approximately 1100 instances. This data is used to

construct three different regression trees corresponding to short survivors, one year survivors and long term survivors.

Since this a real-world application and deﬁning what comprises a “short” survivor in non-trivial, we built models for

feeders that failed in 10 days, 20 days, · · · , 90 days3. When we built a 10 day short survivor model, the one year model

was built on all instances greater than 10 days and less than 365 days and the long term model contained instances that

survived longer than 365 days. These models were ﬁrst tested on a validation data collected between August 2006 -

December 2006. In addition to building the regression models on the training data, we also constructed machine learning

models (decision trees, clusters) for combining regression models as described in Section 5. The blind testing was done

on data collected between January 2007 – February 2009. The metric used to evaluate our models is the Root Mean

Square Error (RMSE)4. Table 1 shows the RMSE values of the Random Forest models with different model combination

techniques in Brooklyn and Queens. Our results indicate that the best model (lowest RMSE value) is obtained when trees

are built by splitting instances that fail within 80 days of the last failure as the short survivors, instances corresponding to

failures greater than 80 days and less than 365 days as one-year survivors and those greater than 365 days as long term

survivors and combining them using weighted averaging. However, the predictions of such a model tend to be restricted

to within a year of the last failure. To avoid this and allow a larger range of predictability, we use the k-Nearest Neighbor

model with euclidean distance measure. Figure 4 shows the plot of actual TBF versus predicted TBF for this model. Our

results indicate that we are able to predict Time Between Failures within approximately 6 months from the last failure.

Future work includes incorporation of seasonal trends in the model and use of other machine learning techniques to built

more robust random forests.

Acknowledgements

This work is supported by the Consolidated Edison Company of New York.

References

[1] David Arthur and Sergei Vassilvitskii. k-means++: the advantages of careful seeding. In SODA ’07: Proceedings

of the eighteenth annual ACM-SIAM symposium on Discrete algorithms, pages 1027–1035, Philadelphia, PA, USA,

2007. Society for Industrial and Applied Mathematics.

2The results from other boroughs are left out for the sake of brevity.

3This was suggested by domain experts.

4The Mean Squared Error (MSE) measures the average of the square of the “error” where error is the amount by which the true value differs from

the quantity estimated. The square root of the MSE yeilds RMSE which is the metric used to evaluate our models.

Combination Method 10d 20d 30d 40d 50d 60d 70d 80d 90d

Averaging 196.35 191.46 193.32 196.45 210.09 201.8 217.07 209.74 218.13

Wt Avg 146.49 133.07 131.85 129.37 139.23 127.19 137.07 126.55 131.78

Decision Tree 251.87 312.26 263.86 268.96 314.87 266.99 276.1 270.86 263.27

k-NN Euclidean 190.7 163.86 168.15 172.42 201.35 181.55 193.43 179.29 183.39

k-NN L1 Norm 187.14 170.26 176.33 179.57 195.83 175.45 192 183.13 184.05

k-NN Cosine 195.37 191.71 190.85 197.88 215.6 204.34 217.29 201.46 202.82

k-means Euclidean 244.51 243.75 243.96 239.29 249.93 247.28 251.87 245.39 257.83

k-means L1 Norm 274.47 271.81 269.81 269.45 276.72 322.58 327.6 322.53 282.59

k-means Cosine 269.06 265.2 267.62 259.67 280.36 274.83 281.28 272.49 255.98

k-means++ Euclidean 244.51 242.32 243.31 239.29 249.93 243.05 251.13 245.23 246.76

k-means++ L1 Norm 195 185.56 183.95 183.03 201.84 194.05 202.73 189 203.88

k-means++ Cosine 230.26 223.75 224.19 226.06 228 225.37 227.59 222.25 226.84

Table 1: RMSE values for Random Forest models built on Brooklyn and Queens data.

[2] Hila Becker and Marta Arias. Real-time ranking with concept drift using expert advice. In Proceedings of the 13th

ACM SIGKDD international conference on Knowledge Discovery and Data Mining (KDD ’07), pages 86–94, New

York, NY, USA, 2007. ACM.

[3] M. Begovic, P. Djuric, J. Perkel, B. Vidakovic, and D. Novosel. New probabilistic method for estimation of equip-

ment failures and development of replacement strategies. Hawaii International Conference on System Sciences,

10:246a, 2006.

[4] Leo Breiman. Random forests. In Machine Learning, volume 45, pages 5–32, 2001.

[5] Leo Breiman, Jerome Friedman, Charles Stone, and R Olshen. Classiﬁcation and Regression Trees. CRC Press,

Boca Raton, Florida, USA., 1998.

[6] D. R. Cox. Regression models and life tables. In J. Roy Statis Soc B, volume 34, pages 187 – 220, 1972.

[7] Charles E Ebeling. An Introduction to Reliability and Maintainability Engineering. McGraw-Hill Companies, Inc.,

Boston, USA., 1997.

[8] P. Gross, A. Boulanger, M. Arias, D. L. Waltz, P. M. Long, C. Lawson, R. Anderson, M. Koenig, M. Mastrocinque,

W. Fairechio, J. A. Johnson, S. Lee, F. Doherty, and A. Kressner. Predicting electricity distribution feeder failures

using machine learning susceptibility analysis. In The Eighteenth Conference on Innovative Applications of Artiﬁcial

Intelligence IAAI-06, Boston, Massachusetts, 2006.

[9] Phil Gross, Ansaf Salleb-Aouissi, Haimonti Dutta, and Albert Boulanger. Ranking electrical feeders of the new york

power grid. In 3rd Annual Machine Learning Symposium at the New York Academy of Sciences (NYAS), New York,

NY, October 2008.

[10] W Zhao H Guo and A Mettas. Practical methods for modeling repairable systems with time trends and repair effects.

Proceedings of Annual Reliability and Maintainability Symposium, 2006.

[11] K. C. Kapur and L. R. Lamberson. Reliability in Engineering Design. John Wiley and Sons, New York, USA., 1977.

[12] T. F. Lawless and K. Thiagarajah. A point-process model incorporating renewals and time trends with applications

to repairable systems. Technometrics, 38(2), 1996.

[13] S. P. Lloyd. Least squares quantization in pcm. IEEE Transactions on Information Theory, 28(2):129–137, 1982.

[14] Philip M. Long and Rocco A. Servedio. Martingale boosting. In Learning Theory: 18th Annual Conference on

Learning Theory, COLT 2005, Bertinoro, Italy, June 27-30, 2005, Proceedings, volume 3559 of Lecture Notes in

Artiﬁcial Intelligence, pages 79–94. Springer, 2005.

[15] M. J. Mondro. Approximation of mean time between failure when a system has periodic maintenance. IEEE

Transactions on Reliability, 51(2), 2002.

[16] V. N. Vapnik. The nature of statistical learning theory. Springer-Verlag New York, Inc., New York, NY, USA, 1995.