Conference Paper

How to identify and estimate the largest traffic matrix elements in a dynamic environment.

DOI: 10.1145/1005686.1005698 Conference: Proceedings of the International Conference on Measurements and Modeling of Computer Systems, SIGMETRICS 2004, June 10-14, 2004, New York, NY, USA
Source: DBLP

ABSTRACT In this paper we investigate a new idea for traffic matrix estimation that makes the basic problem less under-constrained, by deliberately changing the routing to obtain additional measurements. Because all these measurements are collected over disparate time intervals, we need to establish models for each Origin-Destination (OD) pair to capture the complex behaviours of internet traffic. We model each OD pair with two components: the diurnal pattern and the fluctuation process. We provide models that incorporate the two components above, to estimate both the first and second order moments of traffic matrices. We do this for both stationary and cyclo-stationary traffic scenarios. We formalize the problem of estimating the second order moment in a way that is completely independent from the first order moment. Moreover, we can estimate the second order moment without needing any routing changes (i.e., without explicit changes to IGP link weights). We prove for the first time, that such a result holds for any realistic topology under the assumption of . We highlight how the second order moment helps the identification of the top largest OD flows carrying the most significant fraction of network traffic. We then propose a refined methodology consisting of using our variance estimator (without routing changes) to identify the top largest flows, and estimate only these flows. The benefit of this method is that it dramatically reduces the number of routing changes needed. We validate the effectiveness of our methodology and the intuitions behind it by using real aggregated sampled netflow data collected from a commercial Tier-1 backbone.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Confronted with the generalization of monitoring in operational networks, researchers have proposed placement algorithms that can help ISPs deploy their monitoring infrastructure in a cost effective way, while maximizing the benefits of their infrastructure. However, a static placement of monitors cannot be optimal given the short-term and long-term variations in traffic due to re-routing events, anomalies and the normal network evolution. In addition, most ISPs already deploy router embedded monitoring functionalities. Despite some limitations (inherent to being part of a router), these monitoring tools give greater visibility on the network traffic but raise the question on how to configure a network-wide monitoring infrastructure that may contain hundreds of monitoring points. We reformulate the placement problem as follows. Given a network where all links can be monitored, which monitors should be activated and which sampling rate should be set on these monitors in order to achieve a given measurement task with high accuracy and low resource consumption? We provide a formulation of the problem, an optimal algorithm to solve it, and we study its performance on a real backbone network.
    Proceedings of the 2006 ACM Conference on Emerging Network Experiment and Technology, CoNEXT 2006, Lisboa, Portugal, December 4-7, 2006; 01/2006
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: In this paper we propose a new approach for dealing with the ill-posed nature of traffic matrix estimation. We present three solution enhancers: an algorithm for deliberately changing link weights to obtain additional information that can make the underlying linear system full rank; a cyclo-stationary model to capture both long-term and short-term traffic variability, and a method for estimating the variance of origin-destination (OD) flows. We show how these three elements can be combined into a comprehensive traffic matrix estimation procedure that dramatically reduces the errors compared to existing methods. We demonstrate that our variance estimates can be used to identify the elephant OD flows, and we thus propose a variant of our algorithm that addresses the problem of estimating only the heavy flows in a traffic matrix. One of our key findings is that by focusing only on heavy flows, we can simplify the measurement and estimation procedure so as to render it more practical. Although there is a tradeoff between practicality and accuracy, we find that increasing the rank is so helpful that we can nevertheless keep the average errors consistently below the 10% carrier target error rate. We validate the effectiveness of our methodology and the intuition behind it using commercial traffic matrix data from Sprint's Tier-1 backbone.
    IEEE/ACM Transactions on Networking 07/2007; 15(3):485-498. · 2.01 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Dans ce travails nous développons une nouvelle approche pour l'estimation de matrices de trafic. Nous construisons un modèle d'états pour représenter les flots de paires Origine Destination d'un grand réseau d'opérateur. Ce modèle est suffisamment riche pour capturer les corrélations spatio-temporelles des flots. Nous appliquons ensuite un filtre de Kalman a ce système linéaire dynamique. Cette approche diffère des approches précédentes car tandis que les approches précédentes étaient focalisées uniquement sur l'ingénierie de trafic, nous nous concentrerons sur la détection d'anomalies. Notre modèle est suffisamment riche pour définir le comportement normal d'un flot et ainsi détecter les comportements anormaux. De plus la simplicité du modèle permet d'effectuer les calculs en temps réel. Nous avons ensuite validé cette méthode à l'aide de traces d'un grand fournisseur d'accès américain.
    20° Colloque sur le traitement du signal et des images, 2005 ; p. 204-205. 01/2005;

Full-text (2 Sources)

Available from
May 19, 2014