Conference Paper

A Fundamental Trade-Off between the Download Cost and Repair Bandwidth in Distributed Storage Systems

Dept. of Electr. Eng., Shahed Univ., Tehran, Iran
DOI: 10.1109/NETCOD.2010.5487685 Conference: Network Coding (NetCod), 2010 IEEE International Symposium on
Source: IEEE Xplore


Distributed storage systems are mainly justified due to the limited amount of storage capacity and improving the reliability through distributing data over multiple storage nodes. However, it may happen the data is stored in unreliable nodes, while it is desirable the end user to have a reliable access to the stored data. So, in an event that a node is damaged, to prevent the system reliability to regress, it is necessary to regenerate a new node with the same amount of stored data as the damaged node to retain the number of storage nodes, thereby having the previous reliability. This requires the new node to connect to some of existing nodes, and downloads the required information, thereby occupying some bandwidth, called the repair bandwidth. On the other hand, it is more likely the cost of downloading varies across different nodes. This paper aims at investigating the fundamental trade-off between the download cost and repair bandwidth, and more importantly, it is shown any point on this curve can be achieved through the use of the so called generalized regenerating codes which is an enhancement to the regenerating codes introduced by Dimakis et al. in.

23 Reads
  • [Show abstract] [Hide abstract]
    ABSTRACT: Distributed storage systems aim at providing a reliable storage over unreliable nodes thorough introducing redundancy. In these systems when a node is failed to retain the previous reliability a newcomer is connected to existing nodes and downloads the same amount of information as the damage node. Thus, a great deal of data transferring, called the repair bandwidth, is imposed to the system. Recently, regenerating codes are introduced to reduce the repair bandwidth through using the notion of network coding. Furthermore, in the generalized regenerating codes are proposed which is an extension of regenerating codes for the case of having different download cost from surviving nodes. The current paper provides a real world example to explain the main difference between regenerating codes and generalized regenerating codes.
    No preview · Article · Nov 2010
  • [Show abstract] [Hide abstract]
    ABSTRACT: Regenerating codes are mainly justified due to their ability to reduce the repair bandwidth incurred by a newcomer node. This happens when a node fails or leaves the network, thus a new node is initiated, attempting to connect to existing nodes to reconstruct the data. This paper aims to investigate the case in which the newcomer can wisely select some of existing nodes to connect to, so as to reduce the repair bandwidth. Accordingly, selective regenerating codes are proposed, showing the corresponding repair bandwidth is dramatically reduced as compared to that of existing codes.
    No preview · Article · Aug 2011 · IEEE Communications Letters
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Erasure codes are an efficient means of storing data across a network in comparison to data replication, as they tend to reduce the amount of data stored in the network and offer increased resilience in the presence of node failures. The codes perform poorly though, when repair of a failed node is called for, as they typically require the entire file to be downloaded to repair a failed node. A new class of erasure codes, termed as regenerating codes were recently introduced, that do much better in this respect. However, given the variety of efficient erasure codes available in the literature, there is considerable interest in the construction of coding schemes that would enable traditional erasure codes to be used, while retaining the feature that only a fraction of the data need be downloaded for node repair. In this paper, we present a simple, yet powerful, framework that does precisely this. Under this framework, the nodes are partitioned into two types and encoded using two codes in a manner that reduces the problem of node-repair to that of erasure-decoding of the constituent codes. Depending upon the choice of the two codes, the framework can be used to avail one or more of the following advantages: simultaneous minimization of storage space and repair-bandwidth, low complexity of operation, fewer disk reads at helper nodes during repair, and error detection and correction.
    Preview · Conference Paper · Sep 2011
Show more