Conference Paper

Experiences with Hierarchical Storage Management Support in Blue Whale File System.

DOI: 10.1109/PDCAT.2010.35 Conference: 2010 International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2010, Wuhan, China, 8-11 December, 2010
Source: DBLP

ABSTRACT Abstract-In order to meet the challenges of significant storage and application growth, as well as shortened backup windows and limited IT resources, more and more organizations embrace Hierarchical Storage Management (HSM). Parts of SAN file systems provide the ability to support HSM, but they didn't provide more details in design considerations and how to implement. In this paper, we present design considerations in Blue Whale File System (BWFS) which is a high performance SAN file system and implement the system. Event, migration and attribute constitute the three aspects of data management mechanism in BWFS. As for READ/WRITE/TRUNCATE event, the problem is that a client directly accesses to storage devices containing file data, rather than through a metadata server. We monitor LayoutGet operation instead of real I/O path. BWFS is a multi-volume file system which can support single name space across multiple data volumes. When different storage devices are managed by BWFS, migration is between volumes in BWFS. We also implement migration between BWFS and external repository. We believe the solutions to the problem we encountered are useful to file system developers.

0 Bookmarks
 · 
56 Views
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: In this paper we present the analysis of two large-scale network file system workloads. We measured CIFS traf- fic for two enterprise-class file servers deployed in the NetApp data center for a three month period. One file server was used by marketing, sales, and finance de- partments and the other by the engineering department. Together these systems represent over 22TB of storage used by over 1500 employees, making this the first ever large-scale study of the CIFS protocol. We analyzed how our network file system workloads compared to those of previous file system trace studies and took an in-depth look at access, usage, and sharing patterns. We found that our workloads were quite dif- ferent from those previously studied; for example, our analysis found increased read-write file access patterns, decreasedread-writeratios, more randomfile access, and longer file lifetimes. In addition, we found a number of interesting properties regarding file sharing, file re-use, and the access patterns of file types and users, showing that modernfile system workload has changed in the past 5-10 years. This change in workload characteristics has implicationson the futuredesignof networkfile systems, which we describe in the paper.
    2008 USENIX Annual Technical Conference, Boston, MA, USA, June 22-27, 2008. Proceedings; 01/2008
  • Journal of Computer Research and Development 01/2005; 42(6).