Figure 3 - uploaded by Anjus George
Content may be subject to copyright.
Using FID to access a file's Layout EA and corresponding OST objects

Using FID to access a file's Layout EA and corresponding OST objects

Contexts in source publication

Context 1
... Layout EA acts as a map for the client to determine where the file data is actually stored, and it contains a list of the OSTs as well as the FIDs for the objects on those OSTs that hold the actual file data. Figure 3 shows an example of accessing a file with a normal layout of stripe count 3. ...
Context 2
... leads to the invocation of device specific setup routines from various subsystems such as mgc_setup(), lwp_setup(), osc_setup_common() and so on. All of these setup routines invoke a client_obd_setup() routine that acts as a pre-setup stage before the creation of imports for the clients as shown in Figure 13. The client_obd_setup() (see ldlm/ldlm_lib.c) function populates client_obd structure defined in include/obd.h ...

Citations

... A well chosen default file system setup can ensure that newly created files get written to the performance tier and utilize that tier's superior performance. For instance, Lustre [2] allows storage with different characteristics to be configured into separate OST pools while allowing user files to be visible under as a single unified namespace. However, to distribute files to the appropriate tiers, users must make sure that the file's striping layout is set to use the correct pools. ...
... The scan agent in QuickSilver uses the lfs find [4] search utility implemented in Lustre [2] to gather metadata information on all files. Although the current implementation of QuickSilver relies on the NATS messaging system and lfs find, the code uses abstractions that provide flexibility for incorporating any messaging framework and file system search utility. ...
Preprint
Full-text available
Large scale parallel file systems with multiple storage tiers require policy driven data management to facilitate efficient storage and access of large-scale data. However, management of data across the tiers is challenging due to the massive scale of data being stored. In this talk, we present our initial work on QuickSilver, a lightweight flexible distributed policy engine. QuickSilver is composed of many single-purpose agents that handle tasks such as gathering file metadata, enforcing policy decisions, and executing policy actions like purging or data migration. These agents are designed to communicate using distributed message queues, allowing the number of individual agents to be scaled up as needed. Additionally, QuickSilver is designed to function while maintaining minimal state information. We will discuss the architectural details of the policy engine and its use of message queues to enable scaling. Examples of the initial implementation will be shown along with preliminary performance numbers. Since this project is in its infancy, we will also discuss our plans for future work and areas of improvement.
... • Several resources exist to help deploy and configure Lustre Figure 4: Normal RAID0 file striping in Lustre [1] Overview of the Topics Documented Figure 6: Lustre I/O operation: Lustre client requesting file data [1] Overview of the Figure 11: Communication between ost and mdt server obd devices in Lustre [1] Overview of the Overview of the ...
... • Several resources exist to help deploy and configure Lustre Figure 4: Normal RAID0 file striping in Lustre [1] Overview of the Topics Documented Figure 6: Lustre I/O operation: Lustre client requesting file data [1] Overview of the Figure 11: Communication between ost and mdt server obd devices in Lustre [1] Overview of the Overview of the ...
... • Several resources exist to help deploy and configure Lustre Figure 4: Normal RAID0 file striping in Lustre [1] Overview of the Topics Documented Figure 6: Lustre I/O operation: Lustre client requesting file data [1] Overview of the Figure 11: Communication between ost and mdt server obd devices in Lustre [1] Overview of the Overview of the ...