Metadata Workloads for Testing Big Storage Systems

Abad, Cristina L.; Luu, Huong; Lu, Yi; Campbell, Roy H.

Permalink

https://hdl.handle.net/2142/30013 Copy

Description

Title

Metadata Workloads for Testing Big Storage Systems

Author(s)

• Abad, Cristina L.
• Luu, Huong
• Lu, Yi
• Campbell, Roy H.

Issue Date

2012-03-12

Keyword(s)

• metadata
• workload
• storage
• hdfs
• trace
• hadoop
• petascale
• distributed file system

Abstract

Efficient namespace metadata management is becoming more important as next-generation file systems are designed for the peta and exascale era. A number of new metadata management schemes have been proposed. However, evaluation of these designs has been insufficient, mainly due to a lack of appropriate namespace metadata traces. Specifically, no Big Data storage system metadata trace is publicly available, and existing traces are a poor replacement. We studied publicly available traces and one Big Data trace from Yahoo! and note some of the differences and their implications to metadata management studies. We discuss the insufficiency of existing evaluation approaches and present a first step towards a statistical metadata workload model that can capture the relevant characteristics of a workload and is suitable for synthetic workload generation.

Type of Resource

text

Language

en

Permalink

http://hdl.handle.net/2142/30013

Sponsor(s)/Grant Number(s)

This paper is based on research sponsored by the Air Force Research Laboratory and the Air Force Office of Scientific Research, under agreement FA8750-11-2-0084.

Copyright and License Information

The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon.

Owning Collections