Efficient Metadata Management for Cloud Computing applications

Abstract

Cloud computing applications require a scalable, elastic and fault tolerant storage system. In this paper, we describe how metadata management can be improved for a file system built for large scale data-intensive applications. We imple- ment Ring File System (RFS), that uses a single hop Dis- tributed Hash Table, found in peer-to-peer systems, to man- age its metadata and a traditional client server model for managing the actual data. Our solution does not have a single point of failure, since the metadata is replicated and the num- ber of files that can be stored and the throughput of meta- data operations scales linearly with the number of servers. We compare against two open source implementations of Google File System (GFS): HDFS and KFS and show that our prototype performs better in terms of fault tolerance, scalability and throughput.