MHDFS: A Memory-Based Hadoop Framework for Large Data Storage
Joint Authors
Song, Aibo
Zhao, Maoxian
Xue, Yingying
Luo, Junzhou
Source
Issue
Vol. 2016, Issue 2016 (31 Dec. 2016), pp.1-12, 12 p.
Publisher
Hindawi Publishing Corporation
Publication Date
2016-05-09
Country of Publication
Egypt
No. of Pages
12
Main Subjects
Abstract EN
Hadoop distributed file system (HDFS) is undoubtedly the most popular framework for storing and processing large amount of data on clusters of machines.
Although a plethora of practices have been proposed for improving the processing efficiency and resource utilization, traditional HDFS still suffers from the overhead of disk-based low throughput and I/O rate.
In this paper, we attempt to address this problem by developing a memory-based Hadoop framework called MHDFS.
Firstly, a strategy for allocating and configuring reasonable memory resources for MHDFS is designed and RAMFS is utilized to develop the framework.
Then, we propose a new method to handle the data replacement to disk when memory resource is excessively occupied.
An algorithm for estimating and updating the replacement is designed based on the metrics of file heat.
Finally, substantial experiments are conducted which demonstrate the effectiveness of MHDFS and its advantage against conventional HDFS.
American Psychological Association (APA)
Song, Aibo& Zhao, Maoxian& Xue, Yingying& Luo, Junzhou. 2016. MHDFS: A Memory-Based Hadoop Framework for Large Data Storage. Scientific Programming،Vol. 2016, no. 2016, pp.1-12.
https://search.emarefa.net/detail/BIM-1118148
Modern Language Association (MLA)
Song, Aibo…[et al.]. MHDFS: A Memory-Based Hadoop Framework for Large Data Storage. Scientific Programming No. 2016 (2016), pp.1-12.
https://search.emarefa.net/detail/BIM-1118148
American Medical Association (AMA)
Song, Aibo& Zhao, Maoxian& Xue, Yingying& Luo, Junzhou. MHDFS: A Memory-Based Hadoop Framework for Large Data Storage. Scientific Programming. 2016. Vol. 2016, no. 2016, pp.1-12.
https://search.emarefa.net/detail/BIM-1118148
Data Type
Journal Articles
Language
English
Notes
Includes bibliographical references
Record ID
BIM-1118148