Digital Library[ Search Result ]
Performance Analysis of CERN EOS Distributed File System under Bare-metal and Virtualization Environments
Jun-Yeong Lee, Moon-Hyun Kim, Kyeong-Jun Kim, Seo-Young Noh
http://doi.org/10.5626/JOK.2022.49.3.189
To store large amounts of data, the distributed file system has been used in many research facilities and large-scale data centers. Traditional distributed file systems were configured by installing a distributed file system which is referred to as “bare-metal”, directly on server. Recently, with easy management and fast failover capabilities, these systems have been configured and delivered through a virtual environment. In this paper, we analyzed the EOS distributed file system developed and used by CERN(Conseil Européen pour la Recherche Nucléaire), which produces the largest amount of experimental data in the world And using both Bare-Metal environment and KVM(Kernel-based Virtual Machine)-based virtual environment, we analyzed the file system performance of these two environments. We compared the performances and analyzed the different environmental characteristics and presented the advantages of the I/O performance of the distributed file system in the virtual environment from our experimental results.
Streaming Compression Scheme for Reducing Network Resource Usage in Hadoop System
http://doi.org/10.5626/JOK.2018.45.6.516
Recently, the Hadoop system has become one of the most popular large-scale distributed systems used in enterprises, and the amount of data on the system has been increasing continually. As the amount of data in the Hadoop system is increased, the scale of Hadoop clusters is also growing. Resources in a node, such as processor, memory, and storage, are isolated from other nodes, and hence, even though resource usage is increased by data processing requests from clients, it doesn’t affect the performance of other nodes. However, all the nodes in a Hadoop cluster are connected to the network resource, a shared resource in the Hadoop cluster, and so, if some nodes dominate the network resource, other nodes would experience less network resources, which could cause overall performance degradation in the Hadoop system. In this paper, we propose a streaming compression scheme that can decrease the network traffic generated by write operations in the system. We also evaluate the performance of our streaming compression scheme and analyze the overhead of the proposed scheme. Our experimental results with a real-world workload show that our proposed scheme decreases the network traffic in a Hadoop cluster by 56% over the existing HDFS systems.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr