Digital Library[ Search Result ]
Managing DISCARD Commands in F2FS File System for Improving Lifespan and Performance of SSD Devices
Jinwoong Kim, Donghyun Kang, Young Ik Eom
http://doi.org/10.5626/JOK.2024.51.8.669
The DISCARD command is an interface that helps improve the lifespan and performance of SSDs by informing the SSD devices about invalid file system blocks. However, in the F2FS file system, the DISCARD command is only sent to the SSD during idle time, which limits the potential for improving lifespan and performance. In this paper, we propose an EPD scheme to efficiently transfer DISCARD commands during short idle times, as well as a seg-ment allocation scheme called PSA, which replaces DISCARD commands with overwrite commands. To evaluate the effectiveness of these proposed schemes, we conducted several experiments using various workloads to verify the lifespan and performance of real SSD devices. The results showed that the proposed schemes can improve the write amplification factor (WAF) by up to 40% and throughput by up to 160%, when compared to the traditional F2FS file system.
Running IO500 Benchmark and Applying Optimal Parameters on Lustre-based Storage Systems
Hyosil Kim, Byoungjun Seo, Sejin Hwang, Juyeun Han
http://doi.org/10.5626/JOK.2024.51.6.483
In this paper, a comparative study is conducted by executing IO500 benchmarks with various tuning factors in the Lustre-based storage environment, where Lustre is a distributed parallel file system. Contributions of this paper are as follows: 1) enhancing user understanding of IO500 benchmarks and Lustre file system, and 2) establishing a large-scale Lustre environment to analyze factors that can enhance IO500 performance from three perspectives, and 3) presenting results of applied enhancements through experimentation. Experiments were conducted concerning Data-on-MDT (DoM), MPI process count (NP), and MPI CPU affinity parameters. We demonstrated that the performance was improved when DoM was configured, an appropriate number of MPI processes was set, and the '--bind-to core-overload-allowed' option was employed for CPU affinity.
Performance Analysis of CERN EOS Distributed File System under Bare-metal and Virtualization Environments
Jun-Yeong Lee, Moon-Hyun Kim, Kyeong-Jun Kim, Seo-Young Noh
http://doi.org/10.5626/JOK.2022.49.3.189
To store large amounts of data, the distributed file system has been used in many research facilities and large-scale data centers. Traditional distributed file systems were configured by installing a distributed file system which is referred to as “bare-metal”, directly on server. Recently, with easy management and fast failover capabilities, these systems have been configured and delivered through a virtual environment. In this paper, we analyzed the EOS distributed file system developed and used by CERN(Conseil Européen pour la Recherche Nucléaire), which produces the largest amount of experimental data in the world And using both Bare-Metal environment and KVM(Kernel-based Virtual Machine)-based virtual environment, we analyzed the file system performance of these two environments. We compared the performances and analyzed the different environmental characteristics and presented the advantages of the I/O performance of the distributed file system in the virtual environment from our experimental results.
Improving the Performance and Usability of Server Systems through Dynamic Use of Persistent Memory
http://doi.org/10.5626/JOK.2021.48.10.1077
Persistent Memory (PM) has been studied assuming that it can only be used as main memory, storage, or storage cache. In this paper, we present HMMS (Hybrid Memory Management System), which is a memory management system that allows PM to play all these roles, not exclusively, but simultaneously. Specifically, HMMS dynamically and appropriately allocates these three roles by reflecting the state of the computing system and the users’ requests. With HMMS, we aim to improve the functionality and performance of computing systems where DRAM and PM coexist.
A Digital Forensic Process for Ext4 File System in the Flash Memory of IoT Devices
Junho Jeong, Beomseok Kim, Jinsung Cho
http://doi.org/10.5626/JOK.2021.48.8.865
With the recent rapid advances in digital communication technology, the spread of IoT(Internet of Things) has accelerated and IoT devices can be utilized to investigate crimes and accidents due to the close connection between human society and IoT devices. Accordingly, with the increasing importance of digital forensics, numerous studies have been conducted. However, most digital forensics research proposed only abstract methodologies due to the various types of IoT devices. In addition, binwalk, which is actively used as a firmware analysis tool, does not adequately analyze and extract the ext4 file system. To solve these problems, this paper proposes a proper extraction and analysis method and a practical process that could extract the ext4 file system from the flash memory of IoT devices using the binwalk with the proposed method. This study also verifies the proposed process with DJI Phantom 4 Pro V2.0 drone.
A Compression-based Data Consistency Mechanism for File Systems
Dong Hyun Kang, Sang-Won Lee, Young Ik Eom
http://doi.org/10.5626/JOK.2019.46.9.885
Data consistency mechanism is a crucial component in any file system; the mechanism prevents the corruption of data from system crashes or power failures. For the sake of performance, the default journal mode of the Ext4 file system guarantees only the consistency of metadata while compromising with the consistency of normal data. Specially, it does not guarantee full consistency of the whole data of the file system. In this paper, we propose a new crash consistency scheme which guarantees strong data consistency of the data journal mode by still providing higher or comparable performance to the weak default journal mode of the Ext4 file system. By leveraging a compression mechanism, the proposed scheme can halve the amount of write operations as well as the number of fsync() system calls. For evaluation of the performance, we modified the codes related to the jbd2 and compared the proposed scheme with two journaling modes in Ext4 on SSD and HDD. The results clearly confirm that the proposed scheme outperforms the default journal mode by 8.3x times.
An Efficient SLC-buffer Management Scheme for TLC NAND Flash-based Storage
Kirock Kwon, Dong Hyun Kang, Young Ik Eom
http://doi.org/10.5626/JOK.2018.45.7.611
In recent years, almost all consumer devices have adopted NAND flash storage as their main storage, and their performance and capacity requirements are getting higher. To meet these requirements, many researchers have focused on combined SLC-TLC storage consisting of high-speed SLC and high-density TLC. In this paper, we redesign the internal structure of the combined SLC-TLC storage to efficiently manage the SLC region inside the storage and propose a scheme that improves the performance of the storage by employing the I/O characteristics of file system journaling. We implemented our scheme on the real storage platform, the OpenSSD jasmine board, and compared it with the conventional techniques. Our evaluation results show that our technique improves the storage performance by up to 65%, compared with the conventional techniques.
Streaming Compression Scheme for Reducing Network Resource Usage in Hadoop System
http://doi.org/10.5626/JOK.2018.45.6.516
Recently, the Hadoop system has become one of the most popular large-scale distributed systems used in enterprises, and the amount of data on the system has been increasing continually. As the amount of data in the Hadoop system is increased, the scale of Hadoop clusters is also growing. Resources in a node, such as processor, memory, and storage, are isolated from other nodes, and hence, even though resource usage is increased by data processing requests from clients, it doesn’t affect the performance of other nodes. However, all the nodes in a Hadoop cluster are connected to the network resource, a shared resource in the Hadoop cluster, and so, if some nodes dominate the network resource, other nodes would experience less network resources, which could cause overall performance degradation in the Hadoop system. In this paper, we propose a streaming compression scheme that can decrease the network traffic generated by write operations in the system. We also evaluate the performance of our streaming compression scheme and analyze the overhead of the proposed scheme. Our experimental results with a real-world workload show that our proposed scheme decreases the network traffic in a Hadoop cluster by 56% over the existing HDFS systems.
Design and Implementation of an In-Memory File System Cache with Selective Compression
http://doi.org/10.5626/JOK.2017.44.7.658
The demand for large-scale storage systems has continued to grow due to the emergence of multimedia, social-network, and big-data services. In order to improve the response time and reduce the load of such large-scale storage systems, DRAM-based in-memory cache systems are becoming popular. However, the high cost of DRAM severely restricts their capacity. While the method of compressing cache entries has been proposed to deal with the capacity limitation issue, compression and decompression, which are technically difficult to parallelize, induce significant processing overhead and in turn retard the response time. A selective compression scheme is proposed in this paper for in-memory file system caches that rapidly estimates the compression ratio of incoming cache entries with their Shannon entropies and compresses cache entries with low compression ratio. In addition, a description is provided of the design and implementation of an in-kernel in-memory file system cache with the proposed selective compression scheme. The evaluation showed that the proposed scheme reduced the execution time of benchmarks by approximately 18% in comparison to the conventional non-compressing in-memory cache scheme. It also provided a cache hit ratio similar to the all-compressing counterpart and reduced 7.5% of the execution time by reducing the compression overhead. In addition, it was shown that the selective compression scheme can reduce the CPU time used for compression by 28% compared to the case of the all-compressing scheme.
Consideration of fsync() of the Ext4 File System According to Kernel Version
Seongbae Son, Yoenjin Noh, Dokeun Lee, Sungsoon Park, Youjip Won
Ext4 file system is widely used in various computing environments such as those of the PC, the server, and the Linux-based embedded system. Ext4, which uses a buffer for block I/O, provides fsync() system call to applications to guarantee the consistency of a specific file. A log of the analytical studies regarding the operation of Ext4 and the improvement of its performance has been compiled, but it has not been studied in detail in terms of kernel versions. We figure out that the behavior of fsync() system call is different depending on the kernel version. Between the kernel versions of 3.4.0 and 4.7.2, 3.4.0, 3.8.0, and 4.6.2 showed behavioral differences regarding the fsync() system call. The latency of fsync() in kernel 3.4.0 is longer than that of the more-advanced 3.7.10; meanwhile, the characteristics of 3.8.0 enabled the disruption of the Ext4 journaling order, but the ordered defect was solved with 4.6.2.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr