Digital Library[ Search Result ]
Managing DISCARD Commands in F2FS File System for Improving Lifespan and Performance of SSD Devices
Jinwoong Kim, Donghyun Kang, Young Ik Eom
http://doi.org/10.5626/JOK.2024.51.8.669
The DISCARD command is an interface that helps improve the lifespan and performance of SSDs by informing the SSD devices about invalid file system blocks. However, in the F2FS file system, the DISCARD command is only sent to the SSD during idle time, which limits the potential for improving lifespan and performance. In this paper, we propose an EPD scheme to efficiently transfer DISCARD commands during short idle times, as well as a seg-ment allocation scheme called PSA, which replaces DISCARD commands with overwrite commands. To evaluate the effectiveness of these proposed schemes, we conducted several experiments using various workloads to verify the lifespan and performance of real SSD devices. The results showed that the proposed schemes can improve the write amplification factor (WAF) by up to 40% and throughput by up to 160%, when compared to the traditional F2FS file system.
Running IO500 Benchmark and Applying Optimal Parameters on Lustre-based Storage Systems
Hyosil Kim, Byoungjun Seo, Sejin Hwang, Juyeun Han
http://doi.org/10.5626/JOK.2024.51.6.483
In this paper, a comparative study is conducted by executing IO500 benchmarks with various tuning factors in the Lustre-based storage environment, where Lustre is a distributed parallel file system. Contributions of this paper are as follows: 1) enhancing user understanding of IO500 benchmarks and Lustre file system, and 2) establishing a large-scale Lustre environment to analyze factors that can enhance IO500 performance from three perspectives, and 3) presenting results of applied enhancements through experimentation. Experiments were conducted concerning Data-on-MDT (DoM), MPI process count (NP), and MPI CPU affinity parameters. We demonstrated that the performance was improved when DoM was configured, an appropriate number of MPI processes was set, and the '--bind-to core-overload-allowed' option was employed for CPU affinity.
Performance Analysis of CERN EOS Distributed File System under Bare-metal and Virtualization Environments
Jun-Yeong Lee, Moon-Hyun Kim, Kyeong-Jun Kim, Seo-Young Noh
http://doi.org/10.5626/JOK.2022.49.3.189
To store large amounts of data, the distributed file system has been used in many research facilities and large-scale data centers. Traditional distributed file systems were configured by installing a distributed file system which is referred to as “bare-metal”, directly on server. Recently, with easy management and fast failover capabilities, these systems have been configured and delivered through a virtual environment. In this paper, we analyzed the EOS distributed file system developed and used by CERN(Conseil Européen pour la Recherche Nucléaire), which produces the largest amount of experimental data in the world And using both Bare-Metal environment and KVM(Kernel-based Virtual Machine)-based virtual environment, we analyzed the file system performance of these two environments. We compared the performances and analyzed the different environmental characteristics and presented the advantages of the I/O performance of the distributed file system in the virtual environment from our experimental results.
Improving the Performance and Usability of Server Systems through Dynamic Use of Persistent Memory
http://doi.org/10.5626/JOK.2021.48.10.1077
Persistent Memory (PM) has been studied assuming that it can only be used as main memory, storage, or storage cache. In this paper, we present HMMS (Hybrid Memory Management System), which is a memory management system that allows PM to play all these roles, not exclusively, but simultaneously. Specifically, HMMS dynamically and appropriately allocates these three roles by reflecting the state of the computing system and the users’ requests. With HMMS, we aim to improve the functionality and performance of computing systems where DRAM and PM coexist.
A Digital Forensic Process for Ext4 File System in the Flash Memory of IoT Devices
Junho Jeong, Beomseok Kim, Jinsung Cho
http://doi.org/10.5626/JOK.2021.48.8.865
With the recent rapid advances in digital communication technology, the spread of IoT(Internet of Things) has accelerated and IoT devices can be utilized to investigate crimes and accidents due to the close connection between human society and IoT devices. Accordingly, with the increasing importance of digital forensics, numerous studies have been conducted. However, most digital forensics research proposed only abstract methodologies due to the various types of IoT devices. In addition, binwalk, which is actively used as a firmware analysis tool, does not adequately analyze and extract the ext4 file system. To solve these problems, this paper proposes a proper extraction and analysis method and a practical process that could extract the ext4 file system from the flash memory of IoT devices using the binwalk with the proposed method. This study also verifies the proposed process with DJI Phantom 4 Pro V2.0 drone.
A Compression-based Data Consistency Mechanism for File Systems
Dong Hyun Kang, Sang-Won Lee, Young Ik Eom
http://doi.org/10.5626/JOK.2019.46.9.885
Data consistency mechanism is a crucial component in any file system; the mechanism prevents the corruption of data from system crashes or power failures. For the sake of performance, the default journal mode of the Ext4 file system guarantees only the consistency of metadata while compromising with the consistency of normal data. Specially, it does not guarantee full consistency of the whole data of the file system. In this paper, we propose a new crash consistency scheme which guarantees strong data consistency of the data journal mode by still providing higher or comparable performance to the weak default journal mode of the Ext4 file system. By leveraging a compression mechanism, the proposed scheme can halve the amount of write operations as well as the number of fsync() system calls. For evaluation of the performance, we modified the codes related to the jbd2 and compared the proposed scheme with two journaling modes in Ext4 on SSD and HDD. The results clearly confirm that the proposed scheme outperforms the default journal mode by 8.3x times.
Streaming Compression Scheme for Reducing Network Resource Usage in Hadoop System
http://doi.org/10.5626/JOK.2018.45.6.516
Recently, the Hadoop system has become one of the most popular large-scale distributed systems used in enterprises, and the amount of data on the system has been increasing continually. As the amount of data in the Hadoop system is increased, the scale of Hadoop clusters is also growing. Resources in a node, such as processor, memory, and storage, are isolated from other nodes, and hence, even though resource usage is increased by data processing requests from clients, it doesn’t affect the performance of other nodes. However, all the nodes in a Hadoop cluster are connected to the network resource, a shared resource in the Hadoop cluster, and so, if some nodes dominate the network resource, other nodes would experience less network resources, which could cause overall performance degradation in the Hadoop system. In this paper, we propose a streaming compression scheme that can decrease the network traffic generated by write operations in the system. We also evaluate the performance of our streaming compression scheme and analyze the overhead of the proposed scheme. Our experimental results with a real-world workload show that our proposed scheme decreases the network traffic in a Hadoop cluster by 56% over the existing HDFS systems.
Consideration of fsync() of the Ext4 File System According to Kernel Version
Seongbae Son, Yoenjin Noh, Dokeun Lee, Sungsoon Park, Youjip Won
Ext4 file system is widely used in various computing environments such as those of the PC, the server, and the Linux-based embedded system. Ext4, which uses a buffer for block I/O, provides fsync() system call to applications to guarantee the consistency of a specific file. A log of the analytical studies regarding the operation of Ext4 and the improvement of its performance has been compiled, but it has not been studied in detail in terms of kernel versions. We figure out that the behavior of fsync() system call is different depending on the kernel version. Between the kernel versions of 3.4.0 and 4.7.2, 3.4.0, 3.8.0, and 4.6.2 showed behavioral differences regarding the fsync() system call. The latency of fsync() in kernel 3.4.0 is longer than that of the more-advanced 3.7.10; meanwhile, the characteristics of 3.8.0 enabled the disruption of the Ext4 journaling order, but the ordered defect was solved with 4.6.2.
In-Memory File System Backed by Cloud Storage Services as Permanent Storages
Kyungjun Lee, Jiwon Kim, Sungtae Ryu, Hwansoo Han
As network technology advances, a larger number of devices are connected through the Internet. Recently, cloud storage services are gaining popularity, as they are convenient to access anytime and anywhere. Among cloud storage services, object storage is the representative one due to their characteristics of low cost, high availability, and high durability. One limitation of object storage services is that they can access data on the cloud only through the HTTP-based RESTful APIs. In our work, we resolve this limitation with the in-memory file system which provides a POSIX interface to the file system users and communicates with cloud object storages with RESTful APIs. In particular, our flush mechanism is compatible with existing file systems, as it is based on the swap mechanism of the Linux kernel. Our in-memory file system backed by cloud storage reduces the performance overheads and shows a better performance than S3QL by 57% in write operations. It also shows a comparable performance to tmpfs in read operations.
Performance Analysis of Cloud-Backed File Systems with Various Object Sizes
Jiwon Kim, Kyungjun Lee, Sungtae Ryu, Hwansoo Han
Recent cloud infrastructures provide competitive performances and operation costs for many internet services through pay-per-use model. Particularly, object storages are highlighted, as they have unlimited file holding capacity and allow users to access the stored files anytime and anywhere. Several lines of research are based on cloud-backed file systems, which support traditional POSIX interface rather than RESTful APIs via HTTP. However, these existing file systems handle all files with uniform size backing objects. Consequently, the accesses to cloud object storages are likely to be inefficient. In our research, files are profiled according to characteristics, and appropriate backing unit sizes are determined. We experimentally verify that different backing unit sizes for the object storage improve the performance of cloud-backed file systems. In our comparative experiments with S3QL, our prototype cloud-backed file system shows faster performance by 18.6% on average.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr