Digital Library[ Search Result ]
Extracting Instruction Set Architecture Semantics from a Processor Register-transfer Level
http://doi.org/10.5626/JOK.2023.50.10.827
Domain-specific processors have specialized instructions tailored for frequently used operations in a particular domain, which enables them to achieve higher performance. This presents a challenge for program analysis, as the specialized instructions make it difficult to formally describe the instruction semantics. To address this, we present SemTracter, a tool that automatically extracts instruction semantics from a processor implemented in a hardware description language (HDL) at the register-transfer level (RTL). SemTracter obtains the semantics by simulating the processor RTL symbolically and compiling the results into formal instruction semantics using the Sail language. Our evaluation of the SemTracter on a small RISC-V processor RTL showed that it was able to extract the semantics of basic instructions from a 5-stage processor. Most of the RISC-V 32-bit integer base user-level ISA (RV32I) instructions were extracted and the generated semantics matched the manually written version.
Optimizing Computation of Tensor-Train Decomposed Embedding Layer
Seungmin Yu, Hayun Lee, Dongkun Shin
http://doi.org/10.5626/JOK.2023.50.9.729
Personalized recommendation system is ubiquitous in daily life. However, the huge amount of memory requirement to store the embedding tables used by deep learning-based recommendation system models is taking up most of the resources of industrial AI data centers. To overcome this problem, one of the solutions is to use Tensor-Train (TT) decomposition, is promising compression technique in deep neural network. In this study, we analyze unnecessary computations in Tensor-Train Gather and Reduce (TT-GnR) which is the operation of embedding layer applied with TT decomposition. To solve this problem, we define a computational unit called group to bind the item vectors into a group and propose Group Reduced TT-Gather and Reduce operation to reduce unnecessary operations by calculating with groups. Since the GRT-GnR operation is calculated in groups, computational cost varies depending on how item vectors are grouped. Experimental results showed that the GRT-GnR operation had a 41% decrease in latency compared to conventional TT-GnR operation.
Efficient Distributed Training Method Considering the Energy Level of Edge Devices in Solar-powered Edge AIoT Environments
Yeontae Yoo, IKjune Yoon, Dong Kun Noh
http://doi.org/10.5626/JOK.2023.50.8.720
Solar-powered IoT devices periodically harvest energy and therefore can fundamentally solve the energy limitation of battery-based IoT devices. However, a careful energy consumption policy is required due to the variation in the amount of energy harvested. There is a growing interest in the AI-distributed training models that can improve the quality and performance of training by conducting small training at each edge node and sharing the results with neighbors. However, the straggler node problem may occur in such distributed models, significantly decreasing the overall training speed and exponentially reducing the lifespan of the IoT network due to insufficient energy of specific nodes. This study proposes a technique to prevent the occurrence of straggler nodes as much as possible for efficient AI-distributed training in an AIoT environment composed of solar-powered devices. The proposed scheme uses an approximate computing technique that adapts energy consumption by adjusting the accuracy according to each node’s harvested energy while retaining the minimum accuracy required by the application. Among various approximation computing schemes, this study uses a data-level approximation scheme that adjusts the accuracy by adjusting the sampling rate of the sensing data. The experimental results confirm that the proposed scheme reduces the generation of straggler nodes by efficient and balanced use of each node’s harvested energy.
Time-Series Data Augmentation Based on Adversarial Training
http://doi.org/10.5626/JOK.2023.50.8.671
Recently, time series data are being generated in various industries with advancement of the Internet of Things (IoT). Accordingly, demands for time series forecasting in various industries are increasing. With acquisition of a large amount of time-series data, studies on traditional statistical method based time-series forecasting and deep learning-based forecasting methods have become active and the need for data augmentation techniques has emerged. In this paper, we proposed a novel data augmentation method for time series forecasting based on adversarial training. Unlike conventional adversarial training, the proposed method could fix the hyperparameter about the number of adversarial training iterations and utilize blockwise clipping of perturbations. We carried out various experiments to verify the performance of the proposed method. As a result, we were able to confirm that the proposed method had consistent performance improvement effect on various datasets. In addition, unlike conventional adversarial training, the necessity of blockwise clipping and the hyperparameter value fixing proposed in this paper were also verified through comparative experiments.
Multi-Document Summarization Use Semantic Similarity and Information Quantity of Sentence
Yeon-Soo Lim, Sunggoo Kwon, Bong-Min Kim, Seong-Bae Park
http://doi.org/10.5626/JOK.2023.50.7.561
Document summarization task has recently emerged as an important task in natural language processing because of the need for delivering concise information. However, it is difficult to obtain a suitable multi-document summarization dataset. In this paper, rather than training with a multi-document summarization dataset, we propose to use a single-document summarization dataset. That is, we propose a multi-document summarization model which generates multiple single-document summaries with a single-document summarization model and then post-processes these summaries. The proposed model consists of three modules: a summary module, a similarity module, and an information module. When multiple documents are entered into the proposed model, the summary module generates summaries of every single document. The similarity module clusters similar summaries by measuring semantic similarity. The information module selects the most informative summary from each similar summary group and collects selected summaries for the final multi-document summary. Experimental results show that the proposed model outperforms the baseline models and it can generate a high-quality multi-document summary. In addition, the performances of each module also show meaningful results.
Mini-Batching with Similar-Length Sentences to Quickly Train NMT Models
Daniela N. Rim, Richard Kimera, Heeyoul Choi
http://doi.org/10.5626/JOK.2023.50.7.614
The Transformer model has revolutionized Natural Language Processing tasks such as Neural Machine Translation. Many efforts have been made to study the Transformer architecture to increase its efficiency and accuracy. One potential area for improvement is to address the computation of empty tokens that the Transformer computes only to discard them later, leading to an unnecessary computational burden. To tackle this, we propose an algorithm that sorts translation sentence pairs based on their length before batching and mini-batch with similar-length sentences, which minimizes the waste of computing power. Since the amount of sorting could violate the independent and identically distributed (i.i.d) data assumption, we sort the data partially. In experiments, we apply the proposed method to English-Korean and English-Luganda language pairs for machine translation and show that there are gains in computational time while maintaining the performance. Our method is independent of architectures, so that it can be easily integrated into any training process with flexible data lengths.
Multi-task Learning Based Re-ranker for External Knowledge Retrieval in Document-grounded Dialogue Systems
http://doi.org/10.5626/JOK.2023.50.7.606
Document-grounded dialogue systems retrieve external passages related to the dialogue and use them to generate an appropriate response to the user"s utterance. However, the retriever based on the dual-encoder architecture records low performance in finding relevant passages, and the re-ranker to complement the retriever is not sufficiently optimized. In this paper, to solve these problems and perform effective external passage retrieval, we propose a re-ranker based on multi-task learning. The proposed model is a cross-encoder structure that simultaneously learns contrastive learning-based ranking, Masked Language Model (MLM), and Posterior Differential Regularization (PDR) in the fine-tuning stage, enhancing language understanding ability and robustness of the model through auxiliary tasks of MLM and PDR. Evaluation results on the Multidoc2dial dataset show that the proposed model outperforms the baseline model in Recall@1, Recall@5, and Recall@10.
Video Retrieval System Using One-to-One Relation Between Clip-Sentence Sequence
http://doi.org/10.5626/JOK.2023.50.6.476
Video retrieval is a research field that finds videos related to text queries among candidate videos. The previous studies on video retrieval studies have used the learning methods that enforce the embeddings of a text and its paried video to be similar to each other without considering the structures of video and text. In this paper, we propose a novel video retrieval model and a training technique that focus on a pair of a clip sequence and a sentence sequence with a one-to-one relationship. Experimental results show that the performance of the proposed model is improved by 0.3%p in R@1 for sentence-clip retrieval and 5.4%p R@1 for paragraph-video retrieval on YouCook2 datasets compared to baseline models.
Analysis of Adversarial Learning-Based Deep Domain Adaptation for Cross-Version Defect Prediction
Jiwon Choi, Jaewook Lee, Duksan Ryu, Suntae Kim
http://doi.org/10.5626/JOK.2023.50.6.460
Software defect prediction is a helpful technique for effective testing resource allocation. Software cross-version defect prediction reflects the environment in which the software is developed in a continuous version, with software modules added or deleted through a version update process. Repetition of this process can cause differences in data distribution between versions, which can negatively affect defect prediction performance. Deep domain adaptation(DeepDA) techniques are methods used to reduce distribution difference between sources and target data in the field of computer vision. This paper aims to reduce difference in data distribution between versions using various DeepDA techniques and to identify techniques with the best defect prediction performance. We compared performance between deep domain adaptation techniques (i.e., Domain-Adversarial Neural Network (DANN), Adversarial Discriminator Domain Apaptation (ADDA), and Wasserstein Distance Guided Representation Learning (WDGRL)) and identified performance differences according to the pair of source data. We also checked performance difference according to the ratio of target data used in the learning process and performance difference in terms of hyperparameter setting of the DANN model. Experimental results showed that DANN was more suitable for cross-version defect prediction environments. The DANN model performed the best when using all previous versions of data except the target version as a source. In particular, it showed the best performance when setting the number of hidden layers of the DANN model to 3. In addition, when applying the DeepDA technique, the more target data used in the learning process, the better the performance. This study suggests that various DeepDA techniques can be used to predict software cross-version defects in the future.
CommonAI: Quantitative and Qualitative Analysis for Automatic-generation of Commonsense Reasoning Conversations Suitable for AI
Hyeon Gyu Shin, Hyun Jo You, Young Sook Song
http://doi.org/10.5626/JOK.2023.50.5.407
Human-like common sense reasoning is now considered an essential component for improving the quality of natural language generation for chatbots and conversational agents, However, there is no clear consensus at present about to what extent AI systems require common sense. We discussed common sense requirements for AI chatbots based on quantitative and qualitative analysis of results from two experimental surveys to show differences between gender and age groups and variations according to conversation topics. The contribution of this paper is to refine preferences for chatbot conversations that provide useful information and show an appropriate level of empathy.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr