Digital Library[ Search Result ]
Neural Network Learning Method using Weight Mirroring and Direct Feedback Error
Soha Lee, Heesung Yang, Hyeyoung Park
http://doi.org/10.5626/JOK.2024.51.5.445
Error backpropagation algorithm is a core learning algorithm of neural networks and, until recently, has been used in various deep learning models. However, the weight update rule of error backpropagation, in which the error signal of the upper layer is sequentially transmitted to the lower layer and the weight values of the upper layer that are used to update the lower layer weights, has a problem of biological implausibility and computational inefficiency. To address these issues, learning methods using separate backward weights have been proposed, but they are still at an early stage and require further analysis and improvement from various perspectives. In this paper, we proposed a new learning method by combining the direct feedback alignment method, which directly projects the errors of the last layer into each hidden layer, and a weight mirror method with a separate step for updating backward weights. The proposed method overcomes the limitations of learning methods to implement a weight update method that is biologically plausible and allows for more efficient parallel learning. We confirmed the potential of the proposed method through experiments on various benchmark datasets.
Number-based High Fidelity Logical Qubit on the Surface Code FTQC
http://doi.org/10.5626/JOK.2024.51.4.301
To achieve the quantum advantage, the quantum computing size should be increased. However, the quantum computational power cannot be easily increased because of the high error rates on qubits and gates. To overcome such problem, the surface code based on the fault-tolerant quantum computation model has been investigated a lot since it works with relatively higher error rates in theory. However, In practice, we need many improvements on the surface code such as the requirement of a large number of physical qubits. Therefore, in this work, we propose a logical qubit design method, which exploits the multiple lower level qubits unlike the conventional bigger-sized qubit design method. This method uses the concept of the block-code scheme. The analysis result shows that the proposed method achieves a lower error rate than the bigger-sized logical qubits with the same number of physical qubits. In conclusion, we believe this approach can improve the resource efficiency of the surface code FTQC.
Non-autoregressive Korean Morphological Analysis with Word Segment Information
http://doi.org/10.5626/JOK.2023.50.8.653
This paper introduces a non-autoregressive Korean morphological analyzer. The proposed morphological analyzer utilizes a transformer encoder to encode a given sentence and employs two non-autoregressive decoders for morphological analysis. Each decoder generates a morpheme sequence and a corresponding POS tag sequence, which are then combined to produce the final morphological analysis. Additionally, this paper leverages word segment information within the sentence to predict the target sequence length, mitigating performance degradation resulting from incorrect target sequence length predictions. Experimental results show that the proposed non-autoregressive Korean morphological analyzer outperforms all non-autoregressive baselines. It achieves comparable accuracy to an autoregressive Korean morphological analyzer while it performs nearly 14.76 times faster than the autoregressive Korean morphological analyzer.
Knowledge Completion System using Neuro-Symbolic-based Rule Induction and Inference Engine
Won-Chul Shin, Hyun-Kyu Park, Young-Tack Park
http://doi.org/10.5626/JOK.2021.48.11.1202
Recently, there have been several studies on knowledge completion methods aimed to solve the incomplete knowledge graphs problem. Methods such as Neural Theorem Prover (NTP), which combines the advantages of deep learning methods and logic systems, have performed well over existing methods. However, NTP faces challenges in processing large-scale knowledge graphs because all the triples of the knowledge graph are involved in the computation to obtain prediction results for one input. In this paper, we propose an integrated system of deep learning and logic inference methods that can learn vector representations of symbols from improved models of computational complexity of NTP to rule induction, and perform knowledge inference from induced rules using inference engines. In this paper, for rule-induction performance verification of the rule generation model, we compared test data inference ability with NTP using induced rules on Nations, Kinship, and UMLS data set. Experiments with Kdata and WiseKB knowledge inference through inference engines resulted in a 30% increase in Kdata and a 95% increase in WiseKB compared to the knowledge graphs used in experiments.
Survey of EEG Neurofeedback methods for Attention Improvement
Hyunji Kim, Daeun Gwon, Kyungho Won, Sung Chan Jun, Minkyu Ahn
http://doi.org/10.5626/JOK.2021.48.10.1105
Neurofeedback is a method through which a user self-regulates the brain state using the feedback of his/her own brain signal. This can be used to restore or improve brain functions. In this study, we reviewed 108 articles on electroencephalogram (EEG) neurofeedback for attention improvement and surveyed the important parameters. As a result, we found that most studies were conducted with patient subjects and mostly brain signals were recorded from central areas on the scalp by using wet and wire EEG systems. Sensory-motor-rhythm or the ratio between theta and low beta rhythms were used as attention index, and this information was provided to users through auditory or visual stimuli. In addition, Continuous Performance Test or Go/NoGo test was employed for behavior evaluation. Based on these results, we suggest the following directions for the further advancement of the practical neurofeedback system; the future work should target non-patient subjects and utilize wireless/dry EEG devices and virtual/augmented reality for increasing user convenience and building more immersive application. Lastly, a standard or guide for developing usable neurofeedback applications should be established.
Joint Model of Morphological Analysis and Named Entity Recognition Using Shared Layer
Hongjin Kim, Seongsik Park, Harksoo Kim
http://doi.org/10.5626/JOK.2021.48.2.167
Named entity recognition is a natural language processing technology that finds words with unique meanings such as human names, place names, organization names, dates, and time in sentences and attaches them. Morphological analysis in Korean is generally divided into morphological analysis and part-of-speech tagging. In general, named entity recognition and morphological analysis studies conducted in independently. However, in this architecture, the error of morphological analysis propagates to named entity recognition. To alleviate the error propagation problem, we propose an integrated model using Label Attention Network (LAN). As a result of the experiment, our model shows better performance than the single model of named entity recognition and morphological analysis. Our model also demonstrates better performance than previous integration models.
Performance Analysis of Korean Morphological Analyzer based on Transformer and BERT
http://doi.org/10.5626/JOK.2020.47.8.730
This paper introduces a Korean morphological analyzer using the Transformer, which is one of the most popular sequence-to-sequence deep neural models. The Transformer comprises an encoder and a decoder. The encoder compresses a raw input sentence into a fixed-size vector, while the decoder generates a morphological analysis result for the vector. We also replace the encoder with BERT, a pre-trained language representation model. An attention mechanism and a copying mechanism are integrated in the decoder. The processing units of the encoder and the decoder are eojeol-based WordPiece and morpheme-based WordPiece, respectively. Experimental results showed that the Transformer with fine-tuned BERT outperforms the randomly initialized Transformer by 2.9% in the F1 score. We also investigated the effects of the WordPiece embedding on morphological analysis when they are not fully updated in the training phases.
Unified Methodology of Multiple POS Taggers for Large-scale Korean Linguistic GS Set Construction
Tae-Young Kim, Pum-Mo Ryu, Hansaem Kim, Hyo-Jung Oh
http://doi.org/10.5626/JOK.2020.47.6.596
In recent years, there has been national support for constructing, sharing, and spreading a large-scale Korean linguistic GS set for Korean information processing. As part of the corpus construction project, this study proposes the methodology for constructing the Korean linguistic GS set using various Korean language analysis modules developed in Korea. To build a large-scale training set, we referred to automatic tagged candidate answers from the N-modules. We then minimized manual effort by classifying the error types from the candidate responses and semi- automatically correcting the major error types. In this study, we normalized results of the morphological analysis and constructed a large-scale Korean linguistic GS set based on the unified format U-POS. As a result of this study, 348,229 sentences, a total of 9,455,930 words, were constructed as the Korean linguistic GS set. This can be practically applied later as a basic training resource for Korean information processing.
Korean Morphological Analyzer for Neologism and Spacing Error based on Sequence-to-Sequence
Byeongseo Choe, Ig-hoon Lee, Sang-goo Lee
http://doi.org/10.5626/JOK.2020.47.1.70
In order to analyze Internet text data from Korean internet communities, it is necessary to accurately perform morphological analysis even in a sentence with a spacing error and adequate restoration of original form for an out-of-vocabulary input. However, the existing Korean morphological analyzer often uses dictionaries and complicate preprocessing for the restoration. In this paper, we propose a Korean morphological analyzer model which is based on the sequence-to-sequence model. The model can effectively handle the spacing problem and OOV problem. In addition, the model uses syllable bigram and grapheme as additional input features. The proposed model does not use a dictionary and minimizes rule-based preprocessing. The proposed model showed better performance than other morphological analyzers without a dictionary in the experiment for Sejong corpus. Also, better performance was evident for the dataset without space and sample dataset collected from Internet.
Automatic Test Case Generation through Concolic Testing to Improve SW Quality of Defense Weapon System
Kunwoo Park, Joohyun Lee, Hyunggon Song, Kyu Tae Cho, Yunho Kim, Moonzoo Kim
http://doi.org/10.5626/JOK.2019.46.9.926
To improve SW quality of defense weapon system, automatic and systematic generation of test cases is necessary; however, that is not the case in the traditional practice of labor-intensive and manual SW testing. The paper applies concolic testing to the defense weapon system SW, effectively generates test cases that achieve high coverage, and discovers defects which contributes to the improvement in SW quality. Also, two methods are proposed using 4 search strategies in concolic testing and using LIA logic, to increase the efficiency of concolic testing for a program with high complexity. In addition, a symbolic modeling method is proposed as an example to extend concolic testing for practitioners.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr