Digital Library[ Search Result ]
Single Sentence Summarization with an Event Word Attention Mechanism
Ian Jung, Su Jeong Choi, Seyoung Park
http://doi.org/10.5626/JOK.2020.47.2.155
The purpose of summarization is to generate short text that preserves important information in the source sentences. There are two approaches for the summarization task. One is an extractive approach and other is an abstractive approach. The extractive approach is to determine if words in a source sentence are retained or not. The abstractive approach generates the summary of a given source sentence using the neural network such as the sequence-to-sequence model and the pointer-generator. However, these approaches present a problem because such approaches omit important information such as event words. This paper proposes an event word attention mechanism for sentence summarization. Event words serve as the key meaning of a given source sentence, since they express what occurs in the source sentence. The event word attention weights are calculated by event information of each words in the source sentence and then it combines global attention mechanism. For evaluation, we used the English and Korean dataset. Experimental results show that, the model of adopting event attention outperforms the existing models.
A Product Review Summarization Considering Additional Information
Jaeyeun Yoon, Ig-hoon Lee, Sang-goo Lee
http://doi.org/10.5626/JOK.2020.47.2.180
Automatic document summarization is a task that generates the document in a suitable form from an existing document for a certain user or occasion. As use of the Internet increases, the various data including texts are exploding and the value of document summarization technology is growing. While the latest deep learning-based models show reliable performance in document summarization, the problem is that performance depends on the quantity and quality of the training data. For example, it is difficult to generate reliable summarization with existing models from the product review text of online shopping malls because of typing errors and grammatically wrong sentences. Online malls and portal web services are struggling to solve this problem. Thus, to generate an appropriate document summary in poor condition relative to quality and quantity of the product review learning data, this study proposes a model that generates product review summaries with additional information. We found through experiments that this model showed improved performances in terms of relevance and readability than the existing model for product review summaries.
Efficient CNNs with Channel Attention and Group Convolution for Facial Expression Recognition
MyeongOh Lee, Ui Nyoung Yoon, Seunghyun Ko, Geun-Sik Jo
http://doi.org/10.5626/JOK.2019.46.12.1241
Recently, studies using the convolutional neural network have been actively conducted to recognize emotions from facial expressions. In this paper, we propose an efficient convolutional neural network that solves the model complexity problem of the deep convolutional neural network used to recognize the emotions in facial expression. To reduce the complexity of the model, we used group convolution, depth-wise separable convolution to reduce the number of parameters, and the computational cost. We also enhanced the reuse of features and channel information by using Skip Connection for feature connection and Channel Attention. Our method achieved 70.32% and 85.23% accuracy on FER2013, RAF-single datasets with four times fewer parameters (0.39 Million, 0.41 Million) than the existing model.
Korean Movie Review Sentiment Analysis using Self-Attention and Contextualized Embedding
Cheoneum Park, Dongheon Lee, Kihoon Kim, Changki Lee, Hyunki Kim
http://doi.org/10.5626/JOK.2019.46.9.901
Sentiment analysis is the processing task that involves collecting and classifying opinions about a specific object. However, it is difficult to grasp the subjectivity of a person using natural language, so the existing sentimental word dictionaries or probabilistic models cannot solve such a task, but the development of deep learning made it possible to solve the task. Self-attention is a method of modeling a given input sequence by calculating the attention weight of the input sequence itself and constructing a context vector with a weighted sum. In the context, a high weight is calculated between words with similar meanings. In this paper, we propose a method using a modeling network with self-attention and pre-trained contextualized embedding to solve the sentiment analysis task. The experimental result shows an accuracy of 89.82%.
Learning Semantic Features for Dense Video Captioning
http://doi.org/10.5626/JOK.2019.46.8.753
In this paper, we propose a new deep neural network model for dense video captioning. Dense video captioning is an emerging task that aims at both localizing and describing all events in a video. Unlike many existing models, which use only visual features extracted from the given video through a sort of convolutional neural network(CNN), our proposed model makes additional use of high-level semantic features that describe important event components such as actions, people, objects, and backgrounds. The proposed model localizes temporal regions of events by using LSTM, a recurrent neural network(RNN). Furthermore, our model adopts an attention mechanism for caption generation to selectively focus on input features depending on their importance. By conducting experiments using a large-scale benchmark dataset for dense video captioning, AcitivityNet Captions, we demonstrate high performance and superiority of our model.
Grammatical Error Detection for L2 Learners Based on Attention Mechanism
Chanhee Park, Jinuk Park, Minsoo Cho, Sanghyun Park
http://doi.org/10.5626/JOK.2019.46.6.554
Grammar Error Detection refers to the work of discovering the presence and location of grammatical errors in a given sentence, and is considered to be useful for L2 learners to learn and evaluate the language. Systems for grammatical error correction have been actively studied, but there still exist limitations such as lack of training corpus and limited error type correction. Therefore, this paper proposes a model for generalized grammatical error detection through the sequence labeling problem which does not require the determination of error type. The proposed model dynamically decides character-level and word-level representation to deal with unexpected words in L2 learners" writing. Also, based on the proposed model the bias which can occur during the learning process with imbalanced data can be avoided through multi-task learning. Additionally, attention mechanism is applied to efficiently predict errors by concentrating on words for judging errors. To validate the proposed model, three test data were used and the effectiveness of the model was verified through the ablation experiment.
Biomedical Named Entity Recognition using Multi-head Attention with Highway Network
Minsoo Cho, Jinuk Park, Jihwan Ha, Chanhee Park, Sanghyun Park
http://doi.org/10.5626/JOK.2019.46.6.544
Biomedical named entity recognition(BioNER) is the process of extracting biomedical entities such as diseases, genes, proteins, and chemicals from biomedical literature. BioNER is an indispensable technique for the extraction of meaningful data from biomedical domains. The proposed model employs deep learning based Bi-LSTM-CRF model which eliminates the need for hand-crafted feature engineering. Additionally, the model contains multi-head attention to capture the relevance between words, which is used when predicting the label of each input token. Also, in the input embedding layer, the model integrates character-level embedding with word-level embedding and applies the combined word embedding into the highway network to adaptively carry each embedding to the input of the Bi-LSTM model. Two English biomedical benchmark datasets were employed in the present research to evaluate the level of performance. The proposed model resulted in higher f1-score compared to other previously studied models. The results demonstrate the effectiveness of the proposed methods in biomedical named entity recognition study.
Image Caption Generation using Object Attention Mechanism
http://doi.org/10.5626/JOK.2019.46.4.369
Explosive increases in image data have led studies investigating the role of image caption generation in image expression of natural language. The current technologies for generating Korean image captions contain errors associated with object concurrence attributed to dataset translation from English datasets. In this paper, we propose a model of image caption generation employing attention as a new loss function using the extracted nouns of image references. The proposed method displayed BLEU1 0.686, BLEU2 0.557, BLEU3 0.456, BLEU4 0.372, which proves that the proposed model facilitates the resolution of high-frequency word-pair errors. We also showed that it enhances the performance compared with previous studies and reduces redundancies in the sentences. As a result, the proposed method can be used to generate a caption corpus effectively.
Coreference Resolution using Multi-resolution Pointer Networks
Cheoneum Park, Changki Lee, Hyunki Kim
http://doi.org/10.5626/JOK.2019.46.4.334
Multi-resolution RNN is a method of modeling parallel sequences as RNNs. Coreference resolution is a natural language processing task in which several words representing different entities present in a document are defined as one cluster and can be solved by a pointer network. The encoder input sequence of the coreference resolution becomes all the morphemes of the document using the pointer network, and the decoder input sequence becomes all the nouns present in the document. In this paper, we propose three multi-resolution pointer network models that encode all morphemes and noun lists of a document in parallel and perform decoding by using both encoded hidden states in a decoder. We have solved the coreference resolution based on the proposed models. Experimental results show that Multi-resolution1 of the proposed model has 71.44% CoNLL F1, 70.52% CoNLL F1 of Multi-resolution2 and 70.59% CoNLL F1 of Multi-resolution3.
Sentence Similarity Prediction based on Siamese CNN-Bidirectional LSTM with Self-attention
Mintae Kim, Yeongtaek Oh, Wooju Kim
http://doi.org/10.5626/JOK.2019.46.3.241
A deep learning model for semantic similarity between sentences was presented. In general, most of the models for measuring similarity word use level or morpheme level embedding. However, the attempt to apply either word use or morpheme level embedding results in higher complexity of the model due to the large size of the dictionary. To solve this problem, a Siamese CNN-Bidirectional LSTM model that utilizes phonemes instead of words or morphemes and combines long short term memory (LSTM) with 1D convolution neural networks with various window lengths that bind phonemes is proposed. For evaluation, we compared our model with Manhattan LSTM (MaLSTM) which shows good performance in measuring similarity between similar questions in the Naver Q&A dataset (similar to Kaggle Quora Question Pair).
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr