Digital Library[ Search Result ]
AttDRP: Attention Mechanism-based Model for Anti-Cancer Drug Response Prediction
Jonghwan Choi, Sangmin Seo, Sanghyun Park
http://doi.org/10.5626/JOK.2021.48.6.713
Resistance to anti-cancer drugs makes chemotherapy ineffective for cancer patients. Drug resistance is caused by genetic alterations in cancer cells. Many studies have investigated drug responses of diverse cancer cell lines to various anti-cancer drugs to understand drug response mechanisms. Existing studies have proposed machine learning models for drug response prediction to find effective anti-cancer drugs. However, currently there are no models to learn the relationship between anticancer drugs and genes to improve the prediction accuracy. In this paper, we proposed a predictive model AttDRP that could identify important genes associated with anti-cancer drugs and predict drug responses based on identified genes. AttDRP exhibited better predictive accuracy than existing models and we found that the attention scores of AttDRP could be effective tools to analyze molecular structures of anticancer drugs. We hope that our proposed method would contribute to the development of precision medicine for effective chemotherapy. Resistance to anti-cancer drugs makes chemotherapy ineffective for cancer patients. Drug resistance is caused by genetic alterations in cancer cells. Many studies have investigated drug responses of diverse cancer cell lines to various anti-cancer drugs to understand drug response mechanisms. Existing studies have proposed machine learning models for drug response prediction to find effective anti-cancer drugs. However, currently there are no models to learn the relationship between anticancer drugs and genes to improve the prediction accuracy. In this paper, we proposed a predictive model AttDRP that could identify important genes associated with anti-cancer drugs and predict drug responses based on identified genes. AttDRP exhibited better predictive accuracy than existing models and we found that the attention scores of AttDRP could be effective tools to analyze molecular structures of anticancer drugs.
An Explainable Knowledge Completion Model Using Explanation Segments
Min-Ho Lee, Wan-Gon Lee, Batselem Jagvaral, Young-Tack Park
http://doi.org/10.5626/JOK.2021.48.6.680
Recently, a large number of studies that used deep learning have been conducted to predict new links in incomplete knowledge graphs. However, link prediction using deep learning has a major limitation as the inferred results cannot be explained. We propose a high-utility knowledge graph prediction model that yields explainable inference paths supporting the inference results. We define paths to the object from the knowledge graph using a path ranking algorithm and define them as the explanation segments. Then, the generated explanation segments are embedded using a Convolutional neural network (CNN) and a Bidirectional Long short-term memory (BiLSTM). The link prediction model is then trained by applying an attention mechanism, based on the calculation of the semantic similarity between the embedded explanation segments and inferred candidate predicates to be inferred. The explanation segment suitable for link prediction explanation is selected based on the measured attention scores. To evaluate the performance of the proposed method, a link prediction comparison experiment and an accuracy verification experiment are performed to measure the proportion of the explanation segments suitable to explain the link prediction results. We used the benchmark datasets NELL-995, FB15K-237, and countries for the experiment, and accuracy verification experiments showed the accuracies of 89%, 44%, and 97%, respectively. Compared with the existing method, the NELL-995, FB15K-237 data exhibited 35%p and 21%p higher performance on average.
Query-based Abstractive Summarization Model Using Sentence Ranking Scores and Graph Techniques
http://doi.org/10.5626/JOK.2020.47.12.1172
The purpose of the fundamental abstractive summarization model is to generate a short summary document that includes all important contents within the document. Conversely, in the query-based abstractive summarization model, information related to the query should be selected and summarized within the document. The existing query-based summarization models calculates the importance of sentences using only the weight of words through an attention mechanism between words in the document and the query. This method has a disadvantage in that it is difficult to reflect the entire context information of the document to generate an abstractive summary. In this paper, we resolve this problems by calculating the sentence ranking scores and a sentence-level graph structure. Our proposed model shows higher performance than the previous research model, 1.44%p in ROUGE-1 and 0.52%p in ROUGE-L.
Performance Analysis of Korean Morphological Analyzer based on Transformer and BERT
http://doi.org/10.5626/JOK.2020.47.8.730
This paper introduces a Korean morphological analyzer using the Transformer, which is one of the most popular sequence-to-sequence deep neural models. The Transformer comprises an encoder and a decoder. The encoder compresses a raw input sentence into a fixed-size vector, while the decoder generates a morphological analysis result for the vector. We also replace the encoder with BERT, a pre-trained language representation model. An attention mechanism and a copying mechanism are integrated in the decoder. The processing units of the encoder and the decoder are eojeol-based WordPiece and morpheme-based WordPiece, respectively. Experimental results showed that the Transformer with fine-tuned BERT outperforms the randomly initialized Transformer by 2.9% in the F1 score. We also investigated the effects of the WordPiece embedding on morphological analysis when they are not fully updated in the training phases.
A Knowledge Graph Embedding-based Ensemble Model for Link Prediction
http://doi.org/10.5626/JOK.2020.47.5.473
Knowledge bases often suffer from their limited applicability due to missing information in their entities and relations. Link prediction has been investigated to complete the missing information and makes a knowledge base more useful. The existing studies on link prediction often rely on knowledge graph embedding and have shown trade-off in their performance. In this paper, we propose an ensemble model for knowledge graph embedding to improve quality of link prediction. The proposed model combines multiple knowledge graph embeddings that have unique characteristics. In this way, the ensemble model is able to consider various aspects of the entries within a knowledge base and reduce the variation of accuracy depending on hyper-parameters. Our experiment shows that the proposed model outperforms other knowledge graph embedding methods by 13.5% on WN18 and FB15K dataset.
A Product Review Summarization Considering Additional Information
Jaeyeun Yoon, Ig-hoon Lee, Sang-goo Lee
http://doi.org/10.5626/JOK.2020.47.2.180
Automatic document summarization is a task that generates the document in a suitable form from an existing document for a certain user or occasion. As use of the Internet increases, the various data including texts are exploding and the value of document summarization technology is growing. While the latest deep learning-based models show reliable performance in document summarization, the problem is that performance depends on the quantity and quality of the training data. For example, it is difficult to generate reliable summarization with existing models from the product review text of online shopping malls because of typing errors and grammatically wrong sentences. Online malls and portal web services are struggling to solve this problem. Thus, to generate an appropriate document summary in poor condition relative to quality and quantity of the product review learning data, this study proposes a model that generates product review summaries with additional information. We found through experiments that this model showed improved performances in terms of relevance and readability than the existing model for product review summaries.
Single Sentence Summarization with an Event Word Attention Mechanism
Ian Jung, Su Jeong Choi, Seyoung Park
http://doi.org/10.5626/JOK.2020.47.2.155
The purpose of summarization is to generate short text that preserves important information in the source sentences. There are two approaches for the summarization task. One is an extractive approach and other is an abstractive approach. The extractive approach is to determine if words in a source sentence are retained or not. The abstractive approach generates the summary of a given source sentence using the neural network such as the sequence-to-sequence model and the pointer-generator. However, these approaches present a problem because such approaches omit important information such as event words. This paper proposes an event word attention mechanism for sentence summarization. Event words serve as the key meaning of a given source sentence, since they express what occurs in the source sentence. The event word attention weights are calculated by event information of each words in the source sentence and then it combines global attention mechanism. For evaluation, we used the English and Korean dataset. Experimental results show that, the model of adopting event attention outperforms the existing models.
Grammatical Error Detection for L2 Learners Based on Attention Mechanism
Chanhee Park, Jinuk Park, Minsoo Cho, Sanghyun Park
http://doi.org/10.5626/JOK.2019.46.6.554
Grammar Error Detection refers to the work of discovering the presence and location of grammatical errors in a given sentence, and is considered to be useful for L2 learners to learn and evaluate the language. Systems for grammatical error correction have been actively studied, but there still exist limitations such as lack of training corpus and limited error type correction. Therefore, this paper proposes a model for generalized grammatical error detection through the sequence labeling problem which does not require the determination of error type. The proposed model dynamically decides character-level and word-level representation to deal with unexpected words in L2 learners" writing. Also, based on the proposed model the bias which can occur during the learning process with imbalanced data can be avoided through multi-task learning. Additionally, attention mechanism is applied to efficiently predict errors by concentrating on words for judging errors. To validate the proposed model, three test data were used and the effectiveness of the model was verified through the ablation experiment.
Biomedical Named Entity Recognition using Multi-head Attention with Highway Network
Minsoo Cho, Jinuk Park, Jihwan Ha, Chanhee Park, Sanghyun Park
http://doi.org/10.5626/JOK.2019.46.6.544
Biomedical named entity recognition(BioNER) is the process of extracting biomedical entities such as diseases, genes, proteins, and chemicals from biomedical literature. BioNER is an indispensable technique for the extraction of meaningful data from biomedical domains. The proposed model employs deep learning based Bi-LSTM-CRF model which eliminates the need for hand-crafted feature engineering. Additionally, the model contains multi-head attention to capture the relevance between words, which is used when predicting the label of each input token. Also, in the input embedding layer, the model integrates character-level embedding with word-level embedding and applies the combined word embedding into the highway network to adaptively carry each embedding to the input of the Bi-LSTM model. Two English biomedical benchmark datasets were employed in the present research to evaluate the level of performance. The proposed model resulted in higher f1-score compared to other previously studied models. The results demonstrate the effectiveness of the proposed methods in biomedical named entity recognition study.
Image Caption Generation using Object Attention Mechanism
http://doi.org/10.5626/JOK.2019.46.4.369
Explosive increases in image data have led studies investigating the role of image caption generation in image expression of natural language. The current technologies for generating Korean image captions contain errors associated with object concurrence attributed to dataset translation from English datasets. In this paper, we propose a model of image caption generation employing attention as a new loss function using the extracted nouns of image references. The proposed method displayed BLEU1 0.686, BLEU2 0.557, BLEU3 0.456, BLEU4 0.372, which proves that the proposed model facilitates the resolution of high-frequency word-pair errors. We also showed that it enhances the performance compared with previous studies and reduces redundancies in the sentences. As a result, the proposed method can be used to generate a caption corpus effectively.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr