Digital Library[ Search Result ]
Deletion-based Korean Sentence Compression using Graph Neural Networks
Gyoung-Ho Lee, Yo-Han Park, Kong Joo Lee
http://doi.org/10.5626/JOK.2022.49.1.32
Automatic sentence compression aims at generating a concise sentence from a lengthy source sentence. Most common approaches to sentence compression is deletion-based compression. In this paper, we implement deletion-based sentence compression systems based on a binary classifier and long short-term memory (LSTM) networks with attention layers. The binary classifier, which is a baseline model, classifies words in a sentence into words that need to be deleted and words that will remain in a compressed sentence. We also introduce a graph neural network (GNN) in order to employ dependency tree structures when compressing a sentence. A dependency tree is encoded by a graph convolutional network (GCN), one of the most common GNNs, and every node in the encoded tree is input into the sentence compression module. As a conventional GCN deals with only undirected graphs, we propose a directed graph convolutional network (D-GCN) to differentiate between parent and child nodes of a dependency tree in sentence compression. Experimental results show that the baseline model is improved in terms of the sentence compression accuracy when employing a GNN. Regarding the performance comparison of graph networks, a D-GCN achieves higher F1 scores than a GCN when applied to sentence compression. Through experiments, it is confirmed that better performance can be achieved for sentence compression when the dependency syntax tree structure is explicitly reflected.
New Adaptive Matching Order and Performance Comparison for Subgraph Matching Problem
Seunghwan Min, Wonseok Shin, Chaewon Kim, Kunsoo Park
http://doi.org/10.5626/JOK.2022.49.1.1
In recent years, graph analysis has been used in various applications. One of the fundamental problems in graph analysis is the subgraph matching problem. Given a data graph and a query graph, the subgraph matching problem is to find all embeddings of the query graph in the data graph. Many backtracking-based algorithms have been studied to solve this problem. In this paper, we analyzed the problems in adaptive matching order proposed by DAF, a state-of-the-art algorithm that solves this problem, and introduced an improved adaptive matching order. Furthermore, we conducted experiments with real data graphs to demonstrate that the proposed matching order was more effective than the previous matching orders if the pruning technique was not used or the elapsed time was not very short even if the pruning technique was used.
GPT-2 for Knowledge Graph Completion
http://doi.org/10.5626/JOK.2021.48.12.1281
Knowledge graphs become an important resource in many artificial intelligence (AI) tasks. Many studies are being conducted to complete the incomplete knowledge graph. Among them, interest in research that knowledge completion by link prediction and relation prediction is increasing. The most talked-about language models in AI natural language processing include BERT and GPT-2, among which KG-BERT wants to solve knowledge completion problems with BERT. In this paper, we wanted to solve the problem of knowledge completion by utilizing GPT-2, which is the biggest recent issue in the language model of AI. Triple information-based knowledge completion and path-triple-based knowledge completion were proposed and explained as methods to solve the knowledge completion problem using the GPT-2 language model. The model proposed in this paper was defined as KG-GPT2, and experiments were conducted by comparing the link prediction and relationship prediction results of TransE, TransR, KG-BERT, and KG-GPT2 to evaluate knowledge completion performance. For link prediction, WN18RR, FB15k-237, and UMLS datasets were used, and for relation prediction, FB15K was used. As a result of the experiment, in the case of link prediction in the path- triple-based knowledge completion of KG-GPT2, the best performance was recorded for all experimental datasets except UMLS. In the path-triple-based knowledge completion of KG-GPT2, the model"s relationship prediction work also recorded the best performance for the FB15K dataset.
Knowledge Graph Completion using Hyper-class Information and Pre-trained Language Model
http://doi.org/10.5626/JOK.2021.48.11.1228
Link prediction is a task that aims to predict missing links in knowledge graphs. Recently, several link prediction models have been proposed to complete the knowledge graphs and have achieved meaningful results. However, the previous models used only the triples" internal information in the training data, which may lead to an overfitting problem. To address this problem, we propose Hyper-class Information and Pre-trained Language Model (HIP) that performs hyper-class prediction and link prediction through a multi-task learning. HIP learns not only contextual relationship of triples but also abstractive meanings of entities. As a result, it learns general information of the entities and forces the entities connected to the same hyper-class to have similar embeddings. Experimental results show significant improvement in Hits@10 and Mean Rank (MR) compared to KG-BERT and MTL-KGC.
Ensemble of Sentence Interaction and Graph Based Models for Document Pair Similarity Estimation
Seonghwan Choi, Donghyun Son, Hochang Lee
http://doi.org/10.5626/JOK.2021.48.11.1184
Deriving the similarity between two documents, such as, news articles, is one of the most important factors of clustering documents. Sequence similarity models, one of the existing deep-learning based approaches to document clustering, do not reflect the entire context of documents. To address this issue, this paper uses interaction-based and graph-based approaches to construct document pair similarity models suitable for news clustering. This paper proposes four interaction-based models that measures the similarity between two documents through the aggregation of similarity information in the interaction of sentences. The experimental results demonstrated that two out of these four proposed models outperformed SVM and HAN. Ablation studies were conducted on the graph-based model through experiments on the depth of the model’s neural network and its input features. Through error analysis and ensemble of models with an interaction and graph-based approach, this paper showed that these two approaches could be complementarity due to the differences in their prediction tendencies.
Knowledge Completion System using Neuro-Symbolic-based Rule Induction and Inference Engine
Won-Chul Shin, Hyun-Kyu Park, Young-Tack Park
http://doi.org/10.5626/JOK.2021.48.11.1202
Recently, there have been several studies on knowledge completion methods aimed to solve the incomplete knowledge graphs problem. Methods such as Neural Theorem Prover (NTP), which combines the advantages of deep learning methods and logic systems, have performed well over existing methods. However, NTP faces challenges in processing large-scale knowledge graphs because all the triples of the knowledge graph are involved in the computation to obtain prediction results for one input. In this paper, we propose an integrated system of deep learning and logic inference methods that can learn vector representations of symbols from improved models of computational complexity of NTP to rule induction, and perform knowledge inference from induced rules using inference engines. In this paper, for rule-induction performance verification of the rule generation model, we compared test data inference ability with NTP using induced rules on Nations, Kinship, and UMLS data set. Experiments with Kdata and WiseKB knowledge inference through inference engines resulted in a 30% increase in Kdata and a 95% increase in WiseKB compared to the knowledge graphs used in experiments.
Improving Subgraph Isomorphism with Pruning by Bipartite Matching
http://doi.org/10.5626/JOK.2021.48.9.973
In recent years, it has become increasingly important to efficiently solve NP-hard graph problems. One of the fundamental problems in graph analysis is subgraph isomorphism. Given a query graph and a data graph, the subgraph isomorphism problem is to determine whether there is an embedding of the query graph in the data graph. Although a lot of practical algorithms have been developed for the problem, existing algorithms showed limited running time scalability in dealing with many real-world graphs. In this paper, we propose a new pruning technique based on bipartite matching which enables us to capture and remove redundancies in the search space. We also conduct experiments on several real datasets to show effectiveness of our technique.
Community Detection Using Link Attribute-Based Classification
Jeongseon Kim, Soohwan Jeong, Sungsu Lim
http://doi.org/10.5626/JOK.2021.48.8.959
Attempts to discover knowledge through data are becoming gradually diversified to understand a fast and complex world. Graph data analysis, which models and analyzes correlated data as graphs, is drawing much attention as it is combined with the latest machine learning techniques. In this work, we propose a novel methodology for discovering graph community structures. We analyze similarity, curvature-based attributes to allow links existing inside and outside the community to have different attribute values, and exploit them to design and analyze algorithms that eliminate links that affect the community structure less to find better community structures on sparse graphs.
EFA-DTI: Prediction of Drug-Target Interactions Using Edge Feature Attention
Erkhembayar Jadamba, Sooheon Kim, Hyeonsu Lee, Hwajong Kim
http://doi.org/10.5626/JOK.2021.48.7.825
Drug discovery is a high-level field of research requiring the coordination of disciplines ranging from medicinal chemistry, systems biology, structural biology, and increasingly, artificial intelligence. In particular, drug-target interaction (DTI) prediction is central to the process of screening for and optimizing candidate substances to treat disease from a nearly infinite set of compounds. Recently, as computer performance has developed dramatically, studies using artificial intelligence neural networks have been actively conducted to reduce the cost and increase the efficiency of DTI prediction. This paper proposes a model that predicts an interaction value between a given molecule and protein using a learned molecule representation via Edge Feature Attention-applied Graph Net Embedding with Fixed Fingerprints and a protein representation using pre-trained protein embeddings. The paper describes architectures, experimental methods, and findings. The model demonstrated higher performance than DeepDTA and GraphDTA, which had previously demonstrated the best performance in DTI studies.
Knowledge Completion System through Learning the Relationship between Query and Knowledge Graph
Min-Sung Kim, Min-Ho Lee, Wan-Gon Lee, Young-Tack Park
http://doi.org/10.5626/JOK.2021.48.6.649
The knowledge graph is a network comprising of relationships between the entities. In a knowledge graph, there exists a problem of missing or incorrect relationship connection with the specific entities. Numerous studies have proposed learning methods using artificial neural networks based on natural language embedding to solve the problems of the incomplete knowledge graph. Various knowledge graph completion systems are being studied using these methods. In this paper, a system that infers missing knowledge using specific queries and knowledge graphs is proposed. First, a topic is automatically extracted from a query, and topic embedding is obtained from the knowledge graph embedding module. Next, a new triple is inferred by learning the relationship between the topic from the knowledge graph and the query by using Query embedding and knowledge graph embedding. Through this method, the missing knowledge was inferred and the predicate embedding of the knowledge graph related to a specific query was used for good performance. Also, an experiment was conducted using the MetaQA dataset to prove the better performance of the proposed method compared with the existing methods. For the experiment, we used a knowledge graph having movies as a domain. Based on the assumption of the entire knowledge graph and the missing knowledge graph, we experimented on the knowledge graph in which 50% of the triples were randomly omitted. Apparently, better performance than the existing method was obtained.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr