Digital Library[ Search Result ]
Generating Relation Descriptions with Large Language Model for Link Prediction
http://doi.org/10.5626/JOK.2024.51.10.908
The Knowledge Graph is a network consisting of entities and the relations between them. It is used for various natural language processing tasks. One specific task related to the Knowledge Graph is Knowledge Graph Completion, which involves reasoning with known facts in the graph and automatically inferring missing links. In order to tackle this task, studies have been conducted on both link prediction and relation prediction. Recently, there has been significant interest in a dual-encoder architecture that utilizes textual information. However, the dataset for link prediction only provides descriptions for entities, not for relations. As a result, the model heavily relies on descriptions for entities. To address this issue, we utilized a large language model called GPT-3.5-turbo to generate relation descriptions. This allows the baseline model to be trained with more comprehensive relation information. Moreover, the relation descriptions generated by our proposed method are expected to improve the performance of other language model-based link prediction models. The evaluation results for link prediction demonstrate that our proposed method outperforms the baseline model on various datasets, including Korean ConceptNet, WN18RR, FB15k-237, and YAGO3-10. Specifically, we observed improvements of 0.34%p, 0.11%p, 0.12%p, and 0.41%p in terms of Mean Reciprocal Rank (MRR), respecitvely.
Knowledge Graph Embedding with Entity Type Constraints
Seunghwan Kong, Chanyoung Chung, Suheon Ju, Joyce Jiyoung Whang
http://doi.org/10.5626/JOK.2022.49.9.773
Knowledge graph embedding represents entities and relationships in the feature space by utilizing the structural properties of the graph. Most knowledge graph embedding models rely only on the structural information to generate embeddings. However, some real-world knowledge graphs include additional information such as entity types. In this paper, we propose a knowledge graph embedding model by designing a loss function that reflects not only the structure of a knowledge graph but also the entity-type information. In addition, from the observation that certain type constraints exist on triplets based on their relations, we present a negative sampling technique considering the type constraints. We create the SMC data set, a knowledge graph with entity-type restrictions to evaluate our model. Experimental results show that our model outperforms the other baseline models.
Knowledge Graph Embedding for Link Prediction using Node-Link Interaction-based Graph Attention Networks
http://doi.org/10.5626/JOK.2022.49.7.555
Knowledge graphs are structures that express knowledge in the real world in the form of nodes and links-based triple form. These knowledge graphs are incomplete and many embedding techniques have been studied to effectively represent nodes and links in low-dimensional vector spaces to find other missing relationships. Recently, many neural network-based knowledge graph link prediction methods have been studied. However existing models consider nodes and links independently when determining the importance of a triple to a node which makes it difficult to reflect the interaction between nodes and links. In this paper, we propose an embedding method that will be used to analyze the importance of triple units by simultaneously considering nodes and links using composition operators, and at the same time prove that the model outperforms other methods in knowledge graph link prediction.
GPT-2 for Knowledge Graph Completion
http://doi.org/10.5626/JOK.2021.48.12.1281
Knowledge graphs become an important resource in many artificial intelligence (AI) tasks. Many studies are being conducted to complete the incomplete knowledge graph. Among them, interest in research that knowledge completion by link prediction and relation prediction is increasing. The most talked-about language models in AI natural language processing include BERT and GPT-2, among which KG-BERT wants to solve knowledge completion problems with BERT. In this paper, we wanted to solve the problem of knowledge completion by utilizing GPT-2, which is the biggest recent issue in the language model of AI. Triple information-based knowledge completion and path-triple-based knowledge completion were proposed and explained as methods to solve the knowledge completion problem using the GPT-2 language model. The model proposed in this paper was defined as KG-GPT2, and experiments were conducted by comparing the link prediction and relationship prediction results of TransE, TransR, KG-BERT, and KG-GPT2 to evaluate knowledge completion performance. For link prediction, WN18RR, FB15k-237, and UMLS datasets were used, and for relation prediction, FB15K was used. As a result of the experiment, in the case of link prediction in the path- triple-based knowledge completion of KG-GPT2, the best performance was recorded for all experimental datasets except UMLS. In the path-triple-based knowledge completion of KG-GPT2, the model"s relationship prediction work also recorded the best performance for the FB15K dataset.
Knowledge Graph Completion using Hyper-class Information and Pre-trained Language Model
http://doi.org/10.5626/JOK.2021.48.11.1228
Link prediction is a task that aims to predict missing links in knowledge graphs. Recently, several link prediction models have been proposed to complete the knowledge graphs and have achieved meaningful results. However, the previous models used only the triples" internal information in the training data, which may lead to an overfitting problem. To address this problem, we propose Hyper-class Information and Pre-trained Language Model (HIP) that performs hyper-class prediction and link prediction through a multi-task learning. HIP learns not only contextual relationship of triples but also abstractive meanings of entities. As a result, it learns general information of the entities and forces the entities connected to the same hyper-class to have similar embeddings. Experimental results show significant improvement in Hits@10 and Mean Rank (MR) compared to KG-BERT and MTL-KGC.
Knowledge Completion System using Neuro-Symbolic-based Rule Induction and Inference Engine
Won-Chul Shin, Hyun-Kyu Park, Young-Tack Park
http://doi.org/10.5626/JOK.2021.48.11.1202
Recently, there have been several studies on knowledge completion methods aimed to solve the incomplete knowledge graphs problem. Methods such as Neural Theorem Prover (NTP), which combines the advantages of deep learning methods and logic systems, have performed well over existing methods. However, NTP faces challenges in processing large-scale knowledge graphs because all the triples of the knowledge graph are involved in the computation to obtain prediction results for one input. In this paper, we propose an integrated system of deep learning and logic inference methods that can learn vector representations of symbols from improved models of computational complexity of NTP to rule induction, and perform knowledge inference from induced rules using inference engines. In this paper, for rule-induction performance verification of the rule generation model, we compared test data inference ability with NTP using induced rules on Nations, Kinship, and UMLS data set. Experiments with Kdata and WiseKB knowledge inference through inference engines resulted in a 30% increase in Kdata and a 95% increase in WiseKB compared to the knowledge graphs used in experiments.
An Explainable Knowledge Completion Model Using Explanation Segments
Min-Ho Lee, Wan-Gon Lee, Batselem Jagvaral, Young-Tack Park
http://doi.org/10.5626/JOK.2021.48.6.680
Recently, a large number of studies that used deep learning have been conducted to predict new links in incomplete knowledge graphs. However, link prediction using deep learning has a major limitation as the inferred results cannot be explained. We propose a high-utility knowledge graph prediction model that yields explainable inference paths supporting the inference results. We define paths to the object from the knowledge graph using a path ranking algorithm and define them as the explanation segments. Then, the generated explanation segments are embedded using a Convolutional neural network (CNN) and a Bidirectional Long short-term memory (BiLSTM). The link prediction model is then trained by applying an attention mechanism, based on the calculation of the semantic similarity between the embedded explanation segments and inferred candidate predicates to be inferred. The explanation segment suitable for link prediction explanation is selected based on the measured attention scores. To evaluate the performance of the proposed method, a link prediction comparison experiment and an accuracy verification experiment are performed to measure the proportion of the explanation segments suitable to explain the link prediction results. We used the benchmark datasets NELL-995, FB15K-237, and countries for the experiment, and accuracy verification experiments showed the accuracies of 89%, 44%, and 97%, respectively. Compared with the existing method, the NELL-995, FB15K-237 data exhibited 35%p and 21%p higher performance on average.
Knowledge Completion System through Learning the Relationship between Query and Knowledge Graph
Min-Sung Kim, Min-Ho Lee, Wan-Gon Lee, Young-Tack Park
http://doi.org/10.5626/JOK.2021.48.6.649
The knowledge graph is a network comprising of relationships between the entities. In a knowledge graph, there exists a problem of missing or incorrect relationship connection with the specific entities. Numerous studies have proposed learning methods using artificial neural networks based on natural language embedding to solve the problems of the incomplete knowledge graph. Various knowledge graph completion systems are being studied using these methods. In this paper, a system that infers missing knowledge using specific queries and knowledge graphs is proposed. First, a topic is automatically extracted from a query, and topic embedding is obtained from the knowledge graph embedding module. Next, a new triple is inferred by learning the relationship between the topic from the knowledge graph and the query by using Query embedding and knowledge graph embedding. Through this method, the missing knowledge was inferred and the predicate embedding of the knowledge graph related to a specific query was used for good performance. Also, an experiment was conducted using the MetaQA dataset to prove the better performance of the proposed method compared with the existing methods. For the experiment, we used a knowledge graph having movies as a domain. Based on the assumption of the entire knowledge graph and the missing knowledge graph, we experimented on the knowledge graph in which 50% of the triples were randomly omitted. Apparently, better performance than the existing method was obtained.
A Knowledge Completion Approach using Rule Generation based on Neuro-Symbolic Method
Jea-Seung Roh, Won-Chul Shin, Hyun-Kyu Park, Young-Tack Park
http://doi.org/10.5626/JOK.2021.48.4.425
A knowledge graph is a structured representation of real-world knowledge and is designed by collecting information from various sources. These knowledge graphs are networks that represent relationships between data and are applied in various fields of artificial intelligence; however, there exists problems related to incomplete knowledge due to the omission of entities or omission links between the entities. To resolve the problems, research on automatic knowledge completion techniques is necessitated. Consequently, various studies have been examined including embedding techniques, deep learning or symbolic rule inference using ontology. Although automatic knowledge completion can be efficiently performed through the above-mentioned methods, deep learning methods require a large amount of learning data due to data-driven processing methods, and there exist problems related to the results that are hard to explain. Futhermore, ontology-based methods require ontology and rules that are defined by the experts. To overcome this limitation, in this study, we propose an automatic knowledge completion method by explicitly extracting the implicit rules from the data based on the Neuro-Symbolic method. For rule extraction, we have implemented a symbolic unification based embedding path and defined a cost function for it to automatically generate the rules. Compared with the approaches presented in previous embedding studies, the proposed method demonstrates the superiority of the Neuro-Symbolic method concerning speed and performance. To assess the performance of the proposed method, for datasets like Nations, UMLS, and Kinship, experiments were conducted in comparison with the approach of the state-of-the-art knowledge completion studies. Consequently, an immense reduction in the training time and 37.5%p increase in the average performance were observed.
Incorrect Triple Detection Using Knowledge Graph Embedding and Adaptive Clustering
Won-Chul Shin, Jea-Seung Roh, Young-Tack Park
http://doi.org/10.5626/JOK.2020.47.10.958
Recently, with the increase in the amount of information from the development of the Internet, research using large-capacity knowledge graphs is being actively conducted. Additionally, as knowledge graphs are used for various research and services, there is a need to secure quality knowledge graphs. However, there is a lack of research to detect errors within the knowledge graphs to obtain quality knowledge graphs. Previous studies using the embedding and clustering for error triple detection showed good performance. However, in the process of the cluster optimization, there was a problem that the characteristics of each cluster could not be factored using the same threshold collectively. In this paper, to resolve these problems, we propose an adaptive clustering model in which clustering is conducted by finding and applying the optimum threshold for each cluster with the embedding for knowledge graph for error triple detection in the knowledge graph. To evaluate the performance of the method proposed in this paper, the existing error triple detection studies and comparative experiments were conducted on three datasets, DBpeida, Frebase and WiseKB, and the high performance was confirmed by an average of 5.3% based on the F1-Score.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr