Digital Library[ Search Result ]
Generating Relation Descriptions with Large Language Model for Link Prediction
http://doi.org/10.5626/JOK.2024.51.10.908
The Knowledge Graph is a network consisting of entities and the relations between them. It is used for various natural language processing tasks. One specific task related to the Knowledge Graph is Knowledge Graph Completion, which involves reasoning with known facts in the graph and automatically inferring missing links. In order to tackle this task, studies have been conducted on both link prediction and relation prediction. Recently, there has been significant interest in a dual-encoder architecture that utilizes textual information. However, the dataset for link prediction only provides descriptions for entities, not for relations. As a result, the model heavily relies on descriptions for entities. To address this issue, we utilized a large language model called GPT-3.5-turbo to generate relation descriptions. This allows the baseline model to be trained with more comprehensive relation information. Moreover, the relation descriptions generated by our proposed method are expected to improve the performance of other language model-based link prediction models. The evaluation results for link prediction demonstrate that our proposed method outperforms the baseline model on various datasets, including Korean ConceptNet, WN18RR, FB15k-237, and YAGO3-10. Specifically, we observed improvements of 0.34%p, 0.11%p, 0.12%p, and 0.41%p in terms of Mean Reciprocal Rank (MRR), respecitvely.
A Study of Metric and Framework Improving Fairness-utility Trade-off in Link Prediction
Heeyoon Yang, YongHoon Kang, Gahyung Kim, Jiyoung Lim, SuHyun Yoon, Ho Seung Kim, Jee-Hyong Lee
http://doi.org/10.5626/JOK.2023.50.2.179
The advance in artificial intelligence (AI) technology has shown remarkable improvements over the last decade. However, sometimes, AI makes biased predictions based on real-world big data that intrinsically contain discriminative social factors. This problem often arises in friend recommendations in Social Network Services (SNS). In the case of social network datasets, Graph Neural Network (GNN) is utilized for training these datasets, but it has a high tendency to connect similar nodes (Homophily effect). Furthermore, it is more likely to make a biased prediction based on socially sensitive attributes, such as, gender or religion, making it ethically more problematic. To overcome these problems, various fairness-aware AI models and fairness metrics have been proposed. However, most of the studies used different metrics to evaluate fairness and did not consider the trade-off relationship that existed between accuracy and fairness. Thus, we propose a novel fairness metric called Fairβ-metri which takes both accuracy and prediction into consideration, and a framework called FairU that shows outstanding performance in the proposed metric.
Knowledge Graph Embedding with Entity Type Constraints
Seunghwan Kong, Chanyoung Chung, Suheon Ju, Joyce Jiyoung Whang
http://doi.org/10.5626/JOK.2022.49.9.773
Knowledge graph embedding represents entities and relationships in the feature space by utilizing the structural properties of the graph. Most knowledge graph embedding models rely only on the structural information to generate embeddings. However, some real-world knowledge graphs include additional information such as entity types. In this paper, we propose a knowledge graph embedding model by designing a loss function that reflects not only the structure of a knowledge graph but also the entity-type information. In addition, from the observation that certain type constraints exist on triplets based on their relations, we present a negative sampling technique considering the type constraints. We create the SMC data set, a knowledge graph with entity-type restrictions to evaluate our model. Experimental results show that our model outperforms the other baseline models.
Knowledge Graph Embedding for Link Prediction using Node-Link Interaction-based Graph Attention Networks
http://doi.org/10.5626/JOK.2022.49.7.555
Knowledge graphs are structures that express knowledge in the real world in the form of nodes and links-based triple form. These knowledge graphs are incomplete and many embedding techniques have been studied to effectively represent nodes and links in low-dimensional vector spaces to find other missing relationships. Recently, many neural network-based knowledge graph link prediction methods have been studied. However existing models consider nodes and links independently when determining the importance of a triple to a node which makes it difficult to reflect the interaction between nodes and links. In this paper, we propose an embedding method that will be used to analyze the importance of triple units by simultaneously considering nodes and links using composition operators, and at the same time prove that the model outperforms other methods in knowledge graph link prediction.
Knowledge Graph Completion using Hyper-class Information and Pre-trained Language Model
http://doi.org/10.5626/JOK.2021.48.11.1228
Link prediction is a task that aims to predict missing links in knowledge graphs. Recently, several link prediction models have been proposed to complete the knowledge graphs and have achieved meaningful results. However, the previous models used only the triples" internal information in the training data, which may lead to an overfitting problem. To address this problem, we propose Hyper-class Information and Pre-trained Language Model (HIP) that performs hyper-class prediction and link prediction through a multi-task learning. HIP learns not only contextual relationship of triples but also abstractive meanings of entities. As a result, it learns general information of the entities and forces the entities connected to the same hyper-class to have similar embeddings. Experimental results show significant improvement in Hits@10 and Mean Rank (MR) compared to KG-BERT and MTL-KGC.
Path Embedding-Based Knowledge Graph Completion Approach
Batselem Jagvaral, Min-Sung Kim, Young-Tack Park
http://doi.org/10.5626/JOK.2020.47.8.722
Knowledge graphs are widely used in question answering systems. However, in these circumstances most of the relations between the entities in the knowledge graph tend to be missing. To solve this issue, we propose a CNN(Convolutional Neural Network) + BiLSTM(Bidirectional LSTM) based approach to infer missing links in the knowledge graphs. Our method embeds paths connecting two entities into a low-dimensional space via CNN + BiLSTM. Then, an attention operation is used to attentively combine path embeddings to represent two entities. Finally, we measure the similarity between the target relation and representation of the entities to predict whether or not the relation connects those entities. By combining a CNN and BiLSTM, we are able to take advantage of the CNN’s ability to recognize local patterns and the LSTM’s ability to produce entity and relation ordering. In this way, it is possible to effectively identify low-dimensional path features and predict the relationships between entities using the learned features. In our experiments, we performed link prediction tasks on 4 different knowledge graphs and showed that our method achieves comparable results to state-of-the-art methods.
Approach for Managing Multiple Class Membership in Knowledge Graph Completion Using Bi-LSTM
Jae-Seung Roh, Batselem Jagvaral, Wan-Gon Lee, Young-Tack Park
http://doi.org/10.5626/JOK.2020.47.6.559
Knowledge graphs that represent real world information in a structured way are widely used in areas, such as Web browsing and recommendation systems. But there is a problem of missing links between entities in knowledge graphs. To resolve this issue, various studies using embedding techniques or deep learning have been proposed. Especially, the recent study combining CNN and Bidirectional-LSTM has shown high performance compared to previous studies. However, in the previous study, if multiple class types are defined for single entity, the amount of training data exponentially increases with the training time. Also, if class type information for an entity is not defined, training data for that entity cannot be generated. Thus, to enable the generation of training data for such entities and manage multiple class membership in knowledge graph completion, we propose two approaches using pre-trained embedding vectors of knowledge graph and the concept of vector addition. To evaluate the performance of the methods proposed in this paper, we conducted comparative experiments with the existing knowledge completion studies on NELL-995 and FB15K-237 datasets, and obtained MAP 1.6%p and MRR 1.5%p higher than that of the previous studies.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr