Digital Library[ Search Result ]
Contract Eligibility Verification Enhanced by Keyword and Contextual Embeddings
Sangah Lee, Seokgi Kim, Eunjin Kim, Minji Kang, Hyopil Shin
http://doi.org/10.5626/JOK.2022.49.10.848
Contracts need to be reviewed to be verified if they include all the essential clauses for them to be valid. Such clauses are highly formal and repetitive regardless of the kinds of contracts, and automated legal technologies are required for legal text comprehension. In this paper, we have constructed a simple item-by-item classification model for clauses in contracts to estimate contract eligibility by addressing formal and repetitive properties of contract clauses. We have used keyword embeddings based on conventional requirements of contracts and concatenate them to sentence embeddings of clauses, extracted from a BERT model fine-tuned with legal documents. The contract eligibility can be verified by the predicted labels. Based on our methods, we report reasonable performances with the accuracy of 90.57 and 90.64, and an F1-score of 93.27 and 93.26, using additional keyword embeddings with BERT embeddings.
Knowledge Graph Embedding with Entity Type Constraints
Seunghwan Kong, Chanyoung Chung, Suheon Ju, Joyce Jiyoung Whang
http://doi.org/10.5626/JOK.2022.49.9.773
Knowledge graph embedding represents entities and relationships in the feature space by utilizing the structural properties of the graph. Most knowledge graph embedding models rely only on the structural information to generate embeddings. However, some real-world knowledge graphs include additional information such as entity types. In this paper, we propose a knowledge graph embedding model by designing a loss function that reflects not only the structure of a knowledge graph but also the entity-type information. In addition, from the observation that certain type constraints exist on triplets based on their relations, we present a negative sampling technique considering the type constraints. We create the SMC data set, a knowledge graph with entity-type restrictions to evaluate our model. Experimental results show that our model outperforms the other baseline models.
Efficient Compositional Translation Embedding for Visual Relationship Detection
Yu-Jung Heo, Eun-Sol Kim, Woo Suk Choi, Kyoung-Woon On, Byoung-Tak Zhang
http://doi.org/10.5626/JOK.2022.49.7.544
Scene graphs are widely used to express high-order visual relationships between objects present in an image. To generate the scene graph automatically, we propose an algorithm that detects visual relationships between objects and predicts the relationship as a predicate. Inspired by the well-known knowledge graph embedding method TransR, we present the CompTransR algorithm that i) defines latent relational subspaces considering the compositional perspective of visual relationships and ii) encodes predicate representations by applying transitive constraints between the object representations in each subspace. Our proposed model not only reduces computational complexity but also outperformed previous state-of-the-art performance in predicate detection tasks in three benchmark datasets: VRD, VG200, and VrR-VG. We also showed that a scene graph could be applied to the image-caption retrieval task, which is one of the high-level visual reasoning tasks, and the scene graph generated by our model increased retrieval performance.
Knowledge Graph Embedding for Link Prediction using Node-Link Interaction-based Graph Attention Networks
http://doi.org/10.5626/JOK.2022.49.7.555
Knowledge graphs are structures that express knowledge in the real world in the form of nodes and links-based triple form. These knowledge graphs are incomplete and many embedding techniques have been studied to effectively represent nodes and links in low-dimensional vector spaces to find other missing relationships. Recently, many neural network-based knowledge graph link prediction methods have been studied. However existing models consider nodes and links independently when determining the importance of a triple to a node which makes it difficult to reflect the interaction between nodes and links. In this paper, we propose an embedding method that will be used to analyze the importance of triple units by simultaneously considering nodes and links using composition operators, and at the same time prove that the model outperforms other methods in knowledge graph link prediction.
Effective Transfer Learning in Text Classification with the Label-Based Discriminative Feature Learning
http://doi.org/10.5626/JOK.2022.49.3.214
The performance of the natural language processing with transfer learning methodology has improved by pre-training language models with a large amount of general data and applying them on downstream tasks. However, the problem is that it learns general features rather than those specific to the downstream tasks as the data used in pre-training is irrelevant to the downstream tasks. This paper proposes a novel learning method for embeddings of pre-trained models to learn specific features of the downstream tasks. The proposed method is to learn the label feature of the downstream tasks through contrast learning with label embedding and sampled data pairs. To demonstrate the performance of the proposed method, we conducted experiments on sentence classification datasets and evaluated whether features of downstream tasks have been learned through PCA(Principal component analysis) and clustering on embeddings.
Graph Embedding-Based Point-Of-Interest Recommendation Considering Weather Features
Kun Woo Lee, Jongseon Kim, Yon Dohn Chung
http://doi.org/10.5626/JOK.2022.49.3.221
As the Location-Based Services (LBS) grow rapidly, the Point-Of-Interest (POI) recommendation becomes an active research area to provide users appropriate information relevant to their locations. Recently, translation-based recommendation systems using graph embedding, such as TransRec, are attracting great attention. In this paper, we discovered some drawbacks of TransRec; it is limited in expressing the complex relationship between users and POIs, and the relation embedding is fixed without considering weather features. We propose WAPTRec, a graph embedding-based POI recommendation method considering the weather, that overcomes the drawback of TransRec. WAPTRec can rep resent the same POI embedding in different ways according to users by using a category projection matrix and attention mechanism. In addition, it provides better recommendation accuracy by utilizing the users’ movement history, category of POIs and weather features. Experiments using public datasets illustrated that WAPTRec outperformed the conventional translation-based recommendation methods.
Korean Text Summarization using MASS with Copying and Coverage Mechanism and Length Embedding
Youngjun Jung, Changki Lee, Wooyoung Go, Hanjun Yoon
http://doi.org/10.5626/JOK.2022.49.1.25
Text summarization is a technology that generates a summary including important and essential information from a given document, and an end-to-end abstractive summarization model using a sequence-to-sequence model is mainly studied. Recently, a transfer learning method that performs fine-tuning using a pre-training model based on large-scale monolingual data has been actively studied in the field of natural language processing. In this paper, we applied the copying mechanism method to the MASS model, conducted pre-training for Korean language generation, and then applied it to Korean text summarization. In addition, coverage mechanism and length embedding were additionally applied to improve the summarization model. As a result of the experiment, it was shown that the Korean text summarization model, which applied the copying and coverage mechanism method to the MASS model, showed a higher performance than the existing models, and that the length of the summary could be adjusted through length embedding.
Method for the Automatic Generation of Training Sets for Word Embedding Reflecting Sentiment Information
Dahee Lee, Won-Min Lee, Byung-Won On
http://doi.org/10.5626/JOK.2022.49.1.42
Word embedding is a method of expressing a word as a vector. However, since existing word embedding methods predict words that appear together, they are expressed as similar vectors even if they have different emotions. When building a sentiment analysis model using this, sentences with similar patterns may be classified into the same polarity, which is one of the factors that degrade the performance of the emotional analysis model. In this paper, to address the problem, we proposed the automatic generation of a training set for word embedding reflecting sentiment information using morpheme analysis, dependence parsing, and a sentiment dictionary. Using sentiment-specific word embedding vectors generated by the proposed model, we showed that the proposed sentiment-specific word embedding model outperformed the existing word embedding models including CBOW, Skip-Gram, FastText, ELMo, and BERT.
GPT-2 for Knowledge Graph Completion
http://doi.org/10.5626/JOK.2021.48.12.1281
Knowledge graphs become an important resource in many artificial intelligence (AI) tasks. Many studies are being conducted to complete the incomplete knowledge graph. Among them, interest in research that knowledge completion by link prediction and relation prediction is increasing. The most talked-about language models in AI natural language processing include BERT and GPT-2, among which KG-BERT wants to solve knowledge completion problems with BERT. In this paper, we wanted to solve the problem of knowledge completion by utilizing GPT-2, which is the biggest recent issue in the language model of AI. Triple information-based knowledge completion and path-triple-based knowledge completion were proposed and explained as methods to solve the knowledge completion problem using the GPT-2 language model. The model proposed in this paper was defined as KG-GPT2, and experiments were conducted by comparing the link prediction and relationship prediction results of TransE, TransR, KG-BERT, and KG-GPT2 to evaluate knowledge completion performance. For link prediction, WN18RR, FB15k-237, and UMLS datasets were used, and for relation prediction, FB15K was used. As a result of the experiment, in the case of link prediction in the path- triple-based knowledge completion of KG-GPT2, the best performance was recorded for all experimental datasets except UMLS. In the path-triple-based knowledge completion of KG-GPT2, the model"s relationship prediction work also recorded the best performance for the FB15K dataset.
Person Re-Identification Using an Attention Pyramid for Local Multiscale Feature Embedding Extracted from a Person’s Image
http://doi.org/10.5626/JOK.2021.48.12.1305
In this paper, a person re-identification scheme using the dual pyramid adapting attention mechanisms to extract more elaborate local feature embedding by excluding the noises caused by the unnecessary backgrounds in person’s image is proposed. With the dual pyramid of local and scale ones, the spatial attention is used to suppress the noise effects caused by unnecessary backgrounds, and the channel attention is used to emphasize the relatively important multiscale features when the local feature embedding is constructed. In the experiments, the proposed scheme was compared with other cases in which the attention module is not used for each pyramid to confirm the optimal configuration and compared based on the rank-1 accuracy with the state-of-the-art studies for the person re-identification. According to the experimental results, the proposed method showed a maximum rank-1 accuracy of 99.4%, which is higher by at least about 0.2% and at most by about 13.8% than previous works.
Search
Journal of KIISE
- ISSN : 2383-630X(Print)
- ISSN : 2383-6296(Electronic)
- KCI Accredited Journal
Editorial Office
- Tel. +82-2-588-9240
- Fax. +82-2-521-1352
- E-mail. chwoo@kiise.or.kr