Enhancing Molecular Understanding in LLMs through Multimodal Graph-SMILES Representations

http://doi.org/10.5626/JOK.2025.52.5.379

Recent advancements in large language models (LLMs) have shown remarkable performace across various tasks, with increasing focus on multimodal research. Notably, BLIP-2 can enhance performance by efficiently aligning images and text using a Q-Former, aided by an image encoder pre-trained on multimodal data. Inspired by this, the MolCA model extends BLIP-2 to the molecular domain to improve performance. However, the graph encoder in MolCA is pre-trained on unimodal data, necessitating updates during model training, which is a limitation. Therefore, this paper replaced it with a graph encoder pre-trained on multimodal data and frozen while training the model. Experimental results showed that using the graph encoder pre-trained on multimodal data generally enhanced performance. Additionally, unlike the graph encoder pre-trained on unimodal data, which performed better when updated, the graph encoder pre-trained on multimodal data achieved superior results across all metrics when frozen.

Enhancing Retrieval-Augmented Generation Through Zero-Shot Sentence-Level Passage Refinement with LLMs

Taeho Hwang, Soyeong Jeong, Sukmin Cho, Jong C. Park

http://doi.org/10.5626/JOK.2025.52.4.304

This study presents a novel methodology designed to enhance the performance and effectiveness of Retrieval-Augmented Generation (RAG) by utilizing Large Language Models (LLMs) to eliminate irrelevant content at the sentence level from retrieved documents. This approach refines the content of passages exclusively through LLMs, avoiding the need for additional training or data, with the goal of improving the performance in knowledge-intensive tasks. The proposed method was tested in an open-domain question answering (QA) environment, where it demonstrated its ability to effectively remove unnecessary content and outperform over traditional RAG methods. Overall, our approach has proven effective in enhancing performance compared to conventional RAG techniques and has shown the capability to improve RAG's accuracy in a zero-shot setting without requiring additional training data.

KULLM: Learning to Construct Korean Instruction-Following Large Language Models

Seungjun Lee, Yoonna Jang, Jeongwook Kim, Taemin Lee, Heuiseok Lim

http://doi.org/10.5626/JOK.2024.51.9.817

The emergence of Large Language Models (LLMs) has revolutionized the research paradigm in natural language processing. While instruction-tuning techniques have been pivotal in enhancing LLM performance, the majority of current research has focused predominantly on English. This study addresses the need for multilingual approaches by presenting a method for developing and evaluating Korean instruction-following models. We fine-tuned LLM models using Korean instruction datasets and conducted a comprehensive performance analysis using various dataset combinations. The resulting Korean instruction-following model is made available as an open-source resource, contributing to the advancement of Korean LLM research. Our work aims to bridge the language gap in LLM development and promote more inclusive AI technologies.