Yeenheui Yeen  HaeIn Jung  MinJu Kim  Jeong Yang  Minhye Kim  Hyunji Jang  Myoung-Wan Koo

10.5626/JOK.2024.51.7.592

Explainable Artificial Intelligence Large Language Model legal technology prompt engineering

This study introduces a stepwise hierarchical prompting method suitable for large-scale generative language models in complex legal reasoning tasks. Complex logical problems are decomposed into multiple steps, accumulating results from each step to set prompts for subsequent ones. It was confirmed that when this method was applied to the evaluation process of the Korean bar exam's essay-type questions, it achieved better results than fine-tuning with original data. Notably, in the final evaluation by legal experts, both tasks showed a human precision of over 0.70, indicating its capability to produce interpretations based on accurate evidence. This prompting technique suggests a solution to the hallucination issue in large language models and demonstrates its effective application. Future research will consider the introduction of a specialized retriever to reflect more accurate legal knowledge in the large language model, aiming to incorporate more precise evidence into prompts. While the current research applied the prompting method only to the legal field, it is expected to be applicable to other complex logical reasoning tasks that rely on specialized knowledge.