TY  - JOUR
T1  - semantic role labeling, large language model, in-context learning, example selection, example reordering
AU  - Koo, Kyoseong 
AU  - Shin, Hyeong Jin 
AU  - Lee, Jae Sung 
JO  - Journal of KIISE, JOK
PY  - 2025
DA  - 2025/1/14
DO  - 10.5626/JOK.2025.52.11.961
KW  - Bi-directionalization
KW  - encoder language model
KW  - RWKV model
KW  - lightweight model
AB  - In recent years, Transformer-based language models have been extensively developed and have shown substantial effectiveness across various natural language processing tasks. However, these models often incur significant computational costs in terms of time and memory complexity and they are predominantly designed as unidirectional autoregressive architectures. To address these limitations, research has increasingly focused on developing lightweight and bidirectional alternatives. This paper proposes Bi-RWKV, a bidirectional extension of the lightweight RWKV model, specifically designed for encoder-based language tasks. By examining eight configurations of bidirectional integration for RWKV’s time-mixing and channel-mixing modules, we identify the optimal architecture. To ensure a fair comparison of different model architectures, we maintained consistent hyperparameter values and comparable numbers of model parameters, deliberately omitting pretraining. Experimental results on named entity recognition, chunking, and Korean morphological and part-of-speech tagging demonstrate that Bi-RWKV achieves comparable or superior accuracy to Transformer-based encoders while reducing inference time by a factor of 2.7 to 4.