Soha Lee  Heesung Yang  Hyeyoung Park

10.5626/JOK.2024.51.5.445

Backpropagation biologically plausible learning feedback alignment direct feedback alignment weight mirror

Error backpropagation algorithm is a core learning algorithm of neural networks and, until recently, has been used in various deep learning models. However, the weight update rule of error backpropagation, in which the error signal of the upper layer is sequentially transmitted to the lower layer and the weight values of the upper layer that are used to update the lower layer weights, has a problem of biological implausibility and computational inefficiency. To address these issues, learning methods using separate backward weights have been proposed, but they are still at an early stage and require further analysis and improvement from various perspectives. In this paper, we proposed a new learning method by combining the direct feedback alignment method, which directly projects the errors of the last layer into each hidden layer, and a weight mirror method with a separate step for updating backward weights. The proposed method overcomes the limitations of learning methods to implement a weight update method that is biologically plausible and allows for more efficient parallel learning. We confirmed the potential of the proposed method through experiments on various benchmark datasets.