Neural Network Learning Method using Weight Mirroring and Direct Feedback Error

Soha Lee, Heesung Yang, Hyeyoung Park

http://doi.org/10.5626/JOK.2024.51.5.445

Error backpropagation algorithm is a core learning algorithm of neural networks and, until recently, has been used in various deep learning models. However, the weight update rule of error backpropagation, in which the error signal of the upper layer is sequentially transmitted to the lower layer and the weight values of the upper layer that are used to update the lower layer weights, has a problem of biological implausibility and computational inefficiency. To address these issues, learning methods using separate backward weights have been proposed, but they are still at an early stage and require further analysis and improvement from various perspectives. In this paper, we proposed a new learning method by combining the direct feedback alignment method, which directly projects the errors of the last layer into each hidden layer, and a weight mirror method with a separate step for updating backward weights. The proposed method overcomes the limitations of learning methods to implement a weight update method that is biologically plausible and allows for more efficient parallel learning. We confirmed the potential of the proposed method through experiments on various benchmark datasets.