The back-propagation algorithm has a crucial formula:

Enwjiδjzi\frac{\partial E_n}{\partial w_{ji}} \leftarrow \delta_j z_i

(1)

and a formula about δⱼ. Here I would like pointed out that the formula (1) can be considered as Hebbian Law.

Here zᵢ = h(aᵢ) (Ref. 1), it is the output of the neuron i after the activation function. Therefore it is the activity of the neuron i of the forward neural network. At the same time the backward activity of neuron j is δⱼ, i.e. the activities of connection of i and j are zᵢ and δⱼ. And here the back-progation law told us the its update is proportional to δⱼ zᵢ. This is exactly the Hebbian law.

To the best of my knowledge, this observation is not appeared anywhere in the literature, internet, or video. This observation is lectured in my neural network course for graduate students for more than 5 years.

Reference

C. M. Bishop and H. Bishop. Deep Learning: Foundations and Concepts, p. 237. Springer, 2024.