about
The Forward-Forward Algorithm: Some Preliminary Investigations (2022) (arxiv.org)
1 point by lnyan on May 2, 2023 | hide | past | pdf | discuss on HN

In plain words: Instead of the usual backward pass that sends error signals back, each layer runs two forward passes—one on real data, one on fake—and learns to be active for real and quiet for fake. It learned small tasks well enough to study further.

Abstract · The Forward-Forward Algorithm: Some Preliminary Investigations

The aim of this paper is to introduce a new learning procedure for neural networks and to demonstrate that it works well enough on a few small problems to be worth further investigation. The Forward-Forward algorithm replaces the forward and backward passes of backpropagation by two forward passes, one with positive (i.e. real) data and the other with negative data which could be generated by the network itself. Each layer has its own objective function which is simply to have high goodness for positive data and low goodness for negative data. The sum of the squared activities in a layer can be used as the goodness but there are many other possibilities, including minus the sum of the squared activities. If the positive and negative passes could be separated in time, the negative passes could be done offline, which would make the learning much simpler in the positive pass and allow video to be pipelined through the network without ever storing activities or stopping to propagate derivatives.

Geoffrey Hinton
arXiv:2212.13345 · cs.LG · submitted Dec 27, 2022
abstract · pdf · html

add comment on HN
Also discussed: Jan 2023 (2 points, 0 comments)