Create accountLogin

Recent searches

No recent searches

Search options

Only available when logged in.
mastodon.world is one of the many independent Mastodon servers you can use to participate in the fediverse.
Generic Mastodon server for anyone to use.

Administered by:

Server stats:

8.8K
active users

mastodon.world: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.8

#neural

1 post1 participant0 posts today
Public
Philo Sophies

with Dr. : “The of the - or how can learn from ”

There is a away from the purely information-technological-mechanistic, purely data-driven concept of towards increasingly information-biological-polycontextural, structure-driven , () concepts.

More at: philosophies.de/index.php/2024

or: youtu.be/forOGk8k0W8

portrait of Gabriele Scheler
Public
Hacker News

NoProp: Training Neural Networks without Back-propagation or Forward-propagation

arxiv.org/abs/2503.24322

arXiv logo
arXiv.orgNoProp: Training Neural Networks without Back-propagation or Forward-propagationThe canonical deep learning approach for learning requires computing a gradient term at each layer by back-propagating the error signal from the output towards each learnable parameter. Given the stacked structure of neural networks, where each layer builds on the representation of the layer below, this approach leads to hierarchical representations. More abstract features live on the top layers of the model, while features on lower layers are expected to be less abstract. In contrast to this, we introduce a new learning method named NoProp, which does not rely on either forward or backwards propagation. Instead, NoProp takes inspiration from diffusion and flow matching methods, where each layer independently learns to denoise a noisy target. We believe this work takes a first step towards introducing a new family of gradient-free learning methods, that does not learn hierarchical representations -- at least not in the usual sense. NoProp needs to fix the representation at each layer beforehand to a noised version of the target, learning a local denoising process that can then be exploited at inference. We demonstrate the effectiveness of our method on MNIST, CIFAR-10, and CIFAR-100 image classification benchmarks. Our results show that NoProp is a viable learning algorithm which achieves superior accuracy, is easier to use and computationally more efficient compared to other existing back-propagation-free methods. By departing from the traditional gradient based learning paradigm, NoProp alters how credit assignment is done within the network, enabling more efficient distributed learning as well as potentially impacting other characteristics of the learning process.
#HackerNews#NoProp#Neural
ExploreLive feeds
About