Project

Self-normalizing Neural Networks

Goal: Self-normalizing Neural Networks automatically perform normalization like batch normalization or layer normalization via moving towards a stable fixed point. This stable fixed point allows optimal learning without vanishing or exploding gradients. This stable fixed point is moment conserving (mean activation is tends towards zero and the variance tends towards one). Self-normalizing Neural Networks are the key to construct large AI systems since building blocks can be combined while these blocks ensure efficient learning via self-normalization.

Updates
0 new
0
Recommendations
0 new
0
Followers
0 new
107
Reads
0 new
874

Project log

Sepp Hochreiter
added a project goal
Self-normalizing Neural Networks automatically perform normalization like batch normalization or layer normalization via moving towards a stable fixed point. This stable fixed point allows optimal learning without vanishing or exploding gradients. This stable fixed point is moment conserving (mean activation is tends towards zero and the variance tends towards one). Self-normalizing Neural Networks are the key to construct large AI systems since building blocks can be combined while these blocks ensure efficient learning via self-normalization.
 
dear professor Sepp Hochreiter , how can I be in such project