Information geometry of multilayer perceptron

The multilayer perceptron is a brain-inspired model of learning machines. Nowadays, it is used in wide areas of applications, but its ability has not yet been fully explored. Information geometry gives a powerful method to study its capacity, and also gives a new learning method. The present talk gives an historical overview of the perceptron, and then explains how the geometry of neuromanifold of multilayer perceptron reveals the whole structure of such hierarchical systems. Such a hierarchical system includes singularity, which influences its performance very badly, e.g., slow convergence as is in the plateau phenomenon. The Riemannian structure and singularity are well understood from the geometry, and a new efficient learning algorithm called the natural gradient method is introduced.