ARTICLE Thimm-97.1/IDIAP High Order and Multilayer Perceptron Initialization Thimm, Georg Fiesler, Emile activation function comparison of weight initialization methods connectionism high(er) order neural network high(er) order perceptron initial weight initial weight distribution interconnection strength learning rate multilayer perceptron neural computation neural network neural network initialization neurocomputing optimization random weight initialization real-world benchmark sigma-pi connection weight initialization http://publications.idiap.ch/index.php/publications/showcite/Thimm-95.1 Related documents IEEE Transactions on Neural Networks 8 02 1045-9227 1997 Proper initialization is one of the most important prerequisites for fast convergence of feed-forward neural networks like high order and multilayer perceptrons. This publication aims at determining the optimal variance (or range) for the initial weights and biases, which is the principal parameter of random initialization methods for both types of neural networks. An overview of random weight initialization methods for multilayer perceptrons is presented. These methods are extensively tested using eight real- world benchmark data sets and a broad range of initial weight variances by means of more than $30,000$ simulations, in the aim to find the best weight initialization method for multilayer perceptrons. For high order networks, a large number of experiments (more than $200,000$ simulations) was performed, using three weight distributions, three activation functions, several network orders, and the same eight data sets. The results of these experiments are compared to weight initialization techniques for multilayer perceptrons, which leads to the proposal of a suitable initialization method for high order perceptrons. The conclusions on the initialization methods for both types of networks are justified by sufficiently small confidence intervals of the mean convergence times. CHAPTER Thimm-95.1/IDIAP Neural Network Initialization Thimm, Georg Fiesler, Emile Mira, J. Ed. Sandoval, F. Ed. From Natural to Artificial Neural Computation Lecture Notes in Computer Science 930 535-542 3-540-59497-3 1995 Springer Verlag Berlin