CONF Thimm-94.2/IDIAP Weight Initialization for High Order and Multilayer Perceptrons Thimm, Georg Fiesler, Emile Aguilar, Marc Ed. activation function comparison of weight initialization methods connectionism high(er) order neural network high(er) order perceptron initial weight initial weight distribution interconnection strength learning rate multilayer perceptron neural computation neural network neurocomputing optimization sigma-pi connection weight initialization SI Group for Parallel Systems - Proceedings of the '94 SIPAR--Workshop on Parallel and Distributed Computing 1994 Institute of Informatics University, P\'erolles, Chemin du Mus\'ee 3, CH-1700 Fribourg, Switzerland October 1994 87-90 Proper weight initialization is one of the most important prerequisites for fast convergence of feed-forward neural networks like high order and multilayer perceptrons. In order to determine the optimal value of the initial weight variance (or range,',','), which is a important parameter of random weight initialization methods for high order perceptrons, a wide range of experiments (more than $200,000$ simulations) was performed, using seven different data sets, three weight distributions, three activation functions, and several network orders. The results of these experiments are compared to weight initialization techniques for multilayer perceptrons, which leads to the proposal of a suitable weight initialization method for high order perceptrons. Experiments over a large range of initial weight variances are performed (more than $20,000$ simulations) for multilayer perceptrons and compared to weight initialization methods proposed by other authors. The results of this comparison are justified by sufficiently small confidence intervals.