REPORT grandvalet:rr07-76/IDIAP Hierarchical Penalization Szafranski, Marie Grandvalet, Yves Morizet-Mahoudeaux, Pierre EXTERNAL http://publications.idiap.ch/attachments/reports/2007/grandvalet-idiap-rr-07-76.pdf PUBLIC Idiap-RR-76-2007 2007 IDIAP To appear in Advances in Neural Information Processing Systems 21 (NIPS 2007) Hierarchical penalization is a generic framework for incorporating prior information in the fitting of statistical models, when the explicative variables are organized in a hierarchical structure. The penalizer is a convex functional that performs soft selection at the group level, and shrinks variables within each group. This favors solutions with few leading terms in the final combination. The framework, originally derived for taking prior knowledge into account, is shown to be useful in linear regression, when several parameters are used to model the influence of one feature, or in kernel regression, for learning multiple kernels.