2022 - T3 - WS1 - Non-Linear and High Dimensional Inference

Collection 2022 - T3 - WS1 - Non-Linear and High Dimensional Inference

Organisateur(s) Aamari, Eddie ; Aaron, Catherine ; Chazal, Frédéric ; Fischer, Aurélie ; Hoffmann, Marc ; Le Brigant, Alice ; Levrard, Clément ; Michel, Bertrand

Date(s) 03/10/2022 - 07/10/2022

URL associée https://indico.math.cnrs.fr/event/7545/

17 21

Neural networks, wide and deep, singular kernels and Bayes optimality

De Mikhail Belkin

Wide and deep neural networks are used in many important practical settings. In this talk, I will discuss some aspects of width and depth related to optimization and generalization. I will first discuss what happens when neural networks become infinitely wide, giving a general result for the transition to linearity (i.e., showing that neural networks become linear functions of parameters) for a broad class of wide neural networks corresponding to directed graphs. I will then proceed to the ques- tion of depth, showing equivalence between infinitely wide and deep fully connected networks trained with gradient descent and Nadaraya-Watson predictors based on certain singular kernels. Using this connection we show that for certain activation functions these wide and deep networks are (asymptotically) optimal for classifica- tion but, interestingly, never for regression. (Based on joint work with Chaoyue Liu, Adit Radhakrishnan, Caroline Uhler and Libin Zhu.)

Informations sur la vidéo

Date de publication 05/04/2024
Institut IHP
Langue Anglais
Format MP4

Dernières questions liées sur MathOverflow

Pour poser une question, votre compte Carmin.tv doit être connecté à mathoverflow

Poser une question sur MathOverflow

Toutes les vidéos de la collection

Scaling ResNets in the Large-depth Regime

De Adeline Fermanian

Overcoming the curse of dimensionality with deep neural networks

De Sophie Langer

Learning on and near Low-Dimensional Subsets of the Wasserstein Manifold

De Alex Cloninger

Extrinsic and Intrinsic Operator Estimations for Manifold Learning

De John Harlim

A graph coupling view of dimension reduction

De Franck Picard

Topologically penalized regression on manifolds

De Wolfgang Polonik

Bayesian nonparametric estimation of a density living near an unknown manifold

De Judith Rousseau

Linear methods for non-linear inverse problems

De Botond Szabo

Convergence of Sharpness-Aware Minimization

De Peter Bartlett

Dimensionality reduction in reinforcement learning by randomisation

De Denis Belomestny

Stein effect for estimating many vector means: a "blessing of dimensionality" phenomenon

De Gilles Blanchard

On the use of overfitting for estimator selection

De Claire Lacour

Optimal Permutation estimation in crowdsourcing problems

De Nicolas Verzelen

What does LIME really see in images?

De Damien Garreau

A statistical analysis of an image classification problem

De Johannes Schmidt-Hieber

Learning a partial correlation graph using only a few covariance queries

De Vasiliki Velona

publiée le 5 avril 2024

Neural networks, wide and deep, singular kernels and Bayes optimality

De Mikhail Belkin

publiée le 5 avril 2024

Understanding the geometry of high-dimensional data through the reach

De Clément Bérenfeld

publiée le 5 avril 2024

Manifold Learning, Explanations and Eigenflows - Part 1

De Marina Meila

publiée le 5 avril 2024

Manifold Learning, Explanations and Eigenflows - Part 2

De Cécile Mailler

publiée le 5 avril 2024

On high-dimensional Lévy-driven Ornstein-Uhlenbeck processes

De Claudia Strauch

Copyright Carmin.tv 2024

Donner son avis