Meeting in Mathematical Statistics - Machine learning and nonparametric statistics / Rencontres de statistique mathématique

Collection Meeting in Mathematical Statistics - Machine learning and nonparametric statistics / Rencontres de statistique mathématique

Organizer(s) Butucea, Cristina ; Minsker, Stanislav ; Pouet, Christophe ; Spokoiny, Vladimir

Date(s) 13/12/2021 - 17/12/2021

linked URL https://conferences.cirm-math.fr/2581.html

00:00:00 / 00:00:00

1 4

Benign overfitting - Lecture 1

By Peter Bartlett

These lectures present some recent results on two phenomena that have been observed in deep neural networks. The first is benign overfitting: even without any explicit effort to control model complexity, deep learning methods find functions that give a near-perfect fit to noisy training data and yet exhibit good prediction performance in practice. We describe results that characterize this phenomenon in linear regression and in ridge regression. The second phenomenon that we consider is that of adversarial examples: functions computed by deep networkscan be extremely sensitive to small changes in their inputs. We show that this occurs in ReLU networks of constant depth with independent gaussian parameters because the functions that these networks compute are close to linear. The lectures include joint work with Seb Bubeck, Yeshwanth Cherapanamjeri, Phil Long, Gabor, Lugosi, and Alex Tsigler.

Information about the video

Date of recording 14/12/2021
Date of publication 14/01/2022
Institution CIRM
Licence CC BY NC ND
Language English
Audience Researchers
Director(s) Guillaume Hennenfent
Format MP4

Citation data

DOI 10.24350/CIRM.V.19867403
Cite this video Bartlett, Peter (14/12/2021). Benign overfitting - Lecture 1. CIRM. Audiovisual resource. DOI: 10.24350/CIRM.V.19867403
URL https://dx.doi.org/10.24350/CIRM.V.19867403

Domain(s)

Machine Learning

Bibliography

BARTLETT, Peter L., LONG, Philip M., LUGOSI, Gábor, et al. Benign overfitting in linear regression. Proceedings of the National Academy of Sciences, 2020, vol. 117, no 48, p. 30063-30070. - https://arxiv.org/abs/1906.11300
BARTLETT, Peter L., LONG, Philip M., LUGOSI, Gábor, et al. Benign overfitting in linear regression. Proceedings of the National Academy of Sciences, 2020, vol. 117, no 48, p. 30063-30070. - https://arxiv.org/abs/2009.14286
BARTLETT, Peter L. et LONG, Philip M. Failures of model-dependent generalization bounds for least-norm interpolation. arXiv preprint arXiv:2010.08479, 2020. - https://arxiv.org/abs/2010.08479
BARTLETT, Peter L., MONTANARI, Andrea, et RAKHLIN, Alexander. Deep learning: a statistical viewpoint. arXiv preprint arXiv:2103.09177, 2021. - https://arxiv.org/abs/2103.09177

Last related questions on MathOverflow

You have to connect your Carmin.tv account with mathoverflow to add question

Ask a question on MathOverflow

All the collection videos

01:08:14

published on January 14, 2022

Benign overfitting - Lecture 1

By Peter Bartlett

53:47

published on January 14, 2022

High dimensional mean estimation - Lecture 2

By Gábor Lugosi

54:38

published on January 14, 2022

Adversarial examples in deep networks - Lecture 2

By Peter Bartlett

55:11

published on January 14, 2022

Network archeology: a few results and questions - Lecture 1

By Gábor Lugosi

Copyright Carmin.tv 2024

Give feedback