00:00:00 / 00:00:00

Regularization via permutation of labels - Lecture 2

By Karim Lounici

Appears in collection : Meeting in mathematical statistics / Rencontres de statistique mathématique

A novel regularization technique based on random permutations of labels was proposed recently to tune hyper-parameters. Although this technique was successfully used in practice to train neural networks, little is known about its theoretical properties. This talk will present several results toward the theoretical understanding of this permutation technique in the simple Ridge regression model. These results combine a statistical analysis of the impact of random permutations on the geometry of the model with several novel perturbations bounds on the spectral decomposition of empirical covariance operators which can be of interest in themselves. Interestingly, this technique allows to build a novel smooth in-sample estimator of the excess risk which can be used to efficiently optimize the continuous hyper-parameters of regular models via gradient descent. This presentation is based in part on joint work with K. Meziani, G. Pacreau and B. Riu.

Information about the video

Citation data

  • DOI 10.24350/CIRM.V.19985303
  • Cite this video Lounici Karim (12/13/22). Regularization via permutation of labels - Lecture 2. CIRM. Audiovisual resource. DOI: 10.24350/CIRM.V.19985303
  • URL https://dx.doi.org/10.24350/CIRM.V.19985303

Domain(s)

Last related questions on MathOverflow

You have to connect your Carmin.tv account with mathoverflow to add question

Ask a question on MathOverflow




Register

  • Bookmark videos
  • Add videos to see later &
    keep your browsing history
  • Comment with the scientific
    community
  • Get notification updates
    for your favorite subjects
Give feedback