2022 - T3 - WS3 - Measure-theoretic Approaches and Optimal Transportation in Statistics

Collection 2022 - T3 - WS3 - Measure-theoretic Approaches and Optimal Transportation in Statistics

Organizer(s) Aamari, Eddie ; Aaron, Catherine ; Chazal, Frédéric ; Fisher, Aurélie ; Hoffmann, Marc ; Le Brigant, Alice ; Levrard, Clément ; Michel, Bertrand

Date(s) 21/11/2022 - 25/11/2022

linked URL https://indico.math.cnrs.fr/event/7547/

1 14

Gradient Flows on Kernel Divergence Measures

By Arthur Gretton

We construct Wasserstein gradient flows on two measures of divergence, and study their convergence properties.

The first divergence measure is the Maximum Mean Discrepancy (MMD): an integral probability metric defined for a reproducing kernel Hilbert space (RKHS), which serves as a metric on probability measures for a sufficiently rich RKHS. We obtain conditions for convergence of the gradient flow towards a global optimum, and relate this flow to the problem of optimizing neural networks. The second divergence measure on which we define a flow is the KALE (KL Approximate Lower-bound Estimator) divergence. This is a regularized version of the Fenchel dual problem defining the KL over a restricted class of functions (again, a Reproducing Kernel Hilbert Space (RKHS)). We also propose a way to regularize both the MMD and KALE gradient flows, based on an injection of noise in the gradient. This algorithmic fix comes with theoretical and empirical evidence. We compare the MMD and KALE flows, illustrating that the KALE gradient flow is particularly well suited when the target distribution is supported on a low-dimensional manifold.

Information about the video

Date of recording 21/11/2022
Date of publication 19/04/2024
Institution IHP
Language English
Audience Researchers, Graduate Students
Director(s) Alexandre Duplessis, Marco Perez
Format MP4
Venue Amphitheater Hermite, IHP

Citation data

DOI 10.57987/IHP.2022.T3.WS3.001
Cite this video Gretton, Arthur (21/11/2022). Gradient Flows on Kernel Divergence Measures. IHP. Audiovisual resource. DOI: 10.57987/IHP.2022.T3.WS3.001
URL https://dx.doi.org/10.57987/IHP.2022.T3.WS3.001

Domain(s)

Machine Learning

Bibliography

Pierre Glaser, Michael Arbel, Arthur Gretton / KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Support. NeurIPS, 2021
Michael Arbel, Anna Korba, Adil Salim, Arthur Gretton / Maximum Mean Discrepancy Gradient Flow. NeurIPS, 2019

Last related questions on MathOverflow

You have to connect your Carmin.tv account with mathoverflow to add question

Ask a question on MathOverflow

All the collection videos

Gradient Flows on Kernel Divergence Measures

By Arthur Gretton

A varifold perspective on discrete surfaces

By Blanche Buet

Advances in statistics via tools from Stein's Method

By Christophe Ley

On the existence of Monge maps for Gromov-Wasserstein problems

By Théo Lacombe

Distribution-Free Tests for Multi-Dimensional Symmetry Using Optimal Transport

By Bodhisattva Sen

Graphical and uniform consistency of estimated optimal transport plans

By Johan Segers

Unbalanced Optimal Transport across Metric Measured Spaces

By Gabriel Peyré

A Wasserstein-type distance in the space of Gaussian mixture models

By Agnès Desolneux

Sliced Wasserstein on Manifolds: Spherical and Hyperbolical cases - Part 1

By Nicolas Courty

Spectral regularized kernel two-sample tests

By Bharath Sriperumbudur

published on April 5, 2024

Mirror Sinkhorn: Fast Online Optimization on Transport Polytopes

By Quentin Berthet

published on April 5, 2024

Online learning with exponential weights in metric spaces with the measure contraction property

By Quentin Paris

published on April 5, 2024

Some bounds on probabilistic distances via integration by parts formulae

By Giovanni Peccati

published on April 5, 2024

Sliced Wasserstein on Manifolds: Spherical and Hyperbolical cases - Part 2

By Nicolas Courty

Copyright Carmin.tv 2024

Give feedback