Biased Monte Carlo sampling in RBMs | Vidéo | Carmin.tv

00:00:00 / 00:00:00

Chapitres

Biased Monte Carlo sampling in RBMs

De Beatriz Seoane

Apparaît dans la collection : On Future Synergies for Stochastic and Learning Algorithms / Sur les synergies futures autour des algorithmes d'apprentissage et stochastiques

RBMs are generative models capable of fitting complex dataset's probability distributions. Thanks to their simple structure, they are particularly well suited for interpretability and pattern extraction, a feature particularly appealing for scientific use. In this talk, we show that RBMs operate in two distinct regimes, depending on the procedure followed to estimate the log-likelihood gradient during the training. Short sampling times fit machines that are trained to reproduce exactly the dynamics followed to train them, long samplings (as compared to the MCMC mixing time) are need to learn a good model for the data. The non-equilibrium regime should be used to generate high quality samples in short learning and sampling times, but cannot be used to extract the unnormalized data probability of the data necessary for interpretability. In practice, it is hard to extract good equilibrium models for structured datasets (which is the typical case in biological applications) due to a divergence of the Monte Carlo mixing times. In this work, we show this barrier can be surmounted using biased Monte Carlo methods.

Informations sur la vidéo

Date de captation 30/09/2021
Date de publication 18/11/2021
Institut CIRM
Licence CC BY NC ND
Langue Anglais
Audience Chercheurs
Réalisateur(s) Guillaume Hennenfent
Format MP4

Données de citation

DOI 10.24350/CIRM.V.19817903
Citer cette vidéo Seoane, Beatriz (30/09/2021). Biased Monte Carlo sampling in RBMs. CIRM. Audiovisual resource. DOI: 10.24350/CIRM.V.19817903
URL https://dx.doi.org/10.24350/CIRM.V.19817903

Domaine(s)

Bibliographie

DECELLE, Aurélien, FURTLEHNER, Cyril, et SEOANE, Beatriz. Equilibrium and non-Equilibrium regimes in the learning of Restricted Boltzmann Machines. arXiv preprint arXiv:2105.13889, 2021. - https://arxiv.org/abs/2105.13889
DECELLE, Aurélien, FISSORE, Giancarlo, et FURTLEHNER, Cyril. Thermodynamics of restricted Boltzmann machines and related learning dynamics. Journal of Statistical Physics, 2018, vol. 172, no 6, p. 1576-1608. - http://dx.doi.org/10.1007/s10955-018-2105-y
DECELLE, Aurélien et FURTLEHNER, Cyril. Exact training of Restricted Boltzmann machines on intrinsically low dimensional data. arXiv preprint arXiv:2103.10755, 2021. - https://arxiv.org/abs/2103.10755
MARTIN-MAYOR, V., SEOANE, B., et YLLANES, D. Tethered Monte Carlo: Managing rugged free-energy landscapes with a Helmholtz-potential formalism. Journal of Statistical Physics, 2011, vol. 144, no 3, p. 554-596. - http://dx.doi.org/10.1007/s10955-011-0261-4
FERNÁNDEZ, L. A., MARTÍN-MAYOR, Víctor, SEOANE, B., et al. Equilibrium fluid-solid coexistence of hard spheres. Physical review letters, 2012, vol. 108, no 16, p. 165701. - http://dx.doi.org/10.1103/PhysRevLett.108.165701

Codes MSC

Document(s)

https://www.cirm-math.fr/RepOrga/2389/Slides/Seoane.pdf

Dernières questions liées sur MathOverflow

Pour poser une question, votre compte Carmin.tv doit être connecté à mathoverflow

Poser une question sur MathOverflow

Copyright Carmin.tv 2024

Donner son avis