Mikefest : A conference in honor of Michael Douglas’ 60th birthday

Collection Mikefest : A conference in honor of Michael Douglas’ 60th birthday

Organizer(s) Costas Bachas, Semyon Klevtsov, Nikita Nekrasov, Emmanuel Ullmo
Date(s) 09/05/2022 - 13/05/2022
linked URL https://indico.math.cnrs.fr/event/7727/
00:00:00 / 00:00:00
21 27

Embedding of Low-Dimensional Attractor Manifolds by Neural Networks

By Rémi Monasson

Recurrent neural networks (RNN) have long been studied to explain how fixed-point attractors may emerge from noisy, high-dimensional dynamics. Recently, computational neuroscientists have devoted sustained efforts to understand how RNN could embed attractor manifolds of finite dimension, in particular in the context of the representation of space by mammals. A natural issue is the existence of a trade-off between the quantity (number) and the quality (accuracy of encoding) of the stored manifolds. I will here study how to learn the N2 pairwise interactions in a RNN with N neurons to embed L manifolds of dimension D<<N. The capacity, i.e., the maximal ratio L/N, decreases as ~ [log(1/e)]^-D, where e is the error on the position encoded by the neural activity along each manifold. These results derived using a combination of analytical tools from statistical mechanics and random matrix theory show that RNN are flexible memory devices capable of storing a large number of manifolds at high spatial resolution.

Information about the video

  • Date of recording 12/05/2022
  • Date of publication 13/05/2022
  • Institution IHES
  • Language English
  • Audience Researchers
  • Format MP4

Domain(s)

Last related questions on MathOverflow

You have to connect your Carmin.tv account with mathoverflow to add question

Ask a question on MathOverflow




Register

  • Bookmark videos
  • Add videos to see later &
    keep your browsing history
  • Comment with the scientific
    community
  • Get notification updates
    for your favorite subjects
Give feedback