BEGIN:VCALENDAR
VERSION:2.0
X-WR-TIMEZONE:Europe/Paris
CALSCALE:GREGORIAN
PRODID:-//SPIP/Plugin Agenda//NONSGML v1.0//FR
X-WR-CALNAME;VALUE=TEXT: -- Institut de Mathématiques de Marseille\, UMR 7373
X-WR-RELCALID:http://www.i2m.univ-amu.fr/spip.php?page=article&id_article=0
BEGIN:VEVENT
SUMMARY:Maxime GAZEAU - A general system of differential equations to model first order adaptive algorithms. Application to ADAM.
UID:20190125T112800-a128-e2721@https://www.i2m.univ-amu.fr
DTSTAMP:20190125T112800
DTSTART:20190301T140000
DTEND:20190301T150000
CREATED:20190125T112800
ATTENDEE;CN=Maxime GAZEAU:mailto:no-reply@math.cnrs.fr
LAST-MODIFIED:20190218T085515
LOCATION:CMI\, salle de séminaire R164 (1er étage)
DESCRIPTION:A couple of years ago\, adaptive algorithms such as ADAM\, RMSPROP\, AMSGRAD\, ADAGRAD became the default method of choice for training machine learning models. Practitioners commonly observed that the value of the training loss decays faster than for stochastic gradient descent\, but the inherent reason is still not understood. A motivation of our work was to understand what properties make them so well suited for deep learning. In this talk\, I will analyze adaptive algorithms by studying their continuous time counterpart.I will first explain the connection between the optimization algorithms and the continuous differential equations. Then\, I will give sufficient conditions to guarantee convergence of trajectories towards a critical value and will discuss some properties of adaptive algorithms.This is joint work with A. Belotto Da Silva. http://www.math.toronto.edu/gazeauma/ Maxime GAZEAU [
CATEGORIES:(Séminaire Signal et Apprentissage|textebrut|filtrer_ical)]
URL:http://www.i2m.univ-amu.fr/Seminaire-Signal-et-Apprentissage?id_evenement=2721
SEQUENCE:2
STATUS:CONFIRMED
END:VEVENT