📊 Статистика дайджестов
Всего дайджестов: 34022 Добавлено сегодня: 0
Последнее обновление: сегодня
Авторы:
Steffen Dereich, Arnulf Jentzen, Sebastian Kassing
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
The Adam optimizer is currently presumably the most popular optimization
method in deep learning. In this article we develop an ODE based method to
study the Adam optimizer in a fast-slow scaling regime. For fixed momentum
parameters and vanishing step-sizes, we show that the Adam algorithm is an
asymptotic pseudo-trajectory of the flow of a particular vector field, which is
referred to as the Adam vector field. Leveraging properties of asymptotic
pseudo-trajectories, we establish convergence re...