📊 Статистика дайджестов
Всего дайджестов: 34022 Добавлено сегодня: 82
Последнее обновление: сегодня
Авторы:
Elizabeth Collins-Woodfin, Inbar Seroussi
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
We develop a framework for analyzing the training and learning rate dynamics
on a variety of high- dimensional optimization problems trained using one-pass
stochastic gradient descent (SGD) with data generated from multiple anisotropic
classes. We give exact expressions for a large class of functions of the
limiting dynamics, including the risk and the overlap with the true signal, in
terms of a deterministic solution to a system of ODEs. We extend the existing
theory of high-dimensional SGD dyn...
Авторы:
Antonio Álvarez-López, Martín Hernández
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
We study dropout regularization in continuous-time models through the lens of
random-batch methods -- a family of stochastic sampling schemes originally
devised to reduce the computational cost of interacting particle systems. We
construct an unbiased, well-posed estimator that mimics dropout by sampling
neuron batches over time intervals of length $h$. Trajectory-wise convergence
is established with linear rate in $h$ for the expected uniform error. At the
distribution level, we establish stabi...
Авторы:
Damek Davis, Benjamin Recht
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
We show that several popular algorithms for reinforcement learning in large
language models with binary rewards can be viewed as stochastic gradient ascent
on a monotone transform of the probability of a correct answer given a prompt.
In particular, the transformation associated with rejection sampling algorithms
is the logarithm and that associated with the GRPO algorithm is the arcsine of
the square root.
Авторы:
Christophe Roux, Max Zimmer, Alexandre d'Aspremont, Sebastian Pokutta
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
Pruning is a common technique to reduce the compute and storage requirements
of Neural Networks. While conventional approaches typically retrain the model
to recover pruning-induced performance degradation, state-of-the-art Large
Language Model (LLM) pruning methods operate layer-wise, minimizing the
per-layer pruning error on a small calibration dataset to avoid full
retraining, which is considered computationally prohibitive for LLMs. However,
finding the optimal pruning mask is a hard combina...
Авторы:
Bahadur Yadav, Sanjay Kumar Mohanty
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
Forecasting stock prices remains challenging due to the volatile and
non-linear nature of financial markets. Despite the promise of deep learning,
issues such as mode collapse, unstable training, and difficulty in capturing
temporal and feature level correlations have limited the applications of GANs
in this domain. We propose a GRU-based Encoder-Decoder GAN (EDGAN) model that
strikes a balance between expressive power and simplicity. The model introduces
key innovations such as a temporal decod...
📄 Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods
2025-10-16Авторы:
Andrey Veprikov, Arman Bolatov, Samuel Horváth, Aleksandr Beznosikov, Martin Takáč, Slavomir Hanzely
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
Optimization lies at the core of modern deep learning, yet existing methods
often face a fundamental trade-off between adapting to problem geometry and
leveraging curvature utilization. Steepest descent algorithms adapt to
different geometries through norm choices but remain strictly first-order,
whereas quasi-Newton and adaptive optimizers incorporate curvature information
but are restricted to Frobenius geometry, limiting their applicability across
diverse architectures. In this work, we propo...
📄 Improved Central Limit Theorem and Bootstrap Approximations for Linear Stochastic Approximation
2025-10-16Авторы:
Bogdan Butyrin, Eric Moulines, Alexey Naumov, Sergey Samsonov, Qi-Man Shao, Zhuo-Song Zhang
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
In this paper, we refine the Berry-Esseen bounds for the multivariate normal
approximation of Polyak-Ruppert averaged iterates arising from the linear
stochastic approximation (LSA) algorithm with decreasing step size. We consider
the normal approximation by the Gaussian distribution with covariance matrix
predicted by the Polyak-Juditsky central limit theorem and establish the rate
up to order $n^{-1/3}$ in convex distance, where $n$ is the number of samples
used in the algorithm. We also prove...
📄 Cautious Weight Decay
2025-10-16Авторы:
Lizhang Chen, Jonathan Li, Kaizhao Liang, Baiyu Su, Cong Xie, Nuo Wang Pierse, Chen Liang, Ni Lao, Qiang Liu
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
We introduce Cautious Weight Decay (CWD), a one-line, optimizer-agnostic
modification that applies weight decay only to parameter coordinates whose
signs align with the optimizer update. Unlike standard decoupled decay, which
implicitly optimizes a regularized or constrained objective, CWD preserves the
original loss and admits a bilevel interpretation: it induces sliding-mode
behavior upon reaching the stationary manifold, allowing it to search for
locally Pareto-optimal stationary points of th...
Авторы:
Ahmed Ben Yahmed, Hafedh El Ferchichi, Marc Abeille, Vianney Perchet
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
We examine a multi-armed bandit problem with contextual information, where
the objective is to ensure that each arm receives a minimum aggregated reward
across contexts while simultaneously maximizing the total cumulative reward.
This framework captures a broad class of real-world applications where fair
revenue allocation is critical and contextual variation is inherent. The
cross-context aggregation of minimum reward constraints, while enabling better
performance and easier feasibility, introd...
📄 Efficient Group Lasso Regularized Rank Regression with Data-Driven Parameter Determination
2025-10-15Авторы:
Meixia Lin, Meijiao Shi, Yunhai Xiao, Qian Zhang
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
High-dimensional regression often suffers from heavy-tailed noise and
outliers, which can severely undermine the reliability of least-squares based
methods. To improve robustness, we adopt a non-smooth Wilcoxon score based rank
objective and incorporate structured group sparsity regularization, a natural
generalization of the lasso, yielding a group lasso regularized rank regression
method. By extending the tuning-free parameter selection scheme originally
developed for the lasso, we introduce a...
Показано 61 -
70
из 157 записей