📊 Статистика дайджестов
Всего дайджестов: 34022 Добавлено сегодня: 0
Последнее обновление: сегодня
Авторы:
Tiberiu Musat, Tiago Pimentel, Lorenzo Noci, Alessandro Stolfo, Mrinmaya Sachan, Thomas Hofmann
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
Transformers have become the dominant architecture for natural language
processing. Part of their success is owed to a remarkable capability known as
in-context learning (ICL): they can acquire and apply novel associations solely
from their input context, without any updates to their weights. In this work,
we study the emergence of induction heads, a previously identified mechanism in
two-layer transformers that is particularly important for in-context learning.
We uncover a relatively simple an...
Авторы:
Soufiane Hayou
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
We provide the first proof of learning rate transfer with width in a linear
multi-layer perceptron (MLP) parametrized with $\mu$P, a neural network
parameterization designed to ``maximize'' feature learning in the
infinite-width limit. We show that under $\mu P$, the optimal learning rate
converges to a \emph{non-zero constant} as width goes to infinity, providing a
theoretical explanation to learning rate transfer. In contrast, we show that
this property fails to hold under alternative parametr...
Авторы:
Mian Wu, Gavin Zhang, Sewon Min, Sergey Levine, Aviral Kumar
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
Open-ended generation tasks require outputs to satisfy diverse and often
implicit task-specific evaluation rubrics. The sheer number of relevant rubrics
leads to prohibitively high verification costs and incomplete assessments of a
response, making reinforcement learning (RL) post-training with rubric-based
rewards difficult to scale. This problem is exacerbated by the fact that often
the best way to combine these rubrics into one single reward is also highly
prompt-specific. We propose Reinforc...
Авторы:
Vi Retault, Yohaï-Eliel Berreby
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
When fine-tuning language models on new tasks, catastrophic forgetting --
performance degradation on previously-learned tasks -- is a ubiquitous problem.
While Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA address this
through low-rank adapters, sparse adaptation offers an alternative that doesn't
impose rank constraints. We introduce Random Initialization of Gated Sparse
Adapters (RIGSA), which starts from randomly-initialized full-rank adapters,
gates them with a ReZero analog, and ...
Авторы:
Vivswan Shah, Randy Cogill, Hanwei Yue, Gopinath Chennupati, Rinat Khaziev
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
Fine-tuning LLMs for classification typically maps inputs directly to labels.
We ask whether attaching brief explanations to each label during fine-tuning
yields better models. We evaluate conversational response quality along three
axes: naturalness, comprehensiveness, and on-topic adherence, each rated on
5-point scales. Using ensemble-generated data from multiple LLMs, we fine-tune
a 7B-parameter model and test across six diverse conversational datasets.
Across 18 dataset, task settings, labe...
Авторы:
Ziheng Geng, Jiachen Liu, Ran Cao, Lu Cheng, Dan M. Frangopol, Minghui Cheng
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
Flood insurance is an effective strategy for individuals to mitigate
disaster-related losses. However, participation rates among at-risk populations
in the United States remain strikingly low. This gap underscores the need to
understand and model the behavioral mechanisms underlying insurance decisions.
Large language models (LLMs) have recently exhibited human-like intelligence
across wide-ranging tasks, offering promising tools for simulating human
decision-making. This study constructs a benc...
📄 Deep Value Benchmark: Measuring Whether Models Generalize Deep values or Shallow Preferences
2025-11-06Авторы:
Joshua Ashkinaze, Hua Shen, Sai Avula, Eric Gilbert, Ceren Budak
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
We introduce the Deep Value Benchmark (DVB), an evaluation framework that
directly tests whether large language models (LLMs) learn fundamental human
values or merely surface-level preferences. This distinction is critical for AI
alignment: Systems that capture deeper values are likely to generalize human
intentions robustly, while those that capture only superficial patterns in
preference data risk producing misaligned behavior. The DVB uses a novel
experimental design with controlled confoundi...
Авторы:
Yibo Zhao, Yang Zhao, Hongru Du, Hao Frank Yang
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
Decision-making models for individuals, particularly in high-stakes scenarios
like vaccine uptake, often diverge from population optimal predictions. This
gap arises from the uniqueness of the individual decision-making process,
shaped by numerical attributes (e.g., cost, time) and linguistic influences
(e.g., personal preferences and constraints). Developing upon Utility Theory
and leveraging the textual-reasoning capabilities of Large Language Models
(LLMs), this paper proposes an Adaptive Tex...
Авторы:
Weiwei Sun, Xuhui Zhou, Weihua Du, Xingyao Wang, Sean Welleck, Graham Neubig, Maarten Sap, Yiming Yang
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
While existing work focuses primarily on task success, we argue that
effective real-world agents require optimizing three dimensions: productivity
(task completion), proactivity (asking essential questions), and
personalization (adapting to diverse user preferences). We introduce UserVille,
an interactive environment with LLM-based user simulators enabling diverse,
configurable user preferences. Leveraging UserVille, we introduce PPP, a
multi-objective reinforcement learning approach that jointl...
Авторы:
Zhiwei Zhang, Xiaomin Li, Yudi Lin, Hui Liu, Ramraj Chandradevan, Linlin Wu, Minhua Lin, Fali Wang, Xianfeng Tang, Qi He, Suhang Wang
Саммари на русском не найдено
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Доступные поля: ['id', 'arxiv_id', 'title', 'authors', 'abstract', 'summary_ru', 'categories', 'published_date', 'created_at']
Annotation:
Large Language Models (LLMs) trained with reinforcement learning and
verifiable rewards have achieved strong results on complex reasoning tasks.
Recent work extends this paradigm to a multi-agent setting, where a
meta-thinking agent proposes plans and monitors progress while a reasoning
agent executes subtasks through sequential conversational turns. Despite
promising performance, we identify a critical limitation: lazy agent behavior,
in which one agent dominates while the other contributes lit...
Показано 241 -
250
из 1292 записей