LACY: A Vision-Language Model-based Language-Action Cycle for Self-Improving Robotic Manipulation
2511.02239v1
cs.RO, cs.AI
2025-11-06
Авторы:
Youngjin Hong, Houjian Yu, Mingen Li, Changhyun Choi
Abstract
Learning generalizable policies for robotic manipulation increasingly relies
on large-scale models that map language instructions to actions (L2A). However,
this one-way paradigm often produces policies that execute tasks without deeper
contextual understanding, limiting their ability to generalize or explain their
behavior. We argue that the complementary skill of mapping actions back to
language (A2L) is essential for developing more holistic grounding. An agent
capable of both acting and explaining its actions can form richer internal
representations and unlock new paradigms for self-supervised learning. We
introduce LACY (Language-Action Cycle), a unified framework that learns such
bidirectional mappings within a single vision-language model. LACY is jointly
trained on three synergistic tasks: generating parameterized actions from
language (L2A), explaining observed actions in language (A2L), and verifying
semantic consistency between two language descriptions (L2C). This enables a
self-improving cycle that autonomously generates and filters new training data
through an active augmentation strategy targeting low-confidence cases, thereby
improving the model without additional human labels. Experiments on
pick-and-place tasks in both simulation and the real world show that LACY
improves task success rates by 56.46% on average and yields more robust
language-action grounding for robotic manipulation. Project page:
https://vla2026.github.io/LACY/
Ссылки и действия
Дополнительные ресурсы: