Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models
2511.01618v1
cs.CV, cs.CL
2025-11-06
Авторы:
Xiaoyu Zhan, Wenxuan Huang, Hao Sun, Xinyu Fu, Changfeng Ma, Shaosheng Cao, Bohan Jia, Shaohui Lin, Zhenfei Yin, Lei Bai, Wanli Ouyang, Yuanqi Li, Jie Guo, Yanwen Guo
Abstract
Recent advances in Multimodal Large Language Models (MLLMs) have
significantly improved 2D visual understanding, prompting interest in their
application to complex 3D reasoning tasks. However, it remains unclear whether
these models can effectively capture the detailed spatial information required
for robust real-world performance, especially cross-view consistency, a key
requirement for accurate 3D reasoning. Considering this issue, we introduce
Viewpoint Learning, a task designed to evaluate and improve the spatial
reasoning capabilities of MLLMs. We present the Viewpoint-100K dataset,
consisting of 100K object-centric image pairs with diverse viewpoints and
corresponding question-answer pairs. Our approach employs a two-stage
fine-tuning strategy: first, foundational knowledge is injected to the baseline
MLLM via Supervised Fine-Tuning (SFT) on Viewpoint-100K, resulting in
significant improvements across multiple tasks; second, generalization is
enhanced through Reinforcement Learning using the Group Relative Policy
Optimization (GRPO) algorithm on a broader set of questions. Additionally, we
introduce a hybrid cold-start initialization method designed to simultaneously
learn viewpoint representations and maintain coherent reasoning thinking.
Experimental results show that our approach significantly activates the spatial
reasoning ability of MLLM, improving performance on both in-domain and
out-of-domain reasoning tasks. Our findings highlight the value of developing
foundational spatial skills in MLLMs, supporting future progress in robotics,
autonomous systems, and 3D scene understanding.
Ссылки и действия
Дополнительные ресурсы: