SQUARE: Semantic Query-Augmented Fusion and Efficient Batch Reranking for Training-free Zero-Shot Composed Image Retrieval
2509.26330v1
cs.CV, cs.IR, 68U10, 68P20, 68T10, 68T07, 68T45, 68T50, I.4; H.3; I.5
2025-10-02
Авторы:
Ren-Di Wu, Yu-Yen Lin, Huei-Fang Yang
Abstract
Composed Image Retrieval (CIR) aims to retrieve target images that preserve
the visual content of a reference image while incorporating user-specified
textual modifications. Training-free zero-shot CIR (ZS-CIR) approaches, which
require no task-specific training or labeled data, are highly desirable, yet
accurately capturing user intent remains challenging. In this paper, we present
SQUARE, a novel two-stage training-free framework that leverages Multimodal
Large Language Models (MLLMs) to enhance ZS-CIR. In the Semantic
Query-Augmented Fusion (SQAF) stage, we enrich the query embedding derived from
a vision-language model (VLM) such as CLIP with MLLM-generated captions of the
target image. These captions provide high-level semantic guidance, enabling the
query to better capture the user's intent and improve global retrieval quality.
In the Efficient Batch Reranking (EBR) stage, top-ranked candidates are
presented as an image grid with visual marks to the MLLM, which performs joint
visual-semantic reasoning across all candidates. Our reranking strategy
operates in a single pass and yields more accurate rankings. Experiments show
that SQUARE, with its simplicity and effectiveness, delivers strong performance
on four standard CIR benchmarks. Notably, it maintains high performance even
with lightweight pre-trained, demonstrating its potential applicability.