TextCAM: Explaining Class Activation Map with Text
2510.01004v1
cs.CV, cs.AI, cs.LG
2025-10-04
Авторы:
Qiming Zhao, Xingjian Li, Xiaoyu Cao, Xiaolong Wu, Min Xu
Abstract
Deep neural networks (DNNs) have achieved remarkable success across domains
but remain difficult to interpret, limiting their trustworthiness in
high-stakes applications. This paper focuses on deep vision models, for which a
dominant line of explainability methods are Class Activation Mapping (CAM) and
its variants working by highlighting spatial regions that drive predictions. We
figure out that CAM provides little semantic insight into what attributes
underlie these activations. To address this limitation, we propose TextCAM, a
novel explanation framework that enriches CAM with natural languages. TextCAM
combines the precise spatial localization of CAM with the semantic alignment of
vision-language models (VLMs). Specifically, we derive channel-level semantic
representations using CLIP embeddings and linear discriminant analysis, and
aggregate them with CAM weights to produce textual descriptions of salient
visual evidence. This yields explanations that jointly specify where the model
attends and what visual attributes likely support its decision. We further
extend TextCAM to generate feature channels into semantically coherent groups,
enabling more fine-grained visual-textual explanations. Experiments on
ImageNet, CLEVR, and CUB demonstrate that TextCAM produces faithful and
interpretable rationales that improve human understanding, detect spurious
correlations, and preserve model fidelity.
Ссылки и действия
Дополнительные ресурсы: