Investigating Safety Vulnerabilities of Large Audio-Language Models Under Speaker Emotional Variations
2510.16893v1
cs.SD, cs.AI, cs.CL, eess.AS
2025-10-22
Авторы:
Bo-Han Feng, Chien-Feng Liu, Yu-Hsuan Li Liang, Chih-Kai Yang, Szu-Wei Fu, Zhehuai Chen, Ke-Han Lu, Sung-Feng Huang, Chao-Han Huck Yang, Yu-Chiang Frank Wang, Yun-Nung Chen, Hung-yi Lee
Abstract
Large audio-language models (LALMs) extend text-based LLMs with auditory
understanding, offering new opportunities for multimodal applications. While
their perception, reasoning, and task performance have been widely studied,
their safety alignment under paralinguistic variation remains underexplored.
This work systematically investigates the role of speaker emotion. We construct
a dataset of malicious speech instructions expressed across multiple emotions
and intensities, and evaluate several state-of-the-art LALMs. Our results
reveal substantial safety inconsistencies: different emotions elicit varying
levels of unsafe responses, and the effect of intensity is non-monotonic, with
medium expressions often posing the greatest risk. These findings highlight an
overlooked vulnerability in LALMs and call for alignment strategies explicitly
designed to ensure robustness under emotional variation, a prerequisite for
trustworthy deployment in real-world settings.