Who's Asking? Evaluating LLM Robustness to Inquiry Personas in Factual Question Answering
2510.12925v1
cs.CL, cs.LG
2025-10-17
Авторы:
Nil-Jana Akpinar, Chia-Jung Lee, Vanessa Murdock, Pietro Perona
Abstract
Large Language Models (LLMs) should answer factual questions truthfully,
grounded in objective knowledge, regardless of user context such as
self-disclosed personal information, or system personalization. In this paper,
we present the first systematic evaluation of LLM robustness to inquiry
personas, i.e. user profiles that convey attributes like identity, expertise,
or belief. While prior work has primarily focused on adversarial inputs or
distractors for robustness testing, we evaluate plausible, human-centered
inquiry persona cues that users disclose in real-world interactions. We find
that such cues can meaningfully alter QA accuracy and trigger failure modes
such as refusals, hallucinated limitations, and role confusion. These effects
highlight how model sensitivity to user framing can compromise factual
reliability, and position inquiry persona testing as an effective tool for
robustness evaluation.
Ссылки и действия
Дополнительные ресурсы: