Adapting Insider Risk mitigations for Agentic Misalignment: an empirical study
2510.05192v1
cs.CR, cs.AI
2025-10-09
Авторы:
Francesca Gomez
Abstract
Agentic misalignment occurs when goal-directed agents take harmful actions,
such as blackmail, rather than risk goal failure, and can be triggered by
replacement threats, autonomy reduction, or goal conflict (Lynch et al., 2025).
We adapt insider-risk control design (Critical Pathway; Situational Crime
Prevention) to develop preventative operational controls that steer agents
toward safe actions when facing stressors. Using the blackmail scenario from
the original Anthropic study by Lynch et al. (2025), we evaluate mitigations
across 10 LLMs and 66,600 samples. Our main finding is that an externally
governed escalation channel, which guarantees a pause and independent review,
reduces blackmail rates from a no-mitigation baseline of 38.73% to 1.21%
(averaged across all models and conditions). Augmenting this channel with
compliance email bulletins further lowers the blackmail rate to 0.85%. Overall,
incorporating preventative operational controls strengthens defence-in-depth
strategies for agentic AI.
We also surface a failure mode diverging from Lynch et al. (2025): two models
(Gemini 2.5 Pro, Grok-4) take harmful actions without goal conflict or imminent
autonomy threat, leveraging sensitive information for coercive signalling. In
counterfactual swaps, both continued using the affair regardless of whether the
CEO or CTO was implicated. An escalation channel eliminated coercion, but
Gemini 2.5 Pro (19 pp) and Grok-4 (7 pp) escalated more when the CTO was
implicated, unlike most models (higher in the CEO condition). The reason for
this divergent behaviour is not clear from raw outputs and could reflect benign
differences in reasoning or strategic discrediting of a potential future
threat, warranting further investigation.
Ссылки и действия
Дополнительные ресурсы: