Just Do It!? Computer-Use Agents Exhibit Blind Goal-Directedness
2510.01670v1
cs.AI, cs.CL, cs.CR, cs.CY, cs.LG
2025-10-04
Авторы:
Erfan Shayegani, Keegan Hines, Yue Dong, Nael Abu-Ghazaleh, Roman Lutz, Spencer Whitehead, Vidhisha Balachandran, Besmira Nushi, Vibhav Vineet
Abstract
Computer-Use Agents (CUAs) are an increasingly deployed class of agents that
take actions on GUIs to accomplish user goals. In this paper, we show that CUAs
consistently exhibit Blind Goal-Directedness (BGD): a bias to pursue goals
regardless of feasibility, safety, reliability, or context. We characterize
three prevalent patterns of BGD: (i) lack of contextual reasoning, (ii)
assumptions and decisions under ambiguity, and (iii) contradictory or
infeasible goals. We develop BLIND-ACT, a benchmark of 90 tasks capturing these
three patterns. Built on OSWorld, BLIND-ACT provides realistic environments and
employs LLM-based judges to evaluate agent behavior, achieving 93.75% agreement
with human annotations. We use BLIND-ACT to evaluate nine frontier models,
including Claude Sonnet and Opus 4, Computer-Use-Preview, and GPT-5, observing
high average BGD rates (80.8%) across them. We show that BGD exposes subtle
risks that arise even when inputs are not directly harmful. While
prompting-based interventions lower BGD levels, substantial risk persists,
highlighting the need for stronger training- or inference-time interventions.
Qualitative analysis reveals observed failure modes: execution-first bias
(focusing on how to act over whether to act), thought-action disconnect
(execution diverging from reasoning), and request-primacy (justifying actions
due to user request). Identifying BGD and introducing BLIND-ACT establishes a
foundation for future research on studying and mitigating this fundamental risk
and ensuring safe CUA deployment.