LegalSim: Multi-Agent Simulation of Legal Systems for Discovering Procedural Exploits
2510.03405v1
cs.MA, cs.AI, cs.CR
2025-10-08
Авторы:
Sanket Badhe
Abstract
We present LegalSim, a modular multi-agent simulation of adversarial legal
proceedings that explores how AI systems can exploit procedural weaknesses in
codified rules. Plaintiff and defendant agents choose from a constrained action
space (for example, discovery requests, motions, meet-and-confer, sanctions)
governed by a JSON rules engine, while a stochastic judge model with calibrated
grant rates, cost allocations, and sanction tendencies resolves outcomes. We
compare four policies: PPO, a contextual bandit with an LLM, a direct LLM
policy, and a hand-crafted heuristic; Instead of optimizing binary case
outcomes, agents are trained and evaluated using effective win rate and a
composite exploit score that combines opponent-cost inflation, calendar
pressure, settlement pressure at low merit, and a rule-compliance margin.
Across configurable regimes (e.g., bankruptcy stays, inter partes review, tax
procedures) and heterogeneous judges, we observe emergent ``exploit chains'',
such as cost-inflating discovery sequences and calendar-pressure tactics that
remain procedurally valid yet systemically harmful. Evaluation via cross-play
and Bradley-Terry ratings shows, PPO wins more often, the bandit is the most
consistently competitive across opponents, the LLM trails them, and the
heuristic is weakest. The results are stable in judge settings, and the
simulation reveals emergent exploit chains, motivating red-teaming of legal
rule systems in addition to model-level testing.
Ссылки и действия
Дополнительные ресурсы: