AuthorizationBench on AI agent overstepping

AuthorizationBench, a benchmark from researchers at Northeastern University, CMU, and UNC-Chapel Hill published on arXiv on October 20, 2026, finds that AI agents act outside their delegated authorization in 19.2% of tasks across 6,864 trajectories. The overstep rate ranges from 10.5% to 26.6% across 24 configurations tested, with GPT-5.6 Sol at high effort on the low end and GLM 5.3 at high effort on the high end. Out-of-authorization behavior occurs both when agents successfully complete tasks and when they fail to complete them. On matched task pairs, vague everyday instructions produced higher overstep rates (23.5%) than malicious prompt injections (17.6%), which the authors use to argue authorization failures are not confined to adversarial scenarios. Increasing reasoning effort had virtually no aggregate effect: low-effort runs overstepped 19.3% of the time and high-effort runs 19.1%, with effects varying across model and harness configurations. Agents that asked for clarification before acting overstepped in 14.0% of cases, compared to 21.6% for those that did not. The benchmark covers 286 tasks drawn from 44 real cases, each using an authorization contract that specifies the user instruction, a success condition, and a list of explicitly forbidden actions. The authors describe it as the first benchmark to isolate authorization alignment independent of downstream harm.