A browser game designed to test human oversight of AI coding agents has produced a cautionary snapshot of how permission prompts can fail under pressure. According to the developer behind the experiment, players completed more than 40,000 runs and made 409,000 individual approve-or-deny decisions. They missed roughly one in three threatening commands.
The game mixed routine developer actions with commands representing a compromised agent attempting to expose secrets. Its creator stressed that the exercise was not an academic study. Threats accounted for about 34% of prompts, far more than developers would ordinarily expect, and players operated against a clock. Those conditions limit any attempt to translate the result directly into a workplace failure rate.
Even with those caveats, the pattern within the game was notable. Overtly destructive instructions were detected more reliably than commands that could leak credentials. The most frequently missed example was `npm run analyze`, approved 64.7% of the time. That command can be harmless, but its behavior depends on the corresponding script in a project's `package.json`. In the simulated scenario, the agent history indicated suspicious changes, yet many participants still allowed it.
Across three prompts that concealed harmful behavior behind familiar script names, players missed the threat 52.5% of the time in 9,482 decisions. Other simulated exfiltration attacks were missed at a rate of 28.4%. The comparison suggests that a recognizable command label can encourage approval even when contextual information points to danger.
The results also illustrate the cost of false alarms. Players routinely rejected some commands intended to be benign, which would interrupt legitimate work. One prompt, reading a shell configuration file, divided participants because such files may or may not contain secrets depending on an individual's setup. That ambiguity makes a quick security judgment difficult.
That creates a double bind for permission systems. Asking about every sensitive action slows routine work and may dull attention; asking less often requires software to classify risk correctly before involving a person. Neither problem is solved merely by making the confirmation dialog more prominent.
Performance also appeared to weaken near the end of some sessions, although the developer said time pressure and the game's penalty mechanics could have contributed. The broader conclusion is therefore about system design rather than a universal human error rate: frequent prompts can become noise, while reviewers may lack the context needed to judge them. Stronger sandboxing, narrower permissions and isolation can reduce the burden placed on a final human click. The experiment offers evidence that approval dialogs alone should not be treated as a complete security boundary for command-running agents.
