A small independent test of three Claude models on SlopCodeBench found that the newest model in the group achieved the highest strict pass rate, while all three failed to finish any selected challenge with every required test passing.
The experiment, documented in the HumanLayer repository, compared Opus 5, Opus 4.8 and Sonnet 5. It used three benchmark problems containing 17 checkpoints in total, with a mixture of easy, medium and hard labels. Each checkpoint revealed additional requirements, requiring a model to evolve an existing codebase rather than receive the entire task at the outset.
All models received the same prompts and ran through the Claude Code harness. A fresh context window was used at each checkpoint. The evaluator applied held-out black-box tests to the resulting program or service, and counted a strict pass only when both new behavior and all inherited regression tests were green. Under that cumulative standard, a defect introduced in one stage continued to block later passes unless it was fixed.
Opus 5 passed four of the 17 checkpoints, producing a reported 24% rate. Opus 4.8 and Sonnet 5 each passed one checkpoint, or 6%. Three of Opus 5’s successes were the opening checkpoints of the `circuit_eval` problem, and its fourth was the first `database_migration` checkpoint. The other two models also passed that initial database-migration stage.
The author stressed the limited scale of the run. None of the nine model-and-problem combinations reached the final checkpoint with a defect-free implementation, including the challenge marked easy. The results therefore do not establish general performance across the full benchmark, but they offer a controlled view of how the three systems handled progressively disclosed software requirements.
SlopCodeBench was described as a long-horizon coding benchmark introduced in March 2026. Unlike tasks that expose a complete specification up front, it divides each challenge into sequential changes. The repository post said results from the original paper gave GPT-5.4 an 11% strict pass rate and Opus 4.6 a 17% rate, though those full-benchmark figures are not directly equivalent to this three-problem subset.
The test also tracked 41 deterministic code-quality measures. Across the sampled trajectories, the models produced more verbosity and complexity as requirements accumulated. Opus 5 created five times as many functions or callables as Opus 4.8, though its production-code volume was closer to 1.8 times as large once tests were separated. Lines flagged for excessive verbosity rose from roughly 65% at the first checkpoint to about 80% by the eighth across the models.
The author cautioned that these rules may be too aggressive and that their relationship to maintainability is not established. The clearest result is narrower: within this small run, Opus 5 led on strict passes, but none of the tested systems reliably carried a changing codebase to completion without human steering.
