The eval suite passes and fails on the same code

• Re-running the identical eval suite on unchanged code gives pass rates that swing between 78% and 92%.
• The team cannot tell whether a prompt change actually helped or the numbers just moved.
• A "regression" that blocked a deploy disappeared on a re-run.

run 1: 84% pass  run 2: 91% pass  run 3: 79% pass   (same code, same dataset)
judge: LLM-as-judge, temperature=0.7, no rubric, prompt="is this answer good?"
dataset: 22 examples, several with two acceptable answers scored as pass/fail
generation: temperature=0.9, no seed