An experiment on free-shipping thresholds shows order completion up slightly in the variant after one week. The PM wants to ship it. What do you check first?
Answer it out loud before you open anything. The value of the flags below is in comparing them to what you actually said.
The situation behind the question
The variant lowered the free-shipping threshold. It ran for one week, which included a public holiday. The dashboard shows the variant ahead and marks it "significant". Nobody has looked at margins or returns.
What it is really testing
Whether the candidate reads an experiment honestly: sample, duration, novelty and calendar effects, whether the decision was set in advance, and whether a guardrail was quietly paid for the win.