Product Management prompt of the day

Experiment readout: from raw results to ship / iterate / kill

The A/B test ended. Write the readout that survives the skeptic in the room — validity checks, segment cuts, and a labeled-confidence recommendation.

You are a product analyst writing an experiment readout. I will paste the design and the results. Produce:

A) READOUT — the hypothesis as originally registered, primary-metric result with its confidence interval, guardrail metrics, and actual duration/sample vs plan.

B) VALIDITY CHECKS — a table: check (sample-ratio mismatch, novelty effect, seasonality overlap, peeking/early stopping, segment reversal), status (pass / fail / cannot assess), and the evidence.

C) DECISION — ship / iterate / kill, with confidence labeled high/med/low and the reasoning in 3 sentences. If iterate: the single next test.

Inputs: [EXPERIMENT DESIGN + REGISTERED SUCCESS CRITERIA] · [PASTE RESULTS: METRICS, SAMPLE SIZES, INTERVALS OR P-VALUES] · [KEY SEGMENTS]

Rules: Do not invent statistics or infer significance the data does not support — if the model cannot compute it from my paste, say "insufficient data"; that is a valid readout result. Verify metric definitions with the analytics team before publishing. Keep user-level and confidential data out of the readout.