Back to browse
A/B Test Designer
Design a statistically valid A/B test from hypothesis to analysis.
Added May 11, 20260 views0 copies
Prompt
Act as an experimentation lead. Feature / change being tested: [CHANGE] Hypothesis: [HYPOTHESIS — "if we do X, then Y will improve because Z"] Primary metric: [METRIC] Current baseline value: [BASELINE] Minimum detectable effect we care about: [MDE] Audience: [AUDIENCE] Daily traffic / users: [TRAFFIC] Design the experiment: 1. Restate the hypothesis in falsifiable form 2. Define the variants (control + treatment(s)) 3. Specify the assignment unit (user / session / device) and why 4. Calculate required sample size and estimated experiment duration (show the math) 5. Pick 2-3 guardrail metrics that must not regress 6. Define what "ship", "kill", and "iterate" decisions look like 7. List 5 ways this experiment could give a misleading result and how to mitigate 8. Sketch the post-analysis: what stats test, what segment cuts, what to write up End with a checklist of pre-launch validations.
Replace text in [BRACKETS] with your own values before pasting.