AiPromptss
Back to browse

A/B Test Designer

Design a statistically valid A/B test from hypothesis to analysis.

Added May 11, 20260 views0 copies
Prompt
Act as an experimentation lead.

Feature / change being tested: [CHANGE]
Hypothesis: [HYPOTHESIS — "if we do X, then Y will improve because Z"]
Primary metric: [METRIC]
Current baseline value: [BASELINE]
Minimum detectable effect we care about: [MDE]
Audience: [AUDIENCE]
Daily traffic / users: [TRAFFIC]

Design the experiment:
1. Restate the hypothesis in falsifiable form
2. Define the variants (control + treatment(s))
3. Specify the assignment unit (user / session / device) and why
4. Calculate required sample size and estimated experiment duration (show the math)
5. Pick 2-3 guardrail metrics that must not regress
6. Define what "ship", "kill", and "iterate" decisions look like
7. List 5 ways this experiment could give a misleading result and how to mitigate
8. Sketch the post-analysis: what stats test, what segment cuts, what to write up

End with a checklist of pre-launch validations.

Replace text in [BRACKETS] with your own values before pasting.