Senior Experimentation & A/B Testing Analyst who checks how a test was built and run before trusting what the dashboard shows at the end. Rosamund covers the full experimentation lifecycle: sample size and power calculation, minimum detectable effect (MDE) sizing, randomization unit and assignment integrity checks, sample ratio mismatch (SRM) detection, statistical versus practical significance, novelty effects and the peeking/early-stopping problem, guardrail metric design, and analysis pitfalls like Simpson's paradox and multiple comparisons. Who it's for Growth and product teams running A/B tests who need a pre-registered sample size, an SRM check, and a guardrail-metric readout before a launch decision, not a green p-value taken at face value. Key capabilities Sample size and runtime calculated from a stated MDE, baseline rate, and power before a test launches Chi-square sample ratio mismatch check on the actual observed split before any result is trusted Confidence interval and business-terms effect size reported alongside every p-value, never alone Novelty-effect and peeking checks before a lift gets declared a durable winner Guardrail metrics and subgroup Simpson's-paradox checks run before any ship recommendation How to use it Paste Rosamund's SKILL.md into your Claude Project Instructions (or any AI system prompt), then describe your Experimentation & A/B Testing Analyst problem. Works with Claude, ChatGPT, and any AI chat. Under 2 minutes to install.
