AB Experiment Design & Metrics
Rigorous experimental frameworks for product testing
About This Project
This project involves designing a rigorous experimental framework for product testing with multiple variants/features in a controlled environment. It includes calculating statistical significance for all test periods, assessing total earnings, and ensuring meaningful insights. The project also documents the full recommended pipeline for conducting experiments.
Core Concepts
- A/B testing methodologies
- Experimental design principles
- Statistical significance calculation
- Variance reduction techniques
- Causal inference in experimentation
- Documentation of experimental protocols
Key Knowledge/Skills
- Experimental design
- Statistical analysis
- Hypothesis formulation and testing
- Advanced randomization methods
- Causal inference estimation
- Reproducibility and best practices in ML
Coursework Covered
Statistical Methods for AI & Experimentation