It was 1991 at Lands' End. We were greatly ramping-up our testing work. And when we wanted to execute a test, we needed to understand how the results might "vary".
Back then, a test was sampled from the population who would receive a catalog. Maybe that audience was 4,000,000 customers. If the catalog was a productive catalog, it might generate $10.00 per catalog mailed. If the catalog wasn't productive, circulation would be reduced and the catalog might generate $4.00 per catalog mailed.
If you want to measure a 10% difference in sales for two groups performing around $4.00 per book, you need fewer customers than if you are trying to measure at 10% difference in sales for two groups performing around $10.00 per book. This is an issue called "heteroscedasticity".
So I built an equation that measured variability around different dollar-per-book estimates. The equation was a simple one:
- -188 + 192*(Expected Dollar per Book).
- $4.00 = -188 + $192*4.00 = 580.
- $3.60 = -188 + $192*3.60 = 503.
- (4.00 - 3.60) / SQRT(580/25000 + 503/25000).
- T = 1.92.
- (4.00 - 3.60) / SQRT(580/30000 + 503/30000).
- T = 2.11.