As an example, assume a multichannel organization mails a catalog to a housefile list of 1,000,000 names. The database marketer chooses the best 1,100,000 households, and randomly splits them into two groups. The "A" portion of the test are the 1,000,000 households who receive the catalog. The "B" portion of the test are 100,000 households who will not receive the catalog.
Maybe a month after the in-home date, the database marketing analyst is prompted to analyze the results. Within each group, the 1,000,000 who received the catalog, and the 100,000 who didn't receive it, the analyst calculates the average net sales within the catalog/telephone channel, the online channel, and the retail channel.
Here are sample results:
|Did Not Receive Catalog||100,000||$2.50||$7.00||$19.50||$29.00|
In this example, the catalog drove an incremental $3.50 per customer to the catalog/telephone channel, $1.00 per customer to the online channel, and $1.50 per customer to the retail channel, for a total of $6.00 incremental sales per customer.
Because we mailed 1,000,000 households, the total net sales attributed to this mailing is 1,000,000 * $6.00 = $6,000,000.
Some vendors advocate a different methodology --- they advocate allocating any online and retail order generated during the time the catalog was active to the mailing of the catalog. This results in a gross over-estimation of the importance of the catalog. Please don't go down this path.
A similar methodology can be used to test multiple marketing activities at the same time. Assume an e-mail campaign was mailed to the opt-in portion of this audience. Within this audience, you randomly assign customers to one of four test segments. Here are some sample results.
|Catalog + E-Mail||400,000||$5.50||$8.50||$21.25||$35.25|
|No Catalog, No E-Mail||50,000||$3.50||$7.00||$18.50||$29.00|
Tests like these yield interesting and intriguing results. Notice that the best strategy for the catalog/telephone channel was to mail only a catalog. The best strategy for the online channel was to mail a catalog and an e-mail. The best strategy for the retail channel was to mail both a catalog and an e-mail.
Statisticians can assist with significance tests, if you feel that is appropriate. It is more important to simply execute tests of this nature, and learn how all of your marketing activities interact with each other. What you learn about how marketing activities and channels interact with each other within our multichannel ecosystem may surprise you.