Dear Catalog CEOs:
When we chat, we inevitably talk about search, a channel that was top-of-mind during the Google-infused pre-recessionary environment of 2004. Half of the discussions exhibit a contempt with search, including comments like "it doesn't work" or "I'm not paying Google one more penny." Half of the discussions include comments like "it's a fantastic way to acquire new customers" or "we convert existing customers through search".
Who is right?
Those of you who run mail/holdout tests probably know the answer.
Take a look at this test, run for three months.
Look at the "Search" column. Let's assume that we're focusing on Paid Search here, though Natural Search can be analyzed as well.
When you see something like this ... a case where the mailed group outspent the holdout group in search, it tells you that YOU MUST EMBRACE SEARCH. It means that your catalog drives customers to Google, for whatever the reason.
When you see that the mail group and the holdout group have the same spending levels (say $5.00 in each row), then search takes on a different role. It means that the customer does not use search as part of the catalog marketing process. Here, you may or may not need to embrace search.
Holdout tests will tell you the answer. Ask your Chief Marketing Officer for a report, just like the one illustrated above, that will tell you how catalogs and search interact with each other.