Date:         Fri, 9 Sep 2005 14:58:14 -0500
Reply-To:     "Nick ." <ni14@MAIL.COM>
Sender:       "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU>
From:         "Nick ." <ni14@MAIL.COM>
Subject:      DIRECT MARKETING CAMPAIGN--CONTROL GROUP question_UPDATE
Content-Type: text/plain; charset="iso-8859-1"

Hello all,

Management wants to repeat the campaign next week (fresh prospects to campaign to) using my model as we have talked about the past couple of days or so. They will use the vendor again but the vendor will randomly select prospects as a CONTROL group across the entire file. Previously they had chosen the CONTROL group from the high-propensity-to-respond prospects, namely, deciles 7 through 10. According to my model (using a previous campaign of this sort and scoring it and deciling the probabilities from low to high), the average (expected) response rate in deciles 7 through 10 is about 0.26% or so. Previous campaigns of this sort, using randomly selected prospects, generated response rates around 0.13%.  A control group set aside of individual who were not campaigned to but respond on their own brought response rates around 0.05%. (Again these numbers are from previous similar campaigns.)Sometimes I have seen this rate (people responding on their own) as close as 0.11%, close to the 0.13% an indication that in those campaigns the campaign itself was not effective, am I right?

We want to test (null hypothesis?): Will the model discriminate better between responders and non-responders. Put another way, management wants to know if the model response rate will outperfor random response rate (i.e the historical 0.13% or so.). As I said above, we know from previous campaigns of this sort that people randomly campaigned to responded at a rate of about 0.13%.

My model produces the following probability scores. (I used a previous campaign to develop the model, I did not use the CONTROL group--people who responded on theit own, then I scored the campaign file + CONTROL group, I deciled the results which I show below.)

            AVERAGE PROBABILITY
DECILE10 0.56%
DECILE9  0.29%
DECILE8  0.16%
DECILE7  0.10%
DECILE6  0.10%
DECILE5  0.09%
DECILE4  0.04%
DECILE3  0.01%
DECILE2  0.01%
DECILE1  0.00%
The overall average response rate is about 0.13% or so and that in deciles 7 through 10 is about 0.26% or 0.28% or something like that.

The vendor now has another file of about 230K new prospects. It used my model shown above to score this new pouplation. The score distribution is as follows:

            AVERAGE PROBABILITY
DECILE10 0.86%
DECILE9  0.51%
DECILE8  0.49%
DECILE7  0.47%
DECILE6  0.44%
DECILE5  0.27%
DECILE4  0.18%
DECILE3  0.11%
DECILE2  0.09%
DECILE1  0.03%
The overall average response rate is about 0.35% or so and that in deciles 7 through 10 is about 0.58% or something like that.


Q1: I am a little concerned that this new batch of prospects has quite a bit different scoring distribution than the campaign did on which I buit the model. Should I be concerned and should we apply the model for this new campaign?

Q2: Given the above experiment (null hypothesis) and an entire population of about 230K records and about 92K prospects to campaign to(these 92K come from deciles 7 through 10), what do you think the CONTROL group size should be? It would be nice if we could put aside 50% of that, but it won't happen. Management wants something like 10% or about 10K as a control.

Q3: To carry the experiment stated above, the 10% or so control group, should it be campaigned to or not???? I am thinking that it, too, should be campaigned to but management says no. Let the 10% control be a group of individuals to respond on their own. I say not so if you want to test the hypothesis above, they say, yes. So, I ask you bec. I can't afford to screw this one up.

Q4: I (not management or vendor) also want to test the campaign's (creative, promotion, etc.). I know that according to the model the top 4 deciles ought to bring in about 0.26% response or something like that. How do I, at the same time, while campaigning to the 92K individulas (model selected deciles 7 through 10) test the campaign effectiveness? If I don't put aside another control group and ley things go as described above, then, if I see a response rate from the 92K campaigned individuals (much) higher than the expected 0.26% (or, actually, the expected 0.58% according to this newly scored population), then do i conclude also that the campaign was effective? Or do ineed to set aside another control group?

As always, your thoughts are much appreciated. I apologize for the long email but I wish to explain things fully and repeat myself so as to not be any confusion about what information we have, how I got it, and what I want to test.

NICK


--
___________________________________________________________
Sign-up for Ads Free at Mail.com
http://promo.mail.com/adsfreejump.htm