vijay.jayanti@GMAIL.COM wrote: >I need some statistician to respond to this question. > >SITUATION: >In our Direct mail Campaigns, we pull a 10% random control/ Hold out >group. >We send the test/ treatment group to a mail fulfillment house. >They post back a variable to the database which says "Able to mail" >which is a Yes/ NO column, which captures the information, if that >address was mailed or not. > >CURRENT MEASUREMENT METHODOLOGY: >Measure response rate for both treated and control cells and subtract >(TEST - CONTROL) to get the lift. Test for significance. > >While calculating the response rate for TEST (= RESPONSES/ BASE), we >include ALL THE CUSTOMERS in the base (including those who were flagged >"NO" in "ABLE TO MAIL" field. > >The prevalent method in analyzing the responses is to ignore this >variable. The reason being quoted by my senior colleague is that "Since >the control group does not have a "ABLE TO MAIL" field, (since they do >not go thru the mailing process), we have to include all the customers >irrespective of their "ABLE TO MAIL" value. So if we exclude those who >did not get a mail, we are NOT STATISTICALLY CORRECT, AS IT IS NOT >RANDOM ANYMORE. (MEANING CONTROL IS NOT ANYMORE EQUIVALENT TO TEST). > >Also I am told that from a Financial/ ROI point of view, including >everyone in the base gives a more correct picture. > >MY OPINION: >I think it is wrong. I have over 4+ years of experience as a marketing >analyst (I am not a statistician). I think we should exclude those who >did not receive a mail and report the correct response rate. Other wise >we will be depressing the true resp. rate. > >what is right way to compute the response rate? INCLUDE THOSE WHO DID >NOT GET A MAIL IN THE BASE? OR EXCLUDE THEM?

I see that TopKatz took a not-claiming-to-be-an-expert approach on this.

I agree with TopKatz and your 'senior colleague'. And I like to think that I know everything. :-) :-)

The key point (well, *one* key point) is your target population for any future marketing campaigns. You want to be able to extrapolate back to the real target population. And that means including everyone in the database. Particular types of survey sample analyses might warrant this exclusion (with appropriate survey design and weighting consequences), but that's not what your company is expressly after here.

HTH, David -- David L. Cassell mathematical statistician Design Pathways 3115 NW Norwood Pl. Corvallis OR 97330

_________________________________________________________________ Express yourself instantly with MSN Messenger! Download today - it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/