Date: Thu, 10 Oct 2002 01:13:06 0700
ReplyTo: John Hendrickx <john_hendrickx@yahoo.com>
Sender: "SPSSX(r) Discussion" <SPSSXL@LISTSERV.UGA.EDU>
From: John Hendrickx <john_hendrickx@yahoo.com>
Subject: Re: Sample vs Population
InReplyTo: <3DA4B732.9070800@citrus.ucr.edu>
ContentType: text/plain; charset=usascii
Software for complex survey designs can deal with samples from a
"finite population", i.e. a relatively small population whose size is
known. Wesvar and SUDAAN are standalone programs, Stata and SAS are
packages with these facilities.
Of course you'd be making the very dubious assumption that the
nonresponses are missing at random and that your sample therefore
constitutes a random sample of the population. When dealing with a
large nonrandom sample of a population, sampling fluctuations aren't
an issue. The problems you're dealing with now (in my opinion) are:
1. Is selective nonresponse leading to bias in my results?
2. How are confidence intervals affected by model misspecification
(omitted variables, omitted interaction effects).
I don't suppose there's much you could say about (1). You could run a
few simulations assuming a certain amount of bias in your sample to
get an idea of the accuracy of your results in a "worst case"
scenario. Maybe there are better methods available?
With regard to (2), you might get an answer using the software for
comples surveys mentioned above. These can calculate "robust standard
errors" that take model misspecification into account. For example,
in a regression model the standard errors are weighted by the
residuals of the model. This takes heteroscedasticity due to unequal
sampling probabilities (weighted dataset) or omitted variables into
account. But I'm not sure whether these standard errors would be
meaningful if you're dealing with a nonrandom sample.
I've crossposted this to the survey research methods list, perhaps
the folks there can shed some more light on this issue. The easiest
advice of course is to forget statistical inference and to treat your
results as the population parameters. But I wonder if perhaps more
can be said when dealing with population data.
 Jelani Mandara <jelani@citrus.ucr.edu> wrote:
> Alison Neustrom wrote:
>
> >What would you say if you had 80% of a nonrandom sample of a
> population (I
> >sent them to everyone but only got back 80%)? Would you consider
> that a
> >sample or a population?
> >
> >
> >
>
> Inferential stats rely on the size of the sample more than the % of
> the
> population sampled. This is a weakness, but since we rarely know
> the
> actual size of the population, it's understandable.
>
> So, I would argue that it depends on the size of the
> sample/population.
> If you have 80% of 100, then you may wish to increase alpha, since
> you
> risk the chance of Type II errors (i.e., incorrectly concluding a
> null
> result). I think increasing alpha will be more reliable than just
> using
> descriptives to test nulls in small samples when you have 80% of
> the
> population. In large samples (say, N > 1000) I doubt it matters.
> Some
> of the statisticians on the list may know for sure.
>
> >Original Message
> >From: SPSSX(r) Discussion [mailto:SPSSXL@LISTSERV.UGA.EDU]On
> Behalf Of
> >Jelani Mandara
> >Sent: Wednesday, October 09, 2002 11:50 AM
> >To: SPSSXL@LISTSERV.UGA.EDU
> >Subject: Re: Sample vs Population
> >
> >
> >Fink, Steven wrote:
> >
> >
> >
> >>Recently, I've been asked by several analysts about analyzing
> populations,
> >>
> >>
> >that is, data not from a sample.
> >
> >
> >>The purpose of a statistical test is to make estimates about the
> >>
> >>
> >population. If my data set IS the population, are significant
> tests
> >appropriate?
> >
> >
> >>
> >>
> >>
> >No. Descriptives and Effect Sizes are still relevant, but not
> >inferential stats.
> >
> >
> >
> >>What about if my response rate (survey/unit) is low, say 20%.
> For example,
> >>
> >>
> >I sent surveys to everyone in the population, but only received
> 20% of
> >responses. Does this change your answer to the first question?
> >
> >
> >>
> >>
> >>
> >>
> >
> >Then you only have a (probably nonrandom) sample of the
> population. The
> >logic of inferential stats applies.
> >
> >
> >
> >>Thanks
> >>
> >>
> >>
> >>Steve
> >>
> >>
> >>
> >>
> >>
> >
> >
> >Jelani Mandara
> >Assistant Professor
> >Human Development and Social Policy
> >Northwestern University
> >2120 Campus Drive
> >Evanston, Il 60208
> >
> >Office Phone: (847)4913122
> >Web Page:
> http://www.sesp.northwestern.edu/people/sp/j_mandara.html
> >
> >
> >
>
> 
> Jelani Mandara
> Assistant Professor
> Human Development and Social Policy
> Northwestern University
> 2120 Campus Drive
> Evanston, Il 60208
>
> Office Phone: (847)4913122
> Web Page: http://www.sesp.northwestern.edu/people/sp/j_mandara.html
__________________________________________________
Do you Yahoo!?
Faith Hill  Exclusive Performances, Videos & More
http://faith.yahoo.com
