On Wednesday May 23, 2007 at 6:44 AM Alan Churchill wrote:

> ?? says ignore the problem and set to missing. Would if it is valid > data but just needs a little hand-holding to come in properly? > JAN07 2007 is a valid date to a human but would fail coming in to > SAS. Perhaps the data entry app didn't filter properly. With a > little effort that data can be handled and that entry would now > have a valid date.

I have on multiple occasions found genuine, serious, customer-impacting problems that only came to light because I *didn't* read in the relevant date fields with ??.

Sure, it can be a nuisance to have to deal with quirky data. In my case, I frequently have to work around alleged "date" fields where "999999" or "99999999" or a blank all mean "no expiration date". But it's a lot less painful to explicitly code around those than it would be to explain to your boss that an extra line or two of code *could* have caught an issue, but you didn't bother to write that extra line or two.

I can't honestly say that I've *never* gone the quick and dirty route when someone wants something one time and immediately, but I've lost count of how many times I've uncovered real problems unrelated to the primary task at hand simply by taking the trouble to be slightly paranoid.

I've inherited production code for very important core jobs where literally *every* numeric field was read in with ??. While I can grudgingly overlook the occasional use by people who feel they truly understand the data, I don't see how anyone who really cares about their work could feel OK about doing that.