tomclegg.net |
Scientific Survey Posted November 18, 2001 I read something on the back of a newspaper the other day that claimed to be the result of a "scientific survey" about a new recreation facility. Presumably the word "scientific" was thrown in just to make it look important. There are several interesting pieces of information missing from the report. The first thing I looked for was a confidence figure: something along the lines of, "this survey has a 99% chance to be 98% accurate." You can get that out of a formula, and it's supposed to give you an idea of how big the random sample was and stuff like that. If you don't have a confidence figure, you can at least look at the size of the sample and make a subjective judgement about whether it was appropriate. But this particular report said nothing about the sample size. Did they give a questionnaire to 71 people? Or maybe 1000? Or 17? Who knows. Truly random sampling of a human population is practically impossible. The people who don't have telephones are rarely surveyed, for example; or the people who prefer not to respond to questionnaires. The best you can do without a lot of extra trouble is keep track of how many people refused to answer the questionnaire, or didn't answer the phone at all. That would give a rough idea of how random the sample really was. Of course, if you don't know the sample size, the number of refusals doesn't mean much to you. So that's probably why that figure wasn't mentioned either. Of course, the most important thing missing from the article was the questionnaire itself. So perhaps the piece should have been labelled, "City Hall's interpretation of a scientific survey, whose actual results are available at City Hall to anyone who's interested." So I'm off to City Hall to get a copy of the questionnaire. I'll leave you with... |