Identifying Biased Samples
Start by asking: Who is this sample supposed to represent, and how were people or cases chosen? A sample can look large and still give a distorted picture if the people included differ systematically from the wider gr...
Start by asking: Who is this sample supposed to represent, and how were people or cases chosen? A sample can look large and still give a distorted picture if the people included differ systematically from the wider group. Huff’s examples suggest checking: Who had a chance to be included? A poll based on telephone owners, for example, may miss people who don’t have telephones and therefore may not represent all voters. [:cite[1]{ln=4}] Who was missed by the way the sample was gathered? Street or daytime interviews can leave out people who stay home, work, or go out at those times. [:cite[2]{ln=5}][:cite[2]{ln=6}] Could people’s choices or answers skew the results? Interviewers may approach some people rather than others, and respondents may give answers they think will please the interviewer. [:cite[4]{ln=3}][:cite[4]{ln=6}][:cite[3]{ln=2}] Does the conclusion go beyond the group actually sampled? People interviewed at a train station, for instance, represent people at that station—not automatically a whole city or country. [:cite[5]{ln=5}][:cite[5]{ln=6}] A useful rule of thumb: a big sample is not automatically a representative one . Look for how it was selected, who may have been left out, and whether the claim is limited to what the sample can support. [:cite[1]{ln=4}][:cite[6]{ln=2}]