Fallacy - 6

Hasty Generalization

Alias: Converse Accident

Type: Unrepresentative Sample

Quote…

It's a story, say, about the New York City public schools. In the first paragraph a parent, apparently picked at random, testifies that they haven't improved. Readers are clearly expected to draw conclusions from this. But it isn't clear why the individual was picked; it isn't possible to determine whether she's representative; and there's no way of knowing whether she knows what she's talking about. Calling on the individual man or woman on the street to make conclusive judgments is beneath journalistic dignity. If polls involving hundreds of people carry a cautionary note indicating a margin of error of plus-or-minus five points, what kind of consumer warning should be glued to a reporter's ad hoc poll of three or four respondents?

…Unquote

Source: Daniel Okrent, "13 Things I Meant to Write About but Never Did", New York Times, 5/22/2005

Example:

Of course your columnist Michele Slatalla was joking when she wrote about needing to talk with her 58-year-old mother about going into a nursing home. While I admire Slatalla's concern for her parents, and agree that as one approaches 60 it is wise to make some long-term plans, I hardly think that 58 is the right age at which to talk about a retirement home unless there are some serious health concerns. In this era, when people are living to a healthy and ripe old age, Slatalla is jumping the gun. My 85-year-old mother power-walks two miles each day, drives her car (safely), climbs stairs, does crosswords, reads the daily paper and could probably beat Slatalla at almost anything.

Source: Nancy Edwards, "Letters to the Editor", Time, 6/26/00.

Exposition:

This is the fallacy of generalizing about a population based upon a sample which is too small to be representative. If the population is heterogeneous, then the sample needs to be large enough to represent the population's variability. With a completely homogeneous population, a sample of one is sufficiently large, so it is impossible to put an absolute lower limit on sample size. Rather, sample size depends directly upon the variability of the population: the more heterogeneous a population, the larger the sample required. For instance, people tend to be quite variable in their political opinions, so that public opinion polls need fairly large samples to be accurate.

Exposure:

Suppose that you are cooking a pot of spaghetti, and you fish out a single strand to test for doneness. If it is done, then you conclude that all of the spaghetti in the pot is done. Here, your sample is one strand of spaghetti, and the population is the entire potful of pasta. Have you committed the Fallacy of Hasty Generalization? No.

This is a familiar type of inference that most of us engage in whenever we cook something, for instance, when we taste a pot of soup to test whether it is sufficiently seasoned by tasting a single spoonful. We don't feel it necessary to test several spoonfuls, because we have every reason to believe that the spoonful we test is representative of the whole pot of soup. In the same way, a single strand of spaghetti can be representative of a whole pot of noodles.

The reason why these kinds of inference can work is because spaghetti is mass-produced, and every noodle from the same box is virtually identical to all the others. Moreover, if we put a whole boxful of spaghetti into a pot of boiling water, we can be pretty sure that all of them are being cooked for the same amount of time and at the same rate. It is in this way that we can know that the single noodle we test is a representative noodle, that is, it is like all the other noodles in the pot in terms of doneness.

How do such inferences go wrong? Let's return to the soup example: suppose that you season the soup by sprinkling spices onto its surface, but that you forget to stir the pot. Then, if you take your test spoonful from the top of the soup, without stirring, it is unlikely that it will be representative. Instead, you are likely to get much more of the spices in that spoonful than you would get from the bottom of the pot. If you fail to notice this, and conclude from your sample that the soup is sufficiently spicy, then you will have committed the Fallacy of Hasty Generalization. You will probably be disappointed later that the soup is not flavorful enough. This is why we stir a pot of soup after seasoning it, and before tasting it, so that the spices will be evenly distributed throughout the liquid, and a single spoonful will be representative of the entire pot.

When we are dealing with populations that are more variable than soup or spaghetti, we need to be not only careful how we take the sample, but we have to take a sample that is big enough to represent the variability of the population. If we are polling people's political views, then a sample of just one person is guaranteed to be misleading, no matter what opinions that person has. A hasty generalization occurs anytime the sample is not big enough to represent the population.

Source:

S. Morris Engel, With Good Reason: An Introduction to Informal Fallacies (Fifth Edition) (St. Martin's, 1994), pp. 137-140.

Resource:

Douglas Walton, "Rethinking the Fallacy of Hasty Generalization" A technical paper in PDF format.