Quantitative Information in Knowledge Claims

Return to Critical Thinking Reference

As we saw in discussing knowledge claims, one powerful direction that the growth of knowledge has taken has been to quantify our experience. To learn more about quantificational methods in knowledge claims, we begin with the concept of measurement itself. We consider ways that measurements can be misunderstood or used deceptively. People have difficulty, for example, in comparing measurements. We'll look at specific problems that arise from the use of percentages, linear and non-linear relationships, and surveys. Another source of difficulty in the use of quantitative information in knowledge claims comes from our incomplete grasp of probability. So we take some time to discuss some basic ideas in probability and some ways that people makes mistakes in thinking about knowledge by misunderstanding probability. As always, our focus is on using conceptual knowledge about these topics to find practical advice about how to think well.

Surveys

When most people think of surveys, they think of public opinion polls such as those produced by respected polling companies, such as the Gallup Organization (www.gallup.com). Because opinion polls enter into reflective discussions so often, we will focus most of our attention on them. But some of what we will say about good opinion surveying applies as well to any situation in which you want to sample the frequency of some event or property, whether you manufacturing process, or the state of the rush hour traffic jam.

Whether you are studying people's opinions or sampling some other kind of system, the central concept of good surveys is representative sampling. In what is still the textbook example of wnrepresentative sampling, the 1936 Literary Digest poll predicted that Alf Landon would beat Franklin Delano Roosevelt in the presidential election. The sample population, taken from telephone hsts and auto registration lists, was over 2 million people, far more than is needed with current polling techniques. The U.S. population can usually be surveyed accurately by contacting about 1200 people. The prediction was false because the sample contained an unrepresentative number of wealthy Americans (at that time, people who owned cars or telephones) with a class interest in voting for Landon. In other words, wealthy Americans had a far greater chance of appearing in the sample population than in the total population. The general difference in economic status between the sample population and the general population was a relevant difference because it made a difference in how people from each group felt about the presidential race.