- Never take a poll at face value. Gather as much information about the poll as possible. A high quality poll will usually feature a prominent description of its methodology or provide a link to it.
- Research the organization conducting the poll. Is the organization experienced in the polling business? Does it have any biases that could influence the results?
- If available, look at the poll questions and examine the language used. Are the questions asked in such a way as to elicit a particular response?
- Look at the sample size (how many people were interviewed for the poll), margin of error, and confidence level. The margin of error indicates how close you can expect the results to be to the actual value a percentage of time equal to the confidence level. For example, if a poll states that 60% of a population supports a certain position with a margin of error of ±4 points and a confidence level of 95%, you can expect, with 95% certainty, the true figure to be between 56% and 64%. Increasing the sample size will narrow the margin of error and/or make the confidence level higher.
- Look at how the people interviewed for the poll were contacted. Were they contacted randomly, or was an existing panel used? Was the poll conducted by landline phone, cell phone, online, or by mixed methods? Mixed methods are generally the best since each individual method has its own shortcomings. For example, many people do not have landlines any longer, but pollsters are often prohibited from autodialing cell numbers. People also tend not to pick up the phone when they do not recognize the number of the person calling. Polls that are online only would skew the sample toward people who own computers and, in some cases, might bias the poll toward a certain response.
- For polls conducted by phone, was a live interviewer or an automated system used? Sometimes people will be hesitant to tell a live interviewer something that they perceive to be unpopular even if it is what they truly believe.
- Check to see whether the sample is representative of the population being surveyed. For example, in an election poll, do the age, racial, income, gender, and party affiliation breakdowns seem to reflect the population of the geographic area that is being polled? Has the sample been weighted to match the characteristics of the population properly? (This might require some additional research.)
- Look at how current the poll is and the time frame over which it was conducted -- particularly in the case of election polls. A recent poll should be weighted more heavily. Also, a good poll is usually conducted over a modest time frame, such as a few days. If the time frame is too long, then opinions might be change substantially during the period. If the time frame is too short, such as one night, then the sample might not be random since it may not allow enough time to contact a representative sample of people.
- Polls with fewer undecideds (but not too few) tend to be more reliable than those with many. Also, did the poll get a low response rate? If most of the people who refused to respond all had the same characteristics, then the poll could be skewed.
- For election polls, check to see whether the poll is for all adults, registered voters, or likely voters. Polls of registered voters are more appropriate further away from the election, while likely voter surveys are better closer to the election. However, keep in mind that a pollster's assumptions about the identity of a likely voter might be erroneous.
- If more than one poll exists on a given topic or, in the case of elections, for a given race, an average generally will give you a better picture than any individual poll. Watch for outlier polls, ones whose results differ greatly from other polls on the subject.
Read more about opinion polls on this page by Teach Democracy.
LOGIN to Harcum Hatch
