Polls 2016 - History

Polls 2016 - History


We are searching data for your request:

Forums and discussions:
Manuals and reference books:
Data from registers:
Wait the end of the search in all databases.
Upon completion, a link will appear to access the found materials.


A Brief History of Voting Problems on Election Day

T he U.S. has a long history of problems at the polling station&mdashand not just the potentially agonizing choice over whom to vote for. From practical problems to intimidation, there are dozens of difficulties that have befallen Americans trying to have their voices heard. During the 2016 primary-election season, voters encountered problems ranging from long lines to incorrect registration rolls. And, as Tuesday’s general election gets underway, election protection groups are standing by to address any issues that voters may encounter.

A look at the history of voting problems in the U.S. reveals that, while Election Day has always been complicated, the ways in which voting has been made difficult over the years are consistent, and they fall into just two categories: problems of technology and problems of access.

If one thing can be said about the history of American voting technology, it’s that every voting mechanism has flaws.

Over the centuries, technological changes have provided new opportunities for errors and difficulties when voting. Voting by voice or color-coded ticket exposes voters to intimidation or vote-purchasing voting secretly excludes those who cannot read or write. It can be hard to interpret a voter’s intention in a check box on a piece of paper, but mechanical voting machines can also malfunction the hanging and dimpled chads of the 2000 election still loom large. Meanwhile, computerized machines haven’t fixed that problem: in 2004 Senator Barbara Mikulski of Maryland collected reports that voters in three counties never even saw the Senate primary on their voting machine screens.

But, while technology can be problematic, access&mdashmaking sure that everyone who has the right and the desire to vote is able to do so&mdashhas always been the more troubling Election Day problem.

Perhaps the most straightforward form of trouble at the polls is being denied on sight. Women trying to vote in protest before the passage of the 19th Amendment were often turned away, though not unlawfully.

Meanwhile, others who did have the right to vote were often denied that right too&mdashmost notably African Americans.

Though Election Day violence and rioting was relatively common throughout the country in the mid-19th century&mdashhistorian David Grimsted has counted 35 Election Day riots, and 89 deaths as a result of that violence, between 1828 and 1861&mdashit was not always the result of spontaneous voter hysteria. Rather, even after the 14th Amendment guaranteed black men the same rights of citizenship that white Americans enjoyed, violence meant that in practice the right was only briefly enjoyed by the black population. Octavius Catto, a black man who had helped the Union in the Civil War, an intellectual and a skilled teacher, was shot in Philadelphia in 1871 for voting. When Indianapolis held city elections in May 1876, there were reports or African American voters being physically attacked at polling places. And in the presidential election of that year, one white supremacist in South Carolina demanded “every Democrat must feel honor bound to control the vote of at least one Negro, by intimidation, purchase, keeping him away or as each individual may determine.”

A letter from an African-American man headlined in the New York Times as “How They ‘Voted’ in Mississippi” described the intimidation that faced those who did get to vote, who were forced on threat of violence to vote the opposite of what they wanted. Reports of such tactics led to a congressional investigation, during which one man testified that in Louisiana “he saw fifty colored men marched up to the polls by a guard of white men and voted the Democratic ticket, being required to show their tickets before voting,” reported the Detroit Free Press.

And, both before and after that Reconstruction period&mdasheven into the 1960s&mdashnonviolent methods of disenfranchisement were also widespread.

Voters might have to meet financial requirements. In the early 19th century, New York increased the amount of taxable property required for a black man to vote, even as such requirements were eliminated for others. Poll taxes became a ubiquitous method of disenfranchising blacks post-Reconstruction. Later, potential voters were sometimes falsely informed that they might be arrested for unrelated infractions like traffic tickets if they showed up to the polls, a scare tactic used in Texas in 1964 that was apparently still being used in New Jersey in 1996. Some were threatened or served with economic retribution in the form of loss of jobs, eviction or loan denial.

Voters might have to pass a test for literacy or “understanding.” The devious “eight box law” of 1882 in South Carolina was meant to confuse less-educated voters by giving each race for office a different ballot box, so that you had to be able to match your vote to the right box for that candidate, and periodically shuffling the boxes. These hoops developed into literacy tests&mdashwhich were often nearly impossible to pass&mdashthat were long imposed on African Americans in the South. As late as the 1960s, voters might be asked convoluted logic questions that even an expert reader might get wrong. Restrictive laws about literacy weren’t just aimed at the poor or black voters: New York, home to immigrants for generations, adopted an English literacy requirement in 1921.

Or voters might find it was simply impossible to get to the polls. Lerone Bennett Jr. wrote in 1962 in Ebony about the strategies employed to make voting physically impossible for black voters in the south after the Civil War: “Armed white men were stationed on the roads leading to the polls ‘to prevent Negroes from seizing arms.’ In one Louisiana county, the polling places were located in an isolated wilderness. The whites gathered at the white church and were told, in whispers, how to reach the polls.” These weren’t the only manipulations, and tactics of brief or changing polling hours, last minute changes to polling locations or inaccessible locations persisted.

Those concerns also persist today: in 2016 there are hundreds fewer polling places than there were two or four years ago. And, for example, a last-minute change to polling places has already affected Boise, Ida., residents. ProPublica’s Electionland project is monitoring reports of long wait times, problems with machines and registration around the country

In general, however, things have improved.

The 1965 Voting Rights Act helped make sure that tactics like literacy tests could no longer be used to disenfranchise African Americans. After 1975 congress required language assistance for minority groups in districts where they made up 5% of the population or numbered 10,000. A 1982 extension to the Voting Rights Act added protection for blind, disabled and illiterate voters. The 1990 Americans with Disabilities Act further required physical accessibility and accommodations for people with disabilities.


Princeton Election Consortium

November 8th, 2016, 12:45am by Sam Wang

(Updates: 6:06am data for Presidential and Senate, and added confidence intervals. 9:00 am: more description, also variance minimization.)

Here are the final snapshots. Four Senate races are within one percentage point: Indiana, Missouri, New Hampshire, and North Carolina. Partisans there may want to lawyer up for possible recount battles.

Soon I’ll put out a brief Geek’s Guide to the Election. Also, live blogging starting around 8:00 pm.

President: Hillary Clinton (D).

The Presidential estimates are based on the current snapshot in the right sidebar, except for the most-probable-single-outcome map, where variance minimization was done to give a more stable snapshot for North Carolina, Clinton +1.0 ± 1.0% (N=8 polls).

Most probable single outcome (shown on map below): Clinton 323 EV, Trump 215 EV. This is also the mode of the NC-adjusted histogram.

Median: Clinton 307 EV, Trump 231 EV. Meta-Margin: 2.2%. One-sigma range: Clinton 281-326 EV. The win probability is 93% using the revised assumption of polling error, +/- 1.1%.

National popular vote: Clinton +4.0 ± 0.6%.

Where possible, variance minimization was used to identify a time window that gave lower variance than the standard time window.

Mode: 51 Democratic/Independent seats, 49 Republican seats the most likely single combination is shown in the table below.

Median: 50 Democratic/Independent seats, 50 Republican seats. (average=50.4 ± 1.1 the 1-sigma range rounds to 49 to 51 seats)

Generic Congressional ballot: Democratic +1%, about the same as 2012.

Cook Political Report-based expectation: 239 R, 196 D, an 8-seat gain for Democrats.

168 Comments so far &darr

Looking back at the Michigan primary polling error (http://election.princeton.edu/2016/03/09/how-surprising-was-the-sanders-win-in-michigan/) what do we make of the bad news on that side of the coin (MI, -19 AZ, -12 NC, -10 NH, -9 PA, -4 IA, -4)? The good news is: OH, +6 FL, +3. Hard to say about caucus states CO, NV. Forgive me if this ground has been covered, would love a link to it.

Deep breaths, deep breaths // Nov 8, 2016 at 11:46 am

Sam, some at Yale is talking smack about you!

Ed Wittens Cat // Nov 8, 2016 at 11:51 am

The King is dead…
Long live the King!
Nate is REALLY not gunna like this.

Ed Wittens Cat // Nov 8, 2016 at 11:57 am

from Wired article–
Because Wang has sailed True North all along, while Silver has been cautiously trying to tack his FiveThirtyEight data sailboat (weighted down with ESPN gold bars) through treacherous, Category-Five-level-hurricane headwinds in what has easily been the craziest presidential campaign in the modern political era.

Hey, everyone – can you please not do a victory dance over this article? I would like to reduce this kind of comparison. You all know what he has done to create this activity in the public eye. He’s a pioneer.

Ed Wittens Cat // Nov 8, 2016 at 12:46 pm

I told you in 2012 that i was done with Silver
when he disappeared the NYT piece he wrote debunking the dem oversampling myth.
That was pure unadulterated pandering.
Scientists and mathematicians have a deep responsibility to the public good….to not whore themselves out for clicks and eyeballs.
That is why transparency of poll aggregation models is so important.

Cat: regardless, it does not increase the net good in the world to stir up (or egg on) a cat fight. The world would be improved by reconciliation here.

Has anyone else noted the fixed periodicity in the MM plot and those of other poll watchers like the Upshot? Peak-to-peak seems to be a little less than 2 months. Strange because the state plots (Upshot for instance) do not readily show this feature. But the national plot is a convolution of the state plots, so this is some emergent behavior.

Does anyone have FFT code? Be curious to see if this is telling us something.

Bob McConnaughey // Nov 8, 2016 at 3:03 pm

Not FFT but i have some SAS code i can dig up that is set up to evaluate seasonality actually it was borrowed from this site by Ed Stanek:
http://www.umass.edu/seasons/pdffiles/sea05d01.pdf
we converted the dates to Julian dates, obtaining the thetas in radians, getting the sin/cos of the radian measures, but it was really this guy’s SAS code.

Pretty sure it’s going show to spike at k = 1/2months. But what does it mean? What happens on that timescale?

BTW, data is always beautiful, no matter what Wired says! )

Yes, that ‘godawful web design’ line in the article made me think: what are they talking about? That’s when I realized for the first time that I am a nerd.

Old-fashioned web design. Which is not necessarily bad.

Maybe this is covered down-thread, but how do you get 98-99% chance of HRC win if the left tail of the histogram is below 270? Do those only amount to 1-2% of total outcomes?

discussed a bit below and other posts. Short answer: the PEC prediction does not use the histogram to make predictions. Just “what’s the odds that MM>0 on election day” where the MM is calculated only using the median of the histogram. Doesn’t change things much this election, but maybe something to think about in the future.

Good interview on MSNBC. All good points were made.

Hi Sam, why did PEC’s final number of 323 EV for HRC drop this morning to 307? Did you put NC into the DJT column?

What do you want to bet that the Trumpster is going to whine and moan about a rigged election because he won just as many states as Clinton did?

Every election the Democrats win I get post-election e-mail from a cousin that shows that the land area of districts won by the Republicans vastly exceeds the land area of districts won by Democrats, as if land, and not people, voted.

Dr. Wang, I appreciate your steady, stable and classy demeanor during this whole election. Thank you for giving me a place to go to that is sane when it comes to politics. You do wonders for the psyche!


Contents

The first known example of an opinion poll was a tallies of voter preferences reported by the Raleigh Star and North Carolina State Gazette and the Wilmington American Watchman and Delaware Advertiser prior to the 1824 presidential election, [1] showing Andrew Jackson leading John Quincy Adams by 335 votes to 169 in the contest for the United States Presidency. Since Jackson won the popular vote in that state and the whole country, such straw votes gradually became more popular, but they remained local, usually citywide phenomena. In 1916, The Literary Digest embarked on a national survey (partly as a circulation-raising exercise) and correctly predicted Woodrow Wilson's election as president. Mailing out millions of postcards and simply counting the returns, The Literary Digest correctly predicted the victories of Warren Harding in 1920, Calvin Coolidge in 1924, Herbert Hoover in 1928, and Franklin Roosevelt in 1932.

Then, in 1936, its survey of 2.3 million voters suggested that Alf Landon would win the presidential election, but Roosevelt was instead re-elected by a landslide. The error was mainly caused by participation bias those who favored Landon were more enthusiastic about participating in the poll. Furthermore, the survey over-sampled more affluent Americans who tended to have Republican sympathies. [2] At the same time, George Gallup conducted a far smaller (but more scientifically based) survey, in which he polled a demographically representative sample. The Gallup organization correctly predicted Roosevelt's landslide victory, as did another groundbreaking pollster, Archibald Crossley. The Literary Digest soon went out of business, while polling started to take off. [3]

Elmo Roper was another American pioneer in political forecasting using scientific polls. [4] He predicted the reelection of President Franklin D. Roosevelt three times, in 1936, 1940, and 1944. Louis Harris had been in the field of public opinion since 1947 when he joined the Elmo Roper firm, then later became partner.

In September 1938 Jean Stoetzel, after having met Gallup, created IFOP, the Institut Français d'Opinion Publique, as the first European survey institute in Paris and started political polls in summer 1939 with the question "Why die for Danzig?", looking for popular support or dissent with this question asked by appeasement politician and future collaborationist Marcel Déat.

Gallup launched a subsidiary in the United Kingdom that, almost alone, correctly predicted Labour's victory in the 1945 general election, unlike virtually all other commentators, who expected a victory for the Conservative Party, led by Winston Churchill. The Allied occupation powers helped to create survey institutes in all of the Western occupation zones of Germany in 1947 and 1948 to better steer denazification. By the 1950s, various types of polling had spread to most democracies.

In long-term perspective, advertising had come under heavy pressure in the early 1930s. The Great Depression forced businesses to drastically cut back on their advertising spending. Layoffs and reductions were common at all agencies. The New Deal furthermore aggressively promoted consumerism, and minimized the value of (or need for) advertising. Historian Jackson Lears argues that "By the late 1930s, though, corporate advertisers had begun a successful counterattack against their critics." They rehabilitated the concept of consumer sovereignty by inventing scientific public opinion polls, and making it the centerpiece of their own market research, as well as the key to understanding politics. George Gallup, the vice president of Young and Rubicam, and numerous other advertising experts, led the way. Moving into the 1940s, the industry played a leading role in the ideological mobilization of the American people for fighting the Nazis and Japanese in World War II. As part of that effort, they redefined the "American Way of Life" in terms of a commitment to free enterprise. "Advertisers," Lears concludes, "played a crucial hegemonic role in creating the consumer culture that dominated post-World War II American society." [5] [6]

Opinion polls for many years were maintained through telecommunications or in person-to-person contact. Methods and techniques vary, though they are widely accepted in most areas. Over the years, technological innovations have also influenced survey methods such as the availability of electronic clipboards [7] and Internet based polling. Verbal, ballot, and processed types can be conducted efficiently, contrasted with other types of surveys, systematics, and complicated matrices beyond previous orthodox procedures. [ citation needed ]

Opinion polling developed into popular applications through popular thought, although response rates for some surveys declined. Also, the following has also led to differentiating results: [4] Some polling organizations, such as Angus Reid Public Opinion, YouGov and Zogby use Internet surveys, where a sample is drawn from a large panel of volunteers, and the results are weighted to reflect the demographics of the population of interest. In contrast, popular web polls draw on whoever wishes to participate, rather than a scientific sample of the population, and are therefore not generally considered professional.

Recently, statistical learning methods have been proposed in order to exploit social media content (such as posts on the micro-blogging platform Twitter) for modelling and predicting voting intention polls. [8] [9]

Polls can be used in the public relations field as well. In the early 1920s, public relation experts described their work as a two-way street. Their job would be to present the misinterpreted interests of large institutions to public. They would also gauge the typically ignored interests of the public through polls.

Benchmark polls Edit

A benchmark poll is generally the first poll taken in a campaign. It is often taken before a candidate announces their bid for office, but sometimes it happens immediately following that announcement after they have had some opportunity to raise funds. This is generally a short and simple survey of likely voters.

A benchmark poll serves a number of purposes for a campaign, whether it is a political campaign or some other type of campaign. First, it gives the candidate a picture of where they stand with the electorate before any campaigning takes place. If the poll is done prior to announcing for office the candidate may use the poll to decide whether or not they should even run for office. Secondly, it shows them where their weaknesses and strengths are in two main areas. The first is the electorate. A benchmark poll shows them what types of voters they are sure to win, those they are sure to lose, and everyone in-between these two extremes. This lets the campaign know which voters are persuadable so they can spend their limited resources in the most effective manner. Second, it can give them an idea of what messages, ideas, or slogans are the strongest with the electorate. [10]

Brushfire polls Edit

Brushfire polls are polls taken during the period between the benchmark poll and tracking polls. The number of brushfire polls taken by a campaign is determined by how competitive the race is and how much money the campaign has to spend. These polls usually focus on likely voters and the length of the survey varies on the number of messages being tested.

Brushfire polls are used for a number of purposes. First, it lets the candidate know if they have made any progress on the ballot, how much progress has been made, and in what demographics they have been making or losing ground. Secondly, it is a way for the campaign to test a variety of messages, both positive and negative, on themselves and their opponent(s). This lets the campaign know what messages work best with certain demographics and what messages should be avoided. Campaigns often use these polls to test possible attack messages that their opponent may use and potential responses to those attacks. The campaign can then spend some time preparing an effective response to any likely attacks. Thirdly, this kind of poll can be used by candidates or political parties to convince primary challengers to drop out of a race and support a stronger candidate.

Tracking polls Edit

A tracking poll or rolling poll is a poll in which responses are obtained in a number of consecutive periods, for instance daily, and then results are calculated using a moving average of the responses that were gathered over a fixed number of the most recent periods, for example the past five days. [11] In this example, the next calculated results will use data for five days counting backwards from the next day, namely the same data as before, but with the data from the next day included, and without the data from the sixth day before that day.

However, these polls are sometimes subject to dramatic fluctuations, and so political campaigns and candidates are cautious in analyzing their results. An example of a tracking poll that generated controversy over its accuracy, is one conducted during the 2000 U.S. presidential election, by the Gallup Organization. The results for one day showed Democratic candidate Al Gore with an eleven-point lead over Republican candidate George W. Bush. Then, a subsequent poll conducted just two days later showed Bush ahead of Gore by seven points. It was soon determined that the volatility of the results was at least in part due to an uneven distribution of Democratic and Republican affiliated voters in the samples. Though the Gallup Organization argued the volatility in the poll was a genuine representation of the electorate, other polling organizations took steps to reduce such wide variations in their results. One such step included manipulating the proportion of Democrats and Republicans in any given sample, but this method is subject controversy. [12]

Over time, a number of theories and mechanisms have been offered to explain erroneous polling results. Some of these reflect errors on the part of the pollsters many of them are statistical in nature. Others blame the respondents for not giving candid answers (e.g., the Bradley effect, the Shy Tory Factor) these can be more controversial.

Margin of error due to sampling Edit

Polls based on samples of populations are subject to sampling error which reflects the effects of chance and uncertainty in the sampling process. Sampling polls rely on the law of large numbers to measure the opinions of the whole population based only on a subset, and for this purpose the absolute size of the sample is important, but the percentage of the whole population is not important (unless it happens to be close to the sample size). The possible difference between the sample and whole population is often expressed as a margin of error - usually defined as the radius of a 95% confidence interval for a particular statistic. One example is the percent of people who prefer product A versus product B. When a single, global margin of error is reported for a survey, it refers to the maximum margin of error for all reported percentages using the full sample from the survey. If the statistic is a percentage, this maximum margin of error can be calculated as the radius of the confidence interval for a reported percentage of 50%. Others suggest that a poll with a random sample of 1,000 people has margin of sampling error of ±3% for the estimated percentage of the whole population.

A 3% margin of error means that if the same procedure is used a large number of times, 95% of the time the true population average will be within the sample estimate plus or minus 3%. The margin of error can be reduced by using a larger sample, however if a pollster wishes to reduce the margin of error to 1% they would need a sample of around 10,000 people. [13] In practice, pollsters need to balance the cost of a large sample against the reduction in sampling error and a sample size of around 500–1,000 is a typical compromise for political polls. (Note that to get complete responses it may be necessary to include thousands of additional participators.) [14] [15]

Another way to reduce the margin of error is to rely on poll averages. This makes the assumption that the procedure is similar enough between many different polls and uses the sample size of each poll to create a polling average. [16] An example of a polling average can be found here: 2008 Presidential Election polling average. Another source of error stems from faulty demographic models by pollsters who weigh their samples by particular variables such as party identification in an election. For example, if you assume that the breakdown of the US population by party identification has not changed since the previous presidential election, you may underestimate a victory or a defeat of a particular party candidate that saw a surge or decline in its party registration relative to the previous presidential election cycle.

A caution is that an estimate of a trend is subject to a larger error than an estimate of a level. This is because if one estimates the change, the difference between two numbers X and Y, then one has to contend with errors in both X and Y. A rough guide is that if the change in measurement falls outside the margin of error it is worth attention.

Nonresponse bias Edit

Since some people do not answer calls from strangers, or refuse to answer the poll, poll samples may not be representative samples from a population due to a non-response bias. Response rates have been declining, and are down to about 10% in recent years. [17] Because of this selection bias, the characteristics of those who agree to be interviewed may be markedly different from those who decline. That is, the actual sample is a biased version of the universe the pollster wants to analyze. In these cases, bias introduces new errors, one way or the other, that are in addition to errors caused by sample size. Error due to bias does not become smaller with larger sample sizes, because taking a larger sample size simply repeats the same mistake on a larger scale. If the people who refuse to answer, or are never reached, have the same characteristics as the people who do answer, then the final results should be unbiased. If the people who do not answer have different opinions then there is bias in the results. In terms of election polls, studies suggest that bias effects are small, but each polling firm has its own techniques for adjusting weights to minimize selection bias. [18] [19]

Response bias Edit

Survey results may be affected by response bias, where the answers given by respondents do not reflect their true beliefs. This may be deliberately engineered by unscrupulous pollsters in order to generate a certain result or please their clients, but more often is a result of the detailed wording or ordering of questions (see below). Respondents may deliberately try to manipulate the outcome of a poll by e.g. advocating a more extreme position than they actually hold in order to boost their side of the argument or give rapid and ill-considered answers in order to hasten the end of their questioning. Respondents may also feel under social pressure not to give an unpopular answer. For example, respondents might be unwilling to admit to unpopular attitudes like racism or sexism, and thus polls might not reflect the true incidence of these attitudes in the population. In American political parlance, this phenomenon is often referred to as the Bradley effect. If the results of surveys are widely publicized this effect may be magnified - a phenomenon commonly referred to as the spiral of silence.

Use of the plurality voting system (select only one candidate) in a poll puts an unintentional bias into the poll, since people who favor more than one candidate cannot indicate this. The fact that they must choose only one candidate biases the poll, causing it to favor the candidate most different from the others while it disfavors candidates who are similar to other candidates. The plurality voting system also biases elections in the same way.

Some people responding may not understand the words being used, but may wish to avoid the embarrassment of admitting this, or the poll mechanism may not allow clarification, so they may make an arbitrary choice. Some percentage of people also answer whimsically or out of annoyance at being polled. This results in perhaps 4% of Americans reporting they have personally been decapitated. [20]

Wording of questions Edit

Among the factors that impact the results of Opinion Polls, are the wording and order of the questions being posed by the surveyor. Questions that intentionally affect a respondents answer are referred to as leading questions. Individuals and/or groups use these types of questions in surveys to elicit responses favorable to their interests. [21]

For instance, the public is more likely to indicate support for a person who is described by the surveyor as one of the "leading candidates." This description is "leading" as it indicates a subtle bias for that candidate, since it implies that the others in the race are not serious contenders. Additionally, leading questions often contain, or lack, certain facts that can sway a respondent's answer. Argumentative Questions can also impact the outcome of a survey. These types of questions, depending on their nature, either positive or negative, influence respondents’ answers to reflect the tone of the question(s) and generate a certain response or reaction, rather than gauge sentiment in an unbiased manner. [22]

In opinion polling, there are also "loaded questions," otherwise known as "trick questions." This type of leading question may concern an uncomfortable or controversial issue, and/or automatically assume the subject of the question is related to the respondent(s) or that they are knowledgeable about it. Likewise, the questions are then worded in a way that limit the possible answers, typically to yes or no. [23]

Another type of question that can produce inaccurate results are "Double-Negative Questions." These are more often the result of human error, rather than intentional manipulation. One such example is a survey done in 1992 by the Roper Organization, concerning the Holocaust. The question read "Does it seem possible or impossible to you that the Nazi extermination of the Jews never happened?" The confusing wording of this question led to inaccurate results which indicated that 22 percent of respondents believed it seemed possible the Holocaust might not have ever happened. When the question was reworded, significantly fewer respondents (only 1 percent) expressed that same sentiment. [24]

Thus comparisons between polls often boil down to the wording of the question. On some issues, question wording can result in quite pronounced differences between surveys. [25] [26] This can also, however, be a result of legitimately conflicted feelings or evolving attitudes, rather than a poorly constructed survey. [27]

A common technique to control for this bias is to rotate the order in which questions are asked. Many pollsters also split-sample. This involves having two different versions of a question, with each version presented to half the respondents.

The most effective controls, used by attitude researchers, are:

  • asking enough questions to allow all aspects of an issue to be covered and to control effects due to the form of the question (such as positive or negative wording), the adequacy of the number being established quantitatively with psychometric measures such as reliability coefficients, and
  • analyzing the results with psychometric techniques which synthesize the answers into a few reliable scores and detect ineffective questions.

These controls are not widely used in the polling industry. [ why? ] . However, as it is important that questions to test the product have a high quality, survey methodologists work on methods to test them. Empirical tests provide insight into the quality of the questionnaire, some may be more complex than others. For instance, testing a questionnaire can be done by:

  • conducting cognitive interviewing. By asking a sample of potential-respondents about their interpretation of the questions and use of the questionnaire, a researcher can
  • carrying out a small pretest of the questionnaire, using a small subset of target respondents. Results can inform a researcher of errors such as missing questions, or logical and procedural errors.
  • estimating the measurement quality of the questions. This can be done for instance using test-retest, [28] quasi-simplex, [29] or mutlitrait-multimethod models. [30]
  • predicting the measurement quality of the question. This can be done using the software Survey Quality Predictor (SQP). [31]

Involuntary facades and false correlations Edit

One of the criticisms of opinion polls is that societal assumptions that opinions between which there is no logical link are "correlated attitudes" can push people with one opinion into a group that forces them to pretend to have a supposedly linked but actually unrelated opinion. That, in turn, may cause people who have the first opinion to claim on polls that they have the second opinion without having it, causing opinion polls to become part of self-fulfilling prophecy problems. It have been suggested that attempts to counteract unethical opinions by condemning supposedly linked opinions may favor the groups that promote the actually unethical opinions by forcing people with supposedly linked opinions into them by ostracism elsewhere in society making such efforts counterproductive, that not being sent between groups that assume ulterior motives from each other and not being allowed to express consistent critical thought anywhere may create psychological stress because humans are sapient, and that discussion spaces free from assumptions of ulterior motives behind specific opinions should be created. In this context, rejection of the assumption that opinion polls show actual links between opinions is considered important. [32] [33]

Coverage bias Edit

Another source of error is the use of samples that are not representative of the population as a consequence of the methodology used, as was the experience of The Literary Digest in 1936. For example, telephone sampling has a built-in error because in many times and places, those with telephones have generally been richer than those without.

In some places many people have only mobile telephones. Because pollsters cannot use automated dialing machines to call mobile phones in the United States (because the phone's owner may be charged for taking a call [34] ), these individuals are typically excluded from polling samples. There is concern that, if the subset of the population without cell phones differs markedly from the rest of the population, these differences can skew the results of the poll. [35]

Polling organizations have developed many weighting techniques to help overcome these deficiencies, with varying degrees of success. Studies of mobile phone users by the Pew Research Center in the US, in 2007, concluded that "cell-only respondents are different from landline respondents in important ways, (but) they were neither numerous enough nor different enough on the questions we examined to produce a significant change in overall general population survey estimates when included with the landline samples and weighted according to US Census parameters on basic demographic characteristics." [36]

This issue was first identified in 2004, [37] but came to prominence only during the 2008 US presidential election. [38] In previous elections, the proportion of the general population using cell phones was small, but as this proportion has increased, there is concern that polling only landlines is no longer representative of the general population. In 2003, only 2.9% of households were wireless (cellphones only), compared to 12.8% in 2006. [39] This results in "coverage error". Many polling organisations select their sample by dialling random telephone numbers however, in 2008, there was a clear tendency for polls which included mobile phones in their samples to show a much larger lead for Obama, than polls that did not. [40] [41]

The potential sources of bias are: [42]

  1. Some households use cellphones only and have no landline. This tends to include minorities and younger voters and occurs more frequently in metropolitan areas. Men are more likely to be cellphone-only compared to women.
  2. Some people may not be contactable by landline from Monday to Friday and may be contactable only by cellphone.
  3. Some people use their landlines only to access the Internet, and answer calls only to their cellphones.

Some polling companies have attempted to get around that problem by including a "cellphone supplement". There are a number of problems with including cellphones in a telephone poll:

  1. It is difficult to get co-operation from cellphone users, because in many parts of the US, users are charged for both outgoing and incoming calls. That means that pollsters have had to offer financial compensation to gain co-operation.
  2. US federal law prohibits the use of automated dialling devices to call cellphones (Telephone Consumer Protection Act of 1991). Numbers therefore have to be dialled by hand, which is more time-consuming and expensive for pollsters.

1992 UK general election Edit

An oft-quoted example of opinion polls succumbing to errors occurred during the 1992 UK general election. Despite the polling organizations using different methodologies, virtually all the polls taken before the vote, and to a lesser extent, exit polls taken on voting day, showed a lead for the opposition Labour party, but the actual vote gave a clear victory to the ruling Conservative party.

In their deliberations after this embarrassment the pollsters advanced several ideas to account for their errors, including:

Late swing Voters who changed their minds shortly before voting tended to favour the Conservatives, so the error was not as great as it first appeared. Nonresponse bias Conservative voters were less likely to participate in surveys than in the past and were thus under-represented. The Shy Tory Factor The Conservatives had suffered a sustained period of unpopularity as a result of economic difficulties and a series of minor scandals, leading to a spiral of silence in which some Conservative supporters were reluctant to disclose their sincere intentions to pollsters.

The relative importance of these factors was, and remains, a matter of controversy, but since then the polling organizations have adjusted their methodologies and have achieved more accurate results in subsequent election campaigns. [ citation needed ]

A comprehensive discussion of these biases and how they should be understood and mitigated is included in several sources including Dillman and Salant (1994). [43]

A widely publicized failure of opinion polling to date in the United States was the prediction that Thomas Dewey would defeat Harry S. Truman in the 1948 US presidential election. Major polling organizations, including Gallup and Roper, indicated a landslide victory for Dewey. There were also substantial polling errors in the presidential elections of 1952, 1980, 1996, 2000, and 2016. [44]

In the United Kingdom, most polls failed to predict the Conservative election victories of 1970 and 1992, and Labour's victory in February 1974. In the 2015 election virtually every poll predicted a hung parliament with Labour and the Conservatives neck and neck when the actual result was a clear Conservative majority. On the other hand, in 2017, the opposite appears to have occurred. Most polls predicted an increased Conservative majority, even though in reality the election resulted in a hung parliament with a Conservative plurality. However, some polls correctly predicted this outcome.

In New Zealand, the polls leading up to the 1993 general election predicted a comfortable win to the governing National Party. However, the preliminary results on election night showed a hung parliament with National one seat short of a majority, leading to prime minister Jim Bolger exclaiming "bugger the pollsters" on national television. [45] [46] The official count saw National pick up Waitaki to hold a one-seat majority and reform the government.

Social media today is a popular medium for the candidates to campaign and for gauging the public reaction to the campaigns. Social media can also be used as an indicator of the voter opinion regarding the poll. Some research studies have shown that predictions made using social media signals can match traditional opinion polls. [8] [9]

Regarding the 2016 U.S. presidential election, a major concern has been that of the effect of false stories spread throughout social media. Evidence shows that social media plays a huge role in the supplying of news: 62 percent of US adults get news on social media. [47] This fact makes the issue of fake news on social media more pertinent. Other evidence shows that the most popular fake news stories were more widely shared on Facebook than the most popular mainstream news stories many people who see fake news stories report that they believe them and the most discussed fake news stories tended to favor Donald Trump over Hillary Clinton. As a result of these facts, some have concluded that if not for these stories, Donald Trump may not have won the election over Hillary Clinton. [48]

Effect on voters Edit

By providing information about voting intentions, opinion polls can sometimes influence the behavior of electors, and in his book The Broken Compass, Peter Hitchens asserts that opinion polls are actually a device for influencing public opinion. [49] The various theories about how this happens can be split into two groups: bandwagon/underdog effects, and strategic ("tactical") voting.

A bandwagon effect occurs when the poll prompts voters to back the candidate shown to be winning in the poll. The idea that voters are susceptible to such effects is old, stemming at least from 1884 William Safire reported that the term was first used in a political cartoon in the magazine Puck in that year. [50] It has also remained persistent in spite of a lack of empirical corroboration until the late 20th century. George Gallup spent much effort in vain trying to discredit this theory in his time by presenting empirical research. A recent meta-study of scientific research on this topic indicates that from the 1980s onward the Bandwagon effect is found more often by researchers. [51]

The opposite of the bandwagon effect is the underdog effect. It is often mentioned in the media. This occurs when people vote, out of sympathy, for the party perceived to be "losing" the elections. There is less empirical evidence for the existence of this effect than there is for the existence of the bandwagon effect. [51]

The second category of theories on how polls directly affect voting is called strategic or tactical voting. This theory is based on the idea that voters view the act of voting as a means of selecting a government. Thus they will sometimes not choose the candidate they prefer on ground of ideology or sympathy, but another, less-preferred, candidate from strategic considerations. An example can be found in the 1997 United Kingdom general election. As he was then a Cabinet Minister, Michael Portillo's constituency of Enfield Southgate was believed to be a safe seat but opinion polls showed the Labour candidate Stephen Twigg steadily gaining support, which may have prompted undecided voters or supporters of other parties to support Twigg in order to remove Portillo. Another example is the boomerang effect where the likely supporters of the candidate shown to be winning feel that chances are slim and that their vote is not required, thus allowing another candidate to win.

In addition, Mark Pickup, in Cameron Anderson and Laura Stephenson's Voting Behaviour in Canada, outlines three additional "behavioural" responses that voters may exhibit when faced with polling data. The first is known as a "cue taking" effect which holds that poll data is used as a "proxy" for information about the candidates or parties. Cue taking is "based on the psychological phenomenon of using heuristics to simplify a complex decision" (243). [52]

The second, first described by Petty and Cacioppo (1996), is known as "cognitive response" theory. This theory asserts that a voter's response to a poll may not line with their initial conception of the electoral reality. In response, the voter is likely to generate a "mental list" in which they create reasons for a party's loss or gain in the polls. This can reinforce or change their opinion of the candidate and thus affect voting behaviour. Third, the final possibility is a "behavioural response" which is similar to a cognitive response. The only salient difference is that a voter will go and seek new information to form their "mental list", thus becoming more informed of the election. This may then affect voting behaviour.

These effects indicate how opinion polls can directly affect political choices of the electorate. But directly or indirectly, other effects can be surveyed and analyzed on all political parties. The form of media framing and party ideology shifts must also be taken under consideration. Opinion polling in some instances is a measure of cognitive bias, which is variably considered and handled appropriately in its various applications.

Effect on politicians Edit

Starting in the 1980s, tracking polls and related technologies began having a notable impact on U.S. political leaders. [53] According to Douglas Bailey, a Republican who had helped run Gerald Ford's 1976 presidential campaign, "It's no longer necessary for a political candidate to guess what an audience thinks. He can [find out] with a nightly tracking poll. So it's no longer likely that political leaders are going to lead. Instead, they're going to follow." [53]

Some jurisdictions over the world restrict the publication of the results of opinion polls, especially during the period around an election, in order to prevent the possibly erroneous results from affecting voters' decisions. For instance, in Canada, it is prohibited to publish the results of opinion surveys that would identify specific political parties or candidates in the final three days before a poll closes. [54]

However, most Western democratic nations do not support the entire prohibition of the publication of pre-election opinion polls most of them have no regulation and some only prohibit it in the final days or hours until the relevant poll closes. [55] A survey by Canada's Royal Commission on Electoral Reform reported that the prohibition period of publication of the survey results largely differed in different countries. Out of the 20 countries examined, 3 prohibit the publication during the entire period of campaigns, while others prohibit it for a shorter term such as the polling period or the final 48 hours before a poll closes. [54] In India, the Election Commission has prohibited it in the 48 hours before the start of polling.


Election 2016: Exit Polls

Data for 2016 were collected by Edison Research for the National Election Pool, a consortium of ABC News, The Associated Press, CBSNews, CNN, Fox News and NBC News. The voter survey is based on questionnaires completed by 24,537 voters leaving 350 voting places throughout the United States on Election Day including 4,398 telephone interviews with early and absentee voters.

In 2012, 2008 and 2004, the exit poll was conducted by Edison/Mitofsky in 1996 and 2000 by Voter News Services in 1992 by Voter Research and Surveys and in earlier years by The New York Times and CBS News.

Direct comparisons from year to year should factor in differences in how questions were asked. Race and sex were determined by interviewers in surveys before 1984. Independent or third-party candidates are not shown.

Population scaling is representative of the number of voters in each category.

* Change is shown in percentage points. When comparable data are available, this measure combines the change in Republican support and the change in Democratic support from the previous election.


Q&A: Political polls and the 2016 election

Voters cast their ballots at a fire station in Alhambra, California, on Nov. 8, 2016. (Ringo Chiu/AFP/Getty Images)

The outcome of the 2016 presidential election surprised a lot of people – not least the many political pollsters and analysts covering it. Today the American Association for Public Opinion Research (AAPOR), the nation’s leading organization of survey researchers, released a long-awaited report that examines polling during last year’s long primary and general election campaigns.

Courtney Kennedy, Pew Research Center director of survey research

Courtney Kennedy, Pew Research Center’s director of survey research, chaired the AAPOR task force that produced the report. We sat down with Kennedy recently to discuss its findings and recommendations. The conversation has been condensed and edited for clarity and conciseness.

Ever since Donald Trump’s victory over Hillary Clinton last year, there’s been plenty of criticism of the performance and trustworthiness of polls. Was that the impetus for this report?

Actually, this committee was organized back in May 2016, months before any of us had the slightest inkling that last year would be a particularly unusual year for polling. The original intent was pretty straightforward: to evaluate the performance of polls, both in the primary season and the general election to compare how they did relative to past years and, to the extent the data would support it, assess whether certain types of polls – online versus telephone, live versus automated – did better or worse than others.

But as of midnight or so on Nov. 8 , it was clear that what the committee needed to do had changed. We couldn’t just do this very technical, “what was the average deviation” type of report. We needed to, in addition, consider another question: “Why did the polls seem to systematically underestimate support for Donald Trump?” There already were a number of hypotheses floating around – such as the so-called “shy Trump” effect (Trump supporters being less willing than others to disclose their support to an interviewer), differential nonresponse (Trump supporters being less likely than others to participate in surveys), things of that nature – and we felt obligated to take on that additional piece.

The report notes that, while the national polls generally came pretty close to the actual nationwide popular vote (which Clinton won by 2.1 percentage points over Trump), the performance of polls at the state level – where presidential elections actually are decided – was a lot spottier. What reasons did you find for that?

We found evidence for multiple potential causes. One factor that I think affected everybody who was polling in the battleground states, is the legitimate late change in voter preference in the last week before Election Day. The data on this has its limitations, but the best source is the National Election Pool’s exit poll, which has a question about when voters made up their minds about who to vote for in the presidential race. That showed several roughly 20-point swings in favor of Trump among voters making their mind up in the final week. You didn’t really see that nationally, but in Pennsylvania, Michigan, Wisconsin and even Florida, you saw what looks like dramatic movement.

That’s sort of a good news/bad news finding for pollsters. The good news is, if you interviewed people at a certain point in time and they changed their mind several days later, the poll wouldn’t have detected that. That’s not a flaw in the poll, other than perhaps with the field period in which the pollster decided to do the data collection. But there’s fundamentally nothing that was necessarily off if what was generating most of the error was just honest-to-goodness changes of opinion.

What else did you find at the state level?

Another interesting finding had to do with poll respondents’ level of education. A number of studies have shown that in general, people with higher levels of formal education are more likely to take surveys – it’s a very robust finding. Places like Pew Research Center and others have known that for years, and we address that with our statistical weighting – that is, we ask people what their education level is and align our survey data so that it matches the U.S. population on education. And I think a lot of us assumed that was common practice in the industry – that roughly speaking, everybody was doing it. And that’s not what we found. At the state level, more often than not, the polls were not being adjusted for education.

Now in some elections, such as in 2012, that wouldn’t matter, because the very low educated and the very highly educated voted roughly the same way. But 2016 was drastically different – you had a quite strong linear relationship between education and presidential vote. And that meant that if you had too many college graduates in your poll, which virtually all of us do, and you didn’t weight appropriately, you were almost certainly going to overestimate support for Clinton.

Were there any possible factors for which you didn’t find evidence?

Yes. Take the hypothesis that there’s a segment of the Trump support base that does not participate in polls. If that’s true, that’s a huge problem for organizations like ours, and we need to study that and understand it if we’re ever going to fix it. But we looked for evidence of that, and we didn’t find it.

If it’s true that we’re missing a segment of the Trump support base, we would expect to find – without doing any fancy weighting, just looking at the raw data – that people in more rural, deep-red parts of the country would be underrepresented. And we didn’t find that if anything, they were slightly overrepresented. We did a number of things with a critical eye looking for those types of problems, and did not find them. And so that gave me real reassurance that fundamentally, it’s not that the process of doing polls was broken last year.

What, if anything, can the profession do to address the issues the committee found with state and local polls, especially given that so many of the newspapers and TV stations that historically sponsored them can no longer afford to do so at the same level?

There’s lots of evidence to show that the resources that news organizations have for polling seem to be declining over time, and that does two things, I think: There are fewer news organizations doing polling, and those that do – particularly local news organizations – are using very low-cost methodology. What the report shows is that there are important design differences among the national polls, which tend to be pretty well resourced, versus the state polls, which tend to be done a lot more quickly using more automated methods with fewer resources. The state polls are half as likely as national polls to have live interviewers, and they’re about half as likely to have adjusted for education in their weighting, which we know to be important. So there are these structural things that seem to have compounded the gap in performance between those state polls and the national polls. We know that on average they’re doing it differently, and in ways that produced greater error in this election. It’s also true that over time, you just see that there’s more error in the state-level polls.

So I could imagine that a professional association like AAPOR might investigate whether this could be addressed, either by professional education or even by trying to organize funding for more rigorous state-level surveys, conducted very close to Election Day, in order to catch people who change their minds late. This would obviously be done by researchers who use very sophisticated, state-of-the-art weighting protocols, so you don’t have things like this education mishap. It’s unclear if that would completely fix the problem, but at least then you’d have an infusion of higher-quality polls into that set of polls that, on average, are done fairly cheaply.

Another piece of the 2016 election cycle was the prominence, even beyond the individual polls themselves, of the data-analysis operations and news sites that aggregated polls and used them not just to predict the final outcome but to give very precise-sounding probabilities that Clinton or Trump would win. How appropriate or useful is it to use polls as predictive tools?

Polls aren’t designed to produce precision on the order of “so-and-so has X.X% chance of winning.” There was actually quite a bit of diversity of opinion on the committee on that issue: Some leaned toward being more aggressive in emphasizing that distinction between the predictors and the pollsters others less so.

But there is a distinction. Polling and prognosticating really are two different enterprises. A well-done public opinion survey can tell you what opinion was during the time that interviewing was done, but that really doesn’t speak in a precise way to future behavior. It’s been said before, but it bears repeating: A poll is a snapshot in time, not a way of predicting what will happen. As we say in the report, greater caution and humility would seem to be in order for anyone making claims about the likely outcome of an election based in part or in whole on polling data.

Where polls can be useful is in helping answer important questions about what is motivating voters, why people are voting or not voting, how they feel about the policies being debated, how they feel about the candidates themselves. All of those questions are more than deserving of serious answers, and that’s what polls are really best designed to do.

So, can polls still be trusted despite what happened last year?

I believe they can. First off, it’s worth pointing out that the performance of election polls isn’t a good indicator of the quality of surveys in general. Election polls differ from other types of surveys in some key ways: Not only do they have to field a representative sample of the public, but they also have to correctly model who among that sample will actually vote. That’s a very difficult task that non-election polls simply don’t have.

It’s important to dispel the notion that polling writ large is broken – our investigation found that not to be the case. At the same time, we shouldn’t whitewash what happened. There were errors, and the polling industry has taken a reputational hit. But the polling community and poll consumers should take some comfort in the fact that we’ve figured out quite a bit about what went wrong and why, and we all can learn from those errors. Some things were outside of pollsters’ control, namely the late shifts in voter preference other things were in their control and are fixable. The education imbalance, for example, is very fixable.

We as researchers should be talking about the whole story of polling in 2016 – the differences between the national polls and state polls, the fact that we’ve identified major factors that led to the errors – in an open, non-defensive way, to dispel the “polling is broken” narrative. That narrative does a disservice to our democracy. Because polling, imperfect as it is, remains the best available tool for measuring the attitudes of all Americans. And when it’s done well, it can still produce very useful data. No matter which party is in power, it’s important to have independent, objective researchers measuring how the public feels about major issues of the day.


How do we know Trump is in trouble? Look at 2016 polls from the week before the election

Democratic presidential candidate and former Vice President Joe Biden, left, and President Donald Trump participate in the final presidential debate at Belmont University in Nashville, Tenn., on Oct. 22, 2020.

JIm Bourg / AFP / Getty Images

This article is the sixth of a seven-part series. Every Tuesday between now and Election Day, SFGATE will report on how the current 2020 presidential election polling averages compare to the polling averages at the same time in the 2016 presidential election. After the election, SFGATE will examine whether the polls in 2020 were more or less accurate than the 2016 polls.

With one week to go before the 2016 presidential election, Hillary Clinton's strong national lead had all but evaporated following the announcement from former FBI Director James Comey that the bureau was reopening its probe into Clinton's emails &mdash a huge twist in a race with just 11 days before Election Day.

Clinton's national lead had dwindled to 3.5% on FiveThirtyEight and to 1.7% on RealClearPolitics &mdash figures very close to her final national popular vote margin of +2.1%. I'll say it for the 1,000th time since we started this series: National polls were pretty accurate in 2016. They weren't the problem.

In 2020, Joe Biden's national lead has shrunk over the past week, but it's not only contracting at a pace slower than Clinton's evaporated Donald Trump has to make up much more ground vs. Biden than he did vs. Clinton. After briefly leading nationally by double-digits, Biden's lead is down to 9.5% on FiveThirtyEight and 7.8% on RealClearPolitics, which is still a much, much stronger position than Hillary Clinton was in with one week to go.

For this exercise, let's assume that like in 2016, the final national popular vote comes out somewhere between FiveThirtyEight and RealClearPolitics' averages from seven days before Election Day. That means a Biden popular vote victory of about 8% or so. While Trump can most definitely win the Electoral College without winning the popular vote as he did in 2016, it's highly, highly unlikely &mdash if not impossible &mdash for him to win the Electoral College while losing the popular vote by 8%.

When Trump lost the popular vote by 2.1% in 2016, he eked out an Electoral College victory by a margin of fewer than 80,000 votes in Michigan, Wisconsin and Pennsylvania. While the president certainly has an Electoral College advantage that allows him to lose the popular vote by a substantial margin, it's certainly not strong enough to withstand an eight-point loss nationally.

Even when looking at state polls, it's clear Biden has a stronger lead than Clinton had in Michigan, Arizona, Florida and North Carolina. Biden's lead is similar to Clinton's in Wisconsin and Pennsylvania, but that can be at least partially attributed to the fact that neither state was heavily polled in the final weeks of the 2016 race since the two were thought to be somewhat safe states for Democrats.

Of course, the race isn't over yet, and if Biden's national lead narrows further over the final week &mdash something that can't be ruled out following President Trump turning in a much stronger debate performance last week than he did in the first debate &mdash it could certainly be competitive again, assuming a national polling error.

But as of today, it's looking like the president is going to have to hope the polls are more wrong than they were in 2016 &mdash which is not a great spot to be in given the fact that pollsters have made significant adjustments in the past four years and people often forget that 2012 polls were actually too pro-Mitt Romney.

National polls

FiveThirtyEight average one week before Election Day in 2016: Hillary Clinton 45.0%, Donald Trump 41.5% (Clinton +3.5%, was Clinton +6.1% previous week)

FiveThirtyEight average one week before Election Day in 2020: Joe Biden 52.3%, Donald Trump 42.8% (Biden +9.5%, was Biden +10.7% previous week)

RealClearPolitics average one week before Election Day in 2016: Hillary Clinton 47.0%, Donald Trump 45.3% (Clinton +1.7%, was Clinton +5.5% previous week)

RealClearPolitics average one week before Election Day in 2020: Joe Biden 50.8%, Donald Trump 43.0% (Biden +7.8%, was Biden +8.9% previous week)

Actual national popular vote in 2016: Hillary Clinton 48.2%, Donald Trump 46.1% (Clinton +2.1%)

Pennsylvania

FiveThirtyEight projected vote share one week before Election Day in 2016: Hillary Clinton 49.4%, Donald Trump 44.9% (Clinton +4.5%, was Clinton +6.5% previous week)

FiveThirtyEight average one week before Election Day in 2020: Joe Biden 50.2%, Donald Trump 45.1% (Biden +5.1%, was Biden +6.7% previous week)

RealClearPolitics average one week before Election Day in 2016: Hillary Clinton 47.9%, Donald Trump 42.8% (Clinton +5.1%, was Clinton +4.3% previous week)

RealClearPolitics average one week before Election Day in 2020: Joe Biden 49.8%, Donald Trump 45.0% (Biden +4.8%, was Biden +4.4% previous week)

Actual Pennsylvania results in 2016: Donald Trump 48.2%, Hillary Clinton 47.5% (Trump +0.7%)

FiveThirtyEight projected vote share one week before Election Day in 2016: Hillary Clinton 49.3%, Donald Trump 44.9% (Clinton +4.4%, was Clinton +7.1% previous week)

FiveThirtyEight average one week before Election Day in 2020: Joe Biden 51.4%, Donald Trump 44.3% (Biden +7.1%, was Biden +7.3% previous week)

RealClearPolitics average one week before Election Day in 2016: Hillary Clinton 46.7%, Donald Trump 41.3% (Clinton +5.4%, was Clinton +6.5% previous week)

RealClearPolitics average one week before Election Day in 2020: Joe Biden 49.8%, Donald Trump 44.3% (Biden +5.5%, was Biden +6.0% previous week)

Actual Wisconsin results in 2016: Donald Trump 47.2%, Hillary Clinton 46.5% (Trump +0.7%)

FiveThirtyEight projected vote share one week before Election Day in 2016: Hillary Clinton 48.8%, Donald Trump 44.1% (Clinton +4.7%, was Clinton +7.8% previous week)

FiveThirtyEight average one week before Election Day in 2020: Joe Biden 50.9%, Donald Trump 42.5% (Biden +8.4%, was Biden +8.0% previous week)

RealClearPolitics average one week before Election Day in 2016: Hillary Clinton 46.7%, Donald Trump 40.3% (Clinton +6.4%, was Clinton +9.5% previous week)

RealClearPolitics average one week before Election Day in 2020: Joe Biden 50.6%, Donald Trump 41.6% (Biden +9.0%, was Biden +6.8% previous week)


The Science Of Error: How Polling Botched The 2016 Election

On the eve of the 2016 election, Nate Silver's 538 site gave Clinton a 71% chance of winning the presidency. Other sites that used the most advanced aggregating and analytical modeling techniques available had her chances even higher: the New York Times had her odds of winning at 84%, the Princeton Election Consortium had her at 95-99% and ABC News had called that Clinton was a lock for 274 electoral votes -- enough to win -- immediately before voting actually took place. But in a stunning turn-of-events, Trump vastly outperformed what everyone was anticipating from state and national polls, winning nearly all the tossup states plus a number of states predicted to favor Clinton, and he is the new president-elect. Here's the science of how that happened.

The final pre-election predictions from Larry Sabato / University of Virginia Center for Politics. . [+] Image credit: screenshot from 270towin at http://www.270towin.com/maps/crystal-ball-electoral-college-ratings.

We like to think that, with enough data, we can treat any problem scientifically. This may, in principle, be true of voting predictions, and 2012 seems to serve as a great example: where Nate Silver's 538 correctly predicted the results of each individual state: all 50. This time, there were many different high-quality and large-data polls out there, at least as many as there were in 2012. And, most importantly, the science behind it is simple. If you want to know how a sample of, say, a million people are going to vote, you don't need to ask all one million of them to predict the outcome. All you need to do is poll enough people so that you can confidently state the result. So you might decide to poll 100, 500, 2,000 or even 10,000 people, and find that 52% support Clinton in any of those four polls. What they tell you is vastly different, however:

  • 100 people: 52% ± 10%, with 95% (2-sigma) confidence.
  • 500 people: 52% ± 4.5% with 95% confidence.
  • 2,000 people: 52% ± 2.2% with 95% confidence.
  • 10,000 people: 52% ± 1.0% with 95% confidence.

These types of errors are known in science circles as statistical errors. Poll more people and your errors go down, and the greater the odds that the sample you polled will accurately reflect what the electorate will actually do.

A visualization of how your statistical uncertainty drops as your sample size increases. Image . [+] credit: Fadethree at English Wikipedia.

If you have a truly, perfectly random sample of future voters, this is the only type of error that matters. But if you don't, there's another type of error that polling will never catch, and it's a much more insidious type of error: systematic errors. A systematic error is an uncertainty or inaccuracy that doesn't improve or go away as you take more data, but a flaw inherent in the way you collect your data.

  • Maybe the people that you polled aren't reflective of the larger voting population. If you ask a sample of people from Staten Island how they'll vote, that's different from how people in Manhattan -- or Syracuse -- are going to vote.
  • Maybe the people that you polled aren't going to turn out to vote in the proportions you expect. If you poll a sample with 40% white people, 20% black people, 30% Hispanic/Latino and 10% Asian-Americans, but your actual voter turnout is 50% white, your poll results will be inherently inaccurate. [This source-of-error applies to any demographic, like age, income or environment (e.g., urban/suburban/rural.)]
  • Or maybe the polling method is inherently unreliable. If 95% of the people who say they'll vote for Clinton actually do, but 4% vote third-party and 1% vote for Trump, while 100% of those who say they'll vote for Trump actually do it, that translates into a pro-Trump swing of +3%.

Reading the "200" mL line on the left might seem reasonable, but would be an erroneous measurement. . [+] Systematic errors like this do not improve or go away with more data. Image credit: MJCdetroit at the English language Wikipedia under c.c.a.-s.a.-3.0.

None of this is to say that there's anything wrong with the polls that were conducted, or with the idea of polling in general. If you want to know what people are thinking, it's still true that the best way to find out is to ask them. But doing that doesn't guarantee that the responses you get aren't biased or flawed. This is true even of exit polling, which doesn't necessarily reflect how the electorate voted. It's how a reasonable person like Arthur Henning could have written, in 1948,

Dewey and Warren won a sweeping victory in the presidential election yesterday. The early returns showed the Republican ticket leading Truman and Barkley pretty consistently in the western and southern states [. ] complete returns would disclose that Dewey won the presidency by an overwhelming majority of the electoral vote.

and we all learned how that turned out.

Truman holding up a copy of the infamous Chicago Daily Tribune after the 1948 election. Image . [+] credit: flickr user A Meyers 91 of the Frank Cancellare original, via https://www.flickr.com/photos/[email protected]/12894913705 under cc-by-2.0.

I wouldn't go quite as far as Alex Berezow of the American Council on Science and Health does, saying election forecasts and odds of winning are complete nonsense, although he makes some good points. But I will say that it is nonsense to pretend that these systematic errors aren't real. Indeed, this election has demonstrated, quite emphatically, that none of the polling models out there have adequately controlled for them. Unless you understand and quantify your systematics errors -- and you can't do that if you don't understand how your polling might be biased -- election forecasts will suffer from the GIGO problem: garbage in, garbage out.

And despite what the polls indicated, Donald Trump won the 2016 election and will be the next . [+] President of the United States. Image credit: Andrew Harrer/Bloomberg.

It's likely that 2012's successes were a fluke, where either the systematic errors cancelled one another out or the projection models just happened to be right on the nose. 2016 didn't shake out that way at all, indicating there's a long way to go before we have a reliable, robust way to predict election outcomes based on polling. Perhaps it will represent a learning opportunity, and a chance for polls and how they're interpreted to improve. But if analysts change nothing, or learn the wrong lessons from their inaccuracies, we're unlikely to see projections ever achieve 2012's successes again.


Theories about Why Polls Underestimated Support for Trump

A number of theories were put forward as to why many polls missed in 2016. 1

NONRESPONSE BIAS AND DEFICIENT WEIGHTING

Most preelection polls have single-digit response rates or feature an opt-in sample for which a response rate cannot be computed ( Callegaro and DiSogra 2008 AAPOR 2016). While the link between low response rates and bias is not particularly strong (e.g., Merkle and Edelman 2002 Groves and Peytcheva 2008 Pew Research Center 2012, 2017a), such low rates do carry an increased risk of bias (e.g., Burden 2000). Of particular note, adults with weaker partisan strength (e.g., Keeter et al. 2006), lower educational levels ( Battaglia, Frankel, and Link 2008 Chang and Krosnick 2009 Link et al. 2008 Pew Research Center 2012, 2017a), and anti-government views ( U.S. Census Bureau 2015) are less likely to take part in surveys. Given the anti-elite themes of the Trump campaign, Trump voters may have been less likely than other voters to accept survey requests. If survey response was correlated with presidential vote and some factor not accounted for in the weighting, then a deficient weighting protocol could be one explanation for the polling errors.

LATE DECIDING

The notion that preelection polls fielded closer to Election Day tend to be more predictive of the election outcome than equally rigorous polls conducted farther out has been well documented for some time (e.g., Crespi 1988 Traugott 2001 Erikson and Wlezien 2012). The effect of late changes in voters’ decisions can be particularly large in elections with major campaign-related events very close to Election Day ( AAPOR 2009). Both Trump and Clinton had historically poor favorability ratings ( Collins 2016 Yourish 2016). Unhappy with their options, some voters may have waited until the final week or so before deciding. Moreover, late deciders, being less anchored politically, tend to be more influenced by campaign events than voters deciding earlier ( Fournier et al. 2004).

MISSPECIFIED LIKELY VOTER MODELS

Constructing an accurate likely voter model is a tall order for even the most seasoned pollsters ( Erikson, Panagopoulos, and Wlezien 2004). When turnout patterns diverge from recent elections, historical data can be unhelpful or even misleading. Voter turnout in 2016 differed from that in 2012 in ways that advantaged Trump and disadvantaged Clinton. Nationally, turnout among African Americans, the group most supportive of Clinton, dropped seven percentage points while turnout among Hispanics and non-Hispanic whites changed little, according to the Current Population Survey (CPS) Voting and Registration Supplement ( File 2017). Furthermore, analysis by Fraga and colleagues (2017) indicates that the decline in African American turnout was sharpest in states such as Wisconsin and Michigan, which determined the outcome of the election. If pollsters designed their likely voter models around the assumption that 2016 turnout patterns would be similar to 2012, this could have led to underestimation of support for Republicans, including Trump. Such model misspecification could have been exacerbated by skews in the 2012 national exit poll (a popular source for turnout data) overstating turnout among young and non-white voters ( McDonald 2007 Cohn 2016).

THE SHY TRUMP HYPOTHESIS (REPORTING ERROR)

Controversy surrounding Trump’s candidacy raised the possibility that some Trump voters may not have been willing to disclose their support for him in surveys. If a sizable share of Trump voters were reluctant to disclose their support for him, that could explain the systematic underestimation of Trump support in polls (e.g., Enns, Lagodny, and Schuldt 2017). Concern about the possibility of systematic misreporting of vote intention for or against a controversial candidate dates back decades. Studies examining this issue have tended to focus on elections in which either candidate race ( Citrin, Green, and Sears 1990 Finkel, Guterbock, and Borg 1991 Traugott and Price 1992 Hopkins 2009) or gender ( Hopkins 2009 Stout and Kline 2011) was a potential factor in polling error. In the 2016 presidential election, both race and gender were highly salient. Clinton was the first female major-party presidential nominee, and although both candidates were white, Trump’s record on racially charged issues (e.g., housing discrimination, the Central Park Five, birtherism) and open support from white supremacists put race in the forefront of the campaign. However, a recent study suggests that the risk to polls from respondents intentionally misreporting vote choice has diminished considerably or disappeared entirely ( Hopkins 2009).


Hillary Clinton Favorable Rating at New Low

WASHINGTON, D.C. -- Hillary Clinton's image has declined since June and is now the worst Gallup has measured for her to date. Her favorable rating has fallen five percentage points since June to a new low of 36%, while her unfavorable rating has hit a new high of 61%.

Clinton's prior low favorable rating was 38% in late August/early September 2016 during the presidential campaign. She also registered a 38% favorable rating (with a 40% unfavorable rating) in April 1992, when she was much less well-known.

The current results are based on a Dec. 4-11 Gallup poll. Clinton's favorable rating has varied significantly in the 25 years Gallup has measured opinions about her. Her personal best was a 67% favorable rating taken in a December 1998 poll just after the House of Representatives voted to impeach her husband, then President Bill Clinton. She also had favorable ratings in the mid-60s during her time as secretary of state between 2009 and 2013.

At times when she assumed a more overtly political role -- during attempts to reform healthcare in 1994, in her years as a U.S. senator, and during her 2007-2008 and 2015-2016 campaigns for president, her ratings suffered. Her favorable ratings were near 50% when she announced her second bid for the presidency in the spring of 2015, but fell in the summer of 2015 amid controversy over her use of a private email server while she was secretary of state. Throughout 2016, her favorable ratings were generally around 40%, among the worst ever measured for presidential candidates but more positive than Donald Trump's ratings.

Since losing to Trump, Clinton's favorable ratings have not improved, in contrast to what has happened for other recent losing presidential candidates. In fact, her image has gotten worse in recent months as Democratic leaders, political observers and Clinton herself have attempted to explain how she lost an election that she was expected to win. Meanwhile, controversy continues to swirl around Clinton given continuing questions about the fairness of the 2016 Democratic nomination process and her dealings with Russia while secretary of state. There has also been renewed discussion of Bill and Hillary Clinton's handling of past sexual harassment charges made against Bill Clinton in light of heightened public concern about workplace behavior.

Democrats' Image of Hillary Clinton Stable in Past Six Months

In the past six months, Hillary Clinton's image has declined among Republicans and independents but not among Democrats.

June 2017 December 2017 Change
% % pct. pts.
National adults 41 36 -5
Democrats 79 78 -1
Independents 33 27 -6
Republicans 11 5 -6
Gallup

From a longer-term perspective, Hillary Clinton's favorability among Democrats has not held at the level seen during the 2016 election. She had 87% favorable ratings among Democrats both at the beginning (based on a May 2015 poll) and end (based on a November 2016 poll) of her 2016 campaign for president.

The campaign and its aftermath took the greatest toll on independents' views of Clinton. She began with a 51% favorable rating among this group, which fell to 33% in November 2016 and now sits at 27%.

Bill Clinton Image Worst Since 2001

Bill Clinton's image has also slipped over the past year, with his current 45% favorable rating down five points since Gallup last measured Americans' opinions of him in November 2016. Given his 52% unfavorable rating, more U.S. adults now have a negative than a positive opinion of the former president.

His current rating is his lowest since March 2001, when it hit 39% after his rocky exit from the White House that included a series of controversial pardons as well as the Clintons taking, but later returning, gifts intended for the White House. At that time, 59% of Americans had an unfavorable view of Bill Clinton, his highest in Gallup's trend. He did have favorable ratings lower than 39%, but those were measured early in his 1992 presidential campaign when a substantial proportion of Americans were not familiar enough with Clinton to offer an opinion of him.

Bill Clinton's image recovered in the years after he left the White House, as is typical for most ex-presidents. In August 2012, 69% of Americans had a favorable opinion of him, the highest Gallup has measured for him. His ratings began to fall after that, particularly after he began actively campaigning to support his wife's presidential campaign.

In contrast to what has occurred for Hillary Clinton, Bill Clinton's favorable rating is lower among his fellow Democrats than it was in Gallup's prior measurement. Currently, 76% of Democrats have a positive opinion of him, down from 81% in 2016. Independents' positivity has decreased even more -- seven points -- while Republicans' views are steady.

November 2016 December 2017 Change
% % pct. pts.
National adults 50 45 -5
Democrats 81 76 -5
Independents 48 41 -7
Republicans 17 16 -1
Gallup

Implications

Many political experts, and likely the Clintons themselves, thought Bill and Hillary Clinton would be residing in the White House in 2017. But Hillary Clinton's surprise defeat in the 2016 election ended their careers as elected officials. The year away from politics has not caused Americans to view either in a more positive light in fact, the opposite has occurred with Bill Clinton's ratings the worst in 16 years and Hillary Clinton's the worst Gallup has measured to date.

Rather than looking favorably upon their more than 25 years of public service, the past year has been filled with second guessing of the 2016 Clinton campaign strategy and continued allegations of unethical or illegal behavior on Hillary Clinton's part during her time in public service and as a presidential candidate. In addition, the focus on sexual harassment this year has caused some, including Democrats, to question the way Bill Clinton's supporters responded to past allegations that he mistreated women.

In the past, both Bill and Hillary Clinton's ratings improved when they were in less overtly political roles -- she as first lady and secretary of state, and he as a former president and philanthropic leader. For Bill Clinton, though, it took more than two years for his image to recover after his controversial exit from the White House in 2001. Thus, if their ratings are to improve, it may take more time for the political wounds from the 2016 campaign to heal.

Survey Methods

Results for this Gallup poll are based on telephone interviews conducted Dec. 4-11, 2017, with a random sample of 1,049 adults, aged 18 and older, living in all 50 U.S. states and the District of Columbia. For results based on the total sample of national adults, the margin of sampling error is ±4 percentage points at the 95% confidence level. All reported margins of sampling error include computed design effects for weighting.

Each sample of national adults includes a minimum quota of 70% cellphone respondents and 30% landline respondents, with additional minimum quotas by time zone within region. Landline and cellular telephone numbers are selected using random-digit-dial methods.

Learn more about how the Gallup Poll Social Series works.

Gallup News Alerts
Get the latest data-driven news delivered straight to your inbox.