Posts Tagged 'Polling'

How to be an intelligent consumer of political polls

As the days get shorter and the air gets cooler, we are on the edge of a cool, colorful season. We are not talking about autumn — instead, “polling season” is upon us! As the US Presidential race heats up, one thing we can count on is being inundated with polls and pundits spinning polling results.

Most market researchers are interested in polls. Political polling pre-dates the modern market research industry and most market research techniques used today have antecedents from the polling world. And, as we have stated in a previous post, polls can be as important as the election itself.

The polls themselves influence voting behavior which should place polling organizations in an ethical quandary. Our view is that polls, when properly done, are an important facet of modern democracy. Polls can inform our leaders as to what the electorate cares about and keep them accountable. This season, polls are determining which candidates get on the debate stage and are driving which issues candidates are discussing most prominently.

The sheer number of polls that we are about to see will be overwhelming. Some will be well-conducted, some will be shams, and many will be in between. To help, we thought we’d write this post on how be an intelligent consumer of polls and what to look out for when reading the polls or hearing about them in the media.

  • First, and this is harder than it sounds, you have to put your own biases aside. Maybe you are a staunch conservative or liberal or maybe you are in the middle. Whatever your leaning, your political views are likely going to get in the way of you becoming a good reader of the polls. It is hard to not have a confirmation bias when viewing polls, where you tend to accept a polling result that confirms what you believe or hope will happen and question a result that doesn’t fit with your map of the world. I have found the best way to do this is to first try to view the poll from the other side. Say you are a conservative. Start by thinking about how you would view the poll if you leaned left instead.
  • Next, always, and I mean ALWAYS, discover who paid for the poll. If it is an entity that has a vested interest in the results, such as a campaign, a PAC, and industry group or lobbyist, go no further. Don’t even look at the poll. In fact, if the sponsor of the poll isn’t clearly identified, move on and spend your time elsewhere. Good polls always disclose who paid for it.
  • Don’t just look to who released the poll, review which organization executed it. For the most part, polls executed by major polling organizations (Gallup, Harris, ORC, Pew, etc.) will be worth reviewing as will polls done by colleges with polling centers (Marist, Quinnipiac, Sienna, etc.). But there are some excellent polling firms out there you likely have never heard of. When in doubt, remember that Five Thirty Eight gives pollsters grades based on their past performances.  Despite what you may hear, polls done by major media organizations are sound. They have polling editors that understand all the nuances and have standards for how the polls are conducted. These organizations tend to partner with major polling organizations that likewise have the methodological muscle that is necessary.
  • Never, and I mean NEVER, trust a poll that comes from a campaign itself. At their best, campaigns will cherry pick results from well executed polls to make their candidate look better. At their worst, they will implement a biased poll intentionally. Why? Because much of the media, even established mainstream media, will cover these polls. (As an aside, if you are a researcher don’t trust the campaigns either. From my experience, you have about a 1 in 3 chance of being paid by a campaign for conducting their poll.)
  • Ignore any talk about the margin of error. The margin of error on a poll has become a meaningless statistic that is almost always misinterpreted by the media. A margin of error really only makes sense when a random or probability sample is being used. Without going into detail, there isn’t a single polling methodology in use today that can credibly claim to be using a probability sample. Regardless, being within the margin of error does not mean a race is too close to call anyway. It really just means it is too close to call with 95% certainty.
  • When reading stories on polls in the media, read beyond the headline. Remember, headlines are not written by reporters or pollsters. They are written by editors that in many ways have had their journalistic integrity questioned and have become “click hunters.” Their job is to get you to click on the story and not necessarily to accurately summarize the poll. Headlines are bound to be more sensational that the polling results merit.

All is not lost though. There are plenty of good polls out there worth looking at. Here is the routine I use when I have a few minutes and want to discover what the polls are saying.

  • First, I start at the Polling Report. This is an independent site that compiles credible polls. It has a long history. I remember reading it in the 90’s when it was a monthly mailed newsletter. I start here because it is nothing more than raw poll results with no spin whatsoever. Their Twitter feed shows the most recently submitted polls.
  • I sometimes will also look at Real Clear Politics. They also curate polls, but they also provide analysis. I tend to just stay on their poll page and ignore the analysis.
  • FiveThirtyEight doesn’t provide polling results in great detail, but usually draws longitudinal graphs on the probability of each candidate winning the nomination and the election. Their predictions have valid science behind them and the site is non-partisan. This is usually the first site I look at to discover how others are viewing the polls.
  • For fun, I take a peek at BetFair which is an UK online betting site that allows wagers on elections. It takes a little training to understand what the current prices mean, but in essence this site tells you which candidates people are putting their actual money on. Prediction markets fascinate me; using this site to predict who might win is fun and geeky.
  • I will often check out Pew’s politics site. Pew tends to poll more on issues than “horse race” matchups on who is winning. Pew is perhaps the most highly respected source within the research field.
  • Finally, I go to the media. I tend to start with major media sites that seem to be somewhat neutral (the BBC, NPR, USA TODAY). After reviewing these sites, I then look at Fox News and MSNBC’s website because it is interesting to see how their biases cause them to say very different things about the same polls. I stay away from the cable channels (CNN, Fox, MSNBC) just because I can’t stand hearing boomers argue back and forth for hours on end.

This is, admittedly, way harder than it used to be. We used to just be able to let Peter Jennings or Walter Cronkite tell us what the polls said. Now, there is so much out there that to truly get an objective handle on what is going on takes serious work. I truly think that if you can become an intelligent, unbiased consumer of polls it will make you a better market researcher. Reading polls objectively takes a skill that applies well to data analysis and insight generation, which is what market research is all about.

NEW POLL SHOWS THAT IF US PRESIDENTIAL ELECTION WERE HELD AGAIN, INCREASED TURNOUT WOULD LIKELY RESULT IN A CLINTON VICTORY

Crux Research poll shows 92% of Trump voters and 91% of Clinton voters would not change their vote

ROCHESTER, NY – MARCH 12, 2017 – Polling results released today by Crux Research show that if there were a “do over” and the election were held again tomorrow, Hillary Clinton would likely win the Presidency.  But, this would not happen as a result of voters changing their vote – rather voters who didn’t turn out in the fall would provide an edge to Clinton.

In 2016, the popular vote was 48.0% for Hillary Clinton and 45.9% for Donald Trump (a gap of 2.1)[1].  This new poll shows that if the election were held again among these two candidates, the popular vote would be estimated to be 52.9% Clinton and 47.1% Trump (a gap of 5.8).

Further, few Clinton or Trump supporters would change their voting behaviors:

  • 92% of those who voted for Trump in November would vote for him again tomorrow.
  • 91% of those who voted for Clinton in November would vote for her again tomorrow.

A new election would bring out additional voters.  57% of non-voters in 2016 would intend to vote. Their votes would split approximately 60% for Clinton and 40% for Trump.  So, increased turnout would likely provide a decisive edge to Clinton.

A closer look at swing states (the five states where the winner won by 2 percentage points or less[2]), shows that Clinton  would win these states by a gap of 9.3, likely enough to change the election result.

WHO WOULD WIN TOMORROW?
Suppose there was a “do over” and the US presidential election were held again tomorrow. 
Whom would you vote for?
Actual 2016 Election Result March 2017 Crux Research Poll*
Donald Trump 45.9% 47.1%
Hillary Clinton 48.0% 52.9%
Others 6.0%
*2017 Crux Research poll is among those who say they would vote if the election were held again tomorrow.
VOTE SWITCHING BEHAVIOR
Suppose there was a “do over” and the US presidential election were held again tomorrow. 
Whom would you vote for?
Voted for Trump in 2016 Voted for Clinton in 2016
Donald Trump 92% 1%
Hillary Clinton 1% 91%
Others 4% 7%
Wouldn’t vote 2% 1%
SWING STATES RESULTS
Suppose there was a “do over” and the US presidential election were held again tomorrow. 
Whom would you vote for?
Actual 2016 Election Result in Swing States Swing States March 2017 Crux Research Poll*
Donald Trump 48.0% 47.1%
Hillary Clinton 47.2% 52.9%
Others 4.8%
*2017 Crux Research poll is among those who say they would vote if the election were held again tomorrow.
** Swing states are five states where the election was decided by 2 percentage points or less (PA, MI, WI, FL, and NH).

###

Methodology

This poll was conducted online between March 6 and March 10, 2017. The sample size was 1,010 US adults (aged 18 and over). Quota sampling and weighting were employed to ensure that respondent proportions for age group, sex, race/ethnicity, and region matched their actual proportions in the population.  The poll was also balanced to reflect the actual proportion of voters who voted for each candidate in the 2016 election.

This poll did not have a sponsor and was conducted and funded by Crux Research, an independent market research firm that is not in any way associated with political parties, candidates, or the media.

All surveys and polls are subject to many sources of error.  The term “margin of error” is misleading for online polls, which are not based on a probability sample which is a requirement for margin of error calculations.  If this study did use probability sampling, the margin of error would be +/-3%.

About Crux Research Inc.

Crux Research partners with clients to develop winning products and services, build powerful brands, create engaging marketing strategies, enhance customer satisfaction and loyalty, improve products and services, and get the most out of their advertising.

Using quantitative and qualitative methods, Crux connects organizations with their customers in a wide range of industries, including health care, education, consumer goods, financial services, media and advertising, automotive, technology, retail, business-to-business, and non-profits.

Crux connects decision makers with customers, uses data to inspire new thinking, and assures clients they are being served by experienced, senior level researchers who set the standard for customer service from a survey research and polling consultant.

To learn more about Crux Research, visit www.cruxresearch.com.

[1] http://uselectionatlas.org/RESULTS/index.html

[2] PA, MI, WI, FL, and NH were decided by 2 percentage points or less.


Visit the Crux Research Website www.cruxresearch.com

Enter your email address to follow this blog and receive notifications of new posts by email.