A recent issue of The Economist contained an article that describes a potential threat to the accuracy of opinion polling. The latest iPhones have a software feature that doesn’t just block robocalls but sends all calls from unknown callers automatically to voice mail. This feature combats unwanted calls on mobile phones.

Matching sampling frames to populations of interest is increasingly difficult to accomplish in survey research, particularly telephone studies. I will always remember my first day on the job in 1989 when my supervisor was teaching me how to bid projects. The spreadsheet we used assumed our telephone polls would have a 60% cooperation rate. So, at that time about 6 in 10 phone calls we made resulted in a willing respondent. Currently, telephone studies rarely achieve a cooperation rate above 5%. That is 1 in 20 calls. If you are lucky.

The Do Not Call Registry took effect in 2003. At this time, most survey research was still being conducted by telephone (online research was growing but still represented only about 20% of the market research industry’s revenues). Researchers were initially relieved that market research and polls were exempt from the law but in the end that didn’t matter. People stopped cooperating with telephone studies because they thought they had opted out of research calls when they signed up for the Registry. Response rates plummeted.

The rise of mobile phones caused even more headaches for telephone researchers. There was initially no great way to generate random numbers of cell phones in the same way that could be done for land lines and publicly-available directories of cell phone numbers did not exist. For quite some time, telephone studies were underrepresenting mobile phone users and had no great solution for how to interview respondents who did not even have a land line. Eventually, the industry figured this out and methods for including mobile phones became standard.

This new development of automatically routing mobile calls to voice mail could well signify the end of telephone-based research. If consumers like this feature on iPhones it won’t be long until Android-based phones do the same thing. It will preclude pollsters from effectively reaching mobile-only households. Believe it or not, about 45% of US households still have a land line, but the 55% who do not skew young, urban, and liberal.

Pollsters will figure this out and will oversample mobile only households and weight them up in samples. But that won’t really fix the problem. Samples will miss those that have the latest phones and will eventually miss everybody once all current phones are replaced. Oversampling and weighting can help balance under-represented groups, but can’t fix a problem when a group is not represented at all. Weighting can actually magnify biases in samples.

Implications to this?  Here are a few:

1. More polls and market research project will be conducted online. This is a good thing as there is evidence that in the 2016 election the online polls were more accurate than the telephone polls. It is hard to believe, but we are at a stage where telephone polls are almost always slower, more expensive, and less accurate than their online counterparts.
2. Researchers will use more mixed samples, using both telephone and online. In our view this tends to be needlessly complicated and introduces mode effects into these samples. We tend to only recommend mixed-mode data collection in business-to-business projects, where we use the phone to screen to a qualified respondent and then send the questionnaire electronically.
3. Costs of telephone polls will go up. They are already almost criminally expensive and this will get even worse. For those not in the know, the cost per interview for a telephone poll is often 20 to 30 times the cost of an online interview.
4. Address Based Samples (ABS) will gain in popularity. As telephone response rates decline, systematic biases in telephone samples increase. ABS, when properly operationalized, is a good alternative (although ABS has its limitations as well). ABS still isn’t really probability sampling, but it is the closest thing we have.
5. The increased cost of telephone polls will spur even more investment in online panels. The quality of online research will be better off because of it. If there is a silver lining for researchers, this is probably it.

Technology has always tended to move faster than the market research industry has been able to adapt to it, probably because researchers have an academic mindset (thorough, but slow). Research methodologists always seem to eventually come up with a solution, but not always quickly. For now, we’d recommend against trusting any opinion poll that is based on a telephone sample, unless the researchers behind it have specifically made a case for how they are going to address this new issue of software blocking their calls to mobile phones. The good news is push polls and robo polls will soon become almost impossible to conduct.