Technological progress has disrupted market research. Data are available faster and cheaper than ever before. Many traditional research functions have been automated out of existence or have changed significantly because of technology. Projects take half the time to complete that they did just a decade ago. Decision making has moved from an art to a science. Yet, as with most technological disruptions, there are just as many potential pitfalls as efficiencies to be wary of as technology changes market research.

“Passive” data collection is one of these potential pitfalls. It is used by marketers in good ways: the use of passive data helps understand consumers better, target meaningful products and services, and create value for both the consumer and the marketer. However, much of what is happening with passive data collection is done without the full knowledge of the consumer and this process has the potential of being manipulative. The likelihood of backlash towards the research industry is high.

The use of passive data in marketing and research is new and many researchers may not know what is happening so let us explain. A common way to obtain survey research respondents is to tap into large, opt-in online panels that have been developed by a handful of companies. These panels are often augmented with social (river) channels whereby respondents are intercepted while taking part in various online activities. A recruitment email or text is delivered, respondents take a survey, and data are analyzed. Respondents provide information actively and with full consent.

There have been recent mergers which have resulted in fewer but larger and more robust online research panels available. This has made it feasible for some panel companies to gain the scale necessary to augment this active approach with passive data.

It is possible to append information from all sorts of sources to an online panel database. For instance, voter registration files are commonly appended. If you are in one of these research panels, clients likely know if you are registered to vote, if you actually voted, and your political party association. They will have made a prediction of how strong a liberal or conservative you likely are. They may have even run models to predict which issues you care most about. You are likely linked into a PRIZM cluster that associates you with characteristics of the neighborhood where you reside, which in turn can score your potential to be interested in all sorts of product categories. This is all in your file.

These panels also have the potential to link to other publicly-available databases such as car registration files, arrest records, real estate transactions, etc. If you are in these panels, whether you have recently bought a house, how much you paid for it, if you have been convicted of a crime, may all be in your “secret file.”

But, it doesn’t stop there. These panels are now cross-referenced to other consumer databases. There are databases that gather the breadcrumbs you leave behind in your digital life: sites you are visiting, ads you have been served, and even social media posts you have made. There is a tapestry of information available that is far more detailed than most consumers realize. From the research panel company’s perspective, it is just a matter of linking that information to their panel.

This opens up exciting research possibilities. We can now conduct a study among people who are verified to have been served by a specific client’s digital advertising. We can refine our respondent base further by those who are known to have clicked on the ad. As you can imagine, this can take ad effectiveness research to an entirely different level. It is especially interesting to clients because it can help optimize media spending which is by far the largest budget item for most marketing departments.

But, therein lies the ethical problem. Respondents, regardless of what privacy policies they may have agreed to, are unlikely to know that their passive web behavior is being linked into their survey responses. This alone should ring alarm bells for an industry suffering from low response rates and poor data quality. Respondents are bound to push back when they realize there is a secret file panel companies are holding on them.

Panel companies are straying from research into marketing. They are starting to encourage clients to use the survey results to better target individual respondents in direct marketing. This process can close a loop with a media plan. So, say on a survey you report that you prefer a certain brand of a product. That can now get back to you and you’ll start seeing ads for that product, likely without your knowledge that this is happening because you took part in a survey.

To go even further, this can affect advertising people not involved in the survey may see. If you prefer a certain brand and I profile a lot like you, as a result of your participation in a survey I may end up seeing specific ads. Even if I don’t know you or have any connection to you.

In some ways, this reeks of the Cambridge Analytica scandal (which we explain in a blog post here). We’ll be surprised if this practice doesn’t eventually create a controversy in the survey research industry. This sort of sales targeting resulting from survey participation will result in lower response rates and a further erosion of confidence in the market research field. However, it is also clear that these approaches are inevitable and will be used more and more as panel companies and clients gain experience with them.

It is the blurring of the line between marketing and market research that has many old-time researchers nervous. There is a longstanding ethical tenet in the industry that participation in research project should in no way result in the respondent being sold or marketed to. The term for this is SUGGING (Selling Under the Guise of research) and all research industry trade groups have a prohibition against SUGGING embedded in their codes of ethics. It appears that some research firms are ignoring this. But, this concept has always been central to the market research field: we have traditionally assured respondents that they can be honest on our surveys because we will in no way market to them directly because of their answers.

In the novel 1984 George Orwell describes a world where the government places its entire civilization under video surveillance. For most of the time since its publication, this has appeared as a frightening, far-fetched cautionary tale. Recent history has suggested this world may be upon us. The NSA scandal (precipitated by Edward Snowden) showed how much of our passive information is being shared with the government without our knowledge. Rather than wait for the government to surveil the population, we’ve turned the cameras on ourselves. Marketers can do things I don’t feel people realize and research respondents are unknowingly enabling this. The contrails you leave as you simply navigate your life online can be used to follow you and the line between research and marketing is fading, and this will eventually be to the detriment of our field.