AI tools, specifically chatbots such as Chat GBT, are buzzing among market researchers. While chatbots aren’t brand new, Chat GBT’s capabilities have brought the technology to the minds of many for the first time. Chat GBT seems uncannily accurate and less “artificial” than its AI predecessors. It is as if we are watching Frankenstein’s monster become aware in real time.

History shows that there is always a hype curve with new technology. Initially, there is strong enthusiasm that forecasts a greater potential than will ever be realized. Then, as reality sets in, people adjust their expectations downward – usually too far- and disillusionment with the technology ensues. Finally, the technology finds its level – usually below its initial expectations and greater than its disillusionment level.

This is an MBA-like way of saying that chatbots probably won’t take over our field, but they can potentially change how researchers work. They can create efficiencies by saving time and money and may launch new ways of researching consumers that have not been undertaken before.

Chatbots have the potential to significantly improve market research by providing a fast and efficient way to collect data from a large number of people in a short amount of time. Chatbots can be programmed to ask specific questions and record responses in real-time, making the data collection process more efficient and accurate.

Some potential benefits of using chatbots for market research include:

• Improved efficiency: Chatbots can collect data from multiple respondents simultaneously, reducing the time and resources required for data collection.
• Increased accuracy: Chatbots can ask questions in a consistent and unbiased manner, reducing the risk of human error and bias in data collection.
• Greater reach: Chatbots can reach a larger audience, including those who may not typically participate in market research studies.
• Real-time insights: Chatbots can provide real-time insights into consumer preferences and behaviors, allowing businesses to quickly respond to changes in the market.
• Cost-effectiveness: Chatbots can be programmed to automate the data collection process, reducing the need for human researchers and lowering the overall cost of market research.

I see immediate potential for Chat GBT in qualitative research. Question probing can be more instantaneous, automatic, and patient for online bulletin boards than a human interviewer can provide. I can envision automated IDIs being developed, which will be online one-on-one conversations that do not require an interviewer. This will make IDIs more affordable, and more will be done. This is a good thing, as quantitative studies are suffering from data quality issues and IDIs are becoming a more critical element of many research programs.

I see less of an immediate potential for open-ends in quantitative surveys. Yes, chatbots can probe an open-ended response. But, their ability to do this well requires a good initial response from the respondent. The quality of open-ended responses in online surveys tends to be weak and is declining. The bot requires a thoughtful response to probe on, so I don’t see this as a good use of these bots until we improve the initial responses we get. For now, I see this being much more effective when surveying an engaged audience, such as customers, or in business-to-business studies.

At some point, I suspect that chatbots will make quantitative surveys more adaptive. Follow-up survey questions can automatically appear depending on earlier responses. Yes, we can do that now with survey logic and branching, but the bots may be able to ask questions we haven’t even thought of. When analyzing data at the end of every research project, we always discover questions we wish we had asked. Chatbots may be able to develop those for us before we even see the data.

On the downside, chatbots will also beget even more survey fraud. To date, it is pretty easy to look at open-end responses and tell which ones were written by a bot. That may not be as easy in the future. More fraudsters will defeat our defenses. This begs a strange question: will we use AI to detect the presence of AI fraud tools?

It is essential to know how chatbots work. They aren’t human. But, like humans, they are “trained” on past information. A chatbot’s advantage over humans is it can be trained on a vast amount of data, which gives them enormous potential to “think” faster than a human can.

But, because they are trained on past information, chatbots will be most accurate when tasked with researching issues with a past analog. They are likely less accurate in predicting the future for new and novel products and questions. And that is often the most important role for research. The job of market research is to provide information to guide future decisions.

As currently designed, these bots will likely be more accurate and helpful in areas where technical, encyclopedic knowledge prevails. We have heard how Chat GBT can ace the SAT exam. But, at least currently, it does much better on AP exams in Science than in English.

Back to the original point: chatbots are probably over-hyped right now, but don’t discount their potential too much. What if I told you that two of the paragraphs of this post were generated by Chat GBT? Do you think you can tell which ones?

Be afraid. Be very afraid!