Posts Tagged 'Market Research'

Going Mobile

There has been a critical trend happening in market research data collection that is getting little attention. If you are gathering data in online surveys and polls, chances are that most of your respondents are now answering your questionnaires on mobile devices.

This trend snuck up on us. Just three years ago we were advising clients that we were noticing that about 25% of respondents were answering on mobile devices. Of the last 10 projects we have completed, that percentage is now between 75% and 80%. (Our firm conducts a lot of research with younger respondents which likely skews this higher for us than other firms, but it remains the case mobile response has become the norm.)

Survey response tools have evolved considerably. Respondents initially used either the mail or provided responses to an interviewer on the other end of a clipboard. Then, people primarily answered surveys from a tethered land-line phone. The internet revolution made it possible to move data collection to a (stationary) computer. Now, respondents are choosing to answer on a device that is always with them and when and where they choose.

There are always “mode” effects in surveys – whereby the mode itself can influence results. However, the mode effects involved in mobile data collection has not been well-studied. We will sometimes compare mobile versus non-mobile respondents on a specific project, but in our data this is not a fair comparison because there is a self-selection that occurs. Our respondents can choose to respond either on a mobile device or on a desktop/laptop. If we see differences across modes it could simply be due to the nature of the choice respondents make and have little to do with the mode itself.

To study this properly, an experimental design would be needed – where respondents are randomly assigned to a mobile or desktop mode. After searching and asking around to the major panel companies, I wasn’t able to find any such studies that have been conducted.

That is a bit crazy – our respondents are providing data in a new and interesting fashion, and our industry has done little to study how that might influence the usefulness of the information we collect.

Here is what we do know. First, questionnaires do not look the same on mobile devices as they do on laptops. Most types of questions look similar, but grid-style questions look completely different.  Typically, on a mobile device respondents will see one item at a time and on a desktop they will see the entire list. This will create a greater response-set type bias on the desktop version. I’d say that this implies that a mode effect likely does occur and that it doesn’t vary in the same way across all types of questions you are asking.

Second, the limited real estate of a mobile device makes wordy questions and responses look terrible. Depending on the survey system you are using, a lengthy question can require both horizontal and vertical scrolling, almost guaranteeing that respondents won’t attend to it.

Our own anecdotal information suggests that mobile respondents will complete a questionnaire faster, are more likely to suspend the survey part-way, and provide less rich open-ended responses.

So, how can we guard against these mode effects? Well, in the absence of research-on-research that outlines their nature, we have a few suggestions:

  • First and foremost, we need to develop a “mobile-first” mentality when designing questionnaires. Design your questionnaire for mobile and adapt it as necessary for the desktop. This is likely opposite to what you are currently doing.
  • Mobile-first means minimizing wording and avoiding large grid-type questions. If you must use grids, use fewer scale points and keep the number of items to a minimum.
  • Visuals are tough … remember that you have a 5 or 6 inch display to work with when showing images. You are limited here.
  • Don’t expect much from open-ended questions. Open-ends on mobile have to be precisely worded and not vague. We often find that clients expect too much from open-ended responses.
  • Test the questionnaire on mobile. Most researchers who are designing and testing questionnaires are looking at a desktop/laptop screen all day long, and our natural tendency is to only test on a desktop. Start your testing on mobile and then move to the desktop.
  • Shorten your questionnaires. It seems likely that respondents will have more patience for lengthy surveys when they are taking them on stationary devices as opposed to devices that are with them at all (sometimes distracting) times.
  • Finally, educate respondents not to answer these surveys when they themselves are “mobile.” With the millions of invitations and questionnaires our industry is fulfilling, we need to be sure we aren’t distracting respondents while they are driving.

In the long run, as even more respondents choose mobile this won’t be a big issue. But, if you have a tracking study in place you should wonder if the movement to mobile is affecting your data in ways you aren’t anticipating.

Will Big Data Kill Traditional Market Research?

Most of today’s research methods rely on a simple premise:  asking customers questions can yield insights that drive better decisions. This is traditionally called primary research because it involves gathering new data. It is often supplemented with secondary research, which involves looking at information that already exists, such as sales data, publicly available data, etc.  Primary and secondary research yield what I would call active data –individuals are providing data with their knowledge and consent.

We are moving to a passive data world. This involves analyzing data left behind as we live increasingly digital lives. When we breathe online we leave a trail of digital data crumbs everywhere – where we visit, what we post about, link to, the apps we use, etc. We also leave trails as to when and where we are when we do these things, what sequence we do them in, and even what those close to us do.

Our digital shadows are long. And these shadows provide an incredibly accurate version of ourselves. You may not remember what you had to eat a few days ago, but the Internet knows exactly what books you read, how fast you read them, and when you bought them. The Internet knows where you were when you looked up health information, your favorite places to travel, whether you lean liberal or conservative, and much more. Your digital shadow is alarmingly accurate.

Privacy issues aside, this creates exciting possibilities for market research.

The amount of information available is staggering.  It is estimated that the volume of digital information available is doubling about every 18 months. This means in the next year and a half we will create as much data as we have since the Internet was created. Clearly it is easy to drown in the noise of this data, and many certainly do. But, in some ways analyzing this data isn’t unlike what we have been doing for years. It is easy to drown in a data set if you don’t have clear hypotheses that you are pursuing.  Tapping into the power of Big Data is all about formulating the right questions before firing up the laptop.

So, how will Big Data change traditional, “active” research? Why would we need to ask people questions when we can track their actual behaviors more accurately?

Big Data will not obviate the need for traditional survey research. But, it will reposition it. Survey research will change and be reserved for marketing problems it is particularly well suited for.  Understanding underlying motivations of behavior will always require that we talk directly to consumers, if only to probe why their reported behavior differs from their actual behavior.

There are situations when Big Data techniques will triumph. We are noticing compelling examples of how Big Data analysis can save the world.  For instance, medical researchers are looking into diseases that are asymptomatic. Typically, an early doctor’s appointment for these diseases will consist of a patient struggling to remember symptoms and warning signs and when they might have had them.  An analysis of Google searches can look at people who can be inferred to have been diagnosed with the disease from their search behavior. Then, their previous search behavior can be analyzed to see if they were curious about symptoms and when.  In the hands of a skilled analyst, this can lead to new insights regarding the early warning signs of diseases that often are diagnosed too late.

There has been chatter that public health officials can track the early spread of the flu better each year by analyzing search trends than by using their traditional ways, which track doctor visits for the flu and prescriptions dispensed. The reason is that people Google for “flu symptoms” in advance of going to the doctor, and many who have symptoms don’t go to the doctor at all. A search trend analysis can help public health officials react faster to outbreaks.

This is all pretty cool. Marketers are all about delivering the right message to the right people at the right time, and understanding how prior online behavior predicts future decisions will be valued. Big Data is accurate in a way that surveys cannot be because memory is imperfect.

Let’s be clear. I don’t think that people lie on surveys, at least not purposefully. But there are memory errors that harm the ability of a survey to uncover the truth. For instance, I could ask on a survey what books you have read in the past month. But, sales data from the Kindle Store would probably be more accurate.

However, what proponents of “Big Data will take over the world” don’t realize is the errors that respondents make on surveys can be more valuable to marketers than the truth because their recollections are often more predictive of their future behavior than their actual past behavior. What you think you had for dinner two nights ago probably predicts what you will eat tonight better than what you actually may have eaten. Perceptions can be more important than reality and marketing is all about dealing with perceptions.

The key for skilled researchers is going to be to learn when Big Data techniques are superior and when traditional techniques will yield better insights. Big Data is a very big hammer, but isn’t suitable for every size nail.

It is an exciting time for our field. Data science and data analysis skills are going to become even more valuable in the labor market than they are today. While technical database and statistical skills will be important, in a Big Data era it will be even more important to have skills in developing the right questions to pursue in the first place and a solid understanding of the issues our clients face.

How Market Research Contributes to the Opioid Epidemic

Opioid misuse has risen to the level of national emergency. The facts are alarming. More people now die of drug abuse than car accidents in the US. Painkiller prescriptions by doctors are now sufficient for each American adult to have a bottle of pills. And, a US citizen dies about every 10 minutes from an opioid overdose. Perhaps the only thing more stunning than the facts behind this issue is how long it has taken to become considered a crisis.

Like many complex public health issues, the opioid crisis has many causes and owners. This makes it more challenging to solve as there isn’t just one “villain” responsible. There are issues of personal responsibility of the opioid user himself/herself, lack of familial support systems, and lack of education. There are legal causes – a lack of an effective regulatory environment and a continued failure to stem a supply of illegal and prescribed narcotics from getting into the wrong hands. The health care community has focused on pain management, been highly influenced by pharma companies who manufacture opioids, and doctors have willingly over-prescribed opioids (in a few years it is projected that there will be on average one opioid prescription for every person living in the US). And, “aggressive” would be too mild a term to describe how pharma companies have marketed these drugs to doctors and consumers.

I am sure that if you talk about the opioid crisis with others and ask what has caused it, “market research” would not be something that is apt to come up. But, our industry has played an important role in creating an environment in health care that is conducive to over-prescription of opioids. Let me explain.

In the late 1980’s a business trend called Total Quality Management became established in US businesses. It was largely a reaction to a perceived threat to US manufacturing from Japan, and it focused highly on statistical measurement. Put simply, TQM assumed that you can’t improve something if you can’t measure it. TQM first gained hold in US manufacturing (where it is still commonly employed). But, it wasn’t long until it spread like wildfire throughout all types of US businesses, including those in the service sector.

This was a boon for market research. The firm I first joined in market research was a custom research company and customer satisfaction measurement was a primary expertise. It was perfect timing – we were in a great position to conduct surveys supporting TQM efforts taking place in a wide variety of industries.

One of these was health care. The 80’s and 90’s were transitionary times in our health care system, as HMO’s and insurance companies became much more powerful and began to “manage” health care. They more tightly controlled which procedures would be reimbursed, and, for better or worse, exerted much greater oversight over the care that doctor’s provided to patients.

This was happening at the same time TQM became all the rage in business. So, insurance companies, hospital administrations, and regulators all began to insist on TQM measures in health care. One of the most important of these was the patient satisfaction survey.

Research companies responded. A few major players emerged, and the company I worked for became a mid-sized supplier in this area. We had an excellent approach and established a small team to work on it, which I eventually managed for a couple of years in the 90s. Patient satisfaction surveys blossomed, and are still in widespread use. I’d guess that if you have ever been to a hospital for a procedure, you’ve been asked to fill out a questionnaire shortly after your visit.

Health care providers, namely doctors, hated these surveys. With umpteen years of medical training behind them, why were they now being evaluated by what their patients thought of the quality of the care they provided? They resisted these surveys and still do.

At the time, we chalked this resistance up to the fact that no professional wants to be evaluated this way. I know I resisted when we started asking clients what they thought of our work and when the results were incorporated into my performance reviews.

We saw this doctor resistance as a misunderstanding of what we were measuring and also a misuse of our data by hospital administrators. In any service delivery, there are two contributors to outcomes: 1) the quality of the service being delivered and 2) the manner in which it is delivered. In many contexts, including health care, consumers should not be placed in a position to evaluate the former. Health care quality assessment is the province of experts, and peers and medically-trained supervisors are probably in the best position to evaluate it.

However, the latter (how the service is delivered) is best evaluated by patients. Our studies constantly showed that doctors understated the importance of these “bedside manner” measures. In our models, these softer issues dominated a patient’s willingness to comply with the physician’s instructions. And, what use is the doctor’s expertise if the patient doesn’t do what he/she says? (As an aside, the nursing profession loved these surveys, as the analyses often showed that the nurse was more important than the doctor in garnering patient compliance).

We were strong advocates for these surveys and how they empowered patients to become an important part of their own health care. Survey results would cause doctors to become better communicators with patients.

With 25 years of hindsight, I can now say that these surveys had a painful unintended consequence: they contributed to the opioid epidemic in this country.

Just as these surveys were taking hold in health care, two important things occurred:  1) the pharma industry developed new forms of opioid painkillers and marketed them aggressively to doctors, and 2) the medical profession adopted pain as a “fifth vital sign” (along with blood pressure, heart rate, respiratory rate, and temperature.) Prior, pain had been viewed as a symptom, and not an objectively measured sign.

This was a watershed moment for the opioid issue. Now, health care providers were constantly asking patients about their pain level (on a 0 to 10 scale or a smiley-face scale). Doctors focused on pain, and there was a widely held perception that pain was being undertreated in patients. This happened precisely at a time when new forms of opioids were available for prescription.

So, what does the patient satisfaction survey have to do with all this? Well, in most of the analyses we would do, a patient’s satisfaction with a doctor visit or procedure was always highly correlated with one item:  “did you feel better as a result of the appointment?” Reduction of pain and patient satisfaction were largely the same thing. We’d move past this in the analysis and tell clients that beyond pain relief, there were bedside manner measures that did matter and that were in their control to change. This is analogous to studies researchers do on consumer products. Price usually overwhelms our models. So, we mention that, and then give guidance about other things beyond price that you can work on.

Hospitals and doctors were being held accountable to the results of these surveys. They quickly learned that pain relief was paramount to how they would be evaluated. This put pressure on them to prescribe opioids for mild ailments. Prior to this time, opioids were mainly used short-term for acute cases and for patients recovering from surgery. Now, they were being prescribed for every-day ailments – toothaches, back pain, broken bones, etc.  At this time pharma companies were downplaying the potential of addiction to these drugs, marketing them heavily, and as a result prescriptions soared.

Of course, I wouldn’t want to overstate the contribution of the market research community to this national crisis. This is however a good example of the importance and unintended consequences of our work. There was a perfect storm of things brewing – the TQM fad, increased power of insurance companies, the development and marketing of new drugs, and a focus on pain management in health care.

I also don’t advocate that we end the patient survey. It provides important feedback to health care providers and I strongly feel that it has resulted in better communications between providers and patients. But, results shouldn’t be used so prominently in the evaluations of hospitals and their staff and the content of these surveys should shift from pain measures. It will take a lot of effort in a lot of directions to resolve the opioid issue, and our industry has an important role to play.

Will Blockchain Disrupt Marketing and Research?

The field of survey research was largely established in the 1930s and matured in the post WWII era as the US economy boomed and companies became more customer-driven. Many early polls were conducted in the most old-fashioned way possible: by going door-to-door with a clipboard and pestering people with questions. Adoption of the telephone in the US (which happened slowly – telephone penetration was less than 50% before WWII and didn’t hit 90% until 1972) made possible an efficient way to reliably gather projectable samples of consumers and the research industry grew quickly.

Then the Internet changed everything. I was fortunate to be at a firm that was leading the charge for online market research at a time when Internet penetration in the US was only about 20%. By the time I left the firm, Internet penetration had reached over 85% and online market research had pretty much supplanted telephone research. What had taken the telephone 40+ years to do to door-to-door polling had happened in less than 10 years, completely transforming an industry.

So, what is next? What nascent technology might transform the market research industry?

Keep your eyes on Blockchain.

Blockchain is best known as the technology that underpins cryptocurrencies like Bitcoin. The actual technology of Blockchain is complex and difficult for most people to understand. (I’d be lying if I said I understood the technology.) But, Blockchain is conceptually simple. It is a way to exchange value and trust between strangers in an un-hackable way and without the need for middlemen. It allows value to be exchanged and stored securely and privately. Whereas the Internet moves information, Blockchain moves value.

Those interested in the potential for Blockchain technology should read The Blockchain Revolution by Don and Alex Tapscott. Or, if you’d like a shortcut, you can watch Don’s excellent Ted Talk.

If Blockchain gains steam and hits a critical mass of acceptance, it has the potential to transform everything including our financial system, our contracts, our elections, our corporate structures, and our governments. It has applicability for any aspect of life that involves an exchange of value that requires an element of trust – which is pretty much everything we do to interact as human beings.

A simple example of how it works is provided by its first widespread application – as a cryptocurrency like Bitcoin. Currently, if I buy a book online, my transaction passes though many intermediaries that are often transparent to me. My money might move from my credit card company to my bank, to another bank, to Amazon, their bank, to the bookseller, to their bank, and I suppose eventually a few crumbs make their way to the author (via their bank of course). There are markups all along the way that are taken by all the intermediaries who don’t add value beyond facilitating the transaction. And, at every step there is an opportunity for my data to be compromised and hacked. The digital shadow left allows all sorts of third parties to know what I am reading and even where I am reading it.

This is an imperfect system at best and one that a cryptocurrency resolves. Via Bitcoin, I can buy a book directly from an author, securely, with no opportunity for others to see what I am doing or to skim value along the way. In fact, the author and I remain strangers.

Blockchain is mostly known currently as Bitcoin’s technology, but its potential dwarfs its current use. Blockchain will clearly transform the financial services industry, and for the better. Buyers and sellers can transact anonymously and securely without a need for intermediaries. Stocks can be exchanged directly by buyers and sellers, and this could lead to a world devoid of investment banks, brokers, and hedge fund managers, or at least one where their roles become strictly advisory.

A useful way to think of the potential of Blockchain is to think of trust. Trust in an economic sense lowers transactions costs and decreases risk. Why do I need lawyers and a contract if I can fully trust my contractor to do what he/she promises? Why do I need Uber when I can contract directly with the driver? As transactions costs decline, we’ll see a much more “democratized” economy. Smaller entities will no longer be at a disadvantage. The costs of coordinating just about anything will decline, resulting in a smaller and very different role for management. If Blockchain really ignites, I’d expect to see flatter corporate structures, very little middle management, and a greater need for truly inspirational leaders.

Any industry reliant on payment systems or risk is ripe for disruption via Blockchain technology. Retail, insurance, government contracting, etc. will all be affected. But, Blockchain isn’t just about payments.  Payments are just a tangible manifestation of what Blockchain really facilitates – which is an exchange of value. Value isn’t always monetary.

Which brings me (finally!) to our field: marketing and marketing research. Marketers and market researchers are “middlemen” – and any middleman has the potential to be affected by Blockchain technology. We stand between the corporation and its customers.

Marketers should realize Blockchain may have important implications to the brand. A brand is essentially a manifestation of trust. In the current digital world, many marketers struggle to retain control of their brands. This is upsetting to those of us trained in historical brand management. Blockchain will result in a greater focus on the brand by customers. They will seek to trust the brand more because Blockchain can enable this trust.

As a researcher I see Blockchain as making it essential that I add value to the process as opposed to being a conduit for the exchange of value. Put more simply, Blockchain will make it even more important that researchers add insight rather than merely gather data. In custom research about half of the cost of a market research project is wrapped up in data collection and that is the part that seems most ripe for disruption. There won’t be as many financial rewards for researchers for the operational aspects of projects. But, there will always be a need to help marketers make sense of the world.

When we design a survey, we are seeking information from a respondent. This information might be classification information (information about who you are), behavioral information (information about what you do), or attitudinal information (information about what you think and feel). In all cases, as a researcher, I am trusting that respondents will provide this information to me willingly and accurately.  As a respondent, you trust me to keep your identity confidential and to provide you an honorarium or incentive for your time. We are exchanging value – you are providing me with information and your time, and I am providing you with compensation and a comfort that you are helping clients better understand the needs of their customers. Blockchain has the potential to make this process more efficient and beneficial to the respondent. And that is important – our industry is suffering from a severe respondent trust problem right now. We don’t have to look much past our plummeting response rates to see that we have lost the respondent trust. Blockchain may be one way we can earn it back.

Blockchain can also authenticate the information we analyze. It can sort out fake data, such as fake postings on websites. To its core, Blockchain makes data transfers simple, secure, and efficient. It can help us more securely store personal information, which in turn will assure our respondents that they can trust us.

Blockchain can provide individuals with greater control over their “digital beings.” Currently, as we go about our lives (smartphone in pocket) we leave digital traces everywhere. This flotsam of our digital lives has value and is gathered and used by companies and governments, and has spawned new research techniques to mine value from this passive data stream. The burgeoning field of Big Data analysis is dependent on this trail we leave. Privacy concerns aside, it doesn’t seem right that consumers are creating a value they do not get to benefit from. Blockchain technology has the potential to allow individuals to retain control and to benefit from the trail of value they are leaving behind as they negotiate a digital world.

Of course as a research supplier I can also see Blockchain as a threat, as suppliers are middlemen between clients and their customers. Blockchain has the potential to replace, or at least enhance, any third-party relationship.  But, I envision Blockchain as being beneficial to smaller suppliers like Crux Research. Blockchain will require suppliers to be more value-added consultants, and less about reliable data collection. That is precisely what smaller suppliers do better than the larger firms, so I would predict that more smaller firms will be started as a result.

Blockchain is clearly in its infancy for marketers. Its potential may prove to be greater than its reality. But, just as we saw with the rise of the Internet, a technology such as this can grow up quickly, and can transform our industry.

Yes, your CEO has an attention span!

Corporate market research departments are a support function. They support decision makers in marketing and often the C-suite by providing market knowledge and insight. Even though research is a support function, since information truly is power, researchers play a powerful role in many organizations. Those that control key information in an organization have a unique responsibility.

Nothing puts research directors more on edge than a presentation to the CEO. Many times this research presentation represents the only point of contact the researcher will have with the firm’s top manager.  That often makes the researcher understandably nervous about what might happen in the presentation.

I’ve seen research directors make many mistakes in these presentations. The most common is presuming that the CEO is so time-pressed that he/she can’t handle a presentation of the full story of the study. Researchers seem to feel that their presentation has to be boiled down to a few key takeaways, colorfully presented so that anyone can understand them. Either way, there often seems to be an undercurrent of fear among researchers when it comes to CEO presentations.

Why do so many researchers live in fear of their CEOs?

I’ve presented in front of a few dozen CEOs in my career (including three billionaires) and have never felt this fear. Perhaps this is because as an outside supplier, I don’t have as much at stake during the presentation as the internal staff does. I have often found that the CEO will give more careful consideration to what I (who has been studying the issue at hand for a few weeks) have to say than what the internal researchers (who have been studying these issues for years) have to say.

I’ve often wondered why. I think this is partially because when you have paid for a consultant, you feel a bit obligated to listen to what he/she has to say. But I think it is more because of the fear internal staff have when they get in front of the CEO.

I have found that CEO presentations tend to go smoother than presentations to marketing departments. The CEO tends to grasp the study quickly, ask insightful questions, and is almost always an excellent communicator and is good with people. In fact that is the one thing I think they all have in common – you really can’t get to the top without strong people skills. This makes them easy to present to.

It is common for the market research department to feel that they have to somehow “dumb down” the research for the report or presentation that goes to the CEO. That is a mistake. This is a very capable audience. It is true that CEOs are often time-starved but I have found that they value the nuance in the story and grasp it well.

Yes, we don’t want to waste the CEO’s valuable time so we want to be sure to tell a clear story, outline important takeaways, and provide implications. But researchers also should be of the mindset that their top-level manager really shouldn’t have anything more pressing to do than to listen to what his/her customers think. Stop being so nervous – it almost always goes better than you think it will!

Americans value money and brains over looks

We recently posed a question on a national poll which required Americans to make an interesting choice:

If you could have one of the following, which would you choose?

  • I would have more money than I have today
  • I would be smarter than I am today
  • I would be better looking than I am today

This is a provocative cocktail party question. How would you answer it? How might your answer change depending on your life stage – would you answer it differently 15 years ago or 15 years into the future?

Across all ages (18+), 61% of Americans choose more money. It would be interesting to pose this question internationally to learn if this finding reflects American culture and capitalism or if this result reflects something universal to all people. Overall, 26% of US adults choose being smarter and 12% choose being better looking. So, it can be said that Americans value money and brains over looks.

We should note that there wasn’t a gender difference in the results. Males and females were just as likely to say all three options. There were a couple of interesting racial differences. Hispanics were least likely to say they would like more money and most likely to say they would like to be smarter. Blacks were as likely as others to say “money” but were more likely than others to say “better looking” and less likely to say “smarter.”

But, by far the largest and most interesting differences in this question related to the generation of the respondent. We’ve seen the Millennial generation maligned quite a bit recently, hearing that they are entitled and a bit lazy. We’ve never quite believed that, as the perception that a youth generation is disrespectful and lazy has been true since before the term “generation” was coined.

For instance, this is a quote from Socrates, and is about 2,400 years old:

“Children today are tyrants.  They contradict their parents, gobble their food, and tyrannize their teachers.”

Mark Twain, late in his life, had this to say about children:

“When a child turns 12 you should put him in a barrel, nail the lid down, and feed him through a knot hole… When he turns 16, plug the hole.”                                              

One of the more cynical (and unintentionally humorous) quotations about children came from Clarence Darrow, almost a century ago:

“The first half of our lives is ruined by our parents and the second half by our children.”

But, back to our poll question.  There are currently five living generations:

First birth year

Final birth year Current youngest member

Current oldest member

Silent

1925

1942 75

92

Boom

1943

1960 57

74

Gen X

1961

1981 36

56

Milllennials

1982

2004 13

35

Homelanders 2005 2017 0

12

Which one do you think would be the most apt to choose “more money” in our question? We’d presume that most people would predict it would be Millennials. But, in reality, it is Boomers who are most likely to say money:

More Money Smarter Better Looking
Silent

54%

37%

9%

Boom

71%

19%

11%

Gen X

65%

26%

10%

Milllennials

52% 31%

17%

There are fascinating generational differences in this table.  Howe and Strauss have developed an excellent generational theory, and one aspect of it is that a generational cycle recurs through four archetypes. So, typically, a current youth generations will have a similar type and outlook as the oldest living generation. This theory is supported by the table above. It is the oldest (Silent) and youngest (Millennials) generations that are least concerned with money and relatively most concerned with being smarter.

Boomers come across as the most money-obsessed generation, which is interesting as they are in a life stage where personal net worth tends to peak. 71% of Boomers would prefer more money to being smarter or better looking.  Of course, with all generational conclusions, it could be more of a life stage issue at work – Boomers are currently between 57 and 74 years old and perhaps pre- and early-retirement are particularly money-centric life stages. But, we suspect that if we had conducted this poll over time Boomers would have been highly concerned with money compared to other generations throughout all life stages.

Finally, these results underscore a point we like to make with clients. It is challenging to fully understand a generation unless we widen the sampling frame and interview other generations as well. Had this question just been asked of Millennials, we may have concluded that money was an overriding concern for them. It is only when comparing them to other generations that we see that they value intelligence and smarts more than others.

Let’s Make Research and Polling Great Again!

Crux Logo Final 2016

The day after the US Presidential election, we quickly wrote and posted about the market research industry’s failure to accurately predict the election.  Since this has been our widest-read post (by a factor of about 10!) we thought a follow-up was in order.

Some of what we predicted has come to pass. Pollsters are being defensive, claiming their polls really weren’t that far off, and are not reaching very deep to try to understand the core of why their predictions were poor. The industry has had a couple of confabs, where the major players have denied a problem exists.

We are at a watershed moment for our industry. Response rates continue to plummet, clients are losing confidence in the data we provide, and we are swimming in so much data our insights are often not able to find space to breathe. And the public has lost confidence in what we do.

Sometimes it is everyday conversations that can enlighten a problem. Recently, I was staying at an AirBnB in Florida. The host (Dan) was an ardent Trump supporter and at one point he asked me what I did for a living. When I told him I was a market researcher the conversation quickly turned to why the polls failed to accurately predict the winner of the election. By talking with Dan I quickly I realized the implications of Election 2016 polling to our industry. He felt that we can now safely ignore all polls – on issues, approval ratings, voter preferences, etc.

I found myself getting defensive. After all, the polls weren’t off that much.  In fact, they were actually off by more in 2012 than in 2016 – the problem being that this time the polling errors resulted in an incorrect prediction. Surely we can still trust polls to give a good sense of what our citizenry thinks about the issues of the day, right?

Not according to Dan. He didn’t feel our political leaders should pay attention to the polls at all because they can’t be trusted.

I’ve even seen a new term for this bandied about:  poll denialism. It is a refusal to believe any poll results because of their past failures. Just the fact that this has been named should be scary enough for researchers.

This is unnerving not just to the market research industry, but to our democracy in general.  It is rarely stated overtly, but poll results are a key way political leaders keep in touch with the needs of the public, and they shape public policy a lot more than many think. Ignoring them is ignoring public opinion.

Market research remains closely associated with political polling. While I don’t think clients have become as mistrustful about their market research as the public has become about polling, clients likely have their doubts. Much of what we do as market researchers is much more complicated than election polling. If we can’t successfully predict who will be President, why would a client believe our market forecasts?

We are at a defining moment for our industry – a time when clients and suppliers will realize this is an industry that has gone adrift and needs a righting of the course. So what can we do to make research great again?  We have a few ideas.

  1. First and foremost, if you are a client, make greater demands for data quality. Nothing will stimulate the research industry more to fix itself than market forces – if clients stop paying for low quality data and information, suppliers will react.
  2. Slow down! There is a famous saying about all projects.  They have three elements that clients want:  a) fast, b) good, and c) cheap, and on any project you can choose two of these.  In my nearly three decades in this industry I have seen this dynamic change considerably. These days, “fast” is almost always trumping the other two factors.  “Good” has been pushed aside.  “Cheap” has always been important, but to be honest budget considerations don’t seem to be the main issue (MR spending continues to grow slowly). Clients are insisting that studies are conducted at a breakneck pace and data quality is suffering badly.
  3. Insist that suppliers defend their methodologies. I’ve worked for corporate clients, but also many academic researchers. I have found that a key difference between them becomes apparent during results presentations. Corporate clients are impatient and want us to go as quickly as possible over the methodology section and get right into the results.  Academics are the opposite. They dwell on the methodology and I have noticed if you can get an academic comfortable with your methods it is rare that they will doubt your findings. Corporate researchers need to understand the importance of a sound methodology and care more about it.
  4. Be honest about the limitations of your methodology. We often like to say that everything you were ever taught about statistics assumed a random sample and we haven’t seen a study in at least 20 years that can credibly claim to have one.  That doesn’t mean a study without a random sample isn’t valuable, it just means that we have to think through the biases and errors it could contain and how that can be relevant to the results we present. I think every research report should have a page after the methodology summary that lists off the study’s limitations and potential implications to the conclusions we draw.
  5. Stop treating respondents so poorly. I believe this is a direct consequence of the movement from telephone to online data collection. Back in the heyday of telephone research, if you fielded a survey that was too long or was challenging for respondents to answer, it wasn’t long until you heard from your interviewers just how bad your questionnaire was. In an online world, this feedback never gets back to the questionnaire author – and we subsequently beat up our respondents pretty badly.  I have been involved in at least 2,000 studies and about 1 million respondents.  If each study averages 15 minutes that implies that people have spent about 28 and a half years filling out my surveys.  It is easy to lose respect for that – but let’s not forget the tremendous amount of time people spend on our surveys. In the end, this is a large threat to the research industry, as if people won’t respond, we have nothing to sell.
  6. Stop using technology for technology’s sake. Technology has greatly changed our business. But, it doesn’t supplant the basics of what we do or allow us to ignore the laws of statistics.  We still need to reach a representative sample of people, ask them intelligent questions, and interpret what it means for our clients.  Tech has made this much easier and much harder at the same time.  We often seem to do things because we can and not because we should.

The ultimate way to combat “poll denialism” in a “post-truth” world is to do better work, make better predictions, and deliver insightful interpretations. That is what we all strive to do, and it is more important than ever.

 

Battle of the Brands is available for purchase!

boxing-glove

How does your brand compete with others in the battle to win today’s youth?

Crux Research has conducted a syndicated study of 57 youth-oriented brands that is available for purchase on Collaborata.  We have a “data only” option for sale for $4,900 and an option including a full report and consultation/presentation for $9,500.

Brands that succeed with Millennials can enjoy their loyalty for years to come. This study’s 13- to 24-year-old group is often given short shrift by brands that have a more adult target. That can prove to be short-sighted thinking. Teens and young adults not only spend significant amounts of their own money, they also influence the spending of parents, siblings, and other adults in their lives. They are the adult shoppers of the future; building a relationship with them now can translate into loyalty that lasts their lifetime. This study shows you exactly where your brand fares among this critical cohort right now and what you need to do increase young consumers’ engagement with your brand.

More information about this study can be found here.

Objectives for our “Battle of the Brands” project are as follows:

  • Compare and contrast the relative strengths across a variety of measures of 57 youth-oriented brands.
  • See how your brand is “personalized” — learn where it statistically maps across 32 brand personality dimensions.
  • Discover how the 57 brands fare on the key measures of Awareness, Brand Interaction, Brand Connection, Brand Popularity, and Motivation.
  • Take away key insights into why some brand succeed, while others struggle, with these Millennials and Gen Z consumers.
  • These brands have been selected from a wide range of categories, including social causes, media and entertainment, retail, technology, and consumer packaged goods.

Become a co-sponsor of this actionable today! Increase your brand’s youth standing tomorrow.