Posts Tagged 'Crux Research'

Why aren’t there more digital textbooks?

On college campuses, technology is like air – always present, necessary, and only noticed when it is lacking. College networks reach seemingly everywhere. Today’s courses use technology for enrollment, collaboration, communication, etc. Much of the basic research that underlies technological breakthroughs in business and industry is pioneered on college campuses. We find on employee surveys that recent Millennial graduates are often underwhelmed by the technology they have access to at their employers because they became accustomed to a higher standard when they were students.

Why then has a technological revolution that colleges are such a central part of seemingly skipped over what is at the core of most college courses:  the college textbook?

Depending on which source you consult, digital textbooks currently comprise between 10% and 15% of college textbooks and this percentage has been growing glacially … at like 1-2% per year.

Contrast this with other types of books. There are currently nearly half a billion digital trade books sold each year. In the “normal” (non-textbook) book world, there are about two digital books sold for every three printed books sold. In trade publishing the conversation isn’t about whether digital books will continue to grow and dominate (as there is a consensus that they will), but more about how massive Amazon will become in the space and what the impact of a growing audiobook segment will be.

Clearly, penetration of digital books is happening much slower in college textbook world than the trade book world.

But why?

First and foremost, the role of publishers in the college textbook market is different than in the trade book world. About 80% of the college textbook market is controlled by just five publishers and there is a trend towards further market consolidation. Publishers have the lion’s share of market power; after all, they control nearly all the content. So they can also control how this content is distributed.

Publishers’ market power is even greater than one might initially imagine. There might be just one or two viable choices for textbooks to select for a course. The result has been an increase in textbook prices of +1,000% or more since the mid-seventies, and, importantly, little incentive on the part of publishers to innovate. Publishers have created digital options and online learning systems, but these aren’t terribly innovative and largely serve to protect existing (and profitable) print textbook franchises. Textbook publishing is a cash cow and publishers protect it.

A finger can also be pointed at colleges. The college bookstore was once seen as an essential service to provide for students. It is now viewed as a profit center, giving colleges little incentive to push back on publishers to keep prices low and to innovate. The college bookstore’s mission has moved from being educational to being profit-centered.

College professors are unwittingly part of the problem. We have done studies that show that students largely buy the textbooks professors tell them to buy. Publishers market textbooks one professor at a time. There are no buying groups or purchasing departments negotiating prices on behalf of students. Our studies show that professors don’t think much about the cost of a book to a student before putting it on the list for the semester. Textbook costs and innovation just aren’t something professors seem to think much about.

There are a few countervailing forces. Used textbook distributors help recycle books and keep prices down. Textbook rental firms have had a similar effect. Increased online buying options have created price competition. But, these forces are swimming upstream in the face of the power held by publishers. Our data show that although the total textbook market is growing (because more students are going to college) the average number of textbooks obtained is decreasing. But, the average price per textbook continues to increase. This leads us to conclude that students are managing increasing textbook costs by going without some books to compensate for increased prices on books they cannot do without. This clearly isn’t the right thing to do from an educational standpoint. Students should be able to afford the materials they need to learn.

The internet has a way of being a disintermediater – of removing barriers between buyers and sellers and decreasing transaction costs. This effect has taken some market power away from publishers of traditional books. The ease of buying online at Amazon, the growth of digital books, etc., has served to make trade publishers less dominant than they used to be. And, in the non-textbook world, there has been a proliferation of self-publishing. An author no longer needs a publisher to reach an audience. Publishers are still important, but they are getting repositioned.

This hasn’t happened with textbooks. Academic book authors still largely use the traditional route via publishers (although some do self-publish, but mostly for students at their own universities).

What is most troubling about the lack of innovation in college textbooks is the academic impact it can have. There is lots of grumbling among student groups and elected officials about the cost of college textbooks. Few mention how true digital innovation in college textbooks would transform education.

We’ve often talked about how when a new medium arises, it initially isn’t all that innovative from a content standpoint. As an example, when television first became established, its content was largely just adapted from the successful radio content of the day (news, variety shows, serials, etc.). Once the new, innovative delivery mechanism was established, the content itself changed to take advantage of the unique features of the new media. The Internet was similar – initially its popularity was as a new delivery mechanism for content that could be found on other media (information like news, weather, encyclopedias, etc.). Once the mechanism was established, the unique power of the Internet (communication, collaboration, etc.) became evident.

Digital textbooks are following this pattern. Currently, digital textbooks are pretty much printed textbooks forced into a digital format – not much more exciting than a PDF copy of a textbook. But, digital textbooks hold much greater potential than printed textbooks. They can share highlights across students, catalyze students to collaborate on content they don’t understand, link to additional sources of information if an area is unclear, illustrate concepts with animations and video, adapt content based on formative assessments along the way, etc. It is easy to get enthusiastic about what a digital textbook could potentially do. It could transform education and teaching. It is easy to see a future where the textbook is the primary method of instruction and the professor becoming more of a coach and less of a lecturer.

The incredible potential of digital textbooks won’t happen until textbook authors see this and start creating textbooks differently and until publishers move past their reliance on traditional printed textbooks and find a profitable path. This seems to be an industry ripe for disruption.

We’d like to say this change is coming soon and is inevitable – but this entire blog post was based on a presentation we gave eight years ago to an industry event, so we have reservations that this change is impending.

The Cambridge Analytica scandal points to marketing’s future

There has been a lot of press, almost universally bad, regarding Cambridge Analytica recently. Most of this discussion has centered on political issues (how their work may have benefitted the Trump campaign) and on data privacy issues (how this scandal has shined a light on the underpinnings of Facebook’s business model). One thing that hasn’t been discussed is the technical brilliance of this approach to combining segmentation, big data, and targeted communications to market effectively. In the midst of an incredibly negative PR story lurks the story of a controversial future of market research and marketing.

To provide a cursory and perhaps oversimplified recap of what happened, this all began with a psychographic survey which provided input into a segmentation. This is a common type of market research project. Pretty much every brand you can think of has done it. The design usually has a basis in psychology and the end goal is typically to create subgroups of consumers that provide a better customer understanding and ultimately help a client spend marketing resources more efficiently by targeting these subgroups.

Almost every marketer targets demographically – by easy to identify characteristics such as age, gender, race/ethnicity, and geography. Many also target psychographic ally – by personality characteristics and deeper psychological constructs. The general approach taken by Cambridge Analytics has been perfected over decades and is hardly new. I’d say I’ve been involved in about 100 projects that involve segmenting on a psychographic basis.

To give a concrete example, this type of approach is used by public health campaigns seeking to minimize drug and alcohol use. Studies will be done on a demographic basis that indicate things like drug use skews towards males more than females, towards particular age groups, and perhaps even certain regions of the country. But, it can also be shown that those most at risk of addiction also have certain personality types – they are risk takers, sensation seekers, extroverts, etc. Combined with demographic information, this can allow a public health marketer to target their marketing spend as well as help them craft messages that will resound with those most at risk.

Segmentation is essentially stereotyping with another name. It is associating perceived characteristics of a group with an individual. At its best, this approach can provide the consumer with relevant marketing and products customized to his/her needs. At its worst, it can ignore variation within a group and devalue the consumer as an individual. Segmentation can turn to prejudice and profiling fast and marketers can put too much faith in it.

Segmentation is imperfect. Just because you are a male, aged 15-17, and love to skateboard without a helmet and think jumping out of an airplane would be cool does not necessarily mean you are at risk to initiate drug use. But, our study might show that for every 100 people like you, 50 of them are at risk, and that is enough to merit spending prevention money towards reaching you. You might not be at risk for drug use, but we think you have a 50% chance of being so and this is much higher than the general risk in the population. This raises the efficiency of marketing spending.

What Cambridge Analytica did was analogous to this. The Facebook poll users completed provided data needed to establish segments. These segments were then used to predict your likelihood to care about an issue. Certain segments might be more associated with hot button issues in the election campaign, say gun rights, immigration, loss of American jobs, or health care. So, once you filled out the survey, combined with demographic data, it became possible to “score” you on these issues. You might not be a “gun nut” but your data can provide the researcher with the probability that you are, and if it is high enough you might get an inflammatory gun rights ad targeted to you.

Where this got controversial was, first and foremost, regardless of what Facebook’s privacy policy may say, most users had no clue that answering an innocuous quiz might enable them to be targeted in this way. Cambridge Analytica had more than the psychographic survey at their disposal – they also had demographics, user likes and preferred content, and social connections. They also had much of this information on the user’s Facebook friends as well. It is the depth of the information they gathered than has led to the crisis at Facebook.

People tend to associate most strongly with people who are like them. So, if I score you high on a “gun nut scale” chances are reasonably high that your close friends will have a high probability of being like you. So, with access to your friends, a marketer can greatly expand the targeted reach of the campaign.

It is hard to peel away from the controversies to see how this story really points to the future of marketing, and how research will point the way. Let me explain.

Most segmentations suffer from a fatal flaw: they segment with little ability to follow up by targeting. With a well-crafted survey we can almost always create segments help a marketer better understand his/her customers. But, often (and I would even say most of the time) it is next to impossible to target these segments. Back to the drug campaign example, since I know what shows various demographic groups watch, I can tell you to spend your ad dollars on males aged 16-17. But, how the heck do you then target further and find a way to reach the “risk taking” segment you really want? If you can’t target, segmentation is largely an academic exercise.

Traditionally you couldn’t target psychographic segments all that well. But, with what Google and Facebook now know about their users, you can. If we can profile enough of the Facebook teenage user base and have access to who their friends are, we can get incredibly efficient in our targeting.  Ad spend can get to those who have a much higher propensity for drug use and we can avoid wasting money on those who have low propensity.

It is a brilliant approach. But, like most things on the Internet, it can be a force for bad as well as good. If what Cambridge Analytica had done was for the benefit of an anti-drug campaign, I don’t think it would be nearly the story it has become. Once it went into a polarized political climate, it became news gold.

Even when an approach like this is applied to what most would call legitimate marketing, say for a consumer packaged good, it can get a bit creepy and feel manipulative. It is conceivable that via something one of my Facebook friends did, I can get profiled as a drinker of a specific brand of beer. Since Google also knows where my phone is, I can then be sent an ad or a coupon at the exact moment I walk by the beer case in my local grocery store. Or, my friends can be sent the same message. And I didn’t do anything to knowingly opt into being targeted like this.

There are ethical discussions that need to be had regarding whether this is good or bad, if it is a service to the consumer, or if it is too manipulative. But, this sort of targeting and meshing of research and marketing is not futuristic – all of the underpinning technology is there at the ready and it is only a matter of time until marketers really learn how to tap into it. It is a different world for sure and one that is coming fast.

Congrats to Truth Initiative – Wins Gold at Ogilvy Awards!

Congratulations to our client Truth Initiative on winning Gold at the David Ogilvy Awards. The Ogilvy awards are unique in that they celebrate campaigns that effectively use market research to spark an insightful campaign. Truth Initiative won gold in the “Unexpected Targeting and Segmentation” category.

The Truth Campaign was called “Stop Profiling.” It centered on a social justice theme – that today’s youth will ban together if they perceive a segment of the population is being treated unfairly. Truth’s ad (“Market Priority”) can be seen here.

Crux Research partnered with CommSight to provide formative research, copy testing, and campaign tracking. We are excited to be a part of this award-winning effort – and this award is the third Ogilvy we have been involved in for Truth Initiative.

Millennial College Students Are Torn Between Open Speech and Protecting the Vulnerable

We recently completed a poll of 1,000 college students on the topic of free speech on campus. Previous postings (here and here) have shown that students are reticent to support controversial speakers on campus and do not support any speakers who might have viewpoints that some students find to be uncomfortable.

In this final post on our poll results, we take a look at some contradictions in our data that demonstrate that today’s college students are torn between a desire to favor a campus that promotes free and open debate and an ethos that makes them want to protect the vulnerable from feeling uncomfortable.

There has been a long-held belief by conservatives that colleges are bastions of liberal thinking and perhaps indoctrination. Our poll results lend support to this viewpoint, as 52% of college students feel their professors tend to be more liberal in their thinking than the nation as a whole while just 23% feel their professors are more conservative:

Compared to the views of the nation as a whole, would you say that your current professors/instructors tend to be:
More conservative in their thinking 23%
About the same as the nation as a whole 25%
More liberal in their thinking 52%

Students tend to express a desire for their professors to be given a wide latitude to express their views and are largely not in support of administrators censoring how professors express their views to students.

Which statement below comes closest to your opinion?
College administrators should closely monitor what professors/instructors teach to make sure all students are comfortable 33%
College professors/instructors should be given a wide degree of freedom to express their views to students 67%

The result below shows that students report that colleges should encourage students to have an open mind to ideas that they may find uncomfortable. At first glance, college students seem to favor an atmosphere of openness on campus.

Which statement below comes closest to your opinion?
Colleges should attempt to shield students from ideas and opinions they may find unwelcome and offensive 25%
Colleges should encourage students to be exposed to ideas and opinions they may find unwelcome and offensive 75%

Millennial college students also recognize that free and open speech is central to university life. For example:

  • Two-thirds (66%) agree that the intellectual vitality of a university depends on open and free expression of ideas.
  • 63% agree that free speech, including controversial speech, is central to college teaching and learning.
  • 57% agree that student-run newspapers have a first amendment right to publish controversial stories without running afoul of college administrators.

That said, this poll also shows that Millennials also hold some views that run counter to the free speech ethos they express:

  • 57% agree that students should be encouraged to report instances of professor bias to administrators.
  • 48% feel that students should be provided warnings in advance to alert them to potentially troublesome readings.
  • 45% feel that colleges should provide intellectual safe spaces, where students can retreat from ideas and perspectives that are at odds with their own.

And, as we discussed in our previous postings, students shy away from permitting almost any type of speaker on campus that could potentially communicate anything that might cause a subgroup of students discomfort.

So, there are some contradictions in our findings that needs explaining. We feel that there is likely some nuance on Millennial opinion. The Millennial college student seems torn between realizing that exposure to ideas counter to their own is essential to their education and a strong ethos of protecting the vulnerable.

Which statement below comes closest to your opinion?
It is more important that colleges stick up for the vulnerable 50%
It is more important that colleges stand up for a spirit of inquiry 50%

This nuance is difficult for Boomer and Xers (who make up most college administrators and professors) to grasp. Older generations grew up not only at a time when free and open speech was held to a higher standard but also at a time where the college/university campus was the nexus of student opinion and influence. Today’s Millennial student has experienced more cultural diversity on campus and has established digital meeting spaces are their nexus for opinion and community. Millennials are exposed to diverse and controversial opinions constantly, to the point where their desire to protect the campus from controversy and discomfort may be a defense mechanism. It is an environment they can control.

What this all means for the university has yet to be seen. But, campus life is changing, and it will be key that the pendulum that is now swinging towards safety and comfort doesn’t swing so far as to limit student exposure to valuable viewpoints and a well-rounded worldview.

Students Are More Likely to Oppose Campus Speakers Than to Support Them

We recently posted a result from an in-depth poll we conducted among 1,000 college students last fall. In this poll we asked students about specific speakers they may or may not support coming to their campus. Among our conclusions was that students largely aren’t supportive of very many speakers – particularly individuals who might be considered to be controversial or present ideas some might find uncomfortable.

In this same poll, we asked students about types of speakers that might come to a college campus. We included speaker types we felt most observers would feel are appropriate as well as speaker types that we felt even the most passionate free speech advocates might question. Our goal was to see where “the line” might be for today’s college students. The answer is the line is very high – students largely don’t want campus speakers at all.

The table below shows the percentage of US college students who would support each type of speaker coming to their campus to speak:

Support
A leader from the Black Lives Matter movement 50%
An advocate for the legalization of marijuana 46%
An elected official with views that are vastly different than yours 22%
A publisher of pornographic videos 21%
An activist who has a different view on abortion than you do 19%
A speaker who strongly opposes the Black Lives matter movement 19%
A politician who is against gay marriage 17%
A speaker who believes that there are racial differences in intelligence 17%
A tobacco company executive 14%
A speaker who is known to have sexually harassed a colleague in the past 11%
Muslim who advocates hatred towards the United States 10%
A speaker who believes that the Holocaust did not happen 10%
A white supremacist 10%

Some interesting conclusions can be made by looking at whom students are willing to support coming to their campus to speak:

  • Even the most highly supported type of speaker (A leader from the Black Lives Matter movement) is only supported by half (50%) of students. Support for any type of campus speaker is tepid.
  • Two types of speakers stood out as having the most support: Leaders from the Black Lives Matter movement and advocates for the legalization of marijuana.
  • It is perhaps troubling that only about 1 in 5 students (22%) support an elected official with views different from their own.
  • Racially insensitive speakers (white supremacists and Holocaust deniers) are the least supported types of speakers.

We can also look at the same list, but this time sorted by the percentage of students who oppose this type of speaker coming to their campus to speak:

Oppose
A white supremacist 68%
A speaker who believes that the Holocaust did not happen 68%
A speaker who is known to have sexually harassed a colleague in the past 67%
Muslim who advocates hatred towards the United States 66%
A speaker who believes that there are racial differences in intelligence 51%
A politician who is against gay marriage 50%
A tobacco company executive 49%
A speaker who strongly opposes the Black Lives matter movement 46%
A publisher of pornographic videos 39%
An activist who has a different view on abortion than you do 27%
An elected official with views that are vastly different than yours 25%
An advocate for the legalization of marijuana 16%
A leader from the Black Lives Matter movement 16%

Here we see that:

  • In general, students are more passionate in their opposition to speaker types than in their support.
  • Speakers with racially insensitive views and those known to have sexually harassed someone are the most opposed types of speakers. Speakers who have sexually harassed are opposed just as much as white supremacists.
  • About half of students oppose politicians who are against gay marriage and tobacco company executives. This is about the same level of opposition as to a speaker who believes there are racial differences in intelligence.
  • About 1 in 4 students would oppose an elected official that has different views than the student.

Because there have been instances of speakers being shouted down and even physically confronted by college students, we posed a question that asked students what they felt were acceptable ways to protest against a campus speaker.

Which of the following actions would you take if you were strongly opposed to a speaker your college had invited to speak on campus?
Disagree with the speaker during a question-and-answer period 25%
Organize a boycott of the speech 22%
Stage a protest outside of the building where the speech is taking place 21%
Host a concurrent speech from a speaker with an opposing view 16%
Stage a sit-in at an administrative building 12%
Physically confront the speaker 8%
Disrupt the speech while it is going on 7%

For the most part, students don’t support any actions if they strongly oppose a campus speaker. While it is encouraging to see that they do not support disrupting the speech or physically confronting a speaker, it is perhaps just as disheartening to see that only 1 in 4 would be willing to disagree with the speaker during a Q&A period. So, not only do students not want most types of speakers, they aren’t willing to step up and do something if a speaker they find controversial does come to campus.

Just as we found when we looked at specific speakers, students seem to be shying away from not just controversial speakers, but also those that might make some portion of the student body uncomfortable. Based on these results, we predict that there will be fewer speakers invited to college campuses in the future and that attendance at these events will decline.

Who is an appropriate campus speaker? Almost nobody!

US colleges face many free speech challenges. Traditionally, colleges have been places where diverse viewpoints are encouraged even if ideas expressed are seen as controversial. But recently, there have been many instances of invited speakers to college campuses sparking protest, being shouted down, and even being physically confronted by students on campuses. It seems that a generational shift is taking place whereby Millennial students are highly concerned about inclusiveness and protecting vulnerable groups from potentially harmful speech. Prior generations of college students (Xers and especially Boomers) seemed to hold the concept of free speech in higher regard and seemed willing to permit more controversial speech on campus.

This is a fascinating issue and we covered it in depth in a poll of 1,000 US college students conducted last fall. This poll tackled a number of issues regarding how today’s college students view the balance between free speech and protecting vulnerable groups. We will be making a number of posts to share the results of this poll, and our first one relates to who today’s college students view as appropriate speakers to bring to campus.

We brainstormed a number of potential speakers, some liberal and some conservative. We listed government officials who, even though they have strong political opinions, we felt most of academia would say have a legitimate right to be heard. And, we listed celebrities accused of some reprehensible acts, speakers who have already generated controversy on college campuses, and foreign leaders considered to be rivals of the United States. Our goal was to see where Millennials draw a “line” – at what point is a speaker so controversial or so offensive that he/she would not have the support of students to come to campus to speak. In total, we listed 24 individuals.

The table below shows the percentage of US college students who would support each speaker coming to their campus to speak:

Person Support
Barack Obama 71%
Bernie Sanders 59%
Joe Biden 48%
Hillary Clinton 39%
Colin Kaepernick 35%
Elizabeth Warren 27%
Donald Trump 24%
Caitlyn Jenner 23%
Paul Ryan 21%
Mike Pence 20%
Louis CK 20%
Chelsea Manning 19%
Bill Cosby 19%
Vladimir Putin 19%
Al Sharpton 18%
Rachel Maddow 17%
Bill O’Reilly 17%
Kevin Spacey 16%
Milo Yiannopoulos 16%
OJ Simpson 16%
Ann Coulter 14%
Kim Jong-un 13%
Steve Bannon 13%
Betsy DeVos 11%
Harvey Weinstein 10%

Some interesting conclusions can be made from whom students are willing to support coming to their campus to speak:

  • Only two speakers, Barack Obama and Bernie Sanders, receive support from a majority of college students.
  • Liberal politicians lead the way – with 5 of the top 6 most supported speakers being leading Democrats.
  • Donald Trump, our current president, is only supported by about 1 in 4 (24%) college students as a campus speaker.
  • Celebrities accused of sexual harassment (Louis CK, Bill Cosby, Bill O’Reilly, Kevin Spacey, Harvey Weinstein) are among the least supported potential speakers.

We can also look at the same list, but this time sorted by the percentage of students who oppose them coming to their campus to speak:

Person Oppose
Kim Jong-un 61%
Donald Trump 53%
Bill Cosby 47%
Vladimir Putin 47%
OJ Simpson 45%
Harvey Weinstein 45%
Mike Pence 39%
Kevin Spacey 34%
Caitlyn Jenner 33%
Betsy DeVos 33%
Bill O’Reilly 28%
Steve Bannon 28%
Louis CK 27%
Hillary Clinton 27%
Milo Yiannopoulos 25%
Paul Ryan 24%
Ann Coulter 23%
Colin Kaepernick 18%
Al Sharpton 18%
Rachel Maddow 16%
Chelsea Manning 16%
Joe Biden 15%
Elizabeth Warren 13%
Bernie Sanders 12%
Barack Obama 10%

Here we see that:

  • Donald Trump is clearly polarizing among college students, with 53% saying they would oppose him coming to their campus to speak.
  • The most opposed speakers are foreign leaders/rivals (Kim Jong-Un, Vladimir Putin), Donald Trump, and celebrities who have been accused of serious crimes (Bill Cosby, OJ Simpson, Harvey Weinstein).
  • Surprisingly, some speakers who have had challenges when speaking at college campuses in the past (Ann Coulter, Milo Yiannopoulos) don’t have high levels of opposition in this poll.

These results are disheartening to those who feel that open expression of ideas is central to collegiate life. Perhaps the key conclusion from these data is how few speakers students support – showing a clear tendency of students to avoid perspectives they may find uncomfortable. This attitude has caused many college administrators to stop allowing potentially controversial speakers on campus as they worry about security and the unrest it may cause. Free speech advocates are likely to feel that today’s students are missing out on an educational opportunity – to listen to different perspectives to help shape a world view.

In either case, attitudes towards free speech on campus are very different than a generation ago – a topic we will be pursuing as we release other data from this poll.

The types of people you find in a market research presentation

Last summer I led a market research results presentation at a client’s office. I had not met any of the individuals in the meeting prior to the presentation other than my immediate client-contact. During introductions I tried my best to understand who was who and to carefully observe the dynamics between people. “Knowing thy audience” is key to an effective presentation.

And, I have to admit – within a few minutes I found myself stereotyping the members of my audience. I have delivered scores of presentations in the past and I can usually quickly assess what the dynamic of the room is going to be like and categorize attendees. But, I can also be wrong in my assessment and it isn’t healthy to make assumptions about people without taking the time to truly get to know them. I sort of feel guilty that I find myself doing this.

This particular presentation had gathered an interesting cast of characters and I couldn’t help but think about how they each were similar to people I have presented to in the past at various clients. Anyway, the list below is meant to be a bit humorous, and I think that anyone who has been in market research presentations will see people they recognize below.

“The Characters You Find in a Market Research Presentation.”

  • The Introvert. This is a person who says little during the meeting but her mind is racing. She tends to get active late in the meeting and provides insightful comments because she doesn’t feel a need to chime in on every obvious point. Others in the organization often ignore her because she is introverted but she is often the smartest person in the room. However, she has the potential to derail the end of the meeting by starting an entirely new line of conversation as you are trying to wrap up. How to succeed with the Introvert: Try to engage her early and ask for her perspective late in the meeting as this person often has the best things to say and adds a lot to the discussion if you can draw her out.
  • Mr. (Lack of) Attention Span. This is a person who probably comes late to the meeting and forces you to start over and repeat the first 10 minutes. Once in the meeting, he is constantly checking his phone, having side conversations, and asking questions that you just answered. This is also the person that skips ahead in the deck and won’t let you build a story as you would like. How to succeed with Mr. Attention Span: Do not provide handouts beforehand or during this meeting. Keep the presentation short if possible. State ground rules up front as to when you will pause for questions.
  • The Poseur. This person has a clear view of the world in his mind and will find a way to massage every fact you present to make it fit with a pre-conceived view. He uses your facts to illustrate just how insightful he is and what he already knows. This is the marketer that personifies David Ogilvy’s quote that marketers use research “as a drunkard uses a lamp post, for support rather than for illumination.”   He uses the meeting to become the center of attention. He has to provide his view on every slide and every conclusion you have no matter what the size of the meeting. He dominates and other attendees tend to defer to him before offering their own opinions.  How to succeed with the Poseur:  At the onset, set “pause points” in the presentation — at the end of each section you will call for a discussion. Establish ground rules for the meeting. Ask everyone to write down a prediction on how a research result came out on paper before you show the actual result. Then, call on other individuals to discuss their prediction. Look to qualitative techniques for inspiration on how to handle a dominant focus group participant for inspiration.
  • The Jargon Guy. This is a person who talks a lot but doesn’t really say anything. He is a master of business jargon – it is the person who will use words like “bandwidth”, “game changer”, “visioning”, etc.  He will add “ize” onto nouns to turn them into verbs and use acronyms as much as possible. He reads popular business books on the side. You’ll feel like you are in an episode of “The Office” when you meet him. How to succeed with the Jargon Guy: Learn some of the proprietary jargon and acronyms used by your client’s firm beforehand.
  • The Cherry Picker.  Similar to the Poseur, this is the client who also has a clear “map of the world” established in her head and won’t let facts get in the way of a good opinion. She is active in the discussion but what she does is cherry pick results – and criticizes every point that doesn’t fit with her vision, and falls in love with every point that does. How to succeed with the Cherry Picker: Try to get her to buy into your methodology and lead with conclusions you think are likely to fit with how she thinks. That may get her to listen more to findings that don’t fit with her outlook later on.
  • The Naysayer.  This person doesn’t believe in market research and once he learns the study isn’t perfect will challenge everything you say. He straddles a line between “critic” and “cynic”. How to succeed with the Naysayer: This person can be a useful contributor if you can get his negativity to become constructive and establish the right tone. Fortunately, his concerns can often be anticipated beforehand, and you can often address his concerns before he gets a chance to raise them.
  • The Academic.  The academic asks incredibly detailed questions about the methodology and slows down the initial part of the presentation. This person is usually highly educated and understands the details of statistics and experimental design, sometimes better than you do. The good news is she rarely questions your findings if she agrees with the methods you have employed. How to succeed with the Academic: get to her beforehand and share the details of the methodology so she doesn’t get the meeting off to a bad start by bogging it down with methodological details. This person can be a great ally for you during the talk.
  • The Box Checker.  This is a person who is mainly concerned that the research got done because it is part of a larger marketing process that he is responsible for. He is much more of a “process” than an “outcomes” person and tends to be bureaucratic. How to succeed with the Box Checker:  Make sure he knows the project got done efficiently, on time, and within budget.
  • The Enlightened Leader.  This is the person we all want to present to. It is the highest ranking person in the room, but she casts aside all her other responsibilities for the hour you have with her. For at least one hour, you and your client feel that this study is the most important thing in her life.  She truly listens, doesn’t presume anything, and allows the research to add nuance to her view of the world. She usually insists that others in the meeting take action based on the findings.  How to succeed with the Enlightened Leader: Bring her into the conversation early, as it sets the tone for everyone.

I should note, that with very few exceptions, these personalities tend to be respectful and courteous and less challenging to present to than the above descriptions imply. Above all, preparation is key to success with all types of people. You need to deeply know your data set and have well-supported conclusions and implications, as in the end that tends to get you over any rough spots that arise. Your day-to-day contact needs to be your ally, and running through the presentation in advance with him/her often helps stave off any rough moments. Most research presentations go well, but we aim for them to not just go well, but to be effective. While it might not be appropriate to stereotype as I have done here, it is appropriate to realize each individual is coming to your presentation with his/her own perspective. Understanding that perspective can be as important as the study itself in terms of having research inform better decisions.

Going Mobile

There has been a critical trend happening in market research data collection that is getting little attention. If you are gathering data in online surveys and polls, chances are that most of your respondents are now answering your questionnaires on mobile devices.

This trend snuck up on us. Just three years ago we were advising clients that we were noticing that about 25% of respondents were answering on mobile devices. Of the last 10 projects we have completed, that percentage is now between 75% and 80%. (Our firm conducts a lot of research with younger respondents which likely skews this higher for us than other firms, but it remains the case mobile response has become the norm.)

Survey response tools have evolved considerably. Respondents initially used either the mail or provided responses to an interviewer on the other end of a clipboard. Then, people primarily answered surveys from a tethered land-line phone. The internet revolution made it possible to move data collection to a (stationary) computer. Now, respondents are choosing to answer on a device that is always with them and when and where they choose.

There are always “mode” effects in surveys – whereby the mode itself can influence results. However, the mode effects involved in mobile data collection has not been well-studied. We will sometimes compare mobile versus non-mobile respondents on a specific project, but in our data this is not a fair comparison because there is a self-selection that occurs. Our respondents can choose to respond either on a mobile device or on a desktop/laptop. If we see differences across modes it could simply be due to the nature of the choice respondents make and have little to do with the mode itself.

To study this properly, an experimental design would be needed – where respondents are randomly assigned to a mobile or desktop mode. After searching and asking around to the major panel companies, I wasn’t able to find any such studies that have been conducted.

That is a bit crazy – our respondents are providing data in a new and interesting fashion, and our industry has done little to study how that might influence the usefulness of the information we collect.

Here is what we do know. First, questionnaires do not look the same on mobile devices as they do on laptops. Most types of questions look similar, but grid-style questions look completely different.  Typically, on a mobile device respondents will see one item at a time and on a desktop they will see the entire list. This will create a greater response-set type bias on the desktop version. I’d say that this implies that a mode effect likely does occur and that it doesn’t vary in the same way across all types of questions you are asking.

Second, the limited real estate of a mobile device makes wordy questions and responses look terrible. Depending on the survey system you are using, a lengthy question can require both horizontal and vertical scrolling, almost guaranteeing that respondents won’t attend to it.

Our own anecdotal information suggests that mobile respondents will complete a questionnaire faster, are more likely to suspend the survey part-way, and provide less rich open-ended responses.

So, how can we guard against these mode effects? Well, in the absence of research-on-research that outlines their nature, we have a few suggestions:

  • First and foremost, we need to develop a “mobile-first” mentality when designing questionnaires. Design your questionnaire for mobile and adapt it as necessary for the desktop. This is likely opposite to what you are currently doing.
  • Mobile-first means minimizing wording and avoiding large grid-type questions. If you must use grids, use fewer scale points and keep the number of items to a minimum.
  • Visuals are tough … remember that you have a 5 or 6 inch display to work with when showing images. You are limited here.
  • Don’t expect much from open-ended questions. Open-ends on mobile have to be precisely worded and not vague. We often find that clients expect too much from open-ended responses.
  • Test the questionnaire on mobile. Most researchers who are designing and testing questionnaires are looking at a desktop/laptop screen all day long, and our natural tendency is to only test on a desktop. Start your testing on mobile and then move to the desktop.
  • Shorten your questionnaires. It seems likely that respondents will have more patience for lengthy surveys when they are taking them on stationary devices as opposed to devices that are with them at all (sometimes distracting) times.
  • Finally, educate respondents not to answer these surveys when they themselves are “mobile.” With the millions of invitations and questionnaires our industry is fulfilling, we need to be sure we aren’t distracting respondents while they are driving.

In the long run, as even more respondents choose mobile this won’t be a big issue. But, if you have a tracking study in place you should wonder if the movement to mobile is affecting your data in ways you aren’t anticipating.