Posts Tagged 'Crux Research'

Among college students, Bernie Sanders is the overwhelming choice for the Democratic nomination

Crux Research poll of college students shows Sanders at 23%, Biden at 16%, and all other candidates under 10%

ROCHESTER, NY – October 10, 2019 – Polling results released today by Crux Research show that if it was up to college students, Bernie Sanders would win the Democratic nomination the US Presidency. Sanders is the favored candidate for the nomination among 23% of college students compared to 16% for Joe Biden. Elizabeth Warren is favored by 8% of college students followed by 7% support for Andrew Yang.

  • Bernie Sanders: 23%
  • Joe Biden: 16%
  • Elizabeth Warren: 8%
  • Andrew Yang: 7%
  • Kamala Harris: 6%
  • Beto O’Rourke: 5%
  • Pete Buttigieg: 4%
  • Tom Steyer: 3%
  • Cory Booker: 3%
  • Michael Bennet: 2%
  • Tulsi Gabbard: 2%
  • Amy Klobuchar: 2%
  • Julian Castro: 1%
  • None of these: 5%
  • Unsure: 10%
  • I won’t vote: 4%

The poll also presented five head-to-head match-ups. Each match-up suggests that the Democratic candidate currently has a strong edge over President Trump, with Sanders having the largest edge.

  • Sanders versus Trump: 61% Sanders; 17% Trump; 12% Someone Else; 7% Not Sure; 3% would not vote
  • Warren versus Trump: 53% Warren; 18% Trump; 15% Someone Else; 9% Not Sure; 5% would not vote
  • Biden versus Trump: 51% Biden; 18% Trump; 19% Someone Else; 8% Not Sure; 4% would not vote
  • Harris versus Trump: 48% Harris; 18% Trump; 20% Someone Else; 10% Not Sure; 4% would not vote
  • Buttigieg versus Trump: 44% Buttigieg; 18% Trump; 22% Someone Else; 11% Not Sure; 5% would not vote

The 2020 election could very well be determined on the voter turnout among young people, which has traditionally been much lower than among older age groups.

###

Methodology
This poll was conducted online between October 1 and October 8, 2019. The sample size was 555 US college students (aged 18 to 29). Quota sampling and weighting were employed to ensure that respondent proportions for age group, sex, race/ethnicity, and region matched their actual proportions in the US college student population.

This poll did not have a sponsor and was conducted and funded by Crux Research, an independent market research firm that is not in any way associated with political parties, candidates, or the media.

All surveys and polls are subject to many sources of error. The term “margin of error” is misleading for online polls, which are not based on a probability sample which is a requirement for margin of error calculations. If this study did use probability sampling, the margin of error would be +/-4%.

About Crux Research Inc.
Crux Research partners with clients to develop winning products and services, build powerful brands, create engaging marketing strategies, enhance customer satisfaction and loyalty, improve products and services, and get the most out of their advertising.

Using quantitative and qualitative methods, Crux connects organizations with their customers in a wide range of industries, including health care, education, consumer goods, financial services, media and advertising, automotive, technology, retail, business-to-business, and non-profits.
Crux connects decision makers with customers, uses data to inspire new thinking, and assures clients they are being served by experienced, senior level researchers who set the standard for customer service from a survey research and polling consultant.

To learn more about Crux Research, visit http://www.cruxresearch.com.

How to be an intelligent consumer of political polls

As the days get shorter and the air gets cooler, we are on the edge of a cool, colorful season. We are not talking about autumn — instead, “polling season” is upon us! As the US Presidential race heats up, one thing we can count on is being inundated with polls and pundits spinning polling results.

Most market researchers are interested in polls. Political polling pre-dates the modern market research industry and most market research techniques used today have antecedents from the polling world. And, as we have stated in a previous post, polls can be as important as the election itself.

The polls themselves influence voting behavior which should place polling organizations in an ethical quandary. Our view is that polls, when properly done, are an important facet of modern democracy. Polls can inform our leaders as to what the electorate cares about and keep them accountable. This season, polls are determining which candidates get on the debate stage and are driving which issues candidates are discussing most prominently.

The sheer number of polls that we are about to see will be overwhelming. Some will be well-conducted, some will be shams, and many will be in between. To help, we thought we’d write this post on how be an intelligent consumer of polls and what to look out for when reading the polls or hearing about them in the media.

  • First, and this is harder than it sounds, you have to put your own biases aside. Maybe you are a staunch conservative or liberal or maybe you are in the middle. Whatever your leaning, your political views are likely going to get in the way of you becoming a good reader of the polls. It is hard to not have a confirmation bias when viewing polls, where you tend to accept a polling result that confirms what you believe or hope will happen and question a result that doesn’t fit with your map of the world. I have found the best way to do this is to first try to view the poll from the other side. Say you are a conservative. Start by thinking about how you would view the poll if you leaned left instead.
  • Next, always, and I mean ALWAYS, discover who paid for the poll. If it is an entity that has a vested interest in the results, such as a campaign, a PAC, and industry group or lobbyist, go no further. Don’t even look at the poll. In fact, if the sponsor of the poll isn’t clearly identified, move on and spend your time elsewhere. Good polls always disclose who paid for it.
  • Don’t just look to who released the poll, review which organization executed it. For the most part, polls executed by major polling organizations (Gallup, Harris, ORC, Pew, etc.) will be worth reviewing as will polls done by colleges with polling centers (Marist, Quinnipiac, Sienna, etc.). But there are some excellent polling firms out there you likely have never heard of. When in doubt, remember that Five Thirty Eight gives pollsters grades based on their past performances.  Despite what you may hear, polls done by major media organizations are sound. They have polling editors that understand all the nuances and have standards for how the polls are conducted. These organizations tend to partner with major polling organizations that likewise have the methodological muscle that is necessary.
  • Never, and I mean NEVER, trust a poll that comes from a campaign itself. At their best, campaigns will cherry pick results from well executed polls to make their candidate look better. At their worst, they will implement a biased poll intentionally. Why? Because much of the media, even established mainstream media, will cover these polls. (As an aside, if you are a researcher don’t trust the campaigns either. From my experience, you have about a 1 in 3 chance of being paid by a campaign for conducting their poll.)
  • Ignore any talk about the margin of error. The margin of error on a poll has become a meaningless statistic that is almost always misinterpreted by the media. A margin of error really only makes sense when a random or probability sample is being used. Without going into detail, there isn’t a single polling methodology in use today that can credibly claim to be using a probability sample. Regardless, being within the margin of error does not mean a race is too close to call anyway. It really just means it is too close to call with 95% certainty.
  • When reading stories on polls in the media, read beyond the headline. Remember, headlines are not written by reporters or pollsters. They are written by editors that in many ways have had their journalistic integrity questioned and have become “click hunters.” Their job is to get you to click on the story and not necessarily to accurately summarize the poll. Headlines are bound to be more sensational that the polling results merit.

All is not lost though. There are plenty of good polls out there worth looking at. Here is the routine I use when I have a few minutes and want to discover what the polls are saying.

  • First, I start at the Polling Report. This is an independent site that compiles credible polls. It has a long history. I remember reading it in the 90’s when it was a monthly mailed newsletter. I start here because it is nothing more than raw poll results with no spin whatsoever. Their Twitter feed shows the most recently submitted polls.
  • I sometimes will also look at Real Clear Politics. They also curate polls, but they also provide analysis. I tend to just stay on their poll page and ignore the analysis.
  • FiveThirtyEight doesn’t provide polling results in great detail, but usually draws longitudinal graphs on the probability of each candidate winning the nomination and the election. Their predictions have valid science behind them and the site is non-partisan. This is usually the first site I look at to discover how others are viewing the polls.
  • For fun, I take a peek at BetFair which is an UK online betting site that allows wagers on elections. It takes a little training to understand what the current prices mean, but in essence this site tells you which candidates people are putting their actual money on. Prediction markets fascinate me; using this site to predict who might win is fun and geeky.
  • I will often check out Pew’s politics site. Pew tends to poll more on issues than “horse race” matchups on who is winning. Pew is perhaps the most highly respected source within the research field.
  • Finally, I go to the media. I tend to start with major media sites that seem to be somewhat neutral (the BBC, NPR, USA TODAY). After reviewing these sites, I then look at Fox News and MSNBC’s website because it is interesting to see how their biases cause them to say very different things about the same polls. I stay away from the cable channels (CNN, Fox, MSNBC) just because I can’t stand hearing boomers argue back and forth for hours on end.

This is, admittedly, way harder than it used to be. We used to just be able to let Peter Jennings or Walter Cronkite tell us what the polls said. Now, there is so much out there that to truly get an objective handle on what is going on takes serious work. I truly think that if you can become an intelligent, unbiased consumer of polls it will make you a better market researcher. Reading polls objectively takes a skill that applies well to data analysis and insight generation, which is what market research is all about.

Why Your Child Hates Sports

It surprises many to learn that on most measures of well-being today’s youth are the healthiest generation in history. Violent crime against and by young people is historically low. Teen pregnancy and birth rates continue to decline. Most measures of drug and alcohol use among teens and young adults show significant declines from a generation ago. Tobacco use is at a low point. In short, most problems that are a result of choices young people make have shown marked improvement since information on Millennials entered the data sets.

But an important measure of well-being has tracked significantly worse during the Millennial and post-Millennial era:  childhood obesity. According to the CDC, the prevalence of obesity has roughly tripled in the past 40 years. This is a frightful statistic.

This is not new news as many books, documentaries, and scholars have presented possible reasons for the spike in youth obesity. Beyond genetics, there are two likely determinants of obesity: 1) nutrition and 2) physical activity. Discussions of obesity’s “nutritional” causes are fraught with controversy. The food industry involves a lot of interests and money, nutritional science is rarely definitive, and seemingly everyone has their own opinions on what is healthy or unhealthy to eat. The nutritional roots of obesity (while likely very significant) are far from settled.

However, the “physical activity” side of the discussion tends to not be so heated. Nearly everyone agrees that today’s youth aren’t as physically active as they should be. There are likely many causes for this as well, but I believe the way youth sports operate merit some discussion.

When I was young, sports were every bit as important to my life as they became to my Millennial children. The difference is my sports experiences as a child were mostly kid-directed. Almost daily, we gathered in the largest yard in the neighborhood and played whichever sport was in season. It took up an hour or two on most days and sometimes the entire weekend. The biggest difference to today’s youth sports environment is there wasn’t an adult in sight. There were arguments, injuries, and conflicts, all of which got resolved without adult mediation.

Contrast this to today’s youth sports environment. Today’s kids specialize in one sport year-round and from a very young age join travel and elite leagues organized by adults. There is a general dearth of unstructured play time. Correlation and causation are never the same thing but the rise in youth obesity has correlated closely with the rise in youth sports leagues organized by adults. Once adults started making the decisions about sports, our kids got fatter.

As a matter of personal perspective, I have two adult children and I can count six sports (baseball, soccer, ice hockey, track, skiing, cross country) that they played in an adult-organized fashion while growing up. We encountered situations where I had a child who was one of the least talented kids on a team, others where I had a child that was the star of the team, and many others where my child was somewhere in the middle. Between them, my kids were on teams that dominated their leagues and went undefeated, they were on some that lost almost every game, and they were on some teams that both won and lost. I coached for a while and my wife was “team mom” for most teams they were on.

Along the way I noticed that kids seemed to have the most fun when they won just a few more games than they lost. The kids didn’t seem to think it was as fun to dominate the competition and it was even less fun to be constantly on the losing end. 

I remember once when in the car after a hockey game I asked my son what he wanted to happen when he had the puck. He said, “I want to score.” I asked him “suppose you scored every single time you touched the puck. Would that be any fun?” At 10 years old, he didn’t have to think long to say that wouldn’t be very fun at all. But, that is what most hockey dads are hoping will happen.

There seems to be a natural force kids apply to sports equality when adults get out of the way. Left to their own devices, the first things kids will do when choosing up teams is to try to get the teams to be evenly matched. Then, if the game starts to get too one-sided the next thing they will do is swap some players around to balance it out. This seems to be ingrained – nobody teaches kids to do this, but left on their own this is what they tend to do. They will also change the rules of the game to make it more fun.

I’ve encountered many parents who are delusional when it comes to the athletic capabilities of their children. I don’t think I have ever met a dad (including myself) who didn’t think their child was better than he/she really was. We want our kids to succeed of course. But we have to have the right definition of success. Are they having fun? Are they improving? Learning how to work as a team and treat competition with respect? Making friendships? That is what is going to matter down the line.

Far too many parents look to the future too much and don’t let their kids enjoy the moment. They will spend thousands and sacrifice nearly every weekend to send their kid to a camp that might get them noticed by college recruiters. The reality is, their child probably won’t get an athletic scholarship, and if he/she does it probably won’t come close to offsetting the money spent getting him/her to all of the camps and travel league games. Parents also don’t realize that most kids don’t find participating in college sports to be as fun as participating in them was in high school.

When I coached Little League baseball, I used to tell the kids to play catch with their mom or dad every day. I remember a mom once asking me why I was pushing them to do this so much. I told her that playing catch with a baseball in the backyard with your kid is one of the great moments in parenthood. It forces you to talk and listen to your kid. I told her that her son would remember that time with his mom or dad far more than playing on our team.

There are debates over rewards for participation in sports. In my day, you had to win to get the trophy and sometimes you didn’t even get that. Now, kids get trophies for showing up. That is not necessarily a bad thing. As Woody Allen says, “80% of success is showing up.” So, why not reward it?

My youngest son was fortunate to run cross country for a coach that most would classify as a local legend. He has coached the team for 30+ years, has had many state championship teams and individuals, and is widely respected. My favorite memory of him was something I observed when he didn’t know I was looking and it had nothing to do with championships and developing elite athletes.  For the first race of a new season, he took the varsity teams to an out-of-state invitational. The girls team was quite good, and for his 7th (and slowest) runner he brought a freshman girl who was inexperienced and running her very first race. She didn’t do very well and came in about 120th place in the race. I saw the coach come up to her right afterword with a beaming smile on his face. The first thing he said to her was “was that FUN or what?” as he gave her a hug. She smiled, hugged him back and ended up staying on the team for all four years of high school and last weekend (8 years later) I saw her jogging in a local park. She didn’t excel at running in high school, but the coach sparked a lifelong interest in fitness in her.

To me, that signified not just what sports should all be about, but what adults’ role in sports should be all about. We have a real problem with childhood obesity. The cure is to make sports and physical activity more fun, and many times that means getting the adults out of the way.

Will adding a citizenship question to the Census harm the Market Research Industry?

The US Supreme Court appears likely to allow the Department of Commerce to reinstate a citizenship question on the 2020 Census. This is largely viewed as a political controversy at the moment. The inclusion of a citizenship question has proven to dampen response rates among non-citizens, who tend to be people of color. The result will be gains in representation for Republicans at the expense of Democrats (political district lines are redrawn every 10 years as a result of the Census). Federal funding will likely decrease for states with large immigrant populations.

It should be noted that the Census bureau itself has come out against this change, arguing that it will result in an undercount of about 6.5 million people. Yet, the administration has pressed forward and has not committed funds needed by the Census Bureau to fully research the implications. The concern isn’t just about non-response from non-citizens. In tests done by the Census Bureau, non-citizens are also more likely to inaccurately respond to this question than citizens, meaning the resulting data will be inaccurate.

Clearly this is a hot-button political issue. However, there is not much talk of how this change may affect research. Census data are used to calibrate most research studies in the US, including academic research, social surveys, and consumer market research. Changes to the Census may have profound effects on data quality.

The Census serves as a hidden backbone for most research studies whether researchers or clients realize it or not. Census information helps us make our data representative. In a business climate that is becoming more and more data-driven the implications of an inaccurate Census are potentially dire.

We should be primarily concerned that the Census is accurate regardless of the political implications. Adding questions that temper response will not help accuracy. Errors in the Census have a tendency to become magnified in research. For example, in new product research it is common to project study data from about a thousand respondents to a universe of millions of potential consumers. Even a small error in the Census numbers can lead businesses to make erroneous investments. These errors create inefficiencies that reverberate throughout the economy. Political concerns aside, US businesses undoubtably suffer from a flawed Census. Marketing becomes less efficient.

All is not lost though. We can make a strong case that there are better, less costly ways to conduct the Census. Methodologists have long suggested that a sampling approach would be more accurate than the current attempt at enumeration. This may never happen for the decennial Census because the Census methodology is encoded in the US Constitution and it might take an amendment to change it.

So, what will happen if this change is made? I suspect that market research firms will switch to using data that come from the Census’ survey programs, such as the American Community Survey (ACS). Researchers will rely less on the actual decennial census. In fact, many research firms already use the ACS rather than the decennial census (and the ACS currently contains the citizenship question).

The Census bureau will find ways to correct for resulting error, and to be honest, this may not be too difficult from a methodological standpoint. Business will adjust because there will be economic benefits to learning how to deal with a flawed Census, but in the end, this change will take some time for the research industry to address. Figuring things like this out is what good researchers do. While it is unfortunate that this change looks likely to be made, its implications are likely more consequential politically than it will be to the research field.

Long Live the Focus Group!

Market research has changed over the past two decades. Telephone research has faded away, mail studies are rarely considered, and younger researchers have likely never conducted a central location test in a mall. However, there is an old-school type of research that has largely survived this upheaval:  the traditional, in-person focus group.

There has been extensive technological progress in qualitative research. We can now conduct groups entirely online, in real-time, with participants around the globe. We can conduct bulletin board style online groups that take place over days. Respondents can respond via text or live video, can upload assignments we give them, and can take part in their own homes or workplaces. We can intercept them when they enter a store and gather insights “in the moment.” We even use technology to help make sense of the results, as text analytics has come a long way and is starting to prove its use in market research.

These new, online qualitative approaches are very useful. They save on travel costs, can be done quickly, and are often less expensive than traditional focus groups. But we have found that they are not a substitute for traditional focus groups, at least not in the way that online surveys have substituted for telephone surveys. Instead, online qualitative techniques are new tools that can do new things, but traditional focus groups are still the preferred method for many projects.

There is just no real substitute for the traditional focus group that allows clients to see actual customers interact around their product or issue. In some ways, as our world has become more digital traditional focus groups provide a rare opportunity to see and hear from customers. They are often the closest clients get to actually seeing their customers in a live setting.

I’ve attended hundreds of focus groups. I used to think that the key to a successful focus group was the skill of the moderator followed by a cleverly designed question guide. Clients spend a lot of time on the question guide. But they spend very little time on something that is critical to every group’s success: the proper screening of participants.

Seating the right participants is every bit as important as constructing a good question guide. Yet, screening is given passing attention by researchers and clients. Typically, once we decide to conduct groups a screener is turned around within a day because we need to get moving on the recruitment. In contrast, a discussion guide is usually developed over a full week or two.

Developing an outstanding screener starts by having a clear sense of objectives. What decisions are being made as a result of the project? Who is making them? What is already known? How will the decision path differ based on what we find? I am always surprised that in probably half of our qualitative projects our clients don’t have answers to these questions.

Next, it is important to remind clients that focus groups are qualitative research and we shouldn’t be attempting to gather a “representative” sample. Focus groups happen with a limited number of participants in a handful of cities and we shouldn’t be trying to project findings to a larger audience. If that is needed, a follow-up quantitative phase is required. Instead, in groups we are trying to delve deeply into motivations, explore ideas, and develop with new hypotheses we can test later.

It is a common mistake to try to involve enough participants to make findings “valid.” This is important, as we are looking for thoughtful participants and not necessarily “typical” customers. We want folks that will expand our knowledge of a subject and of customers will help us explore deeply into topics and develop new lines of inquiry we haven’t considered.

“Representative” participants can be quiet and reserved and not necessarily useful to this phase of research. For this reason, we always use articulation screening questions which raise the odds that we will get a talkative participant who enjoys sharing his/her opinions.

An important part of the screening process is determining how to segment the groups. It is almost never a good idea to hold all of your sessions with the same audience. We tend to segment on age, potentially gender, and often by the participants’ experience level with the product or issue. Contrasting findings from these groups is often where the key qualitative insights lie.

It is also necessary to over-recruit. Most researchers overrecruit to protect against participants who fail to show up to the sessions. We do it for another reason. We like to have a couple of extra participants in the waiting area. Before the groups start, the moderator spends some time with them. This accomplishes two things. First, the groups are off and running the moment participants enter the focus group room because a rapport with the moderator has been established. Second, spending a few minutes with participants before groups begin allows the moderator to determine in advance which participants are going to be quiet or difficult, and allows us to pay them the incentive and send them home.

Clients tend to insist on group sizes that are too large. I have viewed groups with as many as 12 respondents. Even in a two-hour session, the average participant will be talking for just 10 minutes in this case and that is if there are no silences or the moderator doesn’t talk! In reality, with 12 participants you will get maybe five minutes out of each one. How is that useful?

Group dynamics are different in smaller groups. We like to target having about six participants. This group size is small enough that all must participate and engage, but large enough to get a diversity of views.  We also prefer to have groups run for 90 minutes or less.

We like to schedule some downtime in between groups. The moderator needs this to recharge (and eat!), but this also gives time for a short debrief and to adjust the discussion guide on the fly. I have observed groups where the moderator is literally doing back-to-back sessions for six hours and it isn’t productive. Similarly, it is ideal to have a rest day in between cities to regroup to provide an opportunity to develop new questions. (Although, this is rarely done in practice.)

Clients also need to learn to leave the moderator alone for at least 30 minutes before the first group begins. Moderating is stressful, even for moderators who have led thousands of groups. They need time to review the guide and converse with the participants. Too many times, clients are peppering the moderator with last second changes to the guide and in general are stressing the moderator right before the first session. These discussions need to be held before focus group day.

We’d also caution against conducting too many groups. I remember working on a proposal many years ago when our qualitative director was suggesting we conduct 24 focus groups. She was genuinely angry at me when I asked her “what are we going to learn in that 24th group that we didn’t learn in the first 23?”.

In all candor, in my experience you learn about 80% of what you will learn in the first evening of groups. It is useful to conduct another evening or two to confirm what you have heard. But it is uncommon for a new insight arises after the first few groups. It is a rare project that needs more than about two cities’ worth of groups.

It is also critical to have the right people from the clients attending the sessions. With the right people present discussions behind the mirror become insightful and can be the most important part of the project. Too often, clients send just one or two people from the research team and the internal decision makers stay home. I have attended groups where the client hasn’t shown up at all and it is just the research supplier who is there. If the session isn’t important enough to send decision makers to attend, it probably isn’t important enough to be doing in the first place.

I have mixed feelings about live streaming sessions. This can be really expensive and watching the groups at home is not the same as being behind the mirror with your colleagues. Live streaming is definitely better than not watching them at all. But I would say about half the time our clients pay for live streaming nobody actually logs in to watch them.

Focus groups are often a lead-in to a quantitative study. We typically enter into the groups with an outline of the quantitative questionnaire at the ready. We listen purposefully at the sessions to determine how we need to refine our questionnaire. This is more effective than waiting for the qualitative to be over before starting the quantitative design. We can usually have the quant questionnaire ready for review before the report for the groups is available because we take this approach.

Finally, it is critical to debrief at the end of each evening. This is often skipped. Everyone is tired, has been sitting in the dark for hours, and have to get back to a hotel and get up early for a flight. But, a quick discussion to agree on the key takeaways while they are fresh in mind is very helpful. We try to get clients to agree to these debriefings before the groups are held.

Traditional groups provide more amazing moments and unexpected insights than any other research method. I think this may be why, despite all the new options for qualitative, clients are conducting just as many focus groups as ever.

The Importance of Employee Surveys

America’s CEOs constantly say “Our people are our most important asset.” But how many organizations actually live up to this credo?

When I hear a leader say this I’ll investigate where the head of HR reports. I often discover that the top HR person reports to the COO or the CFO. That is a warning sign, as a leader who truly believes in their workforce as a strategic asset would have HR reporting directly to the CEO.

Another warning sign that this credo may ring hollow is if the organization fails to conduct an annual employee survey. I’ve worked with organizations that spend millions of dollars each year on market research yet never get around to an employee survey. In many cases it is because they are worried about what they might discover and recognize that conducting an employee survey sets up an expectation that actions will result from it.

I have been involved in about 250 employee surveys. Results can be eye opening and help create an open culture in an organization. How they are conducted and their content can tell a lot about what the culture of the organization is like.

The best employee projects I have been involved with have the following characteristics:

  1. They have sincere support and commitment from the CEO and the head of HR. This is essential to an employee survey’s success.
  2. They are developed by researchers and not HR or HR consultants. HR-produced surveys tend to be based on amorphous, general concepts and results tend to not be actionable. Surveys written by research departments and firms tend to be more specific and provide more direction as to what to do with the results.
  3. They are conducted annually. Similarly, they are part of a long term, continuous process of listening to employees and making changes based on their feedback.
  4. Results are used in managerial evaluations but are given the proper weight in these performance reviews and are not over-emphasized.
  5. Results are openly shared with the staff. Not just some of the results, but everything that was asked.
  6. They are conducted with a third-party. An outside firm can be objective in analyzing the results and can place results in a context of other projects the have conducted.

But, by far the most important success criteria for these projects is employees must be confident that changes will happen as a result of the survey. We counsel clients to put our recommended actions into three categories:

  • Changes that can be made quickly and visibly. These are little, inexpensive things that should be done right away, and the staff should know that they were made as a result of the survey.
  • Important changes that will require time and effort. These are more substantive changes that may require more input from employees and investment to make happen. Employees should know these efforts are happening and ideally be a part of them.
  • Changes that are recommended but that leadership will choose not to make. This is a step often skipped. The survey will uncover changes that leadership isn’t prepared to make, that require too much investment, etc. It is important that leaders acknowledge these to the staff. It is often sufficient to mention that the survey uncovered these items, but at this time priority will not be given to them. Employees greatly appreciate this honesty.

By far the biggest mistake that can be made with an employee survey is to conduct it and then make no changes based on its results. In my experience, this happens at least 50% of the time. I have counseled dozens of potential clients away from conducting an employee study because I didn’t feel they were prepared to act on the results. There is nothing worse than asking your entire employee base for feedback, and then ignoring the feedback they provide.

Employee surveys can be an asset to any organization. I honestly wouldn’t recommend working anywhere that doesn’t conduct an annual survey and doesn’t make changes based on the results.

Is getting a driver’s license still a rite of passage for teens?

In the 80’s and 90’s, before the Millennial generation hit their teen years in force, we would use “driver’s license status” as a key classification variable in studies. Rather than split focus groups by age or grade in school, we would often place teens who had their license in one group and those who did not have their license yet in another group. Regardless of the topic of the group. We found that teens with licenses were more independent of their parents and more capable of making decisions without parental input. Drivers license obtention was often better predictor of consumer behavior than age.

Young people experience many rites of passages in a short period of time. These are experiences that signify a change in their development. They ride the school bus for the first time, get their first smartphone, enter high school, go to the prom, leave home to go to college, vote for the first time, etc. As marketers, we have always looked at these inflection points as times when consumer behavior shifts. The obtaining of a driver’s license is traditionally seen as a watershed moment as it signifies a new level of independence.

However, this wisdom no longer holds. Millennials, particularly second wave Millennials, are not as focused on obtaining drivers licenses as their Boomer and Xer parents were. Where I grew up, we couldn’t wait until our 16th birthday so we could get our learner’s permit. My classmates and I usually took our road tests at the first opportunity. Failing the road test was a traumatic experience, as it caused us to remain in our parents’ control for a few more months.

This is no longer the case. In 1983, 46% of America’s 16-year-olds had a driver’s license. That is now less than 25% currently. I was very surprised to notice that my children and their friends seemed to be in no particular rush to get their licenses. Many times, it was the parents that pushed the kids to take their road test, as the parents were tiring of chaperoning the kids from place to place.

There are likely things that have caused this change:

  • Today’s parents are highly protective of children. Parents no longer push their children to be as independent as quickly.
  • There are societal pressures. In most states, there are more stringent requirements in terms of driving experience to be able to take a road test and more restrictions on what a younger driver can do with his/her license. The license simply isn’t as valuable as it used to be.
  • Driving has peaked in the US. People are driving less frequently and fewer miles when they do. There has also been a movement of the population to urban areas which have more mass transit.
  • The decline of retail has played a part. Going to the mall was a common weekend activity for Xer teens. Now, staying home and shopping on Amazon is more common. Millennials never went to the mall to socialize.
  • Online entertainment options have proliferated. Movies and shows are readily streamed. Many teens fulfill a need for socialization via gaming, where they interact with their friends and make new ones. This need could only be met in person in the past.
  • Teens are working less so have less of a need to drive to work. Of course, this means they have less of their own money and that tethers them to their parents even longer.

There are likely many other causes. But the result is clear. Teens are getting licenses later and using them less than they did a generation ago.

As a result, researchers have lost a perfectly good measure! Obtaining a driver’s license is not as strong a rite of passage as it used to be.

We’ve been thinking about what might make a good alternative measure. What life event do young people experience that changes them in terms of granting their independence from parents? Leaving home and living independently for the first time would qualify but seems a bit late to be useful. There may be no clear marker signifying independence for Millennials, as they stay dependent on parents across a much wider time period than in the past. Or, perhaps we need to change our definition of independence.

Why Lori is the Best Shark in the Tank

Shark Tank is one of my favorite TV shows. It showcases aspiring entrepreneurs as they make business presentations to an investor panel, who then choose whether to invest. It is fun to play along and try to predict how the Sharks will react to a business pitch. As a small business owner/entrepreneur, it is fun to imagine how I might do in a Shark Tank presentation. And, it was interesting to watch a college teammate of mine make a successful pitch on the show.

My inner geek came out when watching a recent episode. I got to wondering how much the need to entertain might cloud how the venture capital world is portrayed on the show. How many Shark Tank pitches actually result in successful companies? Is the success rate for Shark Tank businesses any higher than any other small company looking for growth capital? Are there any biases in the way the Shark Tanks choose to invest?

This curiosity led to a wasted work day.

Venture capital, especially at early stages, involves high risk bets. Firms may invest in 100 companies knowing full well that 80 or 90 of them will fail, but that a handful of wild successes will pay off handsomely. It isn’t for the faint of heart. I found an interview with Mark Cuban where he stated he hoped that 15% of his Shark Tank investments would eventually pay off. Even that seems high. Given that he has invested about $32 million so far, that is an admission that $27.5 million of that is expected to be wasted. Gutsy.

I also was able to discover interesting things about the show that are largely hidden from the viewer:

  1. The Sharks themselves are paid to take part. I was able to find discussions that suggested they may make as much as $100K per episode. That is a million dollars or more per season, so perhaps they are playing with house money more than they let on.
  2. Getting on Shark Tank is statistically harder than getting into an Ivy League college. It is estimated that more than 50,000 people apply for each season with less than 1% being successful. That alone should provide some realism as to the probability of success of new businesses.
  3. In the early seasons, an entrepreneur had to give up 2% of revenue or 5% of his/her company to the production firm just to appear on the show. That requirement was removed in later seasons because Mark Cuban refused to remain on the show if it remained.
  4. Many of the deals you see made on the show don’t end up being consummated. Forbes conducted survey research in 2016 that indicated that 43% of Shark Tank deals fell apart in the due diligence stage and 30% of the time the deal changed substantially from what is seen on TV. The deal you see on TV only came to fruition as you saw it about 1 in 4 times (27% of the time).

This makes it challenging to assess the deals and whether or not they paid off. Shark Tank companies are almost all privately-held so their revenue data is tough to come by and we can’t really know for sure what the deal was.

Although we can’t review business outcomes as we might like, we can look closely at the deals themselves. The data we used for this includes all deals and prospective deals from the first nine seasons of the show. So, it does not include the current season, which premiered in October 2018.

In the first nine seasons, there were 803 pitches resulting in 436 closed on-air deals (53% of pitches). Applying the Forbes data would imply that of these 436 deals, 187 of them likely feel apart, and 131 of them likely changed substantially. The net? Our projection would be that 53% of pitches result in handshakes on-air, but post-show only 37% of all original pitches close at all and only 15% of pitches will close at the terms you see on air.

Why would Shark Tank deals fail to close? There is a due diligence stage where investors get to have their accountants review the entrepreneur’s books. I found some articles that indicated that some entrepreneurs got cold feet and refused the deal after the show. Also, some of the deals have contingencies which fail to occur.

It is interesting to look at deals by the gender of the entrepreneur as it shows that Shark Tank entrepreneurs skew heavily male:

  • Men are much more likely than women to appear as entrepreneurs on Shark Tank. Of the 803 pitches, 482 (60%) made by men, 198 (25%) by women, and 119 (15%) by mixed teams of men and women. So, 75% of the time, at least one male is involved in the pitch, and 40% of the time at least one female is involved in the pitch.
  • However, women (59% closed) are more likely than men (51%) to successfully close a deal on air.

There are data that imply that men and women negotiate differently:

  • Men initially ask for higher company valuations ($4.5 million on average) than women ($3.1 million on average).
  • Men also ask for more capital ($342K on average) than women ($238K on average).
  • Men (47%) and women (49%) receive about the same proportion of their initial valuation ask. Men (94%) and women (88%) also receive about the same proportion of cash that they initially ask for.

So, men are far more likely to appear on the show and come with bigger deals on average than women. But they receive (proportionately) about the same discount on their deals as women as they negotiate with the Sharks. If there is a difference in their negotiation skills it is that men start bigger or come to the show with more mature companies.

We can also look at individual Sharks to get a sense of how good of negotiators they are:

  • Mark is the most aggressive Shark. He has the most deals (132, despite not being on the early seasons of the show) as well as the most invested (about $32 million).
  • The cheapest (or most frugal?) Sharks are Barbara and Daymond. Barbara has put forth the least amount of money (about $10 million) and her average deal valuation is $945K. Daymond has put out the second least amount of money (about $12 million) and has an average deal size of $957K. These two Sharks have likely not put much more money into their Shark investments than they have been paid to be on the show.
  • Mr. Wonderful seems to have a “go big or stay home” mentality. He has closed the fewest deals (64) of any Shark. But, his average deal valuation of $2.7 million is the highest of any Shark.
  • Lori and Kevin (31% of pitches) are the most likely to make an offer. Barbara and Daymond (22%) are least likely to make an offer.
  • So, Kevin make the most offers and closes the fewest deals, making him the least desirable Shark from the standpoint of the entrepreneurs.

Barbara is the most likely to invest in a female entrepreneur. She is about as likely to invest in a female entrepreneur as a male entrepreneur despite the fact that so many more men than women appear on the show. Kevin and Robert are the least likely to invest in a female entrepreneur. Mark and Daymond demonstrate no bias, as the invest in about the same proportion as appearances on the show.

ALL

Barbara Lori Mark Daymond Kevin

Robert

Male

60%

44% 53% 60% 57% 67%

71%

Female

25%

42% 33% 27% 25% 19%

17%

Mixed Team

15%

14% 14% 13% 18% 14%

12%

So, who has been the most successful Shark? It can be hard to tell because data are scarce, but my vote would go for Lori. USA Today put out a list of the top 20 best-selling products featured on Shark Tank. Six of the top 10 were from investments Lori made, including the top 3. Eight of the top 10 investments by revenue were made by the two female Sharks, Lori and Barbara.

Who are the worst Sharks in terms of revenue performance? My vote here would be a tie between Mark and Daymond. Mark has just 3 of the top 20 investments and Daymond has just 2. If we can assume that the goal of venture capital is to generate big wins, it is clear that Lori and Barbara are killing it and Mark and Daymond are not.

Shark Tank is a great catalyst for entrepreneurs, but because it is entertainment and not reality it can mischaracterize entrepreneurship in the real world. Sharks may invest for the entertainment value of the show and because investing boosts their personal brand as much as the product. And, it might just be the case that the amount of money they have invested is not much larger than the amount of money they have been paid to be on the show.

Almost all successful people will tell you that learning from their failures was at least as important as their successes, yet Shark Tank never revisits failed investments and it is likely that the bulk of the deals we see on TV do not end up paying off for the investor. The show does not disclose how few of its deals actually come to fruition once the cameras are no longer rolling. Just once I’d like to see an update segment show an investment that failed miserably.

Shark Tank also seems to imply that hard work and grit always triumph, when in reality knowing when to cut losses and having a little bit of luck matters a lot in business success. Grit matters for sure, but not when it’s focus is blind and irrational, and it can be sad to see entrepreneurs who have sacrificed so much and it is clear their business is not going to make it.

At its best, Shark Tank stimulates people to think like an entrepreneur. At its worst, it presents too rosy a picture of small business life which influences people to invent new products and launch companies that are likely to fail, at great consequence to the entrepreneur. It certainly provides great entertainment.


Visit the Crux Research Website www.cruxresearch.com

Enter your email address to follow this blog and receive notifications of new posts by email.