What's the difference between asking customers open-ended questions and close-ended (multiple-choice) questions?
We took something basic like color. Presumably, if you like the color blue you'll indicate it's your favorite color regardless of how you're asked. However, we found that even in something as simple as color, depending on the types of questions you ask, people will give you very different answers.
To demonstrate the difference between these question types, we asked people what their favorite color is. Half of the responders were given the following options to choose from (the order of these choices was randomized for each responder):
Red
Green
Blue
Yellow
Purple
Black
Orange
Other (please specify)
The other half were allowed to answer freely, i.e., given an open-ended question. We did not prompt them with any options beforehand.
The results were quite different between the two groups! For example
116% more people in the open-ended responses listed “pink” as their favorite color compared to those in the closed-ended responses (those people had to list pink under “Other”).
36% fewer people in the open-ended responses listed “green” (or shades of green) compared to those in the closed-ended responses.
43% more people in the open-ended responses listed “red” compared to people that selected “red” in the closed-ended responses.
And on and on …
Now imagine the business impact of getting such huge differences when it comes to the reasons why people churn, why people buy, and what prevents people from buying.
Interestingly, the magnitude of the discrepancy between open-ended and closed-ended results might be impacted by gender. We didn’t thoroughly investigate it (comment below if you want us to look into it for our next blog post), but as an anecdotal example take a look at the results for the color blue:
The results were not only different in the ranking of colors but also in the nuance and details of the results. As you might guess, giving responders the option to write their own answers yielded a lot more nuance. Instead of standard colors, survey takers responded with many more colors and shades (who knew blue had so many variations?).
Translating this to the world of business, your customers are not just made of red, green, and blue, but rather of many shades–whether they be forest green, hot pink, or cobalt blue. In order to succeed, you need to understand all these shades. You also need to allow your customers the option to share these shades with you.
Beehive AI helps you get the right answers and capture the nuance, or shades, in the world.
About Beehive AI:
Beehive AI is an end-to-end, customizable AI platform to design, deploy, and analyze consumer research in real time and at scale. Our platform automatically analyzes qualitative (open-ended) data, combined with quantitative data at scale, and focuses on unlocking the “why” in consumer behavior.
Want to learn more? Contact us at info@beehive.ai
Meet with Beehive AI
Comments