Concept Testing: What Respondents Say Can Be What They Really Mean
It is a widely-accepted belief in qualitative market research that you can’t trust what people tell you.
Insights into nonconscious motivations, System 1 and System 2 processing and the role of emotion in decision-making have caused most in the consumer insights business to question self-reported explanations of consumer behavior and decision-making.
Nowhere is this more evident than in the world of concept testing.
Many market researchers believe you are on shaky ground when you ask people to tell you what they think of a package design, advertising or new product concept. Instead, we employ neuroscience to see what parts of the brain light up when exposed to concepts. Or we use eye-tracking to map people’s gaze when looking at a shelf set, package or ad.
But maybe what people say really is what they mean.
Maybe, instead of outfitting people with neurotransmitters and eye-tracking glasses, all we need to do is ask the right questions and listen the right way.
In his book “The Stuff of Thought: Language as a Window Into Human Nature,” Steven Pinker shows how subtle clues in language reveal how human beings perceive the world. As Pinker writes, “…language is a window into human nature, exposing deep and universal features of our thoughts and feelings.” Language, he continues, is “part of a bigger picture of how the mind works…”
Pinker’s analysis shows that embedded in the words we choose are clues to what’s going on in our hearts and minds. What people say – as well as what they don’t say – can reveal deep insights about how they perceive causation, intention, motivation, space and time.
But market researchers have to ask the right questions and analyze language the right way to unlock the hidden meanings in the words people use.
Whether you’re testing new packaging concepts, optimizing a new ad campaign, or getting a read on new product descriptions, here are three simple questions you can ask respondents to unlock deep insights that reveal the most meaningful language and imagery and have the most potential to express your brand’s narrative power.
What do you remember?
Asking people to describe what they remember after exposure to a concept is a powerful question. All of us, no matter how smart or thorough we think we are, filter what we see and hear. It’s how we deal with complexity. When we see a package design, print or television ad, or new product concept, we take in only a subset of the visual and verbal elements to which we’re exposed. The concept that matters is the one in our head that consists of those filtered elements.
Asking people to describe what they remember seeing, reading or hearing is a simple question. But it can reveal a lot about what elements of your concept stand out, are most relevant or challenge existing beliefs.
What does this make you think about?
People are not passive recipients of the stimuli we put in the front of them. Whenever we human beings are exposed to a packaging, advertising or new product concept, memories and associations are triggered. Meaning is created.
We take the filtered elements of what we’ve seen and heard, combine them with memories and associations from our own lives and experiences, and construct meaning from that mashup. That’s why when we test concepts, we never ask people “What do you think of this concept?” Instead, we ask “What does this concept make you think of?”
The difference between these two questions may seem subtle, but they are a world apart. The first question puts people in an evaluative mindset. It asks for an opinion, which – when it comes to marketing concepts – most people are unqualified to give. This is one of the ways marketers have been misled over the years. They’ve been asking opinion questions. The second question opens a window into people’s personal lives. It elicits the memories and associations that people bring to your concept and that are critical to understanding the meaning they create from it.
The memories and associations triggered by a concept are the best indicators of relevance. If someone sees a concept and begins talking about their own life, you’re on to something. Something about your concept triggered a relevant memory or association. That means you’ve tapped into something deep, human and meaningful.
What story does this tell?
If language is data, then storytelling is the richest way to elicit those data. And we human beings are natural-born storytellers. Psychologists tell us that we make meaning out of the experiences of our lives by telling ourselves stories. The process of making meaning from concepts is no different. Asking people to recount the “story” that a package design, advertising or new product concept tells them taps into the filtered elements they recall, the memories and associations the concept triggers, and the meaning it has for them.
When speaking from a narrative rather than evaluative mindset, people will give you a wealth of descriptive language for analysis – language that can reveal deep insights about the conscious and non-conscious reactions to your concept.
Armed with these three questions, you can elicit a ton of rich language about your concepts. But that’s only half the battle. Now you have to listen carefully to the language these questions produce and analyze that language with intent. Patterns in verb choice and descriptive language provide clues to a concept’s clarity, credibility, relevance, uniqueness and motivational power.
Analyze what’s said and what’s not said. Pay attention to the word choices people make. And understand what filtered visual and verbal elements caused those choices.
Now, all of this is not to minimize advancements in neuroscience and eye-tracking. These methods can provide important pieces of the concept optimization puzzle. But they don’t tell the whole story of the very human process of making meaning from what we see and hear.
So, let’s not be too quick to disregard what people tell us. Evaluating and optimizing your concepts may be a lot simpler than you thought. All you have to do is listen.
Do you trust what people say about concepts? Do you buy the argument that language can be a window into the mind? As always, we’d love to hear what you think.
You can learn more about how RealityCheck applies Human Insights to Concept Testing here
