Text Analytics: 3 Keys to Deeper Consumer Insights

By Jim White PhD. and Esther Park, Insight Analyst

Market research buyers have become increasingly interested in text analytics in recent years. According to the 2022 Greenbook Industry Trends Report survey, 75% of buyers list text analytics when asked “what new methods, or approaches are you considering.” That’s more than any other category.

We think this interest is driven by a few factors.

• First, there has been tremendous growth in the volume of textual data available to brands. Social media, online reviews, blogs, asynchronous qualitative research and auto-transcription of groups and interviews all produce volumes of textual data, all of which is useless without efficient analysis.
• Second, the increasing sophistication of AI text analytics software has made analysis of large text data-sets possible.
• And third, the need for speed in analysis has driven buyers to seek out efficient ways to mine textual data for insights.

But despite the interest of market research buyers in text analytics, suppliers are not rising to the demand. In the survey mentioned above, only 25% of suppliers mentioned text analytics when asked “what new methods, or approaches are you considering.” There appears to be a disconnect between what buyers want and suppliers provide.

Also leaving research buyers a bit cold right now is a lack of deep insight delivered by the text analytics. In many instances, the output of text analytics is expected, uninspiring and superficial.

The power of textual data is its richness and depth. Its value compared to numeric data is its ability to bring human experience to life in a way that inspires creativity and innovation. If we lose that, what’s the point?

At RealityCheck, we’ve been using text analytics software for several years now to aid in the analysis of textual data gathered through asynchronous online qualitative research, transcripts from consumer interviews and other sources. If you’ve ever done asynchronous online qualitative research, you know it produces a ton of unstructured data. Text analytics can help us mine those data for deep insights. But it has to be done the right way.

For us, text analytics has provided two clear benefits. First, as our use of asynchronous online qualitative has increased, so has the volume of data we must analyze. Text analytics software is the only way to efficiently analyze large unstructured datasets. If you care about your analysis being comprehensive, data-driven and valid, but also speedy and cost-effective, you have to use software. There’s no way around it.

Second, text analytics software allows us to find patterns in textual data that we never would have found with the naked eye (in other words, by using human coders alone). We’ve learned that the combination of tech and touch, artificial intelligence and human intelligence, is a powerful combination for revealing deep insights from textual data.

So, here are a few key ways we’ve learned for using text analytics to generate deep human insights. We hope you find these useful.

1. Software should serve your workflow, not the other way around
   In the hands of a skilled analyst, text analytics packages are wonderful. But they have to be built into your analytical workflow to serve your unique analysis objectives. At RealityCheck, our workflow begins with smart study design and data collection. We use different textual analysis software platforms for different kinds of data. And we know when we design our studies the specific study activities that will be analyzed using AI text analytics and what activities will be analyzed by hand.

   While we have made some small adjustments to our data collection methods to accommodate software, and while software capabilities have inspired some new analytical approaches, our basic study design approach and workflow remains the same.

   The point here is simply that the software should be in service to your objectives and process. Put another way, we don’t look at text analytics as a new method. For us, it is a new tool that enables us to do what we’ve always done – reveal deep, human insights – but in ways that are more robust and efficient.

2. At the end of the day, it’s still coding
   Even though you’re using software, you’re still coding. You’re just using software to help you do it. If you’re trying to use text analysis software and you haven’t read a book on textual analysis or coding, I suggest you stop what you’re doing, take a step back and, at a minimum, gain a general understanding of the basic principles.  (“Qualitative Text Analysis” by Udo Kuckartz and “The Coding Manual for Qualitative Researchers” by Johnny Saldana are good places to start.)

   But short of reading a book, here’s a primer. Coding is a process of moving through varying degrees of abstraction from textual data, starting with the literal texts (sometimes called n-vivo codes) to categories to concepts and ultimately, explanatory theory. As soon as the analyst moves from the literal text to categories of text that group similar words, phrases, stories or documents together, they are bringing their own interpretation to the data, making judgments as to what those categories should be called, and what bits of text fit into each category. Moving from categories to concepts – which lend a higher degree of meaning to categories – represent another level of analytical abstraction.

   Now, the AI capabilities of text analytics software packages can aid this process by showing you not only the volume of exact matches in textual datasets but also the textual bits the AI determines is conceptually related. But be careful. AI often misinterprets the nuances of language and can sometimes be flat-out wrong, especially in the relationship between the conceptual matches it finds and the theory of your unique project.

   For that reason, we use AI text analytics as an aid. It helps us begin the process of wrangling data, categorizing and creating conceptual groupings but it doesn’t do all the work for us. Ultimately, it’s a human coder – enabled by AI text analytics – that codes the data. The AI just allows us to do it more comprehensively and quickly.

   One thing we’ve always told our analysts is that the software can help you organize your data set. It will help you get to know it really, really well and allow you to code with confidence. But software won’t find the insights for you.

3. Don’t lose the connection to the text
   The best textual analysis, we believe, builds from the text to create concepts and ultimately a “Grounded Theory” of human beliefs, behaviors and motivations. But your analysis and insights should never lose touch with the rich, textual data from which it came.

   It is easy when using text analytics software to become divorced from the source material; to stay at the level of quantification once textual data are quantified. But I encourage you to move back-and-forth from the high-level view of the data provided by text analytics to the ground-level view of the rich, text. And do this in your reporting as well. The best reports based on textual data blend a comprehensive, quantified picture of the data with rich, textual illustrations that bring the category and concepts to life.

So, there you have it – our take on deeper insights through text analytics. As always, thanks for reading and let us know what you think. Let us know what your experience has been using AI text analytics. How have you used text analytics software to find deeper insights? And what best practices have you developed?

You can reach us here.