Two Principles for Innovation in Market Research (And How AI Can Help)
What do a legendary ad man, a Grammy-winning music producer, and a Nobel Prize-winning psychologist have in common? They all know the keys to innovation in market research.
Our clients describe us as an innovative market research company. They come to us for custom studies to answer complex questions. Recently, I was asked, “How exactly do you do that?” I didn’t have a pithy answer at the ready. Designing custom approaches is just something we do—I never stopped to think about how we do it.
So I sat down and started writing the principles that define our innovation process. As I wrote, I not only unearthed what underpins innovation at RealityCheck, but I also found myself referring to key books—by advertising legend James Webb Young, music producer Rick Rubin, and pioneering psychologist Daniel Kahneman—that have sharpened my thinking.
RealityCheck’s approach to innovation is built on two key principles:
- Innovation market research is fueled by broad exposure to diverse ideas.
- The spark of innovation often emerges from working within constraints.
1. Innovation is fueled by broad exposure to diverse ideas.
On my first day in advertising, I was handed James Webb Young’s A Technique for Producing Ideas. Published in 1965, it showed me that creativity isn’t a mysterious gift—it’s a process.
Webb’s core insight is that new ideas are new combinations of old ones—a principle that has guided generations of creative professionals. Our minds are like vast libraries of specific details and broad concepts. Creativity happens when we connect these in unexpected ways. It’s not about inventing from scratch but about linking abstract ideas to concrete problems to generate fresh perspectives.
In market research, this philosophy is especially powerful. Innovation isn’t just about broad exposure to ideas—it’s about recognizing hidden relationships between concepts and applying them in practical ways. At RealityCheck, we dedicate ourselves to continuous learning, maintaining a curated content library and holding regular “Craftwork” sessions to share, discuss, and catalogue insights. This gives us a rich reservoir to draw from when addressing complex research questions.
But the art—and fun—of study design is applying general theories from this reservoir to specific problems. Here are some examples:
- Narrative Identity: We applied Dan P. McAdam’s theory of Narrative Identity to develop an approach to psychological interviews that reveal deep consumer beliefs, motivations, and tensions.
- The Remembering Self: Using Dan Kahneman’s insights, we developed a storytelling method for a banking client to explore how the remembered experience of the pandemic shapes financial decisions.
- Emotionality and Consumer Behavior: We used research by Matthew Rocklage, Derek Rucker and Loran Nordgren on the relationship between emotionality and consumer behavior to decode the emotional drivers behind brand choice for a major CPG company.
- Cultural Dimensions: We applied Geert Hofstede’s framework to cross-cultural interviews for a global beverage company, creating a validated comparative structure for analysis.
2. The Spark of Innovation Emerges from Working Within Constraints
Rick Rubin believes creativity thrives under constraints. In The Creative Act: A Way of Being, he argues that limitations aren’t barriers but springboards for innovation.
“Whether imposed by design or necessity, it’s helpful to see limitations as opportunities… Novel problems lead to original solutions.”
Rubin compares constraints to setting a palette for a project—when options are limited, you’re forced to be more inventive. He illustrates this with Yves Klein, a painter who limited himself to one color and, in doing so, discovered a shade of blue the world had never seen before.
In market research, our constraints come in the form of standards like validity, reliability, and generalizability. These ensure that we measure what we claim to measure, produce replicable results, and generate findings that extend beyond our study sample. Rather than hindering us, these constraints drive us to be more creative, helping us discover better ways to design research.
Inspired by the Constraint of Noise
One constraint that has sparked innovation in our qualitative research is noise, a concept introduced by Dan Kahneman in Noise: A Flaw in Human Judgment.
Like bias, noise is a form of error—but while bias skews results in one direction, noise is the random variability in human judgment that leads to inconsistent decisions. In qualitative research, especially in business settings, researchers and clients often make on-the-fly judgments about what’s being heard in focus groups or interviews. This introduces noise, which can lead to flawed conclusions.
Rather than seeing noise as a limitation, we treat it as a constraint that forces us to innovate. To address this challenge, we’ve leveraged cutting-edge AI-powered text analytics to sift through transcripts, ensuring that what observers “heard” is actually supported by the data. We also use multivariate statistical analysis to uncover patterns and scale qualitative research, transforming what could be seen as a limitation into an opportunity for more trustworthy insights.
How AI Enhances Innovation in Market Research
AI naturally complements our two principles of market research innovation: broad exposure to diverse ideas and creativity within constraints. By integrating AI into our process, we can amplify both principles—enhancing our ability to absorb, analyze, and apply a wide range of theories while also using structured constraints to improve research rigor.
For innovation through broad exposure, AI helps us process and synthesize vast amounts of information efficiently. We use AI to analyze our curated content library, scanning academic research, industry reports, and consumer insights at scale. AI can surface unexpected connections, suggest relevant frameworks, and even identify gaps where new thinking is needed. This allows us to move quickly from general theory to specific applications, making study design more informed and dynamic.
For creativity within constraints, AI enhances research validity and reliability by reducing bias and noise. AI-powered text analytics help systematically analyze qualitative data, ensuring insights are grounded in patterns rather than individual interpretation. Machine learning models can be trained to recognize inconsistencies, uncover hidden themes, and validate findings across different qualitative and quantitative datasets.
By leveraging AI, we don’t just innovate for the sake of novelty—we innovate with discipline, ensuring our methods remain both creative and methodologically sound.
Bringing It All Together
At RealityCheck, innovation in market research isn’t about chasing trends or creativity for the sake of novelty. It’s about answering complex client questions by applying theory in interesting ways—while staying true to sound social science research practices.
By combining these principles with AI, we take market research innovation even further. AI helps us process and apply vast amounts of knowledge, making it easier to move from broad theories to specific research solutions. At the same time, AI enhances research rigor by reducing error and noise and reinforcing methodological constraints. The result is a research process that is both creative and precise—allowing us to deliver insights our clients can act on with confidence.
Ultimately, innovation in market research isn’t about abandoning structure—it’s about using structure in smarter, more dynamic ways. Whether we’re designing studies rooted in psychological theory, leveraging AI to enhance analysis, or embracing creative constraints to drive better solutions, our goal remains the same: to provide clear, actionable answers to the most complex research questions.
I hope you found this article useful. Let’s hear from you. What’s your process for innovation? Where do you find inspiration? How do you integrate AI into your innovation process?
