It’s Tuesday morning. You have a meeting tomorrow to review next month's product roadmap and priorities. You're staring down a list of requests from sales, issues from support, comments on recent reviews from the CEO, and ideas from that one overly creative colleague, Brad.
What do you do now?
If you value customer feedback, you'll want to organize that pile of feedback by theme and a value metric, then prioritize what fits your product vision and drives the most value.
But unfortunately it's not that easy. If you're lucky enough to have tons of feedback, it's a huge pain to organize. If you're not as fortunate, like in the case of B2B teams, you have to go digging for feedback in longform content like recorded calls, meetings, and CRM notes. And after you've found it, it's a huge pain to organize.
When it comes to analyzing calls, for every 1 hour of text transcript it takes a researcher about 3 hours to analyze. The bulk of that time is spent extracting notes, quotes, and highlights from the text.
In this post we’ll discuss how CustomerIQ uses a family of AI models, including Large Language Models, to shorten the time it takes to extract, organize, and summarize product feedback from hours to seconds.
The Challenge of Extracting Highlights from Longform Text
Extracting highlights from longform text presents significant challenges due to the nature of analyzing lengthy transcripts and reviews. The sheer volume of information contained within these texts can be overwhelming, making it difficult to extract meaningful insights efficiently, especially considering how exhausting it can be to manually read through.
Traditional approaches to analyzing longform text, such as manual reading or basic keyword searches, fall short in extracting the broader context and uncovering valuable insights. Manual reading is subjective, prone to biases, and inefficient when dealing with large volumes of text. Basic keyword searches can be limited in capturing the nuanced meanings and contexts embedded within the feedback.
Without effective techniques in place, organizations may miss out on crucial insights hidden within longform text, impacting their ability to make informed decisions and improve their products or services based on customer feedback.
Fortunately, advancements in natural language processing (NLP) and the utilization of Large Language Models (LLMs) offer a solution to overcome these challenges. LLMs provide powerful capabilities to automatically extract key points, sentiments, and themes from lengthy customer feedback, enabling organizations to unlock valuable insights efficiently and at scale.
What is an LLM and how do they work?
An LLM is an artificial intelligence model that has been trained on a vast amount of text data. These models are designed to generate or synthesize human-like text based on the input they receive. They do this by learning the statistical patterns in the data they're trained on. For example, if a model is trained on a lot of English text data, it will learn the general rules of English grammar, common phrases, and even some factual information, all from the patterns in the data.
At a high level, here's how these models work:
- Data Collection: The model is trained on a large corpus of text data. This data can come from a variety of sources, including books, websites, and other texts.
- Tokenization: The training data is broken down into smaller pieces, called tokens. In English, a token could be as short as one character or as long as one word.
- Model Training: The model is then trained to predict the next token in a sequence based on the tokens it has seen so far. This is done using a type of model called a transformer, which allows the model to pay attention to different parts of the input when making its prediction.
- Prediction: Once the model is trained, it can generate text. You give the model some input (like the start of a sentence), and it generates the next part of the text based on what it learned during training. It does this by calculating the probability of each possible next token and choosing the one with the highest probability.
- Fine-tuning: Sometimes, the model is then fine-tuned on a smaller, more specific dataset. This helps the model perform better on specific tasks or understand specific types of data.
How LLMs can extract key phrases and important points from long text
One of the remarkable capabilities of LLMs is their capacity to understand context. They can comprehend the meaning of words and phrases based on their surrounding context, which enhances their ability to extract meaningful highlights from longform text. LLMs consider the broader context of the feedback, including the relationships between words, sentence structures, and even the overall document, to capture the intended message accurately.
When we say that a large language model (LLM) "synthesizes" text, it typically means the model is being used to produce text that serves a specific purpose or fulfills a particular task. In the case of CustomerIQ, that task is to extract the important phrases out of longer bodies of text (customer feedback).
While the underlying mechanics of the model don't change, the way we use the model does. This use case involves the model taking in some input (the text submission), processing it, and outputting a response that is contextually relevant and fulfills the task - summarizing the most important parts.
When we ask the model to extract highlights from longform text, it doesn't simply generate text based on the patterns it has learned; it tries to understand (in a statistical sense) what we mean by “highlight” and provide a relevant answer. It uses the patterns it has learned to synthesize an answer that's appropriate for the input question.
With this contextual understanding, LLMs can identify not only explicit key points but also subtle nuances, implied sentiments, and underlying themes embedded within the text. This enables organizations to gain a comprehensive understanding of customer feedback and extract deep insights that were previously challenging, or exhausting, to uncover.
LLMs represent a significant breakthrough in text extraction. Their advanced natural language processing capabilities, combined with their contextual understanding and ability to extract meaningful insights, empower organizations to leverage longform text effectively and uncover valuable information that drives product improvements and customer-centric decision-making.
How we use LLMs for insight extraction at CustomerIQ
The magic behind our Folders at CustomerIQ is in our use of LLMs to process and analyze the longform text submissions.
When data is submitted to a folder, a number of key functions are run:
- Non-text media is converted to text
- We split long-form text into digestible chunks called snippets
- We task an LLM to extract key points from each snippet called highlights
- We transform each insight into embeddings
Later, in Views, we're able to organize all these highlights automatically by theme, classify them into different categories (ie. UX issues, feature requests, bugs, etc.), or search and monitor specific topics like "What are people saying about the new pricing model?"
Here’s a bit more detail on each step:
Transform non-text into transcribed and diarized text
CustomerIQ initiates the text extraction process by transforming non-textual feedback into transcribed and diarized text. This involves converting audio recordings, video content, or any other non-textual formats into written text. By transcribing and diarizing the feedback, CustomerIQ ensures that all data is in a text format, ready for further analysis.
Break down longform content into snippets
To tackle the challenge of analyzing lengthy customer feedback, CustomerIQ breaks down the text into manageable chunks called snippets. This step allows for a more focused analysis, prevents information overload, and enables efficient processing of each section. Breaking down the text also enhances the effectiveness of subsequent steps in the extraction process.
Use LLMs to extract highlights from each snippet
CustomerIQ leverages the power of LLMs to extract crucial key points from each text snippet. By utilizing LLMs' advanced natural language processing capabilities, the system can identify the most significant and relevant highlights within the text. These key points capture the essence of the customer feedback, including important themes, sentiments, and specific details that inform product improvements and decision-making.
Transform highlights into embeddings for analysis
The transformative role of CustomerIQ's process lies in the conversion of highlights into embeddings. Embeddings are numerical representations of textual data that capture the semantic meaning and relationships between words or phrases. By transforming key points into embeddings, CustomerIQ unlocks the potential for advanced analysis techniques such as cluster analysis, classification, and semantic search.
What this means for customer research: realizing the value of every customer interaction
Considering the speed, accuracy, and scalability of using LLMs to extract highlights, the question shifts from, “how do we analyze feedback?” to “Where can we find more?”
Consider for a moment all the different interactions members of your team have with customers where customers might provide insight into future product or marketing opportunities. Support calls, sales calls, customer success calls, survey responses, support tickets, product reviews, tweets, Reddit posts, community forums, focus groups… the sources are nearly endless.
Using folders, you could store each of these interactions, pulling out relevant highlights and converting them into embeddings.
Using views you could discover themes and summarize the value of each theme for prioritizing work.
Using a new view, you could monitor a particular theme to watch how it changes over time.
And you can do all of this in minutes.
