Today’s edition of Productify is brough to you by Enterpret:
Get the Coda Template for customer insight reporting
Coda uses Enterpret to consolidate customer feedback, automate tagging, and generate customer insights. These insights help Coda ensure every piece of feedback helps shape a better product experience.
Each month, Coda shares insights from Enterpret in a Voice of Customer (VoC) Report to prioritize their roadmap and answer questions like, "What should we build next?" "How much does this bug bother people?" "How did the launch go?"
For Productify readers, Enterpret is sharing Coda’s Voice of Customer Report Template to make reporting on customer insights easier.
Remember the good old Net Promoter Score (NPS)?
To calculate NPS, simply ask your customers how likely they are to recommend your product on a scale of 0 to 10. Then, group their responses into Promoters (9-10), Passives (7-8), and Detractors (0-6). Finally, subtract the percentage of Detractors from the percentage of Promoters, and voilà—you've got your NPS score!
If you thought no one cares about Net Promoter Score (NPS) anymore when trying to measure customer experience, here are some mind blowing facts😯😯:
According to a Wall Street Journal analysis, “net promoter” or “NPS” was cited more than 150 times in earnings conference calls by 50 S&P 500 companies in 2018. Of all the mentions the Journal tracked, no company ever said its NPS declined.🤥(1)
More recently in 2020, nearly two-thirds of all Fortune 1000 companies use NPS to measure overall customer satisfaction and sentiment (2)
In today’s edition, we will try to make the world a better place.
I am not saying NPS is just pure evil, but that in the list of preferred metrics it is for me at the very last.
And I will explain why you should stop using it already (specifically in a scaled D2C company where you offer your product or service to thousands and millions of customers) or at-least you should track many other things besides just NPS as part of your metrics portfolio.
And unlike others who only crib about NPS, I am going to share an alternative approach that worked for me in a company with millions of active users per month scale. Stay tuned.
Don’t forget to share this post and save your friends in other companies from the NPS black hole!
Register for the Live Session: Product Strategy 101
In this live session of 1.5 hours, you will learn:
1. At which career level of Product Management do you need to start owning product strategy?
2. How to go about forming your first product strategy? (+TEMPLATE included)
3. How to test your product strategy? Is it working?
4. Pitfalls to avoid when strategizing
For paid subscribers of Productify, all product courses and live sessions (once per quarter) are absolutely free + access to all past courses. Upgrade to paid and get this session 100% free:
💥💥Truth Bombs about NPS
Customer’s don’t always have to/need to recommend features
A customer might be highly satisfied with a product's features and functionality but not feel compelled to recommend it to others.
For example, a person who uses a productivity app to manage their personal tasks may find it extremely useful and be very satisfied with their experience. However, they may not actively recommend the app to their friends or colleagues because they don't believe it would be relevant to those individuals' needs.
In this case, the customer's high satisfaction level is not reflected in their NPS score, as the metric focuses solely on the likelihood of recommendation rather than overall satisfaction.Surveys are filled by extremely happy or extremely angry customers
Imagine a scenario where a company sends out an NPS survey after a customer service interaction.The customers who are most likely to respond are those who had a particularly positive or negative experience. A customer who had a neutral experience or was simply satisfied with the interaction may not feel the need to provide feedback.
As a result, the company may receive a disproportionately high number of responses from either very satisfied or very dissatisfied customers, skewing the NPS results. This self-selection bias can lead to an inaccurate representation of the overall customer experience.NPS metric gives zero insights about the Why
Consider a situation where a customer scores a company low on their NPS survey. Without any additional context or qualitative feedback, the company may struggle to understand the reasons behind the customer's dissatisfaction.
For instance, a customer might have had a negative experience with a specific product feature or encountered a bug during their interaction. If the customer doesn't provide further details, the company may not have enough information to address the root cause of the issue and improve the customer's experience going forward.Overemphasis on Promoters and Detractors (Lovers and Haters of your Product)
In a competitive market, a company might focus heavily on converting passive customers into promoters to gain a competitive advantage.
However, this approach can overlook the potential value of passives. For example, a customer might be satisfied with a product but not feel strongly enough to actively recommend it.
With the right engagement strategies, such as targeted communication, personalized offers, or improved customer support, the company could potentially convert this passive customer into a promoter.
By solely focusing on promoters, the company may miss out on opportunities to strengthen relationships with passive customers and drive long-term loyalty.NPS is so static
A company might conduct an NPS survey after launching a new product feature or implementing changes to their customer service process.
If the survey is conducted at a single point in time, the company may not capture the full impact of these changes on customer sentiment.
For instance, a customer might initially be dissatisfied with a new feature due to a learning curve but eventually become satisfied as they become more familiar with it.
By measuring NPS at a single point in time, the company may not recognize this shift in sentiment and miss opportunities to provide additional support or make further improvements to the feature.
Introducing Sentiment Pulse Checks (What we did at Gojek back in 2022)
In my experience working at a large decacorn Gojek (now part of GoTo Group), we made the conscious decision to move away from using Net Promoter Score (NPS) as our primary metric for measuring customer satisfaction in our Direct-to-Consumer (D2C) scaled product.
Instead, we implemented a more nuanced approach that I believe provides deeper insights into customer’s real-time experience with you, rather than static survey results. The approach works better for scaled D2C companies where there is active involvement of your customers across customer service, social media, trustpilot and other channels.
Today, customers don’t always say good or bad things about your products directly to their neighbors or friends, they anonymously rant about it on the internet.
Our Alternative: Sentiment Pulse Measurements
Instead of NPS, we adopted quarterly sentiment or "perception measurements." This approach allowed us to assess user sentiment on key topics over time, providing a more comprehensive view of customer experience. Here’s how it works:
Step 1: Define Categories and Topics
Identify key categories relevant to your product or service. Common categories for my product team at Gojek was something like:
Onboarding
Payments
Refunds
Customer Service
Fees and Transparency
Step 2: Data Collection
Gather feedback from multiple sources, such as:
Social Media: Monitor platforms like X and Facebook for mentions and comments about your service.
App Reviews: Analyze reviews on app stores (Google Play, Apple App Store) to capture user sentiment.
Customer Service Interactions: Review tickets, chat logs, and call transcripts to assess customer sentiment during support interactions.
Step 3: Sentiment Analysis
Utilize sentiment analysis tools to categorize feedback into positive, negative, and neutral sentiments. Here’s how to do it:
Preprocess Text Data: Clean the data by removing irrelevant elements (punctuation, stopwords) and tokenizing the text into manageable pieces.
Apply Sentiment Analysis:
Use sentiment analysis tools (e.g., Qualtrics, Lexalytics) to assign sentiment scores to each piece of feedback. This can be done using predefined sentiment lexicons that classify words as positive, negative, or neutral.
For example, a review stating, "The onboarding process was smooth, but the fees are high," would yield:
Onboarding: Positive
Fees: Negative
Calculate Topic-Specific Sentiment: For each piece of feedback, determine the sentiment for each relevant topic. This allows for a more granular understanding of customer perceptions.
Step 4: Categorize Sentiments
Group the results by topic and sentiment type. For example, after analyzing 1,000 pieces of feedback, you might have:
Step 5: Set Measurable Goals
Based on the categorized sentiments, establish quarterly goals to improve specific areas. For instance:
Onboarding: Maintain or increase positive sentiment above 75%.
Refunds: Decrease negative sentiment from 50% to 40% by implementing clearer refund policies.
Fees: Reduce negative sentiment from 65% to 50% by enhancing transparency about fees.
Step 6: Continuous Monitoring
Implement a system for continuous monitoring of sentiment trends. This can involve:
Regularly scheduled sentiment analysis (e.g., quarterly).
Real-time monitoring of social media and customer service interactions to catch emerging issues early.
By shifting to perception measurements, we gained several advantages over traditional NPS metrics:
Actionable Insights: We could identify specific pain points and address them directly, leading to tangible improvements in customer experience.
Real-Time Feedback: Our approach allowed us to monitor sentiment in real-time, giving us the ability to respond quickly to emerging issues.
Comprehensive Understanding: We moved beyond a single score to a multifaceted view of customer sentiment, which informed our product development and customer service strategies.
Predictive Capabilities: Advanced sentiment analysis can help predict customer churn by identifying negative trends in customer feedback before they lead to attrition. This proactive identification allows businesses to address concerns and retain customers
What do you need in your company to setup sentiment or perception measurements and tracking?
Strong Research Culture
Understanding Research Methodologies: Teams should be familiar with qualitative and quantitative research methods to design surveys that yield reliable and valid data.
Survey Design Expertise: Knowledge of how to craft effective survey questions that minimize bias and maximize clarity is crucial. This includes understanding how to structure questions to elicit useful feedback.
2. Data Analysis Skills
Statistical Analysis: Team members should possess skills in statistical analysis to interpret survey results accurately. This includes understanding metrics like mean, median, mode, and standard deviation.
Sentiment Analysis: Proficiency in sentiment analysis tools is important for interpreting qualitative feedback and understanding customer emotions and perceptions.
3. Tracking and Monitoring
Performance Metrics: Teams need to establish key performance indicators (KPIs) to track changes in perception over time. This involves setting benchmarks and goals based on initial survey results.
Continuous Monitoring: Skills in using analytics tools to continuously track customer sentiment across various channels (e.g., social media, reviews) are essential for ongoing assessment.
4. Cross-Functional Collaboration
Interdepartmental Communication: Effective collaboration between product, marketing, customer service, and research teams ensures that insights from perception surveys are integrated into broader business strategies.
Feedback Loop Implementation: Teams should establish processes for sharing survey results and insights across departments to foster a culture of continuous improvement.
References: