In this post, we are going to go deep on how an analytical vector database can speed up the process of solving a problem in the eCommerce space. In this case, the problem we are focused on is understanding the key drivers for negative reviews. To help demonstrate the capabilities of structured queries using vectors, we will examine this dataset of women’s clothing reviews which is available from Kaggle.
Exploring the data
If we put ourselves in the shoes of an eCommerce clothing supplier, we are trying to find the two or three most impactful things we can do immediately to curb negative reviews on our products. With that in mind, the first thing we want to do is get a sense for the size of potential opportunity.
In the screenshot below, we can see that our catalog has a long tail: our top 7 classes out of 21 total accounts for more than 80 per cent of our reviews. So we’ll start by filtering out all of the classes with fewer than 1,000 reviews so we can focus on the biggest opportunities.
That leaves us with a manageable seven classes to explore. We can see here that Dresses, Knits and Blouses have by far the most reviews. This is likely a function of them also being the highest selling items.
In all likelihood, Dresses, Knits and Blouses will be where we can most quickly identify opportunities just given that they have the highest reviews which, as we already discussed, is likely a proxy for sales. So let’s look at three star reviews and below as a share of total reviews to see if there are any outlier classes. (We included three star reviews in this case because from the prospective of a potential customer, three stars likely puts them off purchasing. Would you be excited to buy something that someone else rated three stars?)
We can see that classes are in a narrow band between 17 and 25 per cent negative reviews. Given that there are no obvious outliers, let’s go deeper into dresses.
The power of querying unstructured data
Up until this point, our analysis has been quick and easy. The above segmentations using Excel and Pivot Tables took about 15 minutes. This is the point where pain starts to enter the process of extracting insights from reviews: you have to read and classify them.
In Dresses alone, we have 222, 459 and 830 reviews with 1, 2 and 3 stars respectively. To gain an analytical insight into those reviews, traditionally we would need to read each of them and assign some kind of taxonomy to generate analytical outputs. For example we might tag reviews as “poor quality”, “overpriced” or “late delivery” and then analyze those totals to understand our biggest opportunities for improvement.
But even those seemingly helpful categories are actually problematic. How, for example, do you disentangle issues of quality and pricing? In some cases, a reviewer might say something is overpriced as a proxy for it being low quality. As in, it is overpriced given the quality. Alternatively, the quality is poor given the price.
What we really want to be able to do is reliably aggregate reviews based on their meaning. And that’s what analytical queries over vectors allow us to do.
In this case, we do still need to read some reviews, but only to get us started. Let’s look at the very first 1 star rating in the dresses category in our dataset:
I was surprised at the positive reviews for this product. its terrible! it cuts you in a weird place to make you look wide. the skirt is also not like the picture. its darker and heavier. the material isnt great. i had to return.
While there is a lot in here, we can quickly identify a critical issue that we should be able to address: the item the purchaser received isn’t what they expected based on the picture on the product display page.
In both physical and online retail, product photos and information about the product such as dimensions are known as “merchandising”. Merchandising is something that a supplier can control. We could take photos of our items in different lighting to help customers understand what they are buying.
But before we do that, we want to understand how widespread is this problem? How many of our negative reviews are the result of the item not matching expectations based on the picture?
Let’s answer this question using a structured search across the vector embeddings of our reviews. We are going to search for the sentiment that the item didn’t match the picture, in the Dresses class, across all reviews of 3 stars or less. Using NNext, this search looks like so:
So, how did we do? Our query returned 21 results in total. Many of these results match quite well, for example:
I returned the dress because the color was not as pictured in the online photo. i did not like the color.
And:
I should have listened to previous reviewers. but the dress looks so good on the model in the two dimensional pictures. when it arrived in the mail, i realized this is a mistake. it has no shape, material is poofy and feels cheap. it is going back.
These two reviews capture what we are looking for: the product that these purchasers received did not match their expectations based on what they saw on the website.
But these reviews also highlight a challenge. There was a mismatch of expectations, but for different reasons. In once case, the item didn’t match the photo because of color. In another, it did not match because of fit! This suggests we should rerun our query, but we should try to be more specific about the cause for mismatched expectations. Let’s be more specific:
We can now target the offending items for new merchandising to see if, by fixing the expectations around fit and color, we can increase their sales volume.
What we were able to achieve in just 30 minutes leveraging semantic search across vector embeddings of reviews might have taken hours of manual inspection to dig out the products that have problems with a mismatch between the product photo and the product itself.
Future capabilities
One of the limitations you can see from this exercise is that you need to bring a hypothesis to customer reviews: if you know what you’re looking for, semantic search can help you find all of those instances.
Beyond manually querying, one of the most requested features we have had is the ability to proactively notify users when various events happen within their dataset. This is a capability on NNext’s roadmap not just in the eCommerce domain, but across unstructured data generally.
In the meantime though, hopefully this exercise has helped demonstrate the power not just of vector embeddings, but of unstructured data and the value of the insights it holds. While NNext is currently pre-release, you can learn more by contacting us at founders@nnext.ai.