Welcome, retail aficionado, this guide is for you! We will outline all the things you need to know about measuring the quality and effectiveness of your store’s site-search with the magic of key performance indicators (KPIs). In three parts, we will walk you through understanding your customers, how they interact with your products, and how they search to find what they need. In a transactional storefront, an increase in search quality can be directly correlated with better outcomes for your business. More narrow than e-commerce KPIs as a whole, these focus on measuring the success of site-search and how it aligns with wider business goals.
Overview
We will cover three sections of KPIs – customer level, item level, and query level. The customer level proposes ways of measuring the overall effort that a customer must go through to find what they are looking for. The item level covers the value delivered by the products themselves. The query level proposes ways of measuring search quality for multiple products together.
We then align these three areas to improve the following business level goals:
- The experience of the customer (to increase conversions and likelihood of returning to the store)
- The quality of the items being presented (to increase satisfaction of purchase)
- The success rate of a purchase through search (to increase revenue)
Two objectives!
You need to balance between making customers happy and meeting business goals. Conventional wisdom says that happy customers will result in a successful business. But if the customers are happy because they are only buying products at low margin, that’s not good for you.
To overcome this challenge, we consider relevance for both the customer and your business. A product may seem perfectly relevant, but it may fail to meet the contextual criteria of the customer or be detrimental towards business goals.
Additionally, while you want to maximize conversions – you also want to maximize the conversion value for the business, as well as customer satisfaction with the product.
Some examples:
- If there are two perfectly relevant products for a query, one of which is a generic brand for the same cost as a partnering brand promotion, you want the latter to convert more than the former. This strengthens the partnership and improves business goals.
- If there are two perfectly relevant products for a query, one of which has poor reviews, and the other more positive reviews, you may want to convert the latter. The former might result in a product return or detriment to your brand.
Knowing these areas through data that you can gather will help increase revenue and satisfy customers.
Let’s quickly hop back into a physical store to draw some parallels. As a store manager, watching customers enter your store, go through a buyer’s journey, and leave. You would be spending a lot of time figuring out what your customers want and placing products on shelves appropriately to make their experience efficient and pleasant. If you did this, they’d come back when they needed something again, and they will tell their friends and family about their experience. If the store was a mess, and items were presented to customers randomly and without forethought, they would be confused and leave the store quickly, never to return.
No more concierge
You don’t need me to tell you this, but I need to make it obvious. You don’t have a personalized customer service agent in your digital store that greets every customer on their way in and can answer any question they have (within reason). BUT, you do have a search bar, which is usually the closest thing you’ll get. It’s not the same – customers don’t talk to a search bar like they talk to a store rep. Customers won’t run into your store, shout SHIRT WITHOUT STRIPES at you, and run away rudely if not answered instantaneously. Well, there’s always that one, but as a whole most people don’t do that. Unfortunately, that’s how people search on your store, and aside from rudeness, the problem is there’s a lot missing in the digital interaction.
Even if someone did come up to you in person, shouting SHIRT WITHOUT STRIPES, waiting impatiently, you could see who they are, what they look like, and make instant recommendations to them based on their appearance, size, likely taste, and escort them efficiently. In other words, you wouldn’t blindly take an anxious person in a tuxedo over to the streetwear department.
So if someone shows up at your search bar, types SHIRT WITHOUT STRIPES, then where should you take them? That’s the problem we’re going to solve over this journey.
Part 1 – Customers
As with all things e-commerce, you need to understand friction. Friction is *the* canonical measure of frustration for customers. High friction, and therefore frustration, will lead to abandoned carts, a lower likelihood of returning, and detraction when discussing your store with their friends and family.
Friction, of course, can happen in any number of places the customer interacts with your store – navigating, onboarding, payment, and of course – search. If you are unable to make your search bar and search results a relevant, fast, and pleasant experience, you will miss opportunities for sales.
I know, there are challenges. But you can fix search! You can measure and react to these things, and you can improve, and it’s easier than you think. First, you need to know this: How satisfied are your visitors? Satisfaction is subjective, so we want to make sure friction is decreasing.
Sessions
To break friction into understandable chunks, we need to know about sessions. We define a session as all the actions a person takes along their buyer’s journey to get a single item they need. So if someone needs both shirts and towels, and they go through several actions until they buy a shirt, that’s one session. When they go through more actions to then buy towels, that’s another separate session.
Since we’re covering site search, we’ll focus on *search* sessions – any session that involves a site search along the buyer’s journey, and we start the session from that first search action.
Identifying when sessions end in analytical data can sometimes be tricky, so we look for various triggers that signify a session has ended, or another has begun. Here is a list of actions that you should use to split your sessions up:
- Conversion (easy!)
- Abandonment (did the visitor close the tab?)
- Idleness for a long period of time, usually 30 minutes of inactivity (this is the default for Google Analytics)
- Reset of search and filters (perhaps by navigating to the store’s home page or to a fresh search page)
Conversions
We’ve mentioned conversions before, but we need to formally define it so we’re all on the same page. A conversion is a successful outcome along the buyer’s journey. That means they got what they needed. Usually, that’s when the visitor ends up paying for all the stuff in their cart. However, there could be things between the cart and the purchase that we shouldn’t measure when thinking about search! For example, what if the customer sees the shipping cost, and abandons their cart? That is usually not the fault of the search experience. So we’re going to do something interesting and important for our purpose – we’re going to use add-to-cart as a conversion! When someone adds a product to their cart, we signify that as success for search – they found what they were looking for and they have the intent to purchase.
Show me the KPIs already!
OK! Enough intro. Let’s get to it. We measure these things to understand friction for searching visitors:
Conversion Rate for Site Search
We start with the most obvious. When someone searches your store, do they purchase more often or less often than other channels? Looking at this metric as a whole is a good overall view of search quality. This is the best known metric for understanding if search is helping or harming the buyer along their journey.
Calculate: You get this indicator by dividing the number of search sessions that convert by the total number of search sessions.
Goal: This should be as high as your organic Google search conversion rate!
Abandonment Rate for Site Search
How often do visitors bail out from a search page or from a product detail page they arrived at from search? Knowing this is extremely important, since a higher the abandonment rate shows something is not going well with search. You may be showing them irrelevant results, or friction can be too overwhelming and the buyer gives up.
Calculate: You get this indicator by dividing the number of search sessions that are abandoned by the total number of search sessions.
Goal: Get this as low as possible. You will never get it to zero, but that’s OK (some abandonment can be attributed to price comparisons or fantasy searching).
Site Search Actions Until Conversion or Abandonment
Do visitors need to search or refine a lot to find what they are looking for? You want this ratio to go down, and the number of conversions to go up! Actions are defined by the following – filtering the search results, adding more terms to the query, or performing any other actions in the same session until they either convert or abandon.
Calculate: You get this indicator by identifying an entire session and counting how many actions were taken during that session. You then take the average number for all sessions over a given time period.
Goal: You want to make sure this number goes down, but for some queries it’s OK if it is high. For example, when people start with broad queries, like ‘TV”, they’re just starting on their exploratory journey and it could take a while for them to decide. However if someone does a targeted search for “65-inch Roku TV”, that narrow search should be completed very quickly with few actions in the session!
Return Rate for Site Search
When someone searches your store, are they more likely or less likely to come back again? This varies depending on the product of course, but for consumables you want them coming back frequently. This indicator will show whether people found your store pleasant or not. If they are coming back, that’s a good thing!
Calculate: You’ll need a way to know if a buyer has returned or not, typically with a cookie assigned to the visitor. When you have that in place, you can count the number of times the buyer has returned based on their activity. You average this over all cookie IDs over a given time period. This should be calculated as a percent – the ratio of buyers that returned at least once.
Goal: You want this number to remain high, as close to 100% as possible, but not everyone will come back. Note that this is a lagging indicator – so you’ll need to wait for a period of time to see if any changes you made have resulted in a successful increase.
Growth Rate for Site Search
Are you gaining more visitors and are they searching successfully? Every visit contributes to the reputation of your business. As more people visit your store, you want them walking away with a good attitude and a likelihood that they will recommend your store to others.
Calculate: This one is easy! Just look at the total number of visitor cookie IDs over a certain period of time, and see if the number is going up or down.
Goal: Growth indicates success as a lagging indicator, and the more people that visit it means that your external campaigns and promoters are having a positive impact on your business.
To be continued…
Those five indicators above are a great start on your quest to data-driven improvement of your store, but they are not everything. They are very useful as an overall health check, but there is plenty more detail that we will need to look at for solving specific problems related to site-search. In the next part of this guide, we will discuss Product Level KPIs, before concluding with a dive into Query Level KPIs. Thanks for reading, and see you again soon!