Interpreting Google Analytics for Site Search

This article provides some guidance on understanding user behavior using what’s provided in Google Analytics. Specifically, it examines the statistics provided under the “search terms” page, found under “Behavior” -> “Site Search” -> “Search Terms”. For example, here’s a sample of OSC’s search terms in Google Analytics

alt text

Interpreting search terms

Perhaps most obviously, the “search terms” column show you what users are searching for. But its worth stopping to interpret what a search term might be telling you. For example, a search for “fancy restaurant dress code” on a restaurant search engine obviously points at a specific kind of use case for search. Instead of focussing on this specific search, think how it might actually represent a broader pattern: users come to search to look up the dress codes for specific fine-dining restaurants. Likely many search terms stand in for these broader use cases.

Search experts like to use the word information need to describe in greater detail what a user is looking for. This is another way of analyzing search terms. Sometimes a user with one information need will use many different searches. You’ve probably experienced this. If you’re shopping for a toy for your child, your information need may be something along the line of

“I want a toy for my son, hopefully educational, below $10, and nothing noisy”

Now usually we don’t sit and write all that down. Much of it may be unconscous. Rather, through the act of searching and refining, we hunt for something that might match our requirements. For example, if we’re trying to satisfy the above information need, we likely will search multiple times, using queries such as:

  • “educational toy”
  • “educational toy for boys”
  • “non electronic educational toy”
  • “non electronic educational toy” (adds price filter)

Eventually we’ll find the right toy using all the facilities of the search inderface and purchase the toy.

The goal in search analytics is to discover use cases and information needs. What sorts of things are people looking for? What patterns do they use to search for them? Can we give users better hints that put them on the right path? Are we dissapointing them or delighting them? Interpreting the analytics depends requires care: your specific knowledge of your users and content must be brought to bare.

Kinds of Information Needs

Its worth noting that search satisfies many kinds of information needs. Some information needs correspond to “single item” searches. They attempt to find the one right most relevant answer. Others are about gathering information and comparing across many relevant results. In other words: some users know exactly what they want; others want to use search to explore, compare, and research.

This is worth pointing out because it directly influences how you interpret analytics. Users that want the “one right answer” expect the first result to match. The ideal case for these users is they click the first result and appear to leave the result satisfied. A happy exploratory searcher, on the other hand, may issue many searches. They may click on every result. They may scan the results and visit the next page.

It’s important to consider whether your searchers are exploratory or single-item. Its likely your search application has a mixture of both: information needs with 1 right answer, and others that involve “shopping around” for different options. For a medical system’s site, trying to pick the right allergist corresponds to exploratory “shopping around” information needs. Support the user making a complicated the decision. Searching for a specific doctor’s name, for example, isn’t about shopping around. Its about navigating to the one exact answer.

Analyzing search terms, in summary

Always analyse search terms by considering:

  • What broader use case might it represent?
  • What might the true information need be?
  • Is the information need “known item” or “exploratory”?

Interpreting each stastistic

With a sense for what the information need behind each search query, let’s begin to take apart the statistics that Google Analytics gives us.

Total unique searches

Common searches should work well. But before focussing excessively on the most popular search term, consider how diverse your search term are. To determine this, examine the percentage of searches (the number in gray).

Some search applications have what some refer to as a long tail. This is a fancy way of saying the application has an unusually diverse set of search terms. The top terms represent a relatively low overall percentage. Other applications are referred to as short tail they are unusual in that only a handful of search terms drive most traffic. Perhaps the top 10 terms in total account for 25% of the searches.

An example of a long tail search application might be Google or Amazon. They must satisfy an extreme number of searches and no one search accounts for a significant proportion of all results. A short term search application might be one for a restaurant site. The top query might be “hours” or “menu.” These may account for 75% of the search traffic.

If you’re a short tail application, use the total unique searches as a way to prioritize those exact searches. For a long tail application, instead of focussing on the specific keyword searches, focus on patterns in information needs and use cases. The specific search terms should stand in as exemplars of a general use case to focus on.

Results pageviews/search

This column indicates how many pages (as in the user clicking “next page”) the searcher viewed. Viewing a single page (a value 1.0) here is often ideal. Viewing many pages is often seen as a negative behavior.

However this depends on the application and information need. Exploratory searchers might simply be performing an exhaustive analysis of the results. In a B2B e-commerce application, for instance, a purchasing officer may wish to comprehensively review every widget to ensure that the widget is a good price and meets all the requirements. This might point to usability improvements you can make to give users a single-glance comprehensive understanding of the results. It might also point at ways your search UI might enable decision making by highlighting important criteria users depend on for decision making.

For exact item searches (search for a specific doctor name, product part number, etc) paging can be deadly. It indicates a terrible case: the specified item wasn’t #1, it wasn’t even in the top 10! Paging for these users should be minimized.

As always, interpreting paging depends on trying to understand your users and their information needs!

Time after search & average search depth

Time after search and average search depth indicate how much the site was interacted with after clicking a search result. Usually higher time and search depth indicate that search isn’t a problem. However, it may indicate a problem with other parts of the site.

Single-item searchers may be experiencing something frustrating with the site. For example a difficult checkout process. Perhaps it should normally take two clicks to checkout, but the search depth oddly averages to 5 clicks. That may indicate a problem.

Users gathering information should be expected to spend considerable time after the search and on the next result. Searching for articles, the user should be expected to have a search depth of 1 following by several minutes reading. Additional search depth could mean clicking on a linked article that’s interesting.

It’s important to note one reason high search depth/time after search might point at poor search. It could indicate users give up on search and try the site’s other navigation features. For example, they might search on “mexican food near me” in a restaurant search. Maybe they didn’t like what they clicked on, so they began using the site’s other navigational features such as a browse to restaurant by genre.

Refinements and exits

Refinements are changes in search terms to alter, narrow, or widen the search. For example, in the “toy search” example above, many refinements were issued to find the toy that matched all the user’s requirements. Refinements aren’t always bad: they should be expected. But lets break down cases of good/bad refinements.

Expect a vague search to have many refinements. For example a user may start with “mexican restaurant” and refine to “mexican restaurant tacos” or “mexican restaurant in Brooklyn.” Just as above for the “toy search” the user refined search to further express the information need. Users refine to explore and understand a data set. Sometimes they know they’re using vague terms and as long as the search engine is keeping the conversation alive and acting in an understandable matter, then users are eager to keep exploring. One important thing to note is users need relevance feedback from the application to refine: they need good hit highlighting that demonstrates reasonably relevant results.

That being said, a hugh number of refinements combined with a high exit rate likely indicate a problem. It indicates users are in an area where the conversation between themselves and the search application has broken down. Also obvious, specific, single item searches (non exploratory) with high level of refinements are also problematic and likely point at a relevance problem.

One aside on refinements: highly performant search encourages more refinements. Fixing a performance problem may lead to more refinements. This is a good thing! The faster your search is, the more ability your users have to explore and find. As long as the application behaves sensibly with good relevance feedback, these refinements may be perfectly fine.

Other stats to track

These are stats not in google analytics that would be wise to track.

  • 0 result searches: Users have information needs they expect you to satisfy, yet you are unable to serve them. May also indicate misspellings or typos that you’re not handling.
  • Conversions: If you have explicit goals (such as a purchase) track whether search leads to these.
  • Click Depth: what position in the search results is clicked on. For example, if the users always click on the 5th result, the clickdepth would be 5. Particularly for single item searches, low click depth is preferred. Statistics such as mean-reciprical rank summarize the click depth across a set of searches.

Let the experts help!

OpenSource Connections wrote the book on search relevance! Let us help you improve your site search with analytics-driven tuneups where we guarantee to move your business metrics forward through search. Get in touch to learn more!

References & further reading