Recently, Advance Auto Parts contacted OSC to improve the search relevancy of their intranet application, Starting Line. Starting Line serves as the knowledge base for every store employee, so having relevant internal search results helps keeps employees connected with resources and company news.
Through our two day Quepid relevancy assessment, we helped bring together content experts and developers. Advance’s field communications team identified several problems with their Solr search results:
- Large number of search results, even for targeted queries Search queries for common forms returned spurious results simply because many items mentioned “form” somewhere in their text
- Important components outside of text matching, such as how recent the document was published or the popularity of a specific document, were not incorporated into the results
To begin our collaboration with the Advance authoring team, we constructed important use cases, and then used Quepid to describe the use cases in the form of executable tests. We then took a snapshot of how well the current search algorithm met up to these expectations, so later we could visually compare our progress over the rest of the two day assessment.
We used Quepid to explore the impact of several changes, including:
- Phrase Boosting so that, for example, if a user searches for “automatic transmission”, documents with the full phrase “automatic transmission” get boosted above documents containing just “automatic” or “transmission”
- Understanding how to correctly apply weights to fields
- Understanding how to allow each matched field to have a vote in the score (dismax usually lets the highest matched field count in the score)
- Strategies for applying a boost for date and popularity
As we tried each algorithm change, the use cases we captured earlier allowed us to easily test the impact of those changes, both positive and negative. More importantly, at every step Quepid gave us insight into why the relevancy algorithm produced the results we were seeing. Using snapshots, Quepid let us easily compare our final results with where we began, letting the customer judge our progress for themselves. To put it in the customer’s words:
As a content editor for our intranet, I’m very concerned when my work doesn’t show up in our Solr search results like it should. Until Quepid, getting to the root cause was rarely worth the effort. Investigation attempts would fizzle out in a confused haze of screen shots and waiting for re-indexes. But with Quepid, we are now working together like speed chess players, tweaking config settings, defining success and swapping out dozens of test queries in a flash – refresh and repeat. And, we don’t even have to worry that our Dev environment doesn’t exactly match Production, because we have a Production sandbox! Will Carter Field Communications, Advance Auto Parts
Do you want a highly relevant search like Advance Auto Parts? Do relevancy problems plague your search and you’re wondering how to start tuning? Contact Us! Whether it’s a quick tune-up like Advance or a long-term project with many requirements, OSC can use Quepid to make your search relevant.