Migrating from one search engine to another can introduce significant business risk. At the KM World conference in Washington DC in November 2023, Annie Michels and Theresa Simek of EY described how they had used OSC’s Quepid tool as part of a successful search migration project. Quepid is open source and also available as a free hosted service.
At EY, the company re-platformed Discover, its global knowledge platform, moving from SharePoint to Elasticsearch. Simek and Michels discuss the pitfalls and lessons learned, as well as how EY tackled relevancy tuning in the new environment.
Abstract from a talk ‘Issues in Migration’ at KMWorld 2023
You can download the slides from their talk where they note that Quepid allowed them to carry out targeted testing on their most popular queries and to calculate overall search quality metrics such as NCDG@10. On the last slide you can see the progression of improvement of the scores recorded in Quepid as the project progressed:
- Using the legacy system, users reported accurate and relevant results at a 61% success rate
- The beta release of the new search system, based on Elasticsearch, gave a 56% succcess rate
- After initial tuning, release 2 achieved a 74% success rate
- Release 3 recorded 89% of queries achieved a Good (2) or Perfect (3) score in Quepid
As Annie writes:
“Quepid jump-started our search relevance tuning program, providing an interface for business analysts to understand and adjust field weight impacts. It captures ratings for each document’s relevance, speeding up iteration cycles with reusable data. Its color-coded scoring visualizes progress for stakeholders. A useful sandbox tool, Quepid and its support from OpenSource Connections exceeded our expectations.“
If you need to de-risk a search engine migration with effective measurement, get in touch today.
