NEW! Watch Shaun Brazendale from Rubix talk about how Rubix learned to “own their search” in this video from MICES EU 2022.
Rubix Group is a multinational distribution company operating only in Europe with a published revenue of 2.2bn Euros in 2020. It distributes spare parts (MRO), tools, and Personal Protective Equipment to B2B customers in many countries including France, the UK and Germany. It also supplies services such as maintenance or repair of gearboxes and motors.
Rubix uses the SAP Hybris e-commerce platform to support web shops for each country or territory. Hybris depends on the open source search engine Apache Solr for search functionality and allows some basic search tuning to be performed.
“We’d reached the limit of what we could do using built-in Hybris features like synonyms and redirects to fix search. Some queries that might have shown zero results in the past were being redirected to category pages, but this doesn’t produce a great search result, just a slightly better one, and it introduces some tracking challenges and may not work with customer-specific assortments. We needed help to take search to the next level and reduce the ‘exit rate’ of unsuccessful search sessions.“
Shaun Brazendale, product owner for search at Rubix Group
Discovery starts the Proven Process
OSC’s engagement with Rubix started in late 2019 with a Discovery engagement, a week-long deep dive into every aspect of search and relevance covering areas including user experience, technology, source data and customer understanding. Using OSC’s Search Maturity Matrix, the team identified key areas for improvement that would lead to a rapid rise in search quality. The Discovery is the first step in OSC’s Proven Process, designed to reduce risk and ensure the success of search quality improvement projects.
The Discovery recommended that Rubix become a Relevance Centered Enterprise to improve how search issues and problems are captured and to develop tools, processes and internal knowledge to allow each country team to have some control over search results using a process of Active Search Management, while the central technical team concentrated on improving overall search quality by developing better search algorithms.
“Moving some of the control of search away from a centralized IT team to those who interact on a daily basis with customers and know exactly what products they are searching for creates a huge opportunity to improve search quality. The central team can then focus on algorithmic improvements to benefit every web shop, constantly iterating new solutions, rather than having to address every new search issue that arises – becoming proactive rather than reactive.“
Eric Pugh, OSC’s co-founder and lead on the Discovery
Accelerating Search Quality with open source tooling
OSC and Rubix then began a 90-day Accelerator Implementation project to address the recommendations from the Discovery. The OSC team started by helping Rubix integrate the Chorus suite of freely available open source tools, a reference implementation platform for high quality, measurable and tunable e-commerce search. These tools include:
Quepid, a ‘search tuning dashboard’ allowing Rubix staff to gather and manually rate the quality of search results and to experiment with different query parameters to see how this affects the overall quality.
Querqy, a query pre- processor plugin that allows for business rules to be created to influence the results of particular queries, e.g. by boosting results from certain categories, handling common mis-spellings or even deleting non-useful words from the query. Querqy rules are maintained using the Search Management User Interface (SMUI).
OSC worked to help Rubix test and deploy these tools into development and production systems, building a custom layer to process the highly complex Solr queries generated by Hybris. OSC audited the existing Solr query structure and helped remove inefficiencies and improve overall accuracy.
Measuring search quality
Rubix had already developed a KPI for search based on the Mean Reciprocal Rank (MRR) measure, but there was a missing element of human judgements to calibrate the big data and to help in de-risking future search changes. OSC’s introduction of Quepid allowed for the quality of results from a small number of key queries to be manually rated by Rubix’ own team, giving a baseline measure of search quality.
In parallel and with input from the OSC team, Rubix have also developed enhanced dashboards for search, which helps identify which queries are most important to the business and which might be underperforming in some way . These queries can then be candidates for improvement using business rules:
Building a practice of Active Search Management
Querqy allows the creation of business rules to be triggered by particular queries – for example, when the user types “power drill”, it might be appropriate – depending on the needs of the business – to boost results showing actual power drills and lower those showing drill bits or other power tools. Querqy can also manage synonyms, mis-spellings and even delete words from the original query. How these rules are created is highly context dependent and relies on an understanding of the user need and the product catalog – matching these should help drive add-to-basket conversions as more relevant results are generated.
SMUI (Search Management User Interface), a web-based front end to Querqy, further eases this process as it stores a list of current rules, allows for rules to be cascaded and with a single click uploaded to production. SMUI and Querqy are open source software used by many large e-commerce companies across Europe and compatible with both Apache Solr and Elasticsearch.
OSC first trained two key members of the Rubix merchandising team, working on the German and French web shops, on how to create and maintain Querqy rules for a small number of known and high-traffic queries. The new system allows far more flexible and powerful search tuning than is possible with Hybris and importantly requires no understanding of the inner workings of Solr. New rules can be tested offline and then deployed within minutes to a live production system. The Rubix team’s reaction to this new capability has been hugely positive as it allows search issues identified by customers or the Rubix team to be fixed very quickly, without the involvement of key technical staff, any search re-indexing or new code deployment. Further dashboards allow Rubix to monitor the impact of the rules on conversions and exits:
“By bringing the Chorus stack into the Rubix search tech stack we have accelerated our capability in rapidly responding to customer issues and allowing local search teams to fine tune and create great search experiences for our customers. It has helped level-up our offline testing and human judgements when implemented alongside a proven methodology.“
Shaun Brazendale, product owner for search at Rubix Group
Early indications are that queries tuned in this way can raise conversions by up to 30%. In the French Rubix store Querqy rules are used on 20% of all searches and have helped to halve the search exit rate for these searches. Following the success of this initial deployment Rubix are now rolling these searchandising tools out to further web shops.
Creating an offline lab for search
Although Active Search Management has addressed many of the most urgent search issues, there is still a need to continuously improve the basic search algorithms. Working with OSC, Rubix has now created an Offline Search Lab, where experiments using new search algorithms can be carried out quickly and verified against the manual ratings created using the Quepid tool. An Offline Lab can feed these experiments into the online, Operations cycle:
Similar queries (or queries representing similar problems, such as the handling of dimensions) can be grouped together so common solutions can be found. An experimental, hypothesis-driven approach is used to rapidly try potential improvements and the best solution can then be promoted for online use. Techniques such as Jaccard similarity are used to reduce the risk of side-effects.
What’s Next
In 2022 Rubix hope to develop further search metrics and use these to improve how the impact of search improvements on KPIs is measured. Now the offline Search Lab is generating new algorithms, it is important to be able to test the best of these online and Rubix plan to expand and improve the organisation’s capacity for online A/B testing. Search Management will also be rolled out across further territories and the partnership with OSC continues.
“The process of working with OSC has been enjoyable, productive, and rewarding. Working with the team and their vast experience in search has helped level-up Rubix’s maturity level in some key areas and deliver better search to our customer with measurable improvements.”
Shaun Brazendale, product owner for search at Rubix Group
If you want to empower your search team to build more relevant search to drive conversions and delight your users contact us today.
