Charlie Hull
Category: Relevancy
Relevance Engineering is a relatively new concept but OSC and others have been carrying out the software engineering practice of tuning search engines for many years. So what is a ‘relevance engineer’ and what do they do? In this series of blogs I’ll try to explain what I see as a new, emerging and important profession.
Understanding the difference
Before a relevance engineer can install or configure a search engine they need to understand the business concerned. I’ve called this ‘learning the business’ and it’s something that we have to do at OSC on a regular basis. One week we may be talking to a recruitment business that thinks and operates in terms of jobs, skills, candidates and roles; the next week it could be a company that sells specialised products and is more concerned with features, prices, availability, stock levels and pack sizes. Even within a single sector, each business will work in a slightly different way, although there will be some common factors.
Context is everything
Example data is key to learning how a business works, but is next to useless without someone to explain it in context. In some cases the business has lost some of the internal knowledge about how their own systems work: “Jeff built that database, but he left two years ago.”. What seems obvious to them may not be obvious to anyone else. Generic terms e.g. “products”, “location”, “keywords” can mean completely different things in each business context. If they exist, corporate glossaries, dictionaries or taxonomies are very useful, but again they may need annotating to explain what each entry means. If a glossary doesn’t exist, it’s a good first step to start one.
Getting a range of opinions
Finding the right people to talk to is also vital. Although relevance engineers are usually engaged or recruited by the IT department, this may not be the best place to learn about the business. The marketing department may have the best view of how the business interacts with its clients; the CEO or Managing Director will know the overall direction and objectives but may not have time for the detail; the content creators (which could be librarians, web editors or product information managers) will know about the items the search engine will need to find.
In many companies there are hierarchies and structures that sometimes actively prevent the sharing of information: it’s common to discover who blames who for past bad decisions and to be used as a sounding board by those with axes to grind. At OSC we try to make sure we talk to people at all levels in the client organisation: sometimes the most junior employees – and especially those who are customer-facing – have the most useful information as they have to deal with problems on a day-to-day basis. As external consultants one of our most useful skills is being able to listen without making sudden judgements or assumptions.
Gaining understanding as a relevance engineer
The end result of these many conversations is an understanding of where source data is created, gathered and stored; what a ‘search result’ is in the context of a particular business (a product on sale? A contract? A CV or resumé?) and how it might be constructed from this data; what a ‘relevant’ result is in this context (a more valuable product to sell? The most recent contract version? The best candidate for a job?) and how good/bad/nonexistent the current search solution is. This is vital information to be gathered before one even begins thinking about how to install, develop and/or configure and test a search solution.
At OSC we formalise the steps above as part of our Discovery engagements, the first step on our Proven Process for successful search projects. One useful side effect is that the people we talk to learn a lot about their colleagues and what they do – often it is the first time that everyone has gathered to focus on search quality improvement.
You can read about The Rise of the Relevance Engineer in Search Insights 2019 from the Search Network – or contact us if you’d like us to learn your business!
Written originally on the Flax blog, I’ve updated this blog for today’s relevance engineers!