Doug Turnbull
Category: Relevancy
In complex Solr relevance work, I find I create a fair amount of function queries to boost & balance results. Solr function queries look like Excel queries from hell. Trying to figure out why something that looks like if(recip(ms(query(...))),query(...),sub(..)) isn’t scoring results the way it should can make your eyes bleed. The debug output is pretty opaque, only showing you the full-on calculation, something like:
{ "match": true, "value": 0.042678382, "description": "1.0/(3.16E-11*float(ms(const(1494960667562),date(release_date)=1994-11-18T00:00:00Z))+1.0)"}If your function query isn’t working, good luck figuring out which part of the calculation was broken.
This debugging pain is why I want to tell you how I debug complex Solr function queries. I use two tricks. First I parameterize my boosts, breaking them into more manageable parts. So if we’re boosting movies by “release date” the classic Solr approach could be broken down into a few parts: msIntoPast=ms(NOW,release_date) and then dateBoost=recip($msIntoPast,3.16e-11,1,1) which could then be used as a multiplicative boost: boost=$dateBoost.
Second, with the parameters broken apart, I can use a Splainer feature to compute the value of each parameter per doc. This lets me dig into parts of the calculation to see what went wrong. Splainer just accepts a little hint to compute something as a function query directly – using Solr computed fields aka “psuedo fields”
In the screenshot of a movie search below, we’ve broken out parts of the release date boost for each doc and asked Splainer to compute the values of each “function query lego block”. We use the “f:” prefix in “Displayed Fields” to refer to each parameter (ie $dateBoost) that Splainer should compute & display:
Viola, if for some reason the date boost doesn’t seem like it’s being applied correctly, now I can go look more directly at the function query math. It’s been a very handy tool in relevance work I’ve done, helping me very efficiently pinpoint problems in my solution.
That’s basically it. If you have any ideas for Splainer, the code is here – don’t hesitate to let us know about them. And as always get in touch if you’d like to chat!