Using Splainer to Debug Complex Solr Function Queries

Doug TurnbullMay 16, 2017

In complex Solr relevance work, I find I create a fair amount of function queries to boost & balance results. Solr function queries look like Excel queries from hell. Trying to figure out why something that looks like if(recip(ms(query(...))),query(...),sub(..)) isn’t scoring results the way it should can make your eyes bleed. The debug output is pretty opaque, only showing you the full-on calculation, something like:

   "match": true,
   "value": 0.042678382,
   "description": "1.0/(3.16E-11*float(ms(const(1494960667562),date(release_date)=1994-11-18T00:00:00Z))+1.0)"

If your function query isn’t working, good luck figuring out which part of the calculation was broken.

This debugging pain is why I want to tell you how I debug complex Solr function queries. I use two tricks. First I parameterize my boosts, breaking them into more manageable parts. So if we’re boosting movies by “release date” the classic Solr approach could be broken down into a few parts: msIntoPast=ms(NOW,release_date) and then dateBoost=recip($msIntoPast,3.16e-11,1,1) which could then be used as a multiplicative boost: boost=$dateBoost.

Second, with the parameters broken apart, I can use a Splainer feature to compute the value of each parameter per doc. This lets me dig into parts of the calculation to see what went wrong. Splainer just accepts a little hint to compute something as a function query directly – using Solr computed fields aka “psuedo fields”

In the screenshot of a movie search below, we’ve broken out parts of the release date boost for each doc and asked Splainer to compute the values of each “function query lego block”. We use the “f:” prefix in “Displayed Fields” to refer to each parameter (ie $dateBoost) that Splainer should compute & display:

showing how to debug function queries in splainer

Viola, if for some reason the date boost doesn’t seem like it’s being applied correctly, now I can go look more directly at the function query math. It’s been a very handy tool in relevance work I’ve done, helping me very efficiently pinpoint problems in my solution.

That’s basically it. If you have any ideas for Splainer, the code is here – don’t hesitate to let us know about them. And as always get in touch if you’d like to chat!

More blog articles:

We've been Solr-istas since day one!

Our founder wrote the first book on Solr, now in 3rd edition. We've helped organizations from the US Patent and Trademark Office to Cisco build smarter search solutions with Solr.

Solr Services Solr Relevance Engineer Training