Exclusions in Field Lists in Solr

Scott StultsNovember 2, 2015

Back in 2012 Luca Cavanna opened an issue on the Solr Jira requesting a field exclusion syntax. Right now it’s the 5th highest-voted ticket, and it’s easy to understand why. Documents consisting of dozens, even hundreds of fields are common and without an exclusion syntax your choices are limited to:

  • All fields (“*”)
  • Some fields that match a specific naming convention (“section_*”)
  • Each field (“section_1, section_2, section_3”…)

With atomic updates you need to store each field, so unwieldy field lists are quickly becoming the norm.

About a year later Andrea Gazzarini added an excellent patch that went above and beyond the original request and added a more flexible expression syntax for both exclusions and inclusions. What’s more he provided excellent documentation about the patch as well. In short, with this patch you can do things like:

  • Exclude a single field: fl=-forbidden_field
  • Exclude similarly named fields: fl=-forbidden_*
  • Use a question mark for a single character wildcard: fl=-forbidden_5?

I should also mention some of the things you could already do with field lists like supply functions, transformers, and aliases (and aliases to functions and transformers). So when you use wt=csv to generate a report on your index you can let Solr do a lot of the busywork for you.

For now this exists only as a patch, so you’ll need to download the source, apply the patch, and build Solr yourself. If you’d like to see this included as part of Solr I encourage you to vote on it!

More blog articles:

We've been Solr-istas since day one!

Our founder wrote the first book on Solr, now in 3rd edition. We've helped organizations from the US Patent and Trademark Office to Cisco build smarter search solutions with Solr.

Learn More about our Solr services.