University of Michigan Health System: Relevant, and Fast Drupal Doctor Search from OSC + MeasuredSearch
OSC Nailed It! Searchstax is the ideal hosted Solr platform!
Josh Coggins — UMHS Architect
The Problem
The University of Michigan’s Health System (UMHS) understands that patients need to be matched to the ideal provider. Finding a physician with the right skills, experience, that convenient to the patient, with scheduling availability takes careful technical implementation.
With search in particular, doctors don’t use the same language as users. So “out of the box” search solutions won’t work. Doctors aren’t “bone doctors” they’re “orthopedists.”
Patients have “heart attacks” while doctors term them “myocardial infarctions.” Mapping these ‘vernaculars’ between lay-person and expert forms a key part of our search relevance practice
Even more mundane: how many times do you hear a physicians name but later are unable to spell it? You might hear “Dr. Rainy” when in reality it’s spelled “Dr. Ranney” — how can a search system cope?
The Assessment: How to build a patient-doctor match platform
UMHS recognized the challenge they faced, so they contacted the search relevance professionals for OpenSource Connections (OSC) — the folks who wrote the book on smarter search. OSC relevance strategists took an inventory UMHS’s physician finder application including:
- Understanding what UMHS hoped to gain from the search application
- Took an inventory of what use cases needed to be supported
UMHS’s the user problems and developed recommendations and a roadmap using OSC’s relevance methodology
The most immediate need of UMHS was to launch a platform for physician matching. UMHS needed OSC’s help to rapidly deliver a solution. UMHS’s platform of choice is Drupal. Drupal provides a tremendous amount of flexibility for authors to manage content. UMHS’s staff identified the following challenges with their Drupal-based platform:
- Data Modeling for Relevance: Directly searching the data in Drupal tends to delivery mediocre search results. To truly be a platform for relevance, relevance engineers must be able to enrich, restructure, join together many forms of source data — far beyond the flexibility of Drupal’s default search API.
- Rapid, Custom Development in Drupal: Drupal’s search API enforces a strict interface between search-engine and application. This makes simple development faster, but doesn’t allow for the customization relevance engineers need to customize the user interaction.
- Hosting Solr Search: Who wants yet another data store to manage? Unfortunately, Drupal-oriented Solr platforms are fairly outdated Solr versions and lack the depth of customization needed for a rich, relevance-focused search application. Standing up and managing your own servers, even with today’s devops capabilities, is a headache! Who wants to get 3AM phone calls?
Implementation: A platform for repeatable relevance improvements
OSC’s roadmap called for a three-pronged approach to rapidly develop the first release of UMHS’s physician finder:
- Augmenting Drupal data for smarter search : OSC integrated with the Drupal Search API by building pre and post versions of the search index. The pre version contained data as output to Solr by Drupal. The post version contained a relevance-focused view on the Drupal data to better support relevance.
- Drupalizing a Javascript Search UI: To wrap a modern, rich and client-side search UI in Drupal: working with UMHS OSC rapidly built an application with AngularJS. This application was “drupalized” along with a simple php script — compiling the Angular application into a custom Drupal module that could be loaded as a Drupal plugin. Drupal was treated like any-other backend UI through Drupal’s exposed variables. This let us configure this search UI to power any number of Drupal search interfaces for UMHS.
- Avoiding the headache of managing search servers : We leveraged SearchStax by MeasuredSearch. SearchStax is hosted Solr product for Solr nerds. It allows integration of plugins and stays up to date with the latest versions of Solr. MeasuredSearch handles the 3AM phone calls, heap sizing, and many other concerns so we could focus on relevance with the UMHS relevance team.
OSC sees these three points as being key reference architecture for Solr-based search with Drupal: SearchStax + Headless Drupal + Pre/Post Solr -> amazing search! The results speak for themselves in the application:
To quote Josh Coggins, the UMHS lead developer:
OSC nailed it. We were in a tight spot: needing to deliver a find a physician feature quickly. OSC knew how to bring together our key platforms: Solr, Drupal, Angular, in a way that worked toward’s each’s strength. They created an innovative approach that helps avoid relevance and search UX problems on classic Drupal search apps.
Releasing & Fine Tuning Search Relevance: the OSC+MeasuredSearch Difference
OSC knew on deploying search it wouldn’t be perfect. This is why Searchstax from MeasuredSearch was chosen. On soft-launching the Find a Physician search, an OSC relevance strategist analyzed whether search was meeting UMHS’s business goals. SearchStax provided crucial insight into how well the implementation performed.
OSC relevance strategists began by analyzing top searches to find problematic patterns for user searches. OSC noticed immediately the top searches involved searching for specializations, and immediately saw how to resolve the problems, as shown below:
OSC began examining these searches and identified some obvious relevance problems. Specifically OSC noticed we needed to optimize the “doctor shopping” experience. Patients “shop” for a doctor by searching for a specialty like “gastroenterology.” This lead us to take actions to improve the likelihood a patient shopping for a doctor would call and make an appointment. Specifically:
- This isn’t article search, it’s doctor search! Specialty searches were improved by changing many of the default text relevance assumptions in Solr, reducing the impact of the number of matches in the specialty field. For example, we know from our work with short snippet search, that more mentions of “medicine” for “internal medicine” didn’t really matter more
- Patients need information to make choices: We prioritize doctors that gave patients more information for making a choice: Doctors without images and fully fleshed out profiles came to the top of the results instead of those with images and
- The small stuff: simple synonyms: We added key synonyms for specialties, including mapping “pediatrist” to equate to “pediatrician.” Simple set of changes around synonym management had a tremendous impact on searches we tested
Josh from UMHS had this to say about the power SearchStax brought to the team:
Searchstax is the ideal hosted Solr platform for search nerds. It stays up to date with Solr, let’s you write your own plugins, while providing tremendous support. If your priority is smart search, why deal with the headache of managing your own servers when you can Searchstax?
Doug Turnbull, OpenSource Connections Lead Consultant had this to say:
We want to focus on tough relevance problems, not servers. That’s why we’re so passionate about MeasuredSearch: MeasuredSearch’s support is stellar. Everytime we had an issue, we had very quick resolution. Increasingly, we’re recommending MeasuredSearch as both a tremendous search analytics platform and a Solr host.
The relevance payoff
In previous projects using a different architecture, iterating quickly on these problems with Drupal+Solr was rather painful. It took bending Drupal to our will, with application-specific hooks. It required working with an operations team to change simple Solr configurations. A month could go by before minor improvements roll out. As we mention in our Search Relevance Methodology, the best way to solve relevance is
- A platform you can measure the quality of search relevance (in this case Searchstax)
- A platform for iterating quickly changes (our Drupal+Solr implementation)
After our initial modifications, we saw improvements immediately in the average click position of the search results. The MRR (mean reciprocal rank) of 0.5 corresponds to an average click position of the second result, an MRR of 1.0 means the first result on average was clicked. After our single day of tuning, we went from a MRR of ~0.4 to 0.6, as shown in the graph below from SearchStax:
Click depth in key specialty searches dropped tremendously, for example the average clicked result for Opthamology dropped tremendously
Having a platform that tightly integrates Drupal+Solr under the control of developers let OSC and UMHS’s team to iterate quickly on these changes.
The Result
UMHS released the first version of their Find a Physician platform on March 20th. With only a few hundred hours of OSC’s time they were able to quickly deliver. With SearchStax and OSC’s relevance skills, OSC relevance team was able to fine tune search after launch based on user feedback.
With this platform in place, we’re ready to help UMHS tackle even more of their relevance problems. Armed with this platform we can begin to react to more complex scenarios, and consider the next level of advanced technologies: machine learning and semantic matching!
