Charlie Hull
Category: AI
The Next Generation of Search at OpenSource Connections
In 2016 Manning published Relevant Search, co-written by Doug Turnbull and John Berryman. Unlike most books on the subject of ‘search’ it focused not just on deep Information Retrieval theory or on the features of a particular engine, but on Relevance itself as a standalone subject – for the first time, showing how to provide relevant search results in data-driven, experimental ways. Unsurprisingly it became a very popular book in the search community, and as Doug Turnbull was CTO at OpenSource Connections at the time, heavily influenced our thinking and how we worked with clients. It is a highly practical book, with examples using the Elasticsearch and Apache Solr search engines.
Learn more with AI-Powered Search
Although Relevant Search remains popular and hugely useful, in the intervening years the more widespread focus on machine learning and other techniques labeled as ‘AI’ has revolutionized the way we search. AI helps us process natural language, automatically learn and balance tuning parameters, infer the user’s intention behind a query and deliver not just a list of results but a single answer. Doug Turnbull, Max Irwin and lead author Trey Grainger have for several years been working on a book to take us further on our search journeys: AI-Powered Search. Drafts have been available under the Manning Early Access program for some time now but we’re very glad to say the book is in the final stages of going to print and should be available to buy within weeks.
AI-Powered Search dives deep into techniques like semantic search query understanding, signals boosting and personalized search, learning to rank, knowledge graph learning, multimodal and hybrid search, vector embeddings, question answering, and generative AI techniques like Retrieval Augmented Generation (RAG), while retaining a data-driven approach familiar from the earlier book. Interactive notebooks are heavily used to demonstrate concepts and techniques, but unlike Relevant Search a more generic approach to technology is shown, making no assumptions about which underlying search engine will be used – a sensible approach, considering that there has been a recent explosion in available technologies. An associated Github project provides Dockerized example code and a growing list of ‘pluggable’ search engines. We expect the book to join Relevant Search on every search developer, search data scientist, and search product owner’s bookshelf as an essential reference and learning resource.
Trey Grainger joins OSC team
At OSC we are already using many of the techniques in the book in client projects, but we’re also very happy to announce that lead author Trey Grainger will join the OSC team from September 2024 as a technical advisor. Trey will help us to continue to deliver the benefits of AI-powered search to our clients – cutting-edge strategic advice, training and expert consulting on everything from data preparation, to selecting and hosting machine learning and Large Language models, to enhanced search, all the way to building complete Generative AI solutions.
Join the AI-Powered Search community
Our team will also be active in a new AI-Powered Search community which aims to bring together the experts, the companies and the people driving this next generation of search to share information and knowledge. Trey will be joining the OSC stand next week at OpenSearchCon US 2024 where we are also speaking and hosting training, and we are also pleased to announce he will give the keynote at Haystack Europe 2024 on September 30th. Come along and meet Trey and the rest of the team!
AI is already changing the world – we look forward to seeing how it changes search! Get in touch to discuss your AI-powered search project.