Is vector search a silver bullet?
Vector search, where both the source data and search query are transformed into multidimensional vectors, promises to revolutionize the world of search. But is this an AI silver bullet – or a distraction? With our vector search consulting we can help you find out how your organisation could benefit from these new techniques and help you build powerful vector search systems to solve real world business problems.
The OSC team have been tracking the rise of these new techniques for several years. Start by reading our Guide to the New Frontier, first published in Search Insights 2022 from The Search Network, or watch industry expert Dmitri Kan explain where vector search is taking us at our Haystack conference.
Where we can help
Our vector search consulting helps you understand:
- How vector search really works
- What are the the use cases for vector search
- Which technologies to consider
- How to ‘vectorize’ your source data with embeddings
- How to create vector queries from user input
- How to combine vector search with traditional text search
- How to measure the impact of vector search
What exactly is vector search?
- Convert text or images into vector (numerical) embeddings using a Transformer model (like BERT)
- Store these embeddings in an index
- Convert a query in a similar way into a vector and search your index
- Traditional search revolves around keywords, while vector search leverages semantic meaning, to find documents most similar to the query
Try it out with Chorus
The reference implementation for open source e-commerce search, Chorus, now comes with vectors! Read how vectors can revolutionize e-commerce search & how we added vectors and how you can try this out yourself.
See vector search in action
Here’s Atita Arora from the OSC team demonstrating vector search in action at Haystack on Tour in Kraków, Poland:
What we’ll do
Explore the options
We’ll help you explore the world of transformers like BERT, models like GPT, vector search providers like Deepset, Milvus, Weaviate, JinaAI and Pinecone (founders of many of these platforms have spoken at our events or collaborated with us in other ways), complete platforms like Vespa and also how established platforms such as Apache Solr, Elasticsearch and OpenSearch have added vector search capabilities.
Examine the use case
Not every search problem can or should be solved by vector search! We’ll identify which of your issues could be addressed with vector search – long tail queries, multimodal search (images and text), misspellings, language mismatch – and help you prototype solutions.
Tame the hybrid
The best solution for many organisations will be a hybrid of traditional and vector search techniques. However, combining the results of two very different systems is hard – we’ve done this successfully at the TREC conference and we can help you do the same.
Start experimenting with vectors
With regular measurement and testing in place, evaluate vector search in an offline environment and start a regular and rapid cycle of search improvement measured against KPIs that drive your business.
Move towards production
Vector search brings a whole new set of challenges – how often to re-train models, whether these models will need fine tuning, increased processing and storage requirements – let us help you plan for success and stability.
Build your search team
We know how to structure a search team for success – let us help you create and fill the roles you’ll need, develop effective processes and foster collaboration. As you build an effective team, let us fill the gaps.
Questions we are asked during vector search consulting:
Which technology to choose?
Will this replace traditional text search?
Can I add this to my existing search platform?
Don’t get left behind by the AI revolution!
We’re always happy to chat – contact our vector search experts for an initial discussion