Doug Turnbull
Category: Ecommerce
In the 2010s “Googling” started to mean asking questions. Now it can feel closer to chatting.
Is the same true of e-commerce? Is it becoming true?
E-commerce stalwartly resisted the switch. Users searched for “blue running shoes”. Not “I want blue running shoes with strong insoles”.
Is that changing? It seems so.
Algolia notes the shift [in their July report][1]
> Sixty-two percent (62%) of consumers expect online search bars to understand natural language like ChatGPT and 34% already use conversational language when searching on retailer websites.
Rezolve.ai [notes the shift too][2]:
> Meanwhile, long-form queries of three or more words rose from 25% of searches in 2024 to 40% by February 2026
The shift is happening. Retailers seem unprepared. If you’re in this situation, what should you do?
## Don’t throw away your current search infrastructure
Compare a Google-like question
> What is the air-speed velocity of an unladen swallow?
to
> I want blue running shoes with strong insoles
The shopping query describes attributes the ideal product must satisfy. A result can be said to either match or not match each requirement.
We’re used to managing queries like “blue shoes”. Translating them to `color:blue item:shoes`. Under the hood, even with a natural language query users have the same requirements. It still requires filtering, understanding products, organizing them into taxonomies, backed by much the same infrastructure.
How? That’s where our next point comes in.
## Small Language Models help process queries
Self-hosting a small LLM has increasingly become feasible. A tiny LLM can turn a natural language query like
> I want blue running shoes with strong insoles
Into a structured query
“`
{
“color”: “blue”,
“item”: “running shoes”,
“attributes”: [“strong insoles”]
}
“`
LLM Query Understanding has become increasingly mainstream. You can easily host a small LLM with 100ms latency able to process millions of queries for hundreds of dollars.
Most teams, however, don’t process queries in real time. They build a cache offline for fast query-time access for their most common queries. That turns 100ms into nearly instant response.
But you might ask, how can a cache even make sense with large natural language queries? It’s rare that this exact full sentence “I want blue running shoes with blue insoles” would be repeated by another user. You might think caching becomes pointless.
Luckily, with vector databases, you can cache semantically. A semantic cache can detect similar queries, like “running shoes, blue, strong insoles” and notice the semantic similarity to an existing query (our “I want blue running shoes with strong insoles”) and avoid the LLM call, returning the structured query nearly instantly.
## Decision models rank and tag
With a set of candidates retrieved, with our hypothetical query:
> I want blue running shoes with strong insoles
We might have a number of questions worth answering about any product shown to the user:
Decision models, like Jev, ask simple questions like “Is this product running shoes?” or “Do they have strong insoles” and give you a probability of “Yes” being true. Add up the P(Yes) for all these questions, and you’ll have the most relevant product.
An emerging approach in e-commerce is to use an LLM not just to ‘parse the query’ but also to [generate good questions to ask about the ideal product][3].
Even better: we can remember how these questions apply to individual products. Essentially tagging the product for “strong insoles”.
## Same retrieval technology, new techniques
One big takeaway for what we’ve discussed: don’t throw away your search engine. Any reasonable search technology combines vector search, lexical retrieval, and metadata filtering. For e-commerce, natural language queries first-and-foremost change our approach to query understanding and reranking.
Interested in learning more about how our e-commerce search and AI experts can help you? We’d love to have a conversation. Contact us today!