An Ordinary RAG Wasn't Going to Cut It

The article is about the RAG system we built for Jedelbom.com, our own Slovenian food app website with almost 10,000 users.

RAG was quite interesting. It started as an idea to build an ordinary RAG system: a vector-embedding search that finds similarities between documents and a search query.

But we soon realized that wasn't going to cut it. Food queries have hard constraints that you must follow. That's why we had to develop new ways to use the semantic and embedding properties of a vector database together with the structural and lexical search of traditional text-based search.

Processing the app's data

For our stack, we use MongoDB Atlas. We primarily use Voyage for embeddings and reranking. We process all of the data available from the app, including CMS data, menu-item data and restaurant data. We also enrich menu items with special cuisines and food styles drawn from a canonical vocabulary.

Once everything is processed, we take city rankings and restaurant rankings within cities into account to influence the results.

Interpreting the query

At a high level, our search combines semantic, structured and lexical search. In the retrieval implementation, we have four lanes because we separate vector search from discovery vector search. But even before retrieval, we have to interpret the query. What does it mean? What does the user want?

This lets people search in natural language and get close to what they are looking for, whether they have a particular mood or a cuisine style in mind. The service can search for semantically relevant ideas and concepts, then present them to the user.

For other searches, price and location are hard constraints. The LLM interpreter identifies them, but an LLM can hallucinate and invent constraints, menu items and other details. So we have to ground it and put some restraint on what it can return.

The second layer of the control protocol is a predefined schema with predefined allowed values. This lets our system identify what the LLM has output and decided.

The interpreter has to identify what action to take, which route or vertical to use, what response mode to make, which retrieval targets and entities were mentioned, and what constraints apply. Are they hard? Are they explicit? How confident is the interpretation, and what is its evidence source?

What is the intent? Is the user searching for menu items, a recommendation, lunches or offers? What is the task and target? What is the relationship? If the user wants a menu item from a restaurant, we need to get the restaurant as well.

We have a semantic control file with a vocabulary for the constraints. They can be numeric, for prices and distances, or categorical, such as a city, region, restaurant, restaurant type or offer. List categories can contain any, all or one of several values, including dietary tags, cuisine types, food styles and restaurant-specific features.

There are also venue attributes: delivery, pickup, parking, card payment, pets allowed, outdoor seating, wheelchair access, reservations and Wi-Fi. Temporal constraints cover times and offer validity: now, after a certain time, today, tomorrow, this week or next week. Text constraints can look for a string, part of a string, a string that starts or ends with something, or a fuzzy match. They can search restaurant names, item names, section descriptions and lunch text.

This means the entity catalogue is preserved in an updated state and provided with every search. The model is currently GPT-6 Luna, running at a temperature of zero, with a generic constraint schema for its output. The model is instructed with every prompt, and the inputs are cached.

In the future, we could fine-tune a model to think the way we train it to think, so it would be wired to answer and perform in a specific way without needing the same instruction on every request. That could cut down latency. For now, some of the latency comes from the model's inference and reasoning, as well as the network round trip.

We resolve entity identities based on existing documents in our system: restaurants, cities, regions, cuisine categories and dishes. Every query gets a purpose. It can be a task to search, describe, recommend, compare or rank. Its target can be provider dishes, lunches, offers, a location or a mix of these. Its objective can be relevance, price, value, quality, popularity, trending, availability or discovery. Its strictness can range from hard constraints to balanced or exploratory recommendations and rankings.

We have more than ten query-planning routes and four retrieval lanes. They all have default limits. The query is embedded so that we can do semantic search, and the results vary depending on how many matches we have, what the user asked for, and how our recommendation system and ranking work.

How ranking works

Ranking plays a big part. First, we use a Voyage reranker, which decides which results are most appropriate for a query.

Result policy and analytics also play a big part. We want to boost results from popular restaurants to the top, based on views. In the future, we may have a chance to boost restaurant views by advertising their sites and pages, and ranking them higher in search. So far, ranking is bypassed only when the user sorts by price; then the cheapest items show up sooner.

There is also personalized search, for accounts only. It scores results based on preferences such as city, cuisine and style, as well as food items and restaurants the user has liked before. General recommendations also show up in a fresh format, mixing recommendations for restaurants, menu items, lunches and offers.

What we have learned so far

The system is constantly expanding and learning from feedback and the searches users make. We have done preliminary testing with about 300 different natural-language queries, and all of them passed. These are predictable queries that we wrote, so the results are predictable too.

Real users make mistakes, type restaurant names incorrectly, use incomplete restaurant names, or send queries that do not describe much. We have safeguards that allow the LLM to ask for clarification before it starts searching. Restaurant nicknames are another challenge. If someone types only part of a nickname they know, the LLM may not be able to infer which restaurant they mean—at least not yet. It may learn those references in the future. This is why the system will keep improving.

A normal, competitor’s LLM like ChatGPT, cannot give you a restaurant's menu items, daily lunch information, offers and other structured data unless it has access to that data. Some menu information is publicly available on our site, but a large part of the data, analytics, user search queries, trends and behaviour, is not visible to the AI. That is why it cannot give you the same quality of search you can get directly from our app.

Data, models and what comes next

We are well positioned to have our own fine-tuned LLM, trained specifically on the domain data of our app. I also believe users could be rewarded for their participation and share in a data-ownership model. That could become an important model and financial structure in a future world of superintelligence, where agents need material to work on and learn from.

There is a finding that an LLM's learning capabilities hit a brick wall after it has ingested all the original data and starts producing its own content. The quality of that content can then diminish when it is used for further learning.

That is why I foresee a future where people work like bees to gather relevant, good information for machine learning and LLMs, so they can perform adequately. I believe users should—and will—be adequately compensated for it.