See how storing documentation as a RAG database, combined with a simple workflow within n8n, can give the squad a significant advantage in terms of efficiency and accuracy. And, perhaps more importantly, how it can help build trust in the responses provided.
The Efficiency Benefit
Same question, same LLM model, same answer, but with very different token and time costs.
When it comes to using AI powered tools, if you do not provide the relevant context to the tool, you will always use more tokens and it will take longer to generate a response.
With access to the exact same information and using the exact same model of LLM, the implementation of conditional filtering within n8n to hand the LLM the most relevant context both reduces token usage and improves response time.
What is more is that the results from this are even more pronounced when the number of documents is larger.
Multi-Source Questions
How many documents are relevant to the question?
If a system for filtering is either not in place or improperly implemented, it can lead to using documents that are not relevant to the question, potentially even the entire pool of documents.
Too much context in the form of documents can lead to bias in the responses, or increased token usage to compensate, and cutoff caused by max context being reached. This can result in answers that contain irrelevant information, old information that isn't up to date because it couldn't fit in the context, and hitting the context limit faster.
Fine-Tuning Matters
Optimizing the filtering for the best results
Sources of information are not always structured the same across squads or departments. This needs to be accounted for.
Through years of experience, I have learned that each squad has unique ways of structuring information. This typically is the result of finding the best approach for their specific needs. Instead of trying to force a squad into a format that is ultimately detrimental to their productivity, this needs to be taken into account when filtering documents. This demo shows how tight or loose the filtering needs to be to give the most correct answers, while pulling in the most relevant information.
How It Works
Typically, when a user submits a question, an LLM searches the database for relevant information on its own, but this ends up handing the LLM far more information than it actually needs. This is illustrated in the red section of the workflow below: the entire document set gets handed directly to the LLM alongside the question.
In the green section, a filtering step is added first. Before any documents reach the LLM, a code node filters out documents that don't contain information relevant to the question, based on configurable options like minimum keyword matches and vector similarity scores. If at least one relevant document remains, that filtered set is handed to the LLM as context. If nothing passes the filter, a simple response indicates that no relevant information was found instead.
