RAG Agent

Answer questions with a chat model that looks up your knowledge base first, and can remember the conversation.

What it does

RAG Agent takes the question in Prompt, searches the knowledge base connected to Knowledge for the most relevant passages, adds them to the prompt and asks the chat model on Model to answer from them. It returns the answer and the sources it used.

Retrieved passages are added inside a block marked as reference data, not instructions. A document that says “ignore your instructions” is treated as text to quote, not as a command.

With a chat memory connected to Memory, the agent has a conversation:

1. Before calling the model, it loads the earlier messages and adds them as chat history.
2. It saves the incoming prompt as a user message.
3. After the model answers, it saves the reply as an assistant message (a structured reply from an output parser is saved as JSON text).

This is how a Chat Trigger gets its replies: connect the same memory to the trigger and to this agent.

When to use it

• A chatbot that answers from your documents, built with a Chat Trigger.
• Answer one question from a knowledge base in the middle of a workflow, with the sources it used.
• Ground a model’s answer in pages you indexed with the Indexer.

To answer fully on-device by quoting your documents, use Q&A Agent with a Local Q&A Model. When the model must also call tools, use Tools Agent, which accepts the same knowledge base.

Inputs and settings

Einstellung	Typ	Was es bewirkt
Prompt PFLICHT(erforderlich)	Text
Options	Group	The options for the node.
Options › System message	Long text	The system message to use.
Options › Knowledge documents (k)	Number	How many documents to retrieve from the connected knowledge base. Standard: 4
Options › Max knowledge context length	Number	Maximum characters of retrieved knowledge to inject into the prompt. Standard: 4000
Options › Fail on knowledge error	Toggle	If enabled, the node fails when knowledge retrieval errors instead of answering without context. Standard: Off
Options › Search type	Choice	How passages are picked from the connected knowledge base. Options: Most similar, Diverse (MMR). Standard: Most similar
Options › Match by	Choice	How a knowledge base is searched with "Most similar". Diverse (MMR) and other vector stores always match by meaning. Options: Keywords and meaning, Meaning only. Standard: Keywords and meaning
Options › Minimum relevance	Number	Drop passages less similar to the question than this (0 to 1). 0 keeps every passage. Standard: 0
Options › Metadata filter	Key-value pairs	Only search passages whose metadata matches every key = value (e.g. sourceType = web). Keys use letters, numbers and _; numbers and true/false are matched as such.
Options › Device-only knowledge with a cloud model	Choice	A knowledge base marked "Device only" is never sent to a cloud model. Choose what the node does when its model is a cloud model. Options: Stop with an error, Answer without it. Standard: Stop with an error

Prompt is the question. Behind a Chat Trigger, use {{ $input.chatInput }}.

Knowledge base options

The knowledge base options are the same on the RAG Agent, the Q&A Agent and the Tools Agent:

• Search type: Most similar picks the passages closest to the question. Diverse (MMR) picks passages close to the question but different from each other, so you get fewer repeats.
• Match by: with Keywords and meaning, exact words, names and codes are found too. Meaning only uses similarity alone.
• Minimum relevance: raise it (for example 0.5) to drop weak passages.
• Metadata filter: useful keys are sourceType (file, web, tab, text, workflow, chat), sourceId, title, url, fileName and page. For example sourceType = web searches only saved web pages.
• Device-only knowledge with a cloud model: a knowledge base marked Device only (see Privacy) is never sent to a cloud model. Models that count as on-device are Web LLM, Transformers Chat, Ollama, Chrome AI, Local Q&A Model and Local Classifier. With any other model, the node stops with an error, or with Answer without it runs without the knowledge base and logs a warning.

Outputs

• Output

Each output item has these fields:

Field	Typ	What it holds
response	Any	The model reply; the parsed result when an output parser is connected
sources	List of entries	The knowledge base sources used for the answer

Shape of an output item

{
  "response": null,
  "sources": []
}

sources is present when a knowledge base is connected (it may be empty). Each entry has title, url (when there is one), knowledgeId, sourceId and score. After a run, the node shows a badge such as 3 sources cited.

Dependencies and credentials

Slot	Accepts	Connections
Model PFLICHT(erforderlich)	chat model	One node.
Memory	chat memory	One node.
Knowledge	vector store	One node.
Output Parser	output parser	One node.

• Model: a chat model such as Chat OpenAI or Web LLM. Cloud models hold their own API key.
• Knowledge: a Local Knowledge node.
• Memory: a Chat Memory, to remember the conversation.
• Output Parser: shapes the answer, for example a Structured Output Parser.

Example workflow

1. Chat Trigger opens a chat window.
2. RAG Agent with Prompt {{ $input.chatInput }} and System message Answer only from the documents. If the answer is not there, say so.
3. Connect an Chat OpenAI model to Model, a Local Knowledge node that points at your “Help center” knowledge base to Knowledge, and one Chat Memory to both Memory and the Chat Trigger.

Every message you send runs the workflow; the agent’s replies appear in the chat window.

Common issues

• “… is on this device only, but … is a cloud model.” The knowledge base is marked Device only. Use an on-device model, change the knowledge base’s privacy, or set Device-only knowledge with a cloud model to Answer without it.
• “Knowledge base “…” doesn’t exist on this device.“ The workflow comes from another device or the knowledge base was deleted. Pick a knowledge base in the Local Knowledge node, or import its .awkb file.
• The answer ignores your documents. No passages were found. Lower Minimum relevance, check the Metadata filter, or try the question in the knowledge base’s Test search tab.
• The chat window stays empty. The Chat Trigger and the agent use different memories. Connect the same Chat Memory to both.
• “Model not found”. The connected model node is not set up: open it and choose a model and its credential.

Related nodes

• Chat Trigger: turn the workflow into a chatbot.
• Q&A Agent: one-shot answers, also fully on-device.
• Tools Agent: knowledge plus tools.
• Indexer: fill the knowledge base from a workflow.