How to Use RAGFlow

This guide covers step by step instructions to get started with RAGFlow.

1. RAGFlow is a RAG engine and needs to work with an LLM to offer grounded, hallucination-free question-answering capabilities. However the VM Comes with preinstalled Ollama LLMs (Deepseek-R1, Qwen 2.5, Mistral, Gemma, Llama, LLaVA) allowing you to get started quickly with these local LLMs and also provide you with the ability to easily add your own models via Ollama or any Other LLM provider, giving you complete control over what models you use and how you run them.

Below is the list of available LLMs on this VM setup.

llava:latest
llava:7b
llava:13b
llava:34b
qwen2.5:14b
qwen2.5:32b
qwen2.5:72b
qwen2.5:7b
deepseek-r1:70b
deepseek-r1:32b
deepseek-r1:14b
deepseek-r1:8b
nomic-embed-text:latest
gemma2:27b
mistral:latest
llama3.3:latest

2. To add and configure an LLM:

• Login to your RAGFlow Web Interface.

• Click on your account icon on the top right of the page.

• Here you can check the list of supported Data Sources.

• Click on “Model providers” tab from left menu to add your choice of model. You can select various models from the list of available models. Here we will add deepseek-r1:8b chat model and nomic-embed-text embedding model. For that type ollama in search box and click on Add button.

Note: From Ragflow V0.22.0, default BAAI and Youdao Text Embedding models are no longer available.

• In the popup window, complete basic settings for Ollama:

  • Select the type of model from the dropdown menu - chat, embedding, vision and image2text. here we will select chat.
  • Ensure that your model name and type match those been pulled on this VM , e.g deepseek-r1:8b. To check the name of the ollama models available on the VM, connect via terminal and run ollama ls command as shown in step 7 of this guide.
  • In Ollama base URL, put the URL as http://host.docker.internal:11434 , make sure not to keep extra space in model name or base url while adding the model.
  • Token size e.g 1024
  • OPTIONAL: Switch on the toggle under Does it support Vision? if your model includes an image-to-text model. e.g Llava
  • Click on Ok to save the model. It takes some time to save. Once done it will show Modified message at the top of this page.
  • Similarly click on Add button on Ollama and add the embedding model here. You can add more than one model by following these steps.

• Once Added you can see the list of added models here:

3. At the top of this page, Under Set Default Models section, set the default models: Chat model, Embedding model, Image-to-text model, and more.

We will set chat and embedding models we just added in above step 2.

4. Create your first knowledge base:

You are allowed to upload files to a knowledge base in RAGFlow and parse them into datasets. A knowledge base is virtually a collection of datasets. Question answering in RAGFlow can be based on a particular knowledge base or multiple knowledge bases. File formats that RAGFlow supports include documents (PDF, DOC, DOCX, TXT, MD, MDX), tables (CSV, XLSX, XLS), pictures (JPEG, JPG, PNG, TIF, GIF), and slides (PPT, PPTX).

• To create your first knowledge base:

  Click the Dataset tab in the top menu of the homepage and click + Create Knowledge base.

  

  

  Input the name of your knowledge base, Embedding model from the dropdown. As we set nomic-embed-text as our default embedding model, this model is available in the dropdown here. Select Built-in if you are going to upload the documents or select Choose Pipeline if your data source is set via the agent pipeline, Select the Build-in from dropdown. Once ready click Save button.

  

• On this page, click + Add file > Local files to start uploading a particular file to the knowledge base.

• In the uploaded file entry, click the play button to start file parsing. To parse the uploaded file, RAGFlow will use default Embedding model set in Set default models at above Step 3.

• Once parsing is completed, you can see chunk result by clicking on the file name.

5. Setting AI Chat in RAGFlow

Once you have created your knowledge base and finished file parsing, you can go ahead and start an AI conversation based on a particular knowledge base or multiple knowledge bases.

• Click the Chat tab in the middle top of the homepage > Create Chat. Give a name to your Chat Assistant and click Save. Click on newly created Chat Assistant to show the Chat Configuration dialogue of your next dialogue.

• Provide different chat setting in setting pop-up. Select your Knowledge base, provide Tavily API Key , set the temperature, select the model etc. Save your Changes and start Chatting with your AI Assistance. Click on + icon from left pane to start the chat.

Note: RAGFlow offer the flexibility of choosing a different chat model for each dialogue, while allowing you to set the default models in System Model Settings.

5. Setting Up AI search

The Key difference between an AI search and an AI chat is that a chat is a multi-turn AI conversation where you can define your retrieval strategy and choose your chat model whereas an AI search is a single-turn AI conversation using a predefined retrieval strategy and the system’s default chat model.

To start the search feature:

• Click on Search option at the top of the page. Click on Create Search.

• Check the Search Settings and set your Dataset and other options here. Save the changes.
• Run the Search.
• Click on output to go to the page for your search query.

6. Create an Agent

Agents and RAG are complementary techniques, each enhancing the other’s capabilities in business applications.

• Click the Agent tab in the middle top of the page to show the Agent page > Create Agent. As shown in the screenshot below, the cards on this page represent the various agent templates, which you can continue to edit. You can also create new agents from Scratch.

• To create an agent from one of the templates, click the desired card, such as SEO Blog, name your agent in the pop-up dialogue, and click OK to confirm.

• It will show no-code workflow editor page. Edit/Update the workflow as per your requirement and Save the Changes. Run the Agent to get the output.

7. If you want to install additional ollama models on this VM then follow below steps:

• Connect to SSH Terminal of this VM. Please refer to Deployment Guide of respective Cloud Platform to launch and connect to the VM.

• Check available Ollama models by running

ollama ls

• To pull new model run

ollama pull modelname

e.g ollama pull bge-m3:latest , Check more models on Ollama Official Site

• Now this model can be added in your RAGFlow Web Interface as shown in above Step2.

8. If you have deployed this solution using GPU instance then you can monitor GPU use by entering below command in terminal while running chat/search/agents in RAGFlow Web Interface.

watch -n 1 nvidia-smi

For more details, please visit Official Documentation page