In a typical AI application trained on a vast corpus of data, the model may not have access to specific, customized data, leading to less accurate or relevant responses. However, with Retrieval-Augmented Generation (RAG), businesses can provide their own data to the AI application. RAG allows the AI to retrieve and utilize this custom data to generate more precise and contextually accurate responses.
Additionally, RAG enhances data security. In many situations, businesses may not want to share proprietary or sensitive data with AI companies due to privacy or security concerns. By using RAG, they can keep their data private and secure while still benefiting from the AI's capabilities. This approach ensures that the data is used solely within a controlled environment, mitigating the risks associated with data sharing.
To understand the process, let's go through the various steps involved in building a Retrieval Augmented Generation (RAG) application step by step.
The Tools We Shall Use
PGVector
First, we’ll convert our data into embeddings, which are numerical representations of the data that capture its semantic meaning. To efficiently manage these embeddings, we’ll use a PGVector container to store them. This approach allows us to retrieve the embeddings whenever needed without having to convert the data each time. Converting data into embeddings repeatedly can be time-consuming and resource-intensive, requiring significant computational power and money. By storing our embeddings in a PGVector container, we avoid this repetitive and costly process, ensuring that our embedded data is readily available for quick and efficient retrieval.
The PGVector extension for PostgreSQL is a tool that helps you work with vectors right inside your database. Here's a breakdown of the key concepts in simpler terms:
What Does PGVector Do?
Storage: It allows you to store vectors in your PostgreSQL database.
Manipulation: You can change and work with these vectors easily.
Querying: It provides tools to search and analyze vector data efficiently.
LangChain
LangChain is a framework designed to streamline the development of applications utilizing large language models (LLMs). As a language model integration framework, LangChain supports a wide range of use cases including document analysis and summarization, and chatbots.
Llama 3
Llama 3 is the next-generation open-source LLM developed by Meta that's been trained on internet-scale data. This allows it to understand and comprehensively respond to language, making it suitable for tasks like writing creative content, translating languages, and answering queries in an informative way.
Gradio UI
Gradio is an open-source Python package that allows you to quickly build a demo or web application for your machine learning model, API, or any Python function. You can then share a link to your demo or web application in just a few seconds using Gradio's built-in sharing features.
E2E Cloud
E2E’s AI development platform, TIR, allows you to host LLMs and fine-tune them easily. TIR has seamless integration with PGVector and Llama 3 – and we can use that as well as an alternative deployment option.
Ollama
Ollama is a streamlined open-source tool used for running open-source LLMs locally, like Llama 2, Mistral, Falcon, etc. Ollama bundles all the necessary ingredients for the model to function locally including model core data (weights), instruction (configuration), and dataset into a neat package managed by a modelfile.
To use Ollama, one simply needs to download it from the website. It's available for macOS, Linux, and Windows.
Let's Code
Since we are working with a locally deployed LLM, we need an advanced GPU for our task. E2E Cloud provides a range of GPUs geared towards building AI applications. You can check out the offerings at https://myaccount.e2enetworks.com/.
Go ahead and register, and then launch a GPU node. Alternatively, you can head to TIR and launch a Llama 3 endpoint and a PGVector endpoint.
Loading all the libraries:
Load the text and split it into manageable chunks.
To simulate real-time data updates, we are using the Faker library to generate dynamic data. Here, we’re creating log files and refreshing the data every 60 seconds. You can customize it with your own real-time data as needed.
Connecting to PGVector database and storing embedding:
You can run the following command to spin up a Postgres container with the PGVector extension:
Once our connection to PGVector is established and we’ve created our container, we can start adding real-time data using the code below. Then we’ll use a retriever to access the stored embeddings when needed.
Initialize the LLM model. Make sure you’ve installed Ollama on your system, launched an Ollama server, and pulled Llama 3. You can follow the instructions here.
Download Ollama - Link
After downloading, you can run the model locally by using simple code on your terminal. -
![](https://cdn.prod.website-files.com/6473d8d02a3cf26273f2787d/6666ea40c7bda3bb3c172abe_AD_4nXcjA5dZSf9hD8tcveXk6MTVRKgKDKeef6xz4UV6VTa3dyJXKabFC0Z9dLudpaomXR6lAEdAI8KUnh_LOcnQU-79NGq-_Ks_G_-jV1ACR6-OrBYC4CNDIBA7BKlKifsCvM3NfE0p4-NdcLy7IM1Q3LVgPd6_.png)
Finally getting the query back and also creating the Gradio UI interface.
Below is the response I received:
![](https://cdn.prod.website-files.com/6473d8d02a3cf26273f2787d/6666ea406b920d1188d1b13f_AD_4nXeLdpxp_vF4U-cXq0_dVx9ZbqlQi6UOT0Paym0PRHOkOqzSky3z3TCrt6LQsVHiaRVlILrPHk5oQMjAKE6GzAt47dwjZZKFyzk_GkUTg5we_h9gB-JZNugL1bA_dL3Wmmmw_cnEF-YTakwY_ESxdCq2RS6A.png)
Conclusion
By leveraging Retrieval Augmented Generation (RAG), you can create a customized chatbot that is both highly accurate and secure, tailored specifically to your data. Tools like Gradio UI and Streamlit facilitate the creation of visually appealing interfaces with ease, enhancing the user experience.In applications like customer support, knowledge management, and information retrieval, users often ask complex questions that require detailed and precise answers. Traditional natural language processing (NLP) systems, including both retrieval-based and generation-based models, face significant challenges in delivering high-quality responses.