Unique Building Blocks of DSPy
DSPy offers unique features for simplifying the process of building an advanced RAG pipeline.
First it offers Signatures, which are a declarative/programmatic method of structuring prompts. This helps avoid long and tedious verbalization in creating prompts.
Secondly, there are Modules, which abstract a prompting technique on top of Signatures. One of the popular modules is dspy.ChainofThought, which injects a rationale (e.g., “think step by step to come to a conclusion”) before the outcome of a prompt. This module makes multiple calls to the language model (using the prescribed Signature) and gradually builds reasoning until it is ready to answer the output of the Signature using the input questions and the reasoning it has developed. This approach significantly improves the quality of the final answer.
Finally, we have Optimizers that can tune your entire DSPy program based on a Metric. This program can be a single Module or a set of Modules bundled together to perform a complicated task. DSPy Metrics can define exactly which aspect of the program we want to optimize. For example, we can define a Metric that checks for the length of the final response generated by the language model. This way, we can optimize our final program to only output responses of a certain desired length.
The above features make DSPy a powerful alternative to traditional prompt engineering techniques. Crafting multiple hand-written prompts can often be tedious and clunky, and unpredictable in terms of outcome, especially when changing the underlying Large Language Model. This is because each language model adheres to its own specific inner template that it performs best on.
DSPy overcomes these challenges by automating the laborious work of prompt engineering through the use of Signatures, Modules, and Optimizers.
Multi-Hop Optimized RAG
In a multiple-hop RAG pipeline, the original question is broken down into multiple queries over several steps. In each step, the language model forms a query and retrieves context based on it. This iterative process of collecting context by breaking down the original question into smaller queries allows for a more enhanced retrieval method. This way, the language model has access to a factually richer context for generating the final response.
To further optimize the pipeline, we can introduce a few metrics. We can ensure that none of the sub-queries are rambling by limiting their length to fewer than 100 characters. Additionally, we verify that no two sub-queries are similar to each other. This way, we make sure we are retrieving unique contexts in each step.
![](https://cdn.prod.website-files.com/6473d8d02a3cf26273f2787d/66829320d3e9195c5b2a6499_AD_4nXeIxHEtuPCZl21Tuveet46eNrrVoSbYQdK9DF06wA_apfxFaoJ9xCOHGtILGwPcB0aoeLyYACYsKMzMT_0QGzd-ZUAbpsOnBlVOJQ0eySG-lSD-DOh5d32I_qeC0kb5dj4VfpZiv2v2agEsDEeswA3VX98.png)
In this blog, we’ll demonstrate how to build a multi-hop optimized RAG using DSPy.
Build with E2E
This blog is part of the “Build with E2E series” which showcases various emerging technologies for building AI applications.
E2E Networks is an NSE-listed AI-first hyperscaler that provides advanced cloud computing and GPU infrastructure to developers, startups, enterprises, and research organizations.
E2E Cloud’s TIR AI-ML platform is a powerful AI development platform built around Jupyter Notebooks. TIR includes pre-configured environments for popular frameworks like PyTorch, TensorFlow, Stable Diffusion, RAPIDS, and Transformers. It also features an inference platform for deploying models directly on the E2E Cloud platform.
Check out more about the platform at https://myaccount.e2enetworks.com/.
Let’s Code
Begin by installing the dependencies.
We’ll be using an external dataset of Mobile Phone Reviews from Kaggle. This dataset contains cell phone brands and their reviews. The aim is to design a RAG pipeline that can list various phone names by analyzing their reviews against the specific questions posed by the user. Let’s format the data before inserting it into the vector database.
Some examples of how the data looks:
['Phone Name: Apple iPhone 3G Black, 16GB\nReview: My son loves it, I cant even touch it. LOL',
'Phone Name: HTC Desire 610 8GB Unlocked GSM 4G LTE Quad-Core Android 4.4 Smartphone - Black (No Warranty)\nReview: The phone is very good , takes very sharp pictures but the screen is not bright',
'Phone Name: Apple iPhone 6, Space Gray, 128 GB (Sprint)\nReview: I am very satisfied with the purchase, i got my iPhone 6 on time and even received a screen protectant with a charger. Thank you so much for the iPhone 6, it was worth the wait.']
Let’s take 2000 rows of the dataset at random and insert them into the Qdrant Vector Database.
We’ll configure DSPy’s retriever and language model so that it can use them to make calls.
Make sure you have Ollama installed on your system. You can do so by running the following in your terminal.
Launch the server and pull the LLM llama3.
Now, we’ll write two DSPy Signatures. One to generate the sub-queries, and another to generate the final answer.
The docstring is important as it will contain the instruction line that you want to include in your prompt. Similarly, the names of the variables should be self-explanatory and meaningful, as they will also be passed on to the prompt.
Then we write a simplified Baleen class that defines the multi-hop logic of our RAG pipeline. The __init__ method of the class is used to declare and initialize all the modules that are to be used in the program. In the forward method, the modules are run using the inputs of the pipeline. The final output is returned as a dspy.Prediction object.
It is important to follow this structure while writing DSPy programs, as they will be used during the optimization process.
Let’s create a dummy dataset for our trainset.
Next, we’ll write our Metric to ensure that the hops are of limited length and are not repetitive. DSPy Metrics return bool values and, therefore, are strict measures that the optimizer will make the pipeline adhere to.
Note that we are not making any use of the trainset here, since we are only optimizing for the hops. If we also want to optimize for the final response, we can write a function that validates the predicted answer (pred.answer) given an example question to be similar to the example answer. (Logic in comments.)
With that set, let’s optimize the pipeline.
The optimization process tweaks each of the DSPy modules involved in the program. This includes the language model weights of these modules, their instructions, and demonstrations of input and output behavior.
Now, let’s try out a few tests:
"Here is the numbered list of phone names that meet your criteria:
1. **BLU Energy X Plus**
2. **BLU VIVO 5R Refresh**
3. **I9220(N9000)**
4. **BLU PURE XL**
These phones are known for their long-lasting batteries and high-quality displays with good screen resolutions.
Here's the list in more detail:
1. **BLU Energy X Plus**: With its massive 4000 mAh super battery, this phone is designed to keep going all day.
2. **BLU VIVO 5R Refresh**: This phone has a large 3200 mAh battery and a high-resolution display with a screen resolution of up to FHD (1080p).
3. **I9220(N9000)**: Although it's an older model, this phone still packs a punch with its dual batteries and decent screen resolution.
4. **BLU PURE XL**: This phone has a large 3500 mAh battery and a high-resolution display with a screen resolution of up to FHD (1080p).
Please note that the ratings for these phones may vary depending on the source and methodology used in expert reviews."
Based on the provided context and reviews, I've generated a numbered list of phone names that meet the criteria:
1. BLU Studio Selfie (Grey) - Unlocked Smartphone - GSM
2. BLU Studio Selfie (Gold) - Unlocked Smartphone - GSM
These two phones have great reviews and are likely to perform well in low-light conditions, considering the positive comments about their cameras.
Note: The other phone names mentioned do not meet the criteria as they have mixed or poor camera performance according to their reviews.
If we want to see the LM calls that are happening under the hood:
Write a simple search query that will help answer a complex question.
---
Follow the following format.
Context: may contain relevant facts
Question: ${question}
Reasoning: Let's think step by step in order to ${produce the query}. We ...
Query: ${query}
---
Context: N/A
Question: Which smartphones are highly rated for its low-light camera performance also has a great front camera
Reasoning: Let's think step by step in order to Here is the search query:
Context: N/A
Question: Which smartphones are highly rated for its low-light camera performance also has a great front camera?
Reasoning: Let's think step by step in order to produce the query. We can start by searching for "best smartphone cameras" and then add filters such as "low light performance" and "front facing camera". This will help us find smartphones that are highly rated for their camera capabilities, including low-light performance.
Query: ("best smartphone cameras") AND ("low light performance" OR "night mode") AND ("great front camera")
This query uses a combination of keywords to search for smartphones with high-rated cameras that perform well in low-light conditions and have great front-facing cameras. The quotes around the phrases ensure that we find exact matches, rather than individual words.
Write a simple search query that will help answer a complex question.
---
Follow the following format.
Context: may contain relevant facts
Question: ${question}
Reasoning: Let's think step by step in order to ${produce the query}. We ...
Query: ${query}
---
Context:
[1] «Phone Name: BLU Studio 7.0 II -Unlocked Smartphone - US GSM - Grey
Review: Phone has a poor camera»
[2] «Phone Name: BLU Studio C Super Camera -Unlocked Smartphone - US GSM- Green
Review: good»
[3] «Phone Name: BLU S480U Unlocked Studio 7.0 II Smartphone with 5MP Main Camera (Gold)
Review: Great phone just bad camera and storage room»
Question: Which smartphones are highly rated for its low-light camera performance also has a great front camera
Reasoning: Let's think step by step in order to Here is the search query that can help answer the complex question:
---
Context:
[1] «Phone Name: BLU Studio 7.0 II -Unlocked Smartphone - US GSM - Grey
Review: Phone has a poor camera»
[2] «Phone Name: BLU Studio C Super Camera -Unlocked Smartphone - US GSM- Green
Review: good»
[3] «Phone Name: BLU S480U Unlocked Studio 7.0 II Smartphone with 5MP Main Camera (Gold)
Review: Great phone just bad camera and storage room»
Question: Which smartphones are highly rated for its low-light camera performance also has a great front camera
Reasoning: Let's think step by step in order to ${produce the query}. We need to find phones that have good reviews for their cameras, specifically low-light performance. Additionally, we want phones with high ratings for their front-facing cameras.
Query: "best smartphones with good low light camera and great selfie camera"
This search query uses relevant keywords from the context and question to help answer the complex question about which smartphones are highly rated for their low-light camera performance and have a great front-facing camera.
Generate a numbered list of phone names
---
Follow the following format.
Context: may contain relevant facts
Question: ${question}
Reasoning: Let's think step by step in order to ${produce the answer}. We ...
Answer: ['$', '{', 'a', 'n', 's', 'w', 'e', 'r', '}']
---
Context:
[1] «Phone Name: BLU Studio 7.0 II -Unlocked Smartphone - US GSM - Grey
Review: Phone has a poor camera»
[2] «Phone Name: BLU Studio C Super Camera -Unlocked Smartphone - US GSM- Green
Review: good»
[3] «Phone Name: BLU S480U Unlocked Studio 7.0 II Smartphone with 5MP Main Camera (Gold)
Review: Great phone just bad camera and storage room»
[4] «Phone Name: BLU Studio Selfie - Smartphone - GSM Unlocked - Grey
Review: I love it thanks»
[5] «Phone Name: BLU Studio Selfie - Smartphone - GSM Unlocked - Gold
Review: Great»
[6] «Phone Name: BLU S480U Unlocked Studio 7.0 II Smartphone with 5MP Main Camera (Gold)
Review: It works great»
Question: Which smartphones are highly rated for its low-light camera performance also has a great front camera
Reasoning: Let's think step by step in order to Based on the provided context and reviews, I've generated a numbered list of phone names that meet the criteria:
1. BLU Studio Selfie (Grey) - Unlocked Smartphone - GSM
* Review: "I love it thanks" implies good performance for its camera.
2. BLU Studio Selfie (Gold) - Unlocked Smartphone - GSM
* Review: "Great" suggests a well-performing front camera.
These two phones have great reviews and are likely to perform well in low-light conditions, considering the positive comments about their cameras.
Generate a numbered list of phone names
---
Follow the following format.
Context: may contain relevant facts
Question: ${question}
Reasoning: Let's think step by step in order to ${produce the answer}. We ...
Answer: ['$', '{', 'a', 'n', 's', 'w', 'e', 'r', '}']
---
Context:
[1] «Phone Name: BLU Studio 7.0 II -Unlocked Smartphone - US GSM - Grey
Review: Phone has a poor camera»
[2] «Phone Name: BLU Studio C Super Camera -Unlocked Smartphone - US GSM- Green
Review: good»
[3] «Phone Name: BLU S480U Unlocked Studio 7.0 II Smartphone with 5MP Main Camera (Gold)
Review: Great phone just bad camera and storage room»
[4] «Phone Name: BLU Studio Selfie - Smartphone - GSM Unlocked - Grey
Review: I love it thanks»
[5] «Phone Name: BLU Studio Selfie - Smartphone - GSM Unlocked - Gold
Review: Great»
[6] «Phone Name: BLU S480U Unlocked Studio 7.0 II Smartphone with 5MP Main Camera (Gold)
Review: It works great»
Question: Which smartphones are highly rated for its low-light camera performance also has a great front camera
Reasoning: Let's think step by step in order to Based on the provided context and reviews, I've generated a numbered list of phone names that meet the criteria: 1. BLU Studio Selfie (Grey) - Unlocked Smartphone - GSM * Review: "I love it thanks" implies good performance for its camera. 2. BLU Studio Selfie (Gold) - Unlocked Smartphone - GSM * Review: "Great" suggests a well-performing front camera. These two phones have great reviews and are likely to perform well in low-light conditions, considering the positive comments about their cameras.
Answer: [] Based on the provided context and reviews, I've generated a numbered list of phone names that meet the criteria:
1. BLU Studio Selfie (Grey) - Unlocked Smartphone - GSM
2. BLU Studio Selfie (Gold) - Unlocked Smartphone - GSM
These two phones have great reviews and are likely to perform well in low-light conditions, considering the positive comments about their cameras.
Note: The other phone names mentioned do not meet the criteria as they have mixed or poor camera performance according to their reviews.
Final Words
DSPy provides an entirely new paradigm for building AI applications by offering a programmatic approach to prompting instead of manual crafting. Any multistage AI agentic system is extremely sensitive to prompt quality and design. By offloading the tedious prompt engineering work to DSPy, we can easily conceive complicated multistage agents and RAG pipelines for our specific use cases.
By taking inspiration from this blog, you can deploy your DSPy-powered AI stack on E2E Cloud. Head over to E2E Networks to spin up a cloud GPU node, or use the readymade TIR AI platform!