In this article, we will showcase how to use a hosted Llama 3 to generate code based on API documentation provided by any platform. The method is fairly simple: use Ollama to create a Llama 3 endpoint, harness prompt engineering, and leverage the Beautiful Soup library to generate API methods from documentation pages. As an example, we will use NewsAPI, which allows you to search worldwide news with code.
The key thing to keep in mind here is - the open-source LLM will not browse, download, and parse content for you; instead, you need to create the scaffolding to get the content, clean it up, and provide it as part of the LLM prompt, in order to finally generate the documentation.
Let’s get started.
Prerequisite: Launch a GPU Node on E2E Cloud
Since we'll be using Llama 3, we need a cloud GPU node. For this, you have two options on E2E Cloud.
First, sign up to E2E Cloud if you haven’t - https://myaccount.e2enetworks.com
Next, launch a cloud GPU node. Pick one which has at least 16GB of GPU RAM available. If you are creating this setup for a large software dev company, you should go for H100 or A100 clusters.
To launch a cloud GPU node, click on ‘Compute’ on the left sidebar.
Guide to Building API Coding Assistant Using Llama 3
Prerequisites
- Python Environment: Ensure you have Python installed.
- Required Libraries: Install the following Python libraries:
The Python Script
Below is the complete Python script that can parse a documentation page, extract the documentation content, and use Llama 3 to generate code snippets.
Explanation
- Grab Documentation Content: The scrape_webpage function sends a GET request to the provided URL and retrieves the HTML content.
- Parsing HTML: Beautiful Soup is used to parse the HTML content and extract the text.
Running the Script
Execute the script to see the generated code snippet:
Results
When you execute the above code, you will see that it will provide you with output on the lines of:
Output:
—-
Now, this was a simple response. Let’s see if the LLM can generate the exact function that we can use.
To do this, we will modify the script.py slightly, so as to update the prompt.
Using this, the result we get is the following:
This looks great. We have essentially built a system which can refer to a documentation endpoint and generate code based on questions you ask. And the best part is this - you control the entire stack, so no API costs, no leakage of sensitive data.
Now, let’s move to the next step, where we integrate this with Visual Studio Code, the preferred development environment for many developers. For this, we will use the extension Continue.dev.
Bonus Step: Integrate with VS Code
Continue.dev is an innovative AI-powered development tool designed to streamline and enhance the software development process. Continue.dev leverages LLMs to provide real-time code suggestions, error detection, and intelligent refactoring recommendations. This tool not only improves coding efficiency but also helps maintain high code quality by identifying potential issues early in the development cycle.
To integrate this script into VSCode using the Continue.dev extension, follow these steps:
- Install Continue.dev Extension:some text
- Open VSCode.
- Go to the Extensions view by clicking on the Extensions icon in the Activity Bar on the side of the window or by pressing Ctrl+Shift+X.
- Search for "Continue.dev" and click Install on the Continue.dev extension.
- Follow the setup instructions provided by the Continue.dev extension documentation to configure it for your project. This typically involves setting up any necessary API keys and configuring the extension to recognize your development environment.
Testing if Continue.dev Is Installed Properly
Select the part of the code you want to edit and press CTRL+L to use continue.dev. For example:
It can also work as a co-pilot for you.
Integrating Llama 3 with Continue
Launch the Ollama server on 0.0.0.0 so that it can be accessed by external ips.
The server will be hosted on the default port 11434. Now pull Llama 3 into your local server.
Open Continue and click on the + sign and then open config.json.
![](https://cdn.prod.website-files.com/6473d8d02a3cf26273f2787d/6683ab57aa21c093fb6e8e7a_AD_4nXczO3rdwKcLu12dAbKK5sdiCQ7O9gAHcVPqx-TUH9ixXIKHVyy8mI79ZriDdozNtTMk81YJL9AkqwPc7maVF9M1BEm6IHcW4QcDWwfDWR4AhDy7YXPK4ubi3mSWd6t_gY_TZYgd7SHl0WEK9tR7vIjKioI.png)
![](https://cdn.prod.website-files.com/6473d8d02a3cf26273f2787d/6683ab5803cf47a825d3d63b_AD_4nXcQbxm93idvm_8fOBOvSA6NcZM205enstWIkj6qQ5RAtJj3cKZ9CWrFwqeT5nwdjj86kbHavZv5_t6OR91HzNL3w3QOS6umWFUgkLtJTEF2yerAhw6h0Unbo-1xQJoIutNTutxqLfz310SZoSrjoiJyo-4.png)
Add the following to the model's list.
![](https://cdn.prod.website-files.com/6473d8d02a3cf26273f2787d/6683ab59dccc9f909b9de0d5_AD_4nXf4WZ0xVwBz3Y4GiVIGZiF_Qx6AkS3xaIbFzBm399l8qKFWfHGXw_IUZD69VjB0WtzBVa9cnsIIm-dGUipsG5U50QCsO7wfdtIvndSZq6BS1s_eOKHUUwkJNCYNqv1NZSb7s5muxnjORcy0BrB35cthHc40.png)
Then we’ll write a custom slash command.
This command will allow Continue to read the API documentation, and then generate sample methods on how to use the API. In the same config.json, add the following to the customCommands list.
Now in the IDE, whenever you type /example_api along with the selected API documentation, the continue chat-interface will help you out with the example functions.
Conclusion
By combining web scraping, real-time documentation extraction, and advanced language models like Llama 3, developers can significantly enhance their productivity. This approach not only ensures that the code is up-to-date but also allows for rapid prototyping and development. Try integrating this method into your workflow and experience the benefits of automated code generation.Open-source LLMs, like Llama 3 or Mistral 7, offer a powerful leverage: companies can fine-tune and create an endpoint, and use it to build AI assistants that are deeply integrated into their internal workflow. This strategy is being used to build AI assistants for developers, analysts, content creators, and others. In this regard, coding assistants are particularly powerful, as they reduce development time and increase developer productivity, thus giving the company a significant edge over competition.