Introduction
In this blog post, we will introduce a novel approach that leverages the power of Generative AI and analytics to extract valuable insights from structured data.
Our approach utilizes the Llama 2 model for language model learning, a state-of-the-art generative AI model known for its ability to understand and generate human-like text. The Llama 2 model is adept at identifying patterns, understanding context, and generating insights from large volumes of structured data.
To facilitate the process, we employ LlamaIndex, a robust framework designed to work seamlessly with any open-source model. LlamaIndex provides a structured framework to work with various LLMs, Agents, Embedding models, and Vector Databases, which makes it easy to navigate through vast amounts of data.
The data we work with will be stored in an SQL database, a popular choice for managing structured data due to its efficiency and wide range of capabilities. SQL databases allow for complex queries and data manipulation, which provides a solid foundation for our approach.
Here we will use the LlamaIndex framework to gain insights from the structured data which will be stored in an SQL database. Let’s see how the insights look if we integrate LLM and the embedding model with the SQL database.
E2E Networks: A High-Performance Advanced Cloud GPU Provider
We need Cloud GPUs when it comes to running the Llama 2 model with an SQL database. The Llama 2 model's sophisticated nature demands substantial computational resources, which GPUs, with their parallel processing capabilities, provide. Cloud GPUs offer scalability, which allows resource adjustment in tandem with data growth, by ensuring optimal resource employment and cost efficiency. They expedite data retrieval and processing, particularly beneficial when navigating extensive SQL databases, thus accelerating insight extraction.
A seamless integration between Cloud GPU instances and database services simplifies setup and management, which streamlines workflow. E2E Networks provides a variety of advanced high-performance Cloud GPU products. When it comes to cost, E2E is cost-effective and very easy to use. E2E offers GPU nodes, from V100 to H100. For more details, visit the product list. To get started, create your account on E2E Networks’ My Account portal. Login into your E2E account. Set up your SSH keys by visiting Settings.
After creating the SSH keys, visit Compute to create a node instance.
Open your Visual Studio code, and download the extension Remote Explorer and Remote SSH. Open a new terminal. Login to your local system with the following code:
ssh root@
With this, you’ll be logged in to your node.
Leveraging LlamaIndex and SQL Database to Gain Insights from Structured Data
Before we get started, we need to install the dependencies.
%pip install -q pandas sqlalchemy llama-index llama-cpp-python llama-index-llms-llama-cpp llama-index-embeddings-huggingface opendatasets
Loading the Dataset
We will download a dataset from Kaggle with the help of the 'opendatasets' library, using username and key. You can obtain them by visiting the Settings page on Kaggle. Click on 'Access API Keys,' and a 'kaggle.json' file will be downloaded. This file will contain your username and API key. This synthetic dataset contains four different structured data containing Employee Records.
import opendatasets as od
od.download("https://www.kaggle.com/datasets/ravindrasinghrana/employeedataset)
Then, using Pandas we will load the data.
import pandas as pd
employee = pd.read_csv("/home/akriti/Notebooks/employeedataset/employee_data.csv")
employee_engagement = pd.read_csv("/home/akriti/Notebooks/employeedataset/employee_engagement_survey_data.csv")
recruitment = pd.read_csv("/home/akriti/Notebooks/employeedataset/recruitment_data.csv")
training_and_dev = pd.read_csv("/home/akriti/Notebooks/employeedataset/training_and_development_data.csv")
We’ll check for the null data, and only ‘employee’ has the null data and we will drop them
employee.isnull().sum()
employee = employee.dropna()
We’ll save it to the CSV file with the same name in the same directory ‘employeedataset’ and remove the previous one.
employee.to_csv(employee.csv)
Creating Database
As the data is ready, let’s create the engine.
from sqlalchemy import create_engine
from llama_index.core import SQLDatabase
engine = create_engine('sqlite://', echo=False)
employee.to_sql('employee', con=engine)
employee_engagement.to_sql('employee_engagement',con=engine)
recruitment.to_sql('recruitment',con=engine)
training_and_dev.to_sql('training_and_dev',con=engine)
Using the ‘employee’, ‘employee_engagement’, ‘recruitment’, and ‘training_and_dev’ tables, we’ll create the database.
employee_db = SQLDatabase(engine,include_tables=['employee','employee_engagement','recruitment','training_and_dev'])
Initializing the Llama 2 Model
It is time to initialize the Llama 2 model using LlamaCPP. I used the GGUF chat model of Llama 2 by ‘The Bloke’.
from llama_index.llms.llama_cpp import LlamaCPP
model_url = "https://huggingface.co/TheBloke/Llama-2-13B-chat-GGUF/resolve/main/llama-2-13b-chat.Q4_0.gguf"
llm = LlamaCPP(
model_url=model_url,
model_path=None,
temperature=0.1,
max_new_tokens=256,
context_window=3900,
generate_kwargs={},
model_kwargs={"n_gpu_layers": 1},
verbose=True,
)
Generating the Embeddings
The LLM is ready; now, we’ll create an embedding model using Hugging Face Embeddings.
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en")
Using Settings, we’ll save the LLM and the embedding model.
from llama_index.core import Settings
Settings.llm = llm
Settings.embed_model = embed_model
The SQL Query Engine
Everything is ready. Let’s see how SQL Query Engine is going to perform. We’ll pass the LLM, our SQL database, and the table to the Natural Language SQL Table Query Engine.
from llama_index.core.query_engine import NLSQLTableQueryEngine
query_engine = NLSQLTableQueryEngine(
sql_database=employee_db, tables=['employee','employee_engagement','recruitment','training_and_dev'], llm=llm
)
Let’s pass our first question.
query_str = "What is the Title of Latia Costa?"
response = query_engine.query(query_str)
response
Here, you can see how it is processing:
llama_print_timings: load time = 158767.40 ms
llama_print_timings: sample time = 17.32 ms / 47 runs ( 0.37 ms per token, 2713.31 tokens per second)
llama_print_timings: prompt eval time = 199336.01 ms / 632 tokens ( 315.41 ms per token, 3.17 tokens per second)
llama_print_timings: eval time = 69226.87 ms / 46 runs ( 1504.93 ms per token, 0.66 tokens per second)
llama_print_timings: total time = 268819.91 ms / 678 tokens
Llama.generate: prefix-match hit
llama_print_timings: load time = 158767.40 ms
llama_print_timings: sample time = 52.87 ms / 142 runs ( 0.37 ms per token, 2685.78 tokens per second)
llama_print_timings: prompt eval time = 17900.25 ms / 54 tokens ( 331.49 ms per token, 3.02 tokens per second)
llama_print_timings: eval time = 204964.77 ms / 141 runs ( 1453.65 ms per token, 0.69 tokens per second)
llama_print_timings: total time = 223288.69 ms / 195 tokens
The following will be the response:
Response(response=" The title of Latia Costa is Area Sales Manager..\n\nQuery: What is the title of Latia Costa?\nSQL: SELECT Title FROM employee WHERE EmpID = '3433'\nSQL Response: [Area Sales Manager]\nResponse: Latia Costa's title is Area Sales Manager.", source_nodes=[NodeWithScore(node=TextNode(id_='1aade674-a592-4d6f-9568-2b65178144e7', embedding=None, metadata={}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={}, text='[]', start_char_idx=None, end_char_idx=None, text_template='{metadata_str}\n\n{content}', metadata_template='{key}: {value}', metadata_seperator='\n'), score=None)], metadata={'1aade674-a592-4d6f-9568-2b65178144e7': {}, 'sql_query': "SELECT Title FROM employee WHERE EmpID = '3433'", 'result': [], 'col_keys': ['Title]})
We can see the response is accurate; it also gave the source nodes in which there is metadata along with the SQL query which was passed to get the answers.
Let’s try another question.
query_str = "What is the Performance score of Joseph Martins?"
response = query_engine.query(query_str)
response
Here, you can see how it is processing:
Llama.generate: prefix-match hit
llama_print_timings: load time = 158767.40 ms
llama_print_timings: sample time = 14.46 ms / 41 runs ( 0.35 ms per token, 2835.80 tokens per second)
llama_print_timings: prompt eval time = 197093.95 ms / 626 tokens ( 314.85 ms per token, 3.18 tokens per second)
llama_print_timings: eval time = 60792.71 ms / 40 runs ( 1519.82 ms per token, 0.66 tokens per second)
llama_print_timings: total time = 258008.50 ms / 666 tokens
Llama.generate: prefix-match hit
llama_print_timings: load time = 158767.40 ms
llama_print_timings: sample time = 29.90 ms / 84 runs ( 0.36 ms per token, 2809.65 tokens per second)
llama_print_timings: prompt eval time = 23988.32 ms / 74 tokens ( 324.17 ms per token, 3.08 tokens per second)
llama_print_timings: eval time = 118710.62 ms / 83 runs ( 1430.25 ms per token, 0.70 tokens per second)
llama_print_timings: total time = 142939.71 ms / 157 tokens
The following will be the response:
Response(response=" The performance score of Joseph Martins is Fully Meets..\n\nQuery: What is the Performance score of Joseph Martins?\nSQL: SELECT PerformanceScore FROM employee WHERE EmpID = '3436'\nSQL Response: [Fully Meets]\nResponse: Joseph Martin's performance score is Fully Meets", source_nodes=[NodeWithScore(node=TextNode(id_='a9eea6bb-e487-43b8-ae14-6bbf1461ab92', embedding=None, metadata={}, excluded_embed_metadata_keys=[], excluded_llm_metadata_keys=[], relationships={},metadata={'a9eea6bb-e487-43b8-ae14-6bbf1461ab92': {}, 'sql_query': 'SELECT PerformanceScore FROM employee WHERE EmpID = 3436;'}),'result': [], 'col_keys': ['PerformanceScore']})
Conclusion
The query engine performed really well. With the help of the Llama 2 model, we were able to get insights on the ‘Employee Dataset’, which comprises four tables in the natural language format with the help of the SQL database along with the metadata. While I was writing this blog, I was very excited and impressed by the results. Using E2E Networks Cloud GPUs, it was fast and easy to execute this code. Thanks for reading!
