Introduction

In today’s corporate world, email has become an essential tool for communication. It's not only used by companies, it has become a part of everyone's life. However, it’s not easy to manage a large email inbox every day, either for individuals or for businesses.

In this blog post, we will explore AI assisted emailing with Large Language Models (LLMs), particularly Radiantloom Email Assist 7b. For there has been mounting evidence suggesting that smaller open access / open-source models are outperforming GPT-4 on specific tasks.

Radiantloom Email Assist 7b is the optimal email assistant. It’s a large language model which can help you summarize, organize, and manage your email. Just by its name, you can understand that this model contains 7 billion parameters. It is fine-tuned on Zephyr-7B Beta model, and it has been trained on a custom-curated dataset of 1,000 email-assistant summarization tasks.

Radiantloom is an AI-powered platform that helps businesses automate their workflows and improve their customer service. It offers a variety of services, like email assistance, customer support products and so on, powered by its proprietary large language models, which have been trained on a massive dataset of text and code.

Let's give a brief introduction to Zephyr-7B Beta as our email assistant LLM is fine-tuned over it.

Zephyr-7B Beta

Zephyr is a series of language models which have been trained to act as helpful assistants. Zephyr-7B-β is the second model in the series, and is a fine-tuned version of mistralai/Mistral-7B-v0.1 that was trained on a mix of publicly available, synthetic datasets using Direct Preference Optimization (DPO).

All of these are decoder type models of transformers. Unlike encoder-decoder models, which process both input and output sequences, decoder-only models process the output sequence only. This makes them well-suited for tasks such as text generation, translation, and question-answering. This requires only two different kinds of input:

An Initial Prompt

An initial prompt is a brief text that offers context for the task at hand. In a generation task, this prompt might describe the task itself, demonstrating how we can employ zero-shot learning, even when some input data is available. This approach allows us to utilize these models in a few-shot manner.

Let's have a look at the prompt template for Radiantloom email assist:


[INST]
<>Given an input, provide a clear and concise summary and action items in less than 50 words.<>
Answer the question below from context below :
Convert the summary into a voice memo, keeping it under one minute.
{{email_text}}
[/INST]

The notation ‘<s>’ usually indicates the start of a section or a tag used to signify a specific type of content or formatting. ‘[INST]’ stands for ‘instruction’, denoting a specific set of instructions or guidelines to follow. ‘<<SYS>>’ could indicate a system message or instruction within a structured conversation or script. These tags help organize and interpret different parts of a document or conversation for easier understanding or processing.

Previous Word/Getting Context

Generating the first word is the most difficult. This type of model first understands the prompt about the task and it already has access to a vast vocabulary, learned from the training data. It utilizes this vocabulary to generate a list of potential first words that align with the context and the overall tone of the text. Then it uses statistical methods to calculate the probability of each potential first word and pick the one with the highest probability.

Once it generates the first word, it only takes the prompt and the previous word.

Radiantloom Email Assist 7b Features

This LLM offers a vast number of features that can help you become efficient and increase your productivity. Just by using a proper prompt you can use these features, which include:

● Email Summarization: This LLM is trained on generating automated crisp summaries of your emails. With this, you can quickly grasp the nature of your emails and identify key action items, saving time and enhancing your productivity.

● Voice Memo Conversion: This model can convert your emails into voice memos, which allows you to catch up on relevant information in the middle of multi-tasking.

● Chat Message Conversion: This model is also capable of converting your emails into chat messages, which you can use to communicate with colleagues or customers.

Use Cases

Radiantloom Email Assist 7b can be used by a large number of individuals and organizations. Here I have given a few examples:

● Busy Professionals: These LLMs can help busy professionals save time and improve their productivity by summarizing emails and organizing inboxes, helping to find the gist of specific emails quickly.

● Salespeople: Radiantloom Email Assist 7b can help salespeople communicate more effectively with customers by providing solid summaries of emails and converting emails into voice memos or chat messages.

● Customer Service Representatives: This model can also help customer service representatives provide better service by helping them find specific emails quickly and providing summaries of customer inquiries.

Now, let’s do a deep dive into using the model and see how it performs with an example email.

AI Assisted Emailing

Launch a GPU node on E2E Networks.

‍

‍

Installing Dependencies


!pip install -q transformers peft accelerate bitsandbytes safetensors sentencepiece langchain sentence-transformers

Import Required Libraries


import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers import BitsAndBytesConfig, pipeline

Loading the Model


model_name = "aigeek0x0/radiantloom-email-assist-7b"

def load_quantized_model(model_name: str):
    bnb_config = BitsAndBytesConfig(
        load_in_4bit=True, 				# loading the model with 4bit quantization
        bnb_4bit_use_double_quant=True, # utilize double quantization for processing
        bnb_4bit_quant_type="nf4",		# quantization type: "nf4"
        bnb_4bit_compute_dtype=torch.bfloat16 
    )

    model = AutoModelForCausalLM.from_pretrained(
        model_name,
        load_in_4bit=True,
        torch_dtype=torch.bfloat16,
        quantization_config=bnb_config
    )
    return model

We are loading the model in 4 bit quantization for causal language modeling. bnb_config defines a BitsandBytesConfig with specific settings for quantization. Have a look at the comments in the above script of the function load_quantized_model for understanding more about bnb_config. This function essentially configures and loads a pre-trained model with specific quantization settings (4-bit quantization using bfloat16 data type) for efficient and optimized causal language modeling tasks.

Initializing the Tokenizer


def initialize_tokenizer(model_name: str):
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    tokenizer.bos_token_id = 1  # Set beginning of sentence token id
    return tokenizer

Loading the Model and Tokenizer


model = load_quantized_model(model_name)
tokenizer = initialize_tokenizer(model_name)

# define stop token ids
stop_token_ids = [0]

Now, as our model is ready for generating some cool output, let's take an example email conversation and instruct the model to provide a clear and concise summary with action items in 50 words.

In the below prompt template you can observe that with <<SYS>> token, we are giving instruction to the model.

Specifying the Prompt


prompt = """

[INST]
<>Given an input, provide a clear and concise summary and action items in less than 50 words.<>

Answer the question below from context below :

Convert the summary into a voice memo, keeping it under one minute:

From: alex@example.com
To: emily@company.com
Subject: Project Update and Next Steps for Marketing Campaign

Hi Emily,

I trust you're doing well. As we progress with our marketing campaign, I wanted to share a brief update on our recent developments.

We've successfully completed the initial phase of market research, identifying key demographics and potential areas for engagement. Our next step involves finalizing the content strategy for social media outreach and refining our targeted messaging.

Your input on the content approach, especially for our upcoming product launch, is crucial. Could you review the proposed social media content plan and suggest enhancements or additions that align with our campaign objectives?

Additionally, let's schedule a brief meeting this week to discuss these updates and finalize the strategy moving forward.

Looking forward to your insights and our upcoming discussion.

Best regards,

Alex

[/INST]

"""

Inferencing the Model

The below code snippet performs text generation using a pre-trained language model based on the input prompt. Let's break down each step:


Tokenization
input_ids = tokenizer.encode(prompt, return_tensors="pt")
tokenizer.encode: Converts the prompt text into tokenized IDs using the provided tokenizer.
return_tensors="pt": Returns PyTorch tensors; the input is converted into tensors suitable for model processing.
Text Generation
output = model.generate(input_ids, max_length=512, temperature=0.1, repetition_penalty=1.1, top_p=0.7, top_k=50)
model.generate: Utilizes the model to generate text based on the provided input_ids.
max_length=512: Specifies the maximum length for the generated text.
temperature=0.1: Controls the randomness of the generated text; lower values lead to more deterministic output.
repetition_penalty=1.1: Modifies the likelihood of repeating tokens; values > 1.0 reduce repetitions.
top_p=0.7: Controls the cumulative probability for token selection in the nucleus sampling method.
top_k=50: Limits the selection of next tokens to the top-k most probable tokens.
Decoding Output
output_text = tokenizer.decode(output[0], skip_special_tokens=True)
tokenizer.decode: Converts the generated output tensor IDs back into human-readable text.
output[0]: Accesses the first generated output sequence.
skip_special_tokens=True: Excludes special tokens like padding or separator tokens from the decoded text.
Displaying Generated Text
print(output_text): Prints the decoded generated text.


input_ids = tokenizer.encode(prompt, return_tensors="pt")
output = model.generate(input_ids, max_length=512,
    	 temperature=0.1, repetition_penalty=1.1, top_p=0.7, top_k=50)
output_text = tokenizer.decode(output[0], skip_special_tokens=True)

print(output_text)

Radianloom Email Assist Output

Summary: Alex provides an update on the marketing campaign, highlighting the completion of market research and the need for content strategy refinement. He requests Emily's feedback on the social media content plan and suggests scheduling a meeting to finalize the strategy.

Action Items

Review the proposed social media content plan.
Suggest improvements or additions aligned with campaign objectives.
Schedule a meeting with Alex to discuss the updates and finalize the strategy.

Wow, beautiful, right? Now let us compare this with the ChatGPT-3.5’s response and see whether a smaller fine-tuned model can outperform an LLM trained on a substantially large amount of data.

‍

‍

We can see the clear difference between the summarization and action item generation capability of Radiantloom and ChatGPT.

Now, let's integrate it with LangChain and build a ready-to-serve pipeline for the email assistant.

LangChain Integration

Importing LangChain Libraries


from langchain import HuggingFacePipeline
from langchain import PromptTemplate, LLMChain

Configuring Hugging Face Pipeline


pipeline = pipeline(
        "text-generation",
        model=model,
        tokenizer=tokenizer,
        use_cache=True,
        device_map="auto",
        max_length=1024,                        
        temperature=0.1,
        do_sample=True,
        top_k=50,
        num_return_sequences=1,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.eos_token_id,
)

Running the Chain


llm = HuggingFacePipeline(pipeline=pipeline)

def email_assist(prompt):

  template = """[INST]
  <>
  Given an input, provide a clear and concise summary and action items in less than 50 	  	words.
  <>

  Answer the question below from context below:
  {context}
  {question} [/INST] 
  """

  question = """Convert the summary into a voice memo, keeping it under one minute:"""


  context = prompt

  prompt = PromptTemplate(template=template, input_variables=["question","context"])
  llm_chain = LLMChain(prompt=prompt, llm=llm)
  response = llm_chain.run({"question":question,"context":context})
  return response


output = email_assist(prompt)

<|assistant|>

"Alex here with a project update. We've completed initial market research and identified key demographics. Next, we're finalizing social media content and messaging for our product launch. Emily, your input on the content strategy is crucial. Please review the plan and suggest improvements. Let's schedule a meeting this week to finalize the strategy. Thanks, Alex."

Action Items:

Review the social media content plan.
Suggest improvements and additions.
Schedule a meeting with Alex this week.

Evaluation

Evaluating large language model (LLM) based chat assistants is challenging due to their broad capabilities and the inadequacy of existing benchmarks in measuring human preferences. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena - this paper established a method to address these issues and the Radiantloom Email Assist model received good results, according to initial assessments, with the utilization of GPT-4.

Conclusion

Radiantloom Email Assist 7b is a powerful tool that can help you to be efficient, improve your productivity, and communicate more effectively. If you are looking to manage your email inbox more effectively, I suggest you try Radiantloom Email Assist 7b.

Siddhant Saxena

POSTS BY THIS AUTHOR

Multi-Model Inference with Triton Inference Server

AI Assisted Emailing with Radiantloom Email Assist 7b