How to Train a Chatbot Using RAG and Custom Data

?

RAG, which stands for Retrieval-Augmented Technology, describes a course of by which an LLM (Massive Language Mannequin) might be optimized by coaching it to drag from a extra particular, smaller information base somewhat than its large unique base. Usually, LLMs like ChatGPT are educated on your entire web (billions of knowledge factors). This implies they’re vulnerable to small errors and hallucinations.

Right here is an instance of a scenario the place RAG may very well be used and be useful:

I need to construct a US state tour information chat bot, which incorporates common details about US states, reminiscent of their capitals, populations, and predominant vacationer points of interest. To do that, I can obtain Wikipedia pages of those US states and practice my LLM utilizing textual content from these particular pages.

Creating your RAG LLM

One of the standard instruments for constructing RAG methods is LlamaIndex, which:

• Simplifies the mixing between LLMs and exterior knowledge sources
• Permits builders to construction, index, and question their knowledge in a approach that’s optimized for LLM consumption
• Works with many sorts of knowledge, reminiscent of PDFs and textual content information
• Helps assemble a RAG pipeline that retrieves and injects related chunks of knowledge right into a immediate earlier than passing it to the LLM for technology

Obtain your knowledge

Begin by getting the info you need to practice your mannequin with. To obtain PDFs from Wikipedia (CC by 4.0) in the best format, ensure you click on Print after which “Save as PDF.”

Don’t simply export the Wikipedia as a PDF — Llama received’t just like the format it’s in and can reject your information.

For the needs of this text and to maintain issues easy, I’ll solely obtain the pages of the next 5 standard states: 

• Florida
• California
• Washington D.C.
• New York
• Texas

Be sure that to save lots of these all in a folder the place your challenge can simply entry them. I saved them in a single referred to as “knowledge”.

Get essential API keys

Earlier than you create your customized states database, there are 2 API keys you’ll must generate.

• One from OpenAI, to entry a base LLM
• One from Llama to entry the index database you add customized knowledge to

After getting these API keys, retailer them in a .env file in your challenge. 

#.env file
LLAMA_API_KEY = ""
OPENAI_API_KEY = ""

Create an Index and Add your knowledge 

Create a LlamaCloud account. When you’re in, discover the Index part and click on “Create” to create a brand new index.

An index shops and manages doc indexes remotely to allow them to be queried through an API while not having to rebuild or retailer them domestically.

Right here’s the way it works:

1. Once you create your index, there can be a spot the place you may add information to feed into the mannequin’s database. Add your PDFs right here.
2. LlamaIndex parses and chunks the paperwork.
3. It creates an index (e.g., vector index, key phrase index).
4. This index is saved in LlamaCloud.
5. You’ll be able to then question it utilizing an LLM via the API.

The subsequent factor you could do is to configure an embedding mannequin. An embedding mannequin is the LLM that can underlie your challenge and be accountable for retrieving the related data and outputting textual content.

Once you’re creating a brand new index you need to choose “Create a brand new OpenAI embedding”:

Once you create your new embedding you’ll have to offer your OpenAI API key and identify your mannequin.

After getting created your mannequin, go away the opposite index settings as their defaults and hit “Create Index” on the backside.

It might take a couple of minutes to parse and retailer all of the paperwork, so make it possible for all of the paperwork have been processed earlier than you attempt to run a question. The standing ought to present on the best aspect of the display screen while you create your index in a field that claims “Index Information Abstract”.

Accessing your mannequin through code

When you’ve created your index, you’ll additionally get an Group ID. For cleaner code, add your Group ID and Index Identify to your .env file. Then, retrieve all the required variables to initialize your index in your code:

index = LlamaCloudIndex(
  identify=os.getenv("INDEX_NAME"), 
  project_name="Default",
  organization_id=os.getenv("ORG_ID"),
  api_key=os.getenv("LLAMA_API_KEY")
)

Question your index and ask a query

To do that, you’ll must outline a question (immediate) after which generate a response by calling the index as such:

question = "What state has the best inhabitants?"
response = index.as_query_engine().question(question)

# Print out simply the textual content a part of the response
print(response.response)

Having an extended dialog along with your bot

By querying a response from the LLM the way in which we simply did above, you’ll be able to simply entry data from the paperwork you loaded. Nevertheless, for those who ask a comply with up query, like “Which one has the least?” with out context, the mannequin received’t keep in mind what your unique query was. It is because we haven’t programmed it to maintain observe of the chat historical past.

As a way to do that, you could:

• Create reminiscence utilizing ChatMemoryBuffer
• Create a chat engine and add the created reminiscence utilizing ContextChatEngine

To create a chat engine:

from llama_index.core.chat_engine import ContextChatEngine
from llama_index.core.reminiscence import ChatMemoryBuffer

# Create a retriever from the index
retriever = index.as_retriever()

# Arrange reminiscence
reminiscence = ChatMemoryBuffer.from_defaults(token_limit=2000)

# Create chat engine with reminiscence
chat_engine = ContextChatEngine.from_defaults(
    retriever=retriever,
    reminiscence=reminiscence,
    llm=OpenAI(mannequin="gpt-4o"),
)

Subsequent, feed your question into your chat engine:

# To question:
response = chat_engine.chat("What's the inhabitants of New York?")
print(response.response)

This offers the response: “As of 2024, the estimated inhabitants of New York is nineteen,867,248.”

I can then ask a comply with up query:

response = chat_engine.chat("What about California?")
print(response.response)

This offers the next response: “As of 2024, the inhabitants of California is 39,431,263.” As you may see, the mannequin remembered that what we had been asking about beforehand was inhabitants and responded accordingly.

Conclusion

Retrieval Augmented Technology is an environment friendly strategy to practice an LLM on particular knowledge. LlamaCloud gives a easy and easy strategy to construct your individual RAG framework and question the mannequin that lies beneath.

The code I used for this tutorial was written in a pocket book, however it can be wrapped in a Streamlit app to create a extra pure forwards and backwards dialog with a chatbot. I’ve included the Streamlit code here on my Github.

Thanks for studying

• Join with me on LinkedIn
• Buy me a coffee to help my work!
• I provide 1:1 knowledge science tutoring, profession teaching/mentoring, writing recommendation, resume opinions & extra on Topmate!