Step-by-Step Guide to Build and Deploy an LLM-Powered Chat with Memory in Streamlit

, I’ll present you step-by-step methods to construct and deploy a chat powered with LLM — Gemini — in Streamlit and monitor the API utilization on Google Cloud Console. Streamlit is a Python framework that makes it tremendous straightforward to show your Python scripts into interactive net apps, with nearly no front-end work.

Just lately, I constructed a venture, bordAI — a chat assistant powered by LLM built-in with instruments I developed to help embroidery tasks. After that, I made a decision to begin this sequence of posts to share suggestions I’ve discovered alongside the way in which. 

Right here’s a fast abstract of the publish:

1 to six — Undertaking Setup

7 to 13 — Constructing the Chat

14 to fifteen— Deploy and Monitor the app

---

1. Create a New GitHub repository

Go to GitHub and create a brand new repository.

---

2. Clone the repository domestically

→ Execute this command in your terminal to clone it:

git clone 

---

3. Set Up a Digital Atmosphere (optionally available)

A Digital Atmosphere is sort of a separate area in your pc the place you may set up a selected model of Python and libraries with out affecting the remainder of your system. That is helpful as a result of completely different tasks would possibly want completely different variations of the identical libraries. 

→ To create a digital atmosphere:

pyenv virtualenv 3.9.14 chat-streamlit-tutorial

→ To activate it:

pyenv activate chat-streamlit-tutorial

---

4. Undertaking Construction

A venture construction is only a approach to set up all of the information and folders in your venture. Ours will appear like this:

chat-streamlit-tutorial/
│
├── .env
├── .gitignore
├── app.py
├── capabilities.py
├── necessities.txt
└── README.md

• .env→ file the place you retailer your API key (not pushed to GitHub)
• .gitignore → file the place you listing the information or folders for git to disregard 
• app.py → essential streamlit app
• capabilities.py → customized capabilities to raised set up the code
• necessities.txt → listing of libraries your venture wants
• README.md → file that explains what your venture is about

→ Execute this inside your venture folder to create these information:

contact .env .gitignore app.py capabilities.py necessities.txt

→ Contained in the file .gitignore, add:

.env
__pycache__/

→ Add this to the necessities.txt:

streamlit
google-generativeai
python-dotenv

→ Set up dependencies:

pip set up -r necessities.txt

---

5. Get API Key

An API Key is sort of a password that tells a service you will have permission to make use of it. On this venture, we’ll use the Gemini API as a result of they’ve a free tier, so you may mess around with it with out spending cash. 

Don’t arrange billing in the event you simply need to use the free tier. It ought to say “Free” below “Plan”, identical to right here:

We’ll use gemini-2.0-flash on this venture. It provides a free tier, as you may see within the desk beneath:

• 15 RPM = 15 Requests per minute
• 1,000,000 TPM = 1 Million Tokens Per Minute
• 1,500 RPD = 1,500 Requests Per Day

Be aware: These limits are correct as of April 2025 and will change over time. 

Only a heads up: if you’re utilizing the free tier, Google might use your prompts to enhance their merchandise, together with human opinions, so it’s not really useful to ship delicate data. If you wish to learn extra about this, verify this link.

---

6. Retailer your API Key

We’ll retailer our API Key inside a .env file. A .env file is an easy textual content file the place you retailer secret data, so that you don’t write it instantly in your code. We don’t need it going to GitHub, so we now have so as to add it to our .gitignore file. This file determines which information git ought to actually ignore whenever you push your modifications to the repository. I’ve already talked about this partly 4, “Undertaking Construction”, however simply in case you missed it, I’m repeating it right here.

This step is actually necessary, don’t neglect it!
→ Add this to .gitignore: 

.env
__pycache__/

→ Add the API Key to .env:

API_KEY= "your-api-key"

Should you’re working domestically, .env works effective. Nonetheless, in the event you’re deploying in Streamlit later, you’ll have to use st.secrets and techniques. Right here I’ve included a code that may work in each eventualities. 

→Add this perform to your capabilities.py:

import streamlit as st
import os
from dotenv import load_dotenv

def get_secret(key):
    """
    Get a secret from Streamlit or fallback to .env for native growth.

    This permits the app to run each on Streamlit Cloud and domestically.
    """
    attempt:
        return st.secrets and techniques[key]
    besides Exception:
        load_dotenv()
        return os.getenv(key)

→ Add this to your app.py:

import streamlit as st
import google.generativeai as genai
from capabilities import get_secret

api_key = get_secret("API_KEY")

---

7. Select the mannequin 

I selected gemini-2.0-flash for this venture as a result of I feel it’s an incredible mannequin with a beneficiant free tier. Nonetheless, you may discover different mannequin choices that additionally provide free tiers and select your most popular one.

• Professional: fashions designed for excessive–high quality outputs, together with reasoning and creativity. Usually used for advanced duties, problem-solving, and content material technology. They’re multimodal — this implies they will course of textual content, picture, video, and audio for enter and output.
• Flash: fashions projected for pace and value effectivity. Can have lower-quality solutions in comparison with the Professional for advanced duties. Usually used for chatbots, assistants, and real-time purposes like computerized phrase completion. They’re multimodal for enter, and for output is presently simply textual content, different options are in growth.
• Lite: even sooner and cheaper than Flash, however with some decreased capabilities, resembling it’s multimodal just for enter and text-only output. Its essential attribute is that it’s extra economical than the Flash, best for producing giant quantities of textual content inside value restrictions.

This link has loads of particulars concerning the fashions and their variations.

Right here we’re establishing the mannequin. Simply substitute “gemini-2.0-flash” with the mannequin you’ve chosen. 

→ Add this to your app.py:

genai.configure(api_key=api_key)
mannequin = genai.GenerativeModel("gemini-2.0-flash")

---

8. Construct the chat

First, let’s talk about the important thing ideas we’ll use:

• st.session_state: this works like a reminiscence in your app. Streamlit reruns your script from prime to backside each time one thing modifications — whenever you ship a message or click on a button —  so usually, all of the variables can be reset. This permits Streamlit to recollect values between reruns. Nonetheless, in the event you refresh your net web page you’ll lose the session_state. 
• st.chat_message(identify, avatar): Creates a chat bubble for a message within the interface. The primary parameter is the identify of the message writer, which could be “consumer”, “human”, “assistant”, “ai”, or str. Should you use consumer/human and assistant/ai, it already has default avatars of consumer and bot icons. You’ll be able to change this if you wish to. Take a look at the documentation for extra particulars.
• st.chat_input(placeholder): Shows an enter field on the backside for the consumer to kind messages. It has many parameters, so I like to recommend you try the documentation. 

First, I’ll clarify every a part of the code individually, and after I’ll present you the entire code collectively. 

This preliminary step initializes your session_state, the app’s “reminiscence”, to maintain all of the messages inside one session. 

if "chat_history" not in st.session_state:
    st.session_state.chat_history = []

Subsequent, we’ll set the primary default message. That is optionally available, however I like so as to add it. You might add some preliminary directions if appropriate in your context. Each time Streamlit runs the web page and st.session_state.chat_history is empty, it’ll append this message to the historical past with the position “assistant”.

if not st.session_state.chat_history:
    st.session_state.chat_history.append(("assistant", "Hello! How can I aid you?"))

In my app bordAI, I added this preliminary message giving context and directions for my app:

For the consumer half, the primary line creates the enter field. If user_message comprises content material, it writes it to the interface after which appends it to chat_history. 

user_message = st.chat_input("Kind your message...")

if user_message:
    st.chat_message("consumer").write(user_message)
    st.session_state.chat_history.append(("consumer", user_message))

Now let’s add the assistant half:

• system_prompt is the immediate despatched to the mannequin. You might simply ship the user_message instead of full_input (have a look at the code beneath). Nonetheless, the output may not be exact. A immediate supplies context and directions about how you need the mannequin to behave, not simply what you need it to reply. An excellent immediate makes the mannequin’s response extra correct, constant, and aligned along with your targets. As well as, with out telling how our mannequin ought to behave, it’s weak to immediate injections. 

Immediate injection is when somebody tries to govern the mannequin’s immediate in an effort to alter its conduct. One approach to mitigate that is to construction prompts clearly and delimit the consumer’s message inside triple quotes. 

We’ll begin with a easy and unclear system_prompt and within the subsequent session we’ll make it higher to check the distinction. 

• full_input: right here, we’re organizing the enter, delimiting the consumer message with triple quotes (“””). This doesn’t forestall all immediate injections, however it’s one approach to create higher and extra dependable interactions. 
• response: sends a request to the API, storing the output in response. 
• assistant_reply: extracts the textual content from the response.

Lastly, we use st.chat_message() mixed to write() to show the assistant reply and append it to the st.session_state.chat_history, identical to we did with the consumer. 

if user_message:
    st.chat_message("consumer").write(user_message)
    st.session_state.chat_history.append(("consumer", user_message))
    
    system_prompt = f"""
    You might be an assistant.
    Be good and type in all of your responses.
    """
    full_input = f"{system_prompt}nnUser message:n"""{user_message}""""

    response = mannequin.generate_content(full_input)
    assistant_reply = response.textual content

    st.chat_message("assistant").write(assistant_reply)
    st.session_state.chat_history.append(("assistant", assistant_reply))

Now let’s see the whole lot collectively!

→ Add this to your app.py:

import streamlit as st
import google.generativeai as genai
from capabilities import get_secret

api_key = get_secret("API_KEY")
genai.configure(api_key=api_key)
mannequin = genai.GenerativeModel("gemini-2.0-flash")

if "chat_history" not in st.session_state:
    st.session_state.chat_history = []

if not st.session_state.chat_history:
    st.session_state.chat_history.append(("assistant", "Hello! How can I aid you?"))

user_message = st.chat_input("Kind your message...")

if user_message:
    st.chat_message("consumer").write(user_message)
    st.session_state.chat_history.append(("consumer", user_message))

    system_prompt = f"""
    You might be an assistant.
    Be good and type in all of your responses.
    """
    full_input = f"{system_prompt}nnUser message:n"""{user_message}""""

    response = mannequin.generate_content(full_input)
    assistant_reply = response.textual content

    st.chat_message("assistant").write(assistant_reply)
    st.session_state.chat_history.append(("assistant", assistant_reply))

To run and take a look at your app domestically, first navigate to the venture folder, then execute the next command.

→ Execute in your terminal:

cd chat-streamlit-tutorial
streamlit run app.py

Yay! You now have a chat working in Streamlit!

---

9. Immediate Engineering 

Immediate Engineering is a means of writing directions to get the very best output from an AI mannequin. 

There are many strategies for immediate engineering. Listed here are 5 suggestions:

1. Write clear and particular directions.
2. Outline a job, anticipated conduct, and guidelines for the assistant.
3. Give the correct quantity of context.
4. Use the delimiters to point consumer enter (as I defined partly 8).
5. Ask for the output in a specified format.

The following pointers could be utilized to the system_prompt or whenever you’re writing a immediate to work together with the chat assistant.

Our present system immediate is:

system_prompt = f"""
You might be an assistant.
Be good and type in all of your responses.
"""

It’s tremendous imprecise and supplies no steerage to the mannequin. 

• No clear course for the assistant, what sort of assist it ought to present
• No specification of the position or what’s the subject of the help
• No pointers for structuring the output
• No context on whether or not it ought to be technical or informal
• Lack of boundaries 

We will enhance our immediate based mostly on the guidelines above. Right here’s an instance.

→ Change the system_prompt within the app.py: 

system_prompt = f"""
You're a pleasant and a programming tutor.
At all times clarify ideas in a easy and clear method, utilizing examples when doable.
If the consumer asks one thing unrelated to programming, politely convey the dialog again to programming subjects.
"""
full_input = f"{system_prompt}nnUser message:n"""{user_message}""""

If we ask “What’s python?” to the outdated immediate, it simply offers a generic brief reply:

With the brand new immediate, it supplies a extra detailed response with examples:

Attempt altering the system_prompt your self to see the distinction within the mannequin outputs and craft the perfect immediate in your context!

---

10. Select Generate Content material Parameters

There are numerous parameters you may configure when producing content material. Right here I’ll reveal how temperature and maxOutputTokens work. Verify the documentation for extra particulars.

• temperature: controls the randomness of the output, starting from 0 to 2. The default is 1. Decrease values produce extra deterministic outputs, whereas increased values produce extra inventive ones.
• maxOutputTokens: the utmost variety of tokens that may be generated within the output. A token is roughly 4 characters. 

To vary the temperature dynamically and take a look at it, you may create a sidebar slider to regulate this parameter.

→ Add this to app.py:

temperature = st.sidebar.slider(
    label="Choose the temperature",
    min_value=0.0,
    max_value=2.0,
    worth=1.0
)

→ Change the response variable to:

response = mannequin.generate_content(
    full_input,
    generation_config={
        "temperature": temperature,
        "max_output_tokens": 1000
    }
)

The sidebar will appear like this:

Attempt adjusting the temperature to see how the output modifications!

---

11. Show chat historical past 

This step ensures that you just preserve observe of all of the exchanged messages within the chat, so you may see the chat historical past. With out this, you’d solely see the newest messages from the assistant and consumer every time you ship one thing.

This code accesses the whole lot appended to chat_history and shows it within the interface.

→ Add this earlier than the if user_message in app.py:

for position, message in st.session_state.chat_history:
    st.chat_message(position).write(message)

Now, all of the messages inside one session are saved seen within the interface:

Obs: I attempted to ask a non-programming query, and the assistant tried to alter the topic again to programming. Our immediate is working!

---

12. Chat with reminiscence 

Moreover having messages saved in chat_history, our mannequin isn’t conscious of the context of our dialog. It’s stateless, every transaction is impartial. 

To resolve this, we now have to go all this context inside our immediate so the mannequin can reference earlier messages exchanged. 

Create context which is an inventory containing all of the messages exchanged till that second. Including lastly the newest consumer message, so it doesn’t get misplaced within the context.

system_prompt = f"""
You're a pleasant and educated programming tutor.
At all times clarify ideas in a easy and clear method, utilizing examples when doable.
If the consumer asks one thing unrelated to programming, politely convey the dialog again to programming subjects.
"""
full_input = f"{system_prompt}nnUser message:n"""{user_message}""""

context = [
    *[
        {"role": role, "parts": [{"text": msg}]} for position, msg in st.session_state.chat_history
    ],
    {"position": "consumer", "elements": [{"text": full_input}]}
]

response = mannequin.generate_content(
    context,
    generation_config={
        "temperature": temperature,
        "max_output_tokens": 1000
    }
)

Now, I instructed the assistant that I used to be engaged on a venture to research climate knowledge. Then I requested what the theme of my venture was and it appropriately answered “climate knowledge evaluation”, because it now has the context of the earlier messages. 

In case your context will get too lengthy, you may think about summarizing it to save lots of prices, because the extra tokens you ship to the API, the extra you’ll pay.

---

13. Create a Reset Button (optionally available) 

I like including a reset button in case one thing goes improper or the consumer simply desires to clear the dialog. 

You simply have to create a perform to set de chat_history as an empty listing. Should you created different session states, it is best to set them right here as False or empty, too. 

→ Add this to capabilities.py: 

def reset_chat():
    """
    Reset the Streamlit chat session state.
    """
    st.session_state.chat_history = []
    st.session_state.instance = False # Add others if wanted

→ And if you would like it within the sidebar, add this to app.py:

from capabilities import get_secret, reset_chat

if st.sidebar.button("Reset chat"):
    reset_chat()

It is going to appear like this:

Every part collectively:

import streamlit as st
import google.generativeai as genai
from capabilities import get_secret, reset_chat

api_key = get_secret("API_KEY")
genai.configure(api_key=api_key)
mannequin = genai.GenerativeModel("gemini-2.0-flash")

temperature = st.sidebar.slider(
    label="Choose the temperature",
    min_value=0.0,
    max_value=2.0,
    worth=1.0
)

if st.sidebar.button("Reset chat"):
    reset_chat()

if "chat_history" not in st.session_state:
    st.session_state.chat_history = []

if not st.session_state.chat_history:
    st.session_state.chat_history.append(("assistant", "Hello! How can I aid you?"))

for position, message in st.session_state.chat_history:
    st.chat_message(position).write(message)

user_message = st.chat_input("Kind your message...")

if user_message:
    st.chat_message("consumer").write(user_message)
    st.session_state.chat_history.append(("consumer", user_message))

    system_prompt = f"""
    You're a pleasant and a programming tutor.
    At all times clarify ideas in a easy and clear method, utilizing examples when doable.
    If the consumer asks one thing unrelated to programming, politely convey the dialog again to programming subjects.
    """
    full_input = f"{system_prompt}nnUser message:n"""{user_message}""""

    context = [
        *[
            {"role": role, "parts": [{"text": msg}]} for position, msg in st.session_state.chat_history
        ],
        {"position": "consumer", "elements": [{"text": full_input}]}
    ]

    response = mannequin.generate_content(
        context,
        generation_config={
            "temperature": temperature,
            "max_output_tokens": 1000
        }
    )
    assistant_reply = response.textual content

    st.chat_message("assistant").write(assistant_reply)
    st.session_state.chat_history.append(("assistant", assistant_reply))

---

14. Deploy

In case your repository is public, you may deploy with Streamlit free of charge. 

MAKE SURE YOU DO NOT HAVE API KEYS ON YOUR PUBLIC REPOSITORY.

First, save and push your code to the repository.

→ Execute in your terminal:

git add .
git commit -m "tutorial chat streamlit"
git push origin essential

Pushing instantly into the essential isn’t a greatest apply, however because it’s only a easy tutorial, we’ll do it for comfort. 

1. Go to your streamlit app that’s working domestically.
2. Click on on “Deploy” on the prime proper.
3. In Streamlit Group Cloud, click on “Deploy now”.
4. Fill out the data.

5. Click on on “Superior settings” and write API_KEY="your-api-key", identical to you probably did with the .env file. 

6. Click on “Deploy”.

All achieved! Should you’d like, try my app here! 🎉

---

15. Monitor API utilization on Google Console 

The final a part of this publish exhibits you methods to monitor API utilization on the Google Cloud Console. That is necessary in the event you deploy your app publicly, so that you don’t have any surprises.

1. Entry Google Cloud Console.
2. Go to “APIs and companies”.
3. Click on on “Generative Language API”.

• Requests: what number of occasions your API was referred to as. In our case, the API known as every time we run mannequin.generate_content(context).
• Error (%): the proportion of requests that failed. Errors can have the code 4xx which is normally the consumer’s/requester’s fault — for example, 400 for unhealthy enter, and 429 means you’re hitting the API too steadily. As well as, errors with the code 5xx are normally the system’s/server’s fault and are much less frequent. Google usually retries internally or recommends retrying after just a few seconds — e.g. 500 for Inner Server Error and 503 for Service Unavailable.
• Latency, median (ms): This exhibits how lengthy (in milliseconds) it takes in your service to reply, on the fiftieth percentile — that means half the requests are sooner and half are slower. It’s a very good basic measure of your service’s pace, answering the query, “How briskly is it usually?”.
• Latency, 95% (ms): This exhibits the response time on the ninety fifth percentile — that means 95% of requests are sooner than this time, and solely 5% slower. It helps to establish how your system behaves below heavy load or with slower instances, answering the query, “How unhealthy is it getting for some customers?”.

A fast instance of the distinction between Latency median and Latency p95:
Think about your service normally responds in 200ms:

• Median latency = 200ms (good!)
• p95 latency = 220ms (additionally good)

Now below heavy load:

• Median latency = 220ms (nonetheless seems to be OK)
• p95 latency = 1200ms (not good)

The metric p95 exhibits that 5% of your customers are ready greater than 1.2 seconds — a a lot worse expertise. If we had appeared simply on the median, we’d assume the whole lot was effective, however p95 exhibits hidden issues.

Persevering with within the “Metrics” web page, you’ll discover graphs and, on the backside, the strategies referred to as by the API. Additionally, in “Quotas & System Limits”, you may monitor the API utilization in comparison with the free tier restrict.

Click on “Present utilization chart” to check utilization daily.

---

I hope you loved this tutorial. 

Yow will discover all of the code for this venture on my GitHub.

I’d love to listen to your ideas! Let me know within the feedback what you assume.

Observe me on: