What’s Google Mesop?
Mesop is a comparatively latest (2023) Python net framework used at Google for fast AI app growth.
“Mesop offers a flexible vary of 30 elements, from low-level constructing blocks to high-level, AI-focused elements. This flexibility permits you to quickly prototype ML apps or construct customized UIs, all inside a single framework that adapts to your challenge’s use case.” — Mesop Homepage
What’s an AI Agent?
The origins of the Agent software program paradigm comes from the phrase Company, a software program program that may observe its surroundings and act upon it.
“A man-made intelligence (AI) agent is a software program program that may work together with its surroundings, accumulate knowledge, and use the info to carry out self-determined duties to satisfy predetermined targets.
People set targets, however an AI agent independently chooses one of the best actions it must carry out to attain these targets.
AI brokers are rational brokers. They make rational selections primarily based on their perceptions and knowledge to supply optimum efficiency and outcomes.
An AI agent senses its surroundings with bodily or software program interfaces.” — AWS Website
What’s CO-STAR prompting?
This can be a information to the formatting of prompts such that the next headers are included; context, goal, model, tone, viewers and response. That is broadly accepted to enhance mannequin output for LLMs.
“The CO-STAR framework, a brainchild of GovTech Singapore’s Information Science & AI workforce, is a useful template for structuring prompts.
It considers all the important thing features that affect the effectiveness and relevance of an LLM’s response, resulting in extra optimum responses.” — Sheila Teo’s Medium Post
What’s Chain-of-Thought (CoT) prompting?
Initially proposed in a Google paper; Wei et al. (2022). Chain-of-Thought (CoT) prompting means to supply few-shot prompting examples of intermediate reasoning steps. Which was confirmed to enhance common sense reasoning of the mannequin output.
What’s Django?
Django is without doubt one of the extra refined broadly used Python frameworks.
“Django is a high-level Python net framework that encourages fast growth and clear, pragmatic design. It’s free and open supply.” — Django Homepage
What’s LangChain?
LangChain is without doubt one of the higher know open supply libraries for supporting a LLM functions, as much as and together with brokers and prompting related to this challenge.
“LangChain’s versatile abstractions and AI-first toolkit make it the #1 selection for builders when constructing with GenAI.
Be a part of 1M+ builders standardizing their LLM app growth in LangChain’s Python and JavaScript frameworks.” — LangChain website
I’ve used Streamlit extensively professionally for internet hosting Generative AI functions, an instance of my work could be discovered here.
At a excessive degree, Streamlit is a comparable open-source Python net framework
For extra on Streamlit, please see my different Medium article the place it’s mentioned at size.
This was the primary alternative to make use of Mesop in anger — so I believed a comparability could be helpful.
Mesop is designed to present extra fine-grained management over the CSS styling of elements and natively integrates with JS net feedback. Mesop additionally has helpful debugging instruments when working domestically. I’d additionally say from expertise that the multi-page app performance is simpler to make use of.
Nonetheless, this does imply that there’s a bigger barrier to entry for say machine studying practitioners much less well-versed in CSS styling (myself included). Streamlit additionally has a bigger neighborhood for help.
From the code snippet, we will arrange completely different web page routes. The challenge solely accommodates two pages. The primary web page and an error web page.
import mesop as me# native imports
attempt:
from .utils import ui_components
besides Exception:
from utils import ui_components
@me.web page(path="/")
def web page(security_policy=me.SecurityPolicy(dangerously_disable_trusted_types=True)):
with me.field(
model=me.Fashion(
background="#fff",
min_height="calc(100% - 48px)",
padding=me.Padding(backside=16),
)
):
with me.field(
model=me.Fashion(
width="min(800px, 100%)",
margin=me.Margin.symmetric(horizontal="auto"),
padding=me.Padding.symmetric(
horizontal=16,
),
)
):
ui_components.header_text()
ui_components.example_row()
ui_components.chat_input()
ui_components.output()
ui_components.clear_output()
ui_components.footer()
@me.web page(path="/error")
def error(security_policy=me.SecurityPolicy(dangerously_disable_trusted_types=True)):
with me.field(
model=me.Fashion(
background="#fff",
min_height="calc(100% - 48px)",
padding=me.Padding(backside=16),
)
):
with me.field(
model=me.Fashion(
width="min(720px, 100%)",
margin=me.Margin.symmetric(horizontal="auto"),
padding=me.Padding.symmetric(
horizontal=16,
),
)
):
ui_components.header_text()
ui_components.render_error_page()
ui_components.footer()
The error web page features a button to redirect to the homepage.
The code to set off the redirect to the homepage is included right here.
def navigate_home(occasion: me.ClickEvent):
me.navigate("/")def render_error_page():
is_mobile = me.viewport_size().width < 640
with me.field(
model=me.Fashion(
place="sticky",
width="100%",
show="block",
top="100%",
font_size=50,
text_align="middle",
flex_direction="column" if is_mobile else "row",
hole=10,
margin=me.Margin(backside=30),
)
):
me.textual content(
"AN ERROR HAS OCCURRED",
model=me.Fashion(
text_align="middle",
font_size=30,
font_weight=700,
padding=me.Padding.all(8),
background="white",
justify_content="middle",
show="flex",
width="100%",
),
)
me.button(
"Navigate to dwelling web page",
sort="flat",
on_click=navigate_home
)
We should additionally create the State class, this enables knowledge to persist inside the occasion loop.
import mesop as me@me.stateclass
class State:
enter: str
output: str
in_progress: bool
To clear the mannequin output from the interface, we will then assign the output variable to an empty string. There are additionally completely different button supported types, as of writing are; default, raised, flat and stroked.
def clear_output():
with me.field(model=me.Fashion(margin=me.Margin.all(15))):
with me.field(model=me.Fashion(show="flex", flex_direction="row", hole=12)):
me.button("Clear output", sort="flat", on_click=delete_state_helper)def delete_state_helper(ClickEvent):
config.State.output = ""
To robotically populate the chat interface with the instance prompts supplied, we use the button onclick occasion, by updating the state.
def example_row():
is_mobile = me.viewport_size().width < 640
with me.field(
model=me.Fashion(
show="flex",
flex_direction="column" if is_mobile else "row",
hole=10,
margin=me.Margin(backside=40),
)
):
for instance in config.EXAMPLE_PROMPTS:
prompt_box(instance, is_mobile)def prompt_box(instance: str, is_mobile: bool):
with me.field(
model=me.Fashion(
width="100%" if is_mobile else 200,
top=250,
text_align="middle",
background="#F0F4F9",
padding=me.Padding.all(16),
font_weight=500,
line_height="1.5",
border_radius=16,
cursor="pointer",
),
key=instance,
on_click=click_prompt_box,
):
me.textual content(instance)
def click_prompt_box(e: me.ClickEvent):
config.State.enter = e.key
Equally, to ship the request to the Django service we use the code snippet beneath. We use a Walrus Operator (:=) to find out if the request has acquired a legitimate response as not None (standing code 200) and append the output to the state so it may be rendered within the UI, in any other case we redirect the person to the error web page as beforehand mentioned.
def chat_input():
with me.field(
model=me.Fashion(
padding=me.Padding.all(8),
background="white",
show="flex",
width="100%",
border=me.Border.all(me.BorderSide(width=0, model="stable", coloration="black")),
border_radius=12,
box_shadow="0 10px 20px #0000000a, 0 2px 6px #0000000a, 0 0 1px #0000000a",
)
):
with me.field(
model=me.Fashion(
flex_grow=1,
)
):
me.native_textarea(
worth=config.State.enter,
autosize=True,
min_rows=4,
placeholder="Enter your immediate",
model=me.Fashion(
padding=me.Padding(prime=16, left=16),
background="white",
define="none",
width="100%",
overflow_y="auto",
border=me.Border.all(
me.BorderSide(model="none"),
),
),
on_blur=textarea_on_blur,
)
with me.content_button(sort="icon", on_click=click_send):
me.icon("ship")def click_send(e: me.ClickEvent):
if not config.State.enter:
return
config.State.in_progress = True
enter = config.State.enter
config.State.enter = ""
yield
if end result := api_utils.call_jira_agent(enter):
config.State.output += end result
else:
me.navigate("/error")
config.State.in_progress = False
yield
def textarea_on_blur(e: me.InputBlurEvent):
config.State.enter = e.worth
For completeness, I’ve supplied the request code to the Django endpoint for working the AI Jira Agent.
import requests# native imports
from . import config
def call_jira_agent(request):
attempt:
knowledge = {"request": request}
if (response := requests.submit(f"{config.DJANGO_URL}api/jira-agent/", knowledge=knowledge)) and
(response.status_code == 200) and
(output := response.json().get("output")):
return f"Request: {request}
Output: {output}
"
besides Exception as e:
print(f"ERROR call_jira_agent: {e}")
For this to run domestically, I’ve included the related Docker and Docker compose information.
This Docker file for working Mesop was supplied by way of the Mesop challenge homepage.
The Docker compose file consists of three companies. The back-end Django software, the front-end Mesop software and a PostgreSQL database occasion for use along with the Django software.
I needed to name out the surroundings variable being handed into the Mesop Docker container, PYTHONUNBUFFERED=1 ensures Python output, stdout, and stderr streams are despatched to the terminal. Having used the really useful Docker picture for Mesop functions it took me a while to find out the foundation explanation for not seeing any output from the appliance.
The DOCKER_RUNNING=true surroundings variable is a conference to easily decide if the appliance is being run inside Docker or for instance inside a digital surroundings.
It is very important level out that surroundings variables will likely be populated by way of the config file ‘config.ini’ inside the config sub-directory referenced by the env_file aspect within the Docker compose file.
To run the challenge, you need to populate this config file along with your Open AI and Jira credentials.
Django is a Python net framework with a number of helpful performance out of the field.
It’s akin to frameworks akin to Flask or FastAPI, although does require some extra setup and a steeper studying curve to get began.
If you wish to study extra about Flask, please see my article beneath.
On this article, I’ll cowl apps, fashions, serializers, views and PostgreSQL database integration.
An app is a logically separated net software that has a selected function.
In our occasion, we now have named the app “api” and is created by working the next command.
django-admin startapp api
Throughout the views.py file, we outline our API endpoints.
“A view operate, or view for brief, is a Python operate that takes an online request and returns an online response. This response could be the HTML contents of an online web page, or a redirect, or a 404 error, or an XML doc, or a picture . . . or something, actually. The view itself accommodates no matter arbitrary logic is important to return that response.” — Django website
The endpoint routes to Django views are outlined within the app urls.py file as beneath. The urls.py file is created on the initialization of the app. We’ve three endpoints on this challenge; a well being test endpoint, an endpoint for returning all data saved inside the database and an endpoint for dealing with the decision out to the AI agent.
The views are declared courses, which is the usual conference inside Django. Please see the file in its completeness.
Many of the code is self-explanatory although this snippet is critical as it’s going to saves the fashions knowledge to the database.
modelRequest = fashions.ModelRequest(request=request, response=response)
modelRequest.save()
The snippet beneath returns all data within the DB from the ModelRequest mannequin, I’ll cowl fashions subsequent.
class GetRecords(APIView):
def get(self, request):
"""Get request data endpoint"""
knowledge = fashions.ModelRequest.objects.all().values()
return Response({'end result': str(knowledge)})
“A mannequin is the only, definitive supply of details about your knowledge. It accommodates the important fields and behaviors of the info you’re storing. Usually, every mannequin maps to a single database desk.” — Django website
Our mannequin for this challenge is straightforward as we solely need to retailer the person request and the ultimate mannequin output, each of that are textual content fields.
The __str__ technique is a typical Python conference which for instance, is known as by default within the print operate. The aim of this technique is to return a human-readable string illustration of an object.
The serializer maps fields from the mannequin to validate inputs and outputs and switch extra advanced knowledge sorts in Python knowledge sorts. This may be seen within the views.py detailed beforehand.
“A ModelSerializer usually refers to a part of the Django REST framework (DRF). The Django REST framework is a well-liked toolkit for constructing Net APIs in Django functions. It offers a set of instruments and libraries to simplify the method of constructing APIs, together with serializers.
The ModelSerializer class offers a shortcut that permits you to robotically create a Serializer class with fields that correspond to the Mannequin fields.
The ModelSerializer class is similar as an everyday Serializer class, besides that:
It would robotically generate a set of fields for you, primarily based on the mannequin.
It would robotically generate validators for the serializer, akin to unique_together validators.
It contains easy default implementations of .create() and .replace().” — Geeks for geeks
The whole serializers.py file for the challenge is as follows.
For the PostgreSQL database integration, the config inside the settings.py file should match the databse.ini file.
The default database settings should be modified to level on the PostgreSQL database, as this isn’t the default database integration for Django.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'vectordb',
'USER': 'testuser',
'PASSWORD': 'testpwd',
'HOST': 'db' if DOCKER_RUNNING else '127.0.0.1',
'PORT': '5432',
}
}
The database.ini file defines the config for the PostgreSQL database at initialization.
To make sure database migrations are utilized as soon as the Docker container has been run, we will use a bash script to use the migrations after which run the server. Operating migrations robotically will imply that the database is at all times modified with any change in definitions inside supply management for Django, which saves time in the long term.
The entry level to the Dockerfile is then modified to level on the bash script utilizing the CMD instruction.
I’m utilizing the prevailing LangChain agent performance mixed with the Jira toolkit, which is a wrapper across the Atlassian Python API.
The default library is kind of helpful out of the field, typically requiring some trial and error on the immediate although I’d assume it ought to enhance over time as analysis into the realm progresses.
For this challenge nevertheless, I needed so as to add some customized tooling to the agent. This may be seen because the operate ‘triage’ beneath with the @device decorator.
The operate sort hints and remark description of the device are vital to speak to the agent what is anticipated when a name is made. The returned string of the operate is noticed by the agent, on this occasion, we merely return “Process full” such that the agent then ceases to conduct one other step.
The customized triage device performs the next steps;
- Get all unresolved Jira tickets for the challenge
- Get the outline and abstract for the Jira situation key the agent is conducting the triage on
- Makes asynchronous LLM-based comparisons with all unresolved tickets and robotically tags those that seem associated from a text-to-text comparability, then makes use of the Jira API to hyperlink them
- An LLM is then used to generate; person tales, acceptance standards and precedence, leaving this mannequin end result as a touch upon the first ticket
from langchain.brokers import AgentType, initialize_agent
from langchain_community.agent_toolkits.jira.toolkit import JiraToolkit
from langchain_community.utilities.jira import JiraAPIWrapper
from langchain_openai import OpenAI
from langchain.instruments import device
from langchain_core.prompts import ChatPromptTemplate, FewShotChatMessagePromptTemplatellm = OpenAI(temperature=0)
@device
def triage(ticket_number:str) -> None:
"""triage a given ticket and hyperlink associated tickets"""
ticket_number = str(ticket_number)
all_tickets = jira_utils.get_all_tickets()
primary_issue_key, primary_issue_data = jira_utils.get_ticket_data(ticket_number)
find_related_tickets(primary_issue_key, primary_issue_data, all_tickets)
user_stories_acceptance_criteria_priority(primary_issue_key, primary_issue_data)
return "Process full"
jira = JiraAPIWrapper()
toolkit = JiraToolkit.from_jira_api_wrapper(jira)
agent = initialize_agent(
toolkit.get_tools() + [triage],
llm,
agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
verbose=True,
max_iterations=5,
return_intermediate_steps=True
)
Each LLM duties use a CO-STAR system immediate and chain-of-thought few-shot prompting technique. Subsequently I’ve abstracted these duties into an LLMTask class.
They’re instantiated within the following code snippet. Arguably, we may experiment with completely different LLMs for every duties although within the curiosity of time I’ve not performed any experimentation round this — please be happy to remark beneath if you happen to do pull the repo and have any expertise to share.
class LLMTask:
def __init__(self, system_prompt, examples, llm):
self.system_prompt = system_prompt
self.examples = examples
self.llm = llmdef construct_prompt(self):
example_prompt = ChatPromptTemplate.from_messages(
[
("human", "{input}"),
("ai", "{output}"),
]
)
few_shot_prompt = FewShotChatMessagePromptTemplate(
example_prompt=example_prompt,
examples=self.examples,
)
return ChatPromptTemplate.from_messages(
[
("system", self.system_prompt),
few_shot_prompt,
("human", "{input}"),
]
)
def run_llm(self, enter):
chain = self.construct_prompt() | self.llm
return chain.invoke({"enter": enter})
product_model = LLMTask(system_prompts.get("system_prompt_product"), example_prompts.get("examples_product"), llm)
linking_model = LLMTask(system_prompts.get("system_prompt_linking"), example_prompts.get("examples_linking"), llm)
For the linking duties, the CO-STAR system immediate is beneath. The headings of Context, Goal, Fashion, Tone, Viewers and Response are the usual headings for the CO-STAR technique. We outline the context and outputs together with the tagging of every aspect of the mannequin outcomes.
Explicitly defining the viewers, model and tone helps to make sure the mannequin output is acceptable for a enterprise context.
# CONTEXT #
I need to triage newly created Jira tickets for our software program firm by evaluating them to earlier tickets.
The primary ticket will likely be in tags and the second ticket will likely be in tags. # OBJECTIVE #
Decide if two tickets are associated if the difficulty describes related duties and return True in tags, additionally embrace your pondering in tags.
# STYLE #
Maintain reasoning concise however logical.
# TONE #
Create an informative tone.
# AUDIENCE #
The viewers will likely be enterprise stake holders, product stakeholders and software program engineers.
# RESPONSE #
Return a boolean if you happen to assume the tickets are associated in tags and in addition return your pondering as to why you assume the tickets are associated in tags.
For performing the product model ticket analysis (person tales, acceptance standards, and precedence), the system immediate is beneath. We explicitly outline the precedence as both LOW, MEDIUM, or HIGH.
We additionally dictate that the mannequin has the model of a product proprietor/ supervisor for which this job would have historically been carried out.
# CONTEXT #
You're a product proprietor working in a big software program firm, you triage new tickets from their descriptions in tags as they're raised from customers.# OBJECTIVE #
From the outline in tags, you must write the next; person tales in tags, acceptance standards in tags and precedence in .
Precedence should be both LOW, MEDIUM OR HIGH relying on the what you deem is most acceptable for the given description.
Additionally embrace your pondering in tags for the precedence.
# STYLE #
Must be within the model of a product proprietor or supervisor.
# TONE #
Use knowledgeable and enterprise oriented tone.
# AUDIENCE #
The viewers will likely be enterprise stake holders, product stakeholders and software program engineers.
# RESPONSE #
Reply with the next format.
Person tales in tags.
Acceptance standards in tags.
Precedence in tags.
I’ll now present the Chain-of-thought few-shot immediate for linking Jira tickets, we append each the abstract and outline for each tickets in
The few-shot designation comes from the purpose that a number of examples are being fed into the mannequin.
The
We later regex parse the mannequin output and have a helper operate to hyperlink the associated tickets by way of the Jira API, all Jira API helper capabilities for this challenge are supplied later within the article.
"examples_linking": [
{
"input": "Add Jira integration ticket creation Add a Jira creation widget to the front end of the websiteAdd a widget to the front end to create a Jira Add an integration to the front end to allow users to generated Jira tickets manually",
"output": "TrueBoth tickets relate to a Jira creation widget, they must be duplicate tickets."
},
{
"input": "Front end spelling error There is a spelling error for the home page which should read 'Welcome to the homepage' rather than 'Wellcome to the homepage'Latency issue there is a latency issue and the calls to the Open AI should be made asynchronous",
"output": "FalseThe first ticket is in relation to a spelling error and the second is a latency, therefore they are not related."
},
{
"input": "Schema update We need to add a column for model requests and responsesUpdate schema to include both model requests and model responses Add to two new additional fields to the schema",
"output": "TrueBoth tickets reference a schema update with two new fields for model requests and model responses, therefore they must be related."
}
]
Equally for ticket analysis, the person story is supplied in
The entire examples had been annotated manually on my own.
"examples_product": [
{
"input": "Add Jira integration ticket creation Add a Jira creation widget to the front end of the website",
"output": "As a user, I want to be able to create a Jira ticket directly from the website.nAs a product owner, I want to streamline the process of creating Jira tickets for our team.nThe Jira creation widget should be easily accessible on the front end of the website.nThe widget should allow users to input all necessary information for a Jira ticket, such as title, description, and assignee.nOnce submitted, the widget should create a new Jira ticket in the designated project.nThe widget should have proper error handling in case of any issues with creating the ticket.nMEDIUMnBy adding a Jira integration and creation widget to the website, we can improve the efficiency and accuracy of creating Jira tickets for our team. This will ultimately save time and resources for both the business and the development team. Though there is no immediate need for this to be completed. Therefore, I believe this ticket should be prioritized as MEDIUM."
},
{
"input": "Urgent schema update We need to add a column for model requests and responses for compliance",
"output": "As a developer, I want to be able to view and track model requests and responses for compliance purposes.nAs a product owner, I want to ensure our system is compliant with regulations and standards.nA new column should be added to the schema for model requests and responses.nThe column should accurately track and store all relevant information for compliance purposes.nThe column should be easily accessible and viewable for authorized users.nThe update should not disrupt any existing functionality of the system.nHIGHnThis ticket is marked as urgent, indicating that it is a high priority for the business. Compliance is a critical aspect of our business and it is important that we address this issue as soon as possible. By adding a new column for model requests and responses, we can ensure that our system is compliant and avoid any potential legal or regulatory issues. Therefore, I believe this ticket should be prioritized as HIGH."
},
{
"input": "Homepage CSS error There is a CSS error for the homepage which is affecting a call to action button and negatively impacting conversion",
"output": "As a user, I want the CSS error on the homepage to be fixed so that I can easily access the call to action button.nAs a product owner, I want to improve the conversion rate of our website by fixing any CSS errors.nThe CSS error should be identified and fixed on the homepage.nThe call to action button should be easily visible and accessible for users.nThe fix should not affect any other functionality of the website.nHIGHnThis CSS error is directly impacting the conversion rate of our website, which is a key metric for our business. It is important that we address this issue as soon as possible to improve the user experience and ultimately increase conversions. Therefore, I believe this ticket should be prioritized as HIGH."
}
],
This code snippet makes use of a muti-threaded strategy to linking Jira points concurrently. This can vastly cut back the time it takes to make pair comparisons with all of the open tickets inside a challenge to find out if they’re associated.
def check_issue_and_link_helper(args):
key, knowledge, primary_issue_key, primary_issue_data = args
if key != primary_issue_key and
llm_check_ticket_match(primary_issue_data, knowledge):
jira_utils.link_jira_issue(primary_issue_key, key) def find_related_tickets(primary_issue_key, primary_issue_data, points):
args = [(key, data, primary_issue_key, primary_issue_data) for key, data in issues.items()]
with concurrent.futures.ThreadPoolExecutor(os.cpu_count()) as executor:
executor.map(check_issue_and_link_helper, args)
def llm_check_ticket_match(ticket1, ticket2):
llm_result = linking_model.run_llm(f"{ticket1}{ticket2}")
if ((end result := jira_utils.extract_tag_helper(llm_result)))
and (end result == 'True'):
return True
An instance workflow of the device, making a ticket and triaging it.
The results of these actions is captured in Jira ticket. Associated tickets have been linked robotically, the person tales, acceptance standards, precedence and thought have been captured as a Jira remark.
We are able to see the agent intermediate steps within the print statements of the Docker container.
All examples on this challenge the place I’ve explicitly used the Jira REST API have been included beneath for visibility.
The regex extraction helper operate used to parse mannequin outcomes can be included. There may be additionally a Python SDK for Jira although I elected to make use of the requests library on this occasion such that’s extra simply translated into different programming languages.
The pure subsequent step can be to incorporate code era by integrating with supply management for a close to totally automated software program growth lifecycle, with a human within the loop this could possibly be a possible resolution.
We are able to already see that AI code era is making an impression on the enterprise — if BAU duties could be partially automated then software program builders/product practitioners can concentrate on extra fascinating and significant work.
If there may be plenty of curiosity on this text then maybe I may look into this as a follow-up challenge.
I hope you discovered this text insightful, as promised — you could find all of the code within the Github repo here, and be happy to attach with me on LinkedIn additionally.
*All pictures, until in any other case famous, are by the writer.
