For importing Kaggle information into Google Colab, the wanted credentials such as Kaggle API might be downloaded from Kaggle profile. This json file needs to be uploaded to the information part in Google Colab. After importing the file, the next snippet can be use for getting the info.
# Setting the Kaggle API configuration listing to the place 'kaggle.json' is
os.environ['KAGGLE_CONFIG_DIR'] = '/content material'# Altering the file permission of 'kaggle.json' to be readable by the proprietor
!chmod 600 /content material/kaggle.json
# Downloading the dataset from Kaggle utilizing the dataset identifier
!kaggle datasets obtain -d paramaggarwal/fashion-product-images-small
# Unzipping the downloaded dataset archive into the folder named 'fashion_data'
!unzip -q fashion-product-images-small.zip -d ./fashion_data
The csv file containing information about every merchandise doesn’t embrace costs. Subsequently, a brand new column is created with random costs between $20 and $100 for every merchandise.
#Studying the CSV file
csv_file_path='fashion_data/types.csv'
df=pd.read_csv(csv_file_path,nrows=200)#Producing Random costs between 20 and 101.
df['Price'] = np.random.randint(20, 101, measurement=len(df))
#Show the primary 5 rows
df.head()
Step one within the picture embedding course of is to outline a operate that takes a picture path as enter and returns a 1408-dimensional vector representing the picture’s embedding. This operate calls a multimodal embedding mannequin from Vertex AI, which takes the picture and the specified embedding vector size as inputs.
def get_image_embedding(image_path: str,dimension: int | None = 1408,) -> listing[float]:#Load the Picture
picture = VMImage.load_from_file(image_path)
#Get the Embedding
embedding = mm_embedding_model.get_embeddings(
picture=picture,
dimension=dimension,
)
return embedding.image_embedding
After defining the operate, the subsequent step is to name the Multimodal Embedding mannequin, outline an inventory to retailer all of the embeddings, and retrieve the IDs of all out there merchandise. Then, we loop over all of the picture IDs, generate an embedding for every picture, and retailer every embedding within the listing.
# Calling the Multimodel Embedding Mannequin
mm_embedding_model = MultiModalEmbeddingModel.from_pretrained("multimodalembedding@001")#Outline an inventory that may take all of the embedding
store_embedding_list=[]
#Get the Ids of all of the pictures within the dataframe
photo_id=listing(df['id'])
#Loop over all of the picture ids and create the embedding via the get_image_embedding
for identify in photo_id:
image_path='fashion_data/photos/'+str(identify)+'.jpg'
image_emb = get_image_embedding(image_path=image_path)
store_embedding_list.append(image_emb)
The retrieved embeddings can be saved in a Chroma vector area together with their corresponding IDs. This vector area will later be used to retrieve probably the most related merchandise to a picture uploaded by the consumer.
#Outline a reputation for the Vector House
DB_NAME = "Fashion_Store_Embedding"#Provoke a Shopper
chroma_client = chromadb.Shopper()
#Specify the Database Title
assortment = chroma_client.create_collection(identify=DB_NAME )
#Add Knowledge To the Assortment
assortment.add(embeddings= store_embedding_list,ids=[str(i) for i in photo_id])
The first instrument which can be referred to as add_row that the Agent will use is to save lots of the Title of the Shopper, the Product the shopper needs to purchase, in addition to the worth of this product as a brand new row in a beforehand specified information body, which can be referred to as records_df.
#Outline the dataframe that the agent will use to retailer interactions between the purchasers
records_df= pd.DataFrame(columns=['Name of the Client', 'Product','Price'])#Outline the operate
def add_row(client_name: str, product: str, worth: float) :
"""Takes the shopper identify, product identify and the worth of the product and saves the data as a row in a dataframe"""
international records_df
row_append=pd.DataFrame({'Title of the Shopper': [client_name],'Product':[product],'Worth':[price]})
records_df=pd.concat([records_df,row_append])
The second instrument which can be referred to as similar_products takes as enter the Picture Path together with the variety of related pictures that should be fetched after which returns a dictionary with the identify of the product together with their costs sorted in lowering order of their similarity to the inputted photograph. So, the primary key within the dictionary would be the most related product to the inputted photograph, whereas the final key would be the farthest picture to the inputted photograph.
def similar_products(image_query_path: str, number_products:int) -> dict :""" Takes picture path as an enter and the variety of related outcomes wanted and return a dictionary that incorporates the identify of the same merchandise and their worth in US {Dollars}"""
#Get the Embedding Vector for the inputted photograph
image_emb = get_image_embedding(image_path=image_query_path)
#Get the highest 5 related photos to the inputted one
outcomes = assortment.question(query_embeddings=image_emb,n_results=number_products)
#Get the ids of the pictures retreived
image_ids=outcomes['ids'][0]
image_ids=listing(map(int, image_ids))
#Get the rows of the pictures retreived
filtered_df = df[df['id'].isin(image_ids)]
filtered_df=filtered_df[['productDisplayName','Price','id']].reset_index()
del filtered_df['index']
#Rename the columns
filtered_df.columns=['Product Name','Price in US Dollars','id']
#Kind the dataframe within the order of similarity
filtered_df['id'] = pd.Categorical(filtered_df['id'], classes=image_ids, ordered=True)
filtered_df = filtered_df.sort_values('id')[['Product Name','Price in US Dollars', 'id']].reset_index(drop=True)
#Rework the dataframe to a dictionary
dict_price=dict(zip(filtered_df['Product Name'], filtered_df['Price in US Dollars']))
return dict_price
The instruction is a immediate that can be despatched to the Mannequin, this immediate explains all the mandatory logic for the Agent and when to make use of every instrument.
#The Immediate Useinstruction = """You're a useful chatbot that works in a retail retailer, you are job is to assist the client with figuring out the identify of probably the most related
product to the picture supplied throughout the funds they've. The shopper will connect the picture and specify that quantity he/she is keen to pay and you then'll use the name_of_similar_product instrument to get as a dictionary the identify and the worth of
the highest 5 related merchandise to the picture hooked up sorted within the lowering orer of the similarity, so the primary key would be the most related merchandise whereas the final secret is the farthest merchandise.
You'll look first on the most related product and if the worth of this product is increased than the funds of the shopper you may go to the second and so till
you attain the fifth product, if the costs of high 5 related merchandise are increased than the funds then inform the shopper that no merchandise can be found with the worth supplied. Notice that generally,
the client will present the funds with totally different currencies so make sure that to remodel to US greenback and present the conversion within the reply.
if the shopper confirmed that he/she needs to purchase the merchandise proposed then save the shopper identify, the product identify and the worth utilizing add_row"""
After defining the instruments and writing the instruction for the mannequin to observe, now we are going to mix all this setup via creating a brand new customized chat session.
#Saving all of the instruments that can be used the Agent in an inventory
all_tools=[similar_products,add_row]# Create a brand new chat session utilizing the Gemini 2.0 Flash mannequin
#with a customized system instruction and a set of predefined instruments
chat = shopper.chats.create(
mannequin="gemini-2.0-flash",
config=sorts.GenerateContentConfig(
system_instruction=instruction,
instruments=all_tools,
),
)
Now, all the things is about! For experimentation, we are going to randomly choose a picture and ship it to the agent together with a specified funds in GBP (or some other forex of your selection), then observe the agent’s response.
#Select a random photograph from the listing
identify=random.selection(photo_id)
image_path='fashion_data/photos/'+str(identify)+'.jpg'#Show the Picture
picture = PImage.open(image_path)
show(picture) # Renders the picture within the pocket book
#We cross to the agent, the picture in addition to the Price range
message_prompt=[image_path, "My name is Alex and my budget is 100 GBP "]
#Get the response
resp = chat.send_message(message_prompt)
resp.textual content
For instance, if the uploaded picture is appears to be like like the next picture:
The response of the agent can be:
I've the 5 most related merchandise to the picture you supplied.
Your funds is 100 GBP which is roughly 120 USD.
Probably the most related product is "Puma Males's Stripe Polo Black T-shirt"
and it prices $99. Would you want to purchase it?'
If the consumer replied again as
resp = chat.send_message(["Yes I want to buy this item"])
Then the agent will reply as
Nice, I’ve saved your order. You might have purchased “Puma Males’s Stripe Polo Black T-shirt” for $99.
If we have a look at the info body outlined on the Starting, we are going to discover {that a} new row was added.
Many enhancements might be made to this agent. Some recommendations for additional growth embrace:
- Join the orders to a stay database.
- Allow voice messaging.
- Enable internet search so the agent can search for different merchandise if none can be found throughout the funds.
- Strive pictures with Greater Decision and take a look at an enormous quantity of pictures
