Building a Simple LLM Agent with a Simple Front-End

This is a continuation of my previous post.

Summary

The goal is to build an LLM agent that will utilize the tools provided when required, and add a simple Front-end webpage so the user can interact with it.

Key modules/methods used:

• create_react_agent() from the langgraph.prebuilt module to create the agent
• Streamlit to quickly add a simple Front-end webpage.

Main Challenge/Learning:

• How to parse the output from the LangGraph prebuilt ReAct agent into a format that can handled by the front end.

Starting with the Agent

The official LangGraph documentation has a good basic tutorial to start with the prebuilt agent. I changed the search tool into a TavilySearchResults tool. The outputs were pretty impressive as the agent was able to determine if it needed to use the Tavily tool to get the result.

Output when the agent used the Tavily tool

Output when the agent didn't used the Tavily tool

Adding an in-memory store using MemorySaver as a LangGraph checkpoint is pretty straightforward as well. You can check the full code and execution results at this link.

Try Adding a Front End using LangServe

I first tried following some of the examples in LangServe. However, most of the client codes shown are on Jupyter notebooks which is not a front-end that the user can interact and play with. Although LangServe has a simple playground as a front-end, it offers little clue or flexibility in formatting the output from the agent. This resulted in ugly output as shown below:

Example LangServe Playground for Agent output

Try Adding a Front End using Streamlit

Then I came across Streamlit where you can create a Front End page easily without dealing with all the HTML, and CSS stuff. For example, I need only a few lines of code as shown below, to generate a simple webpage.

import streamlit as st

# Setting up the streamlit page
st.set_page_config(page_title="Weather Assistant")
st.header("Ask anything about weather")

# First row
r1c1, r1c2 = st.columns(2)
with r1c1:
    st.markdown("Ask anything about the weather in your city")

with r1c2:
    st.write(
        "Contact with [Author](https://github.com/sjchin88) to build your AI Projects")

# Get User Input
st.markdown("## Enter your question")

Simple Webpage by using Streamlit

Formatting LangGraph Agent Output

Streamlit will need a string to write the output. The challenge remains to be parsing the output of the LangGraph agent into a string. This challenge will need a little bit of software engineering to solve.

First, we need to understand what is actually returned when we run the agent. With a little digging, the official documentations say that the create_react_agent() method returns a CompiledGraph object. Using the stream method of the CompiledGraph, the object should return a dictionary. BUT the return object is actually not the normal Python dict. Let's use the following printout to investigate

Investigating the type of output

It turns out

• output from calling the method CompiledGraph.stream() is a Pregel.stream object.
• each item in the stream is an AddableValuesDict object.
• All interaction messages can be identified using the keyword “messages”. with the actual messages in the list
• the messages are stored in a BaseMessage object, where the BaseMessage class is inherited by the HumanMessage, and AIMessage classes. The BaseMessage class provides a pretty_repr() method to turn the output into a pretty formatted string. This is the method we can use.

By filtering for AI messages only, we can obtain the following output.

Output from the Agent

Final Step — Adding Memory with Streamlit

Now there is one problem left: how do we add the memory with the Streamlit application?

To recap, when we enter a new input via the streamlit textbox, it will run the whole file script over. That means all objects created by the script will be removed and recreated. The MemorySaver class from langgraph.checkpoint.memory is an in-memory database that lives on the RAM memory, if it is recreated, it won’t remember any history we saved before.

I can think of two solutions to this problem:

Solution 1: Use a singleton pattern where we import the agent from another module. Every time the Streamlit scripts were run over again, it would always fetch the agent created during the first run.

Solution 2: Use a persistent database as a memory store like SQLite.

I decided to try out solution 1 and the result is impressive. You can see in the picture below the comparison of answers with and without the memory. The full codes are available here.

Agent’s answer without the memory

Agent’s answer with the memory