This example is pulled from Langchain Docs and most of the code is just LangChain implementation of a simple chain.

If you are using Vertex AI through langchain, concurrent requests to Vertex AI LLMs will fail to compute node outputs. Use one worker for best results.

​
Creating a simple chain with LangChain

First let’s build the components of our chain

we want to ask a chat model a question about hallucinations, but we want to give it the context to answer correctly, so naturally we set up a vector db and use RAG. In this case we’ll get the context from a Galileo blog post


from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Chroma
from langchain.chat_models import ChatOpenAI
from langchain.document_loaders import WebBaseLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from typing import List
from langchain.prompts import ChatPromptTemplate
from langchain.schema import StrOutputParser
from langchain.schema.runnable import RunnablePassthrough
from langchain.schema.document import Document

# Load text from webpage
loader = WebBaseLoader("https://www.rungalileo.io/blog/deep-dive-into-llm-hallucinations-across-generative-tasks")
data = loader.load()

# Split text into documents
text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)
splits = text_splitter.split_documents(data)

# Add text to vector db
embedding = OpenAIEmbeddings()
vectordb = Chroma.from_documents(documents=splits, embedding=embedding)

# Create a retriever
retriever = vectordb.as_retriever()

Now we have the retriever, we can build our chain. The chain will

  1. Take in a question.

  2. Feed that question to our retriever for some context.

  3. Fill out the prompt with the question and context.

  4. Feed the prompt to a chat model.

  5. output the answer from the model.


def format_docs(docs: List[Document]) -> str:
    return "\n\n".join([d.page_content for d in docs])

template = """Answer the question based only on the following context:

    {context}

    Question: {question}
    """
prompt = ChatPromptTemplate.from_template(template)
model = ChatOpenAI()

chain = {"context": retriever | format_docs, "question": RunnablePassthrough()} | prompt | model | StrOutputParser()

​
Integrating our chain with promptquality

Now all we have to do to integrate with promptquality, is to add our callback. In just 3 lines of code we can integrate promptquality into any existing LangChain experiments


import promptquality as pq

# Create callback handler
prompt_handler = pq.GalileoPromptCallback(
    scorers=[pq.Scorers.latency, pq.Scorers.groundedness, pq.Scorers.factuality]
)

# Run your chain experiments across multiple inputs with the galileo callback
inputs = [
    "What are hallucinations?",
    "What are intrinsic hallucinations?",
    "What are extrinsic hallucinations?"
]
chain.batch(inputs, config=dict(callbacks=[prompt_handler]))

# publish the results of your run
prompt_handler.finish()

​
Adding Tools and Agents

More complex chains including LangChain Tools and Agents, also integrate well with Galileo Evaluate.

Here’s another example pulled from the LangChain Docs.

​
Creating the tool from our retriever

First we can use our retriever, created above, and convert it to a tool.


from langchain.tools.retriever import create_retriever_tool

# Create retreiver tool
retriever_tool = create_retriever_tool(
    retriever,
    "hallucination_search",
    "Search for information about hallucinations. Use this tool for any questions about hallucinations",
)

Now let’s create a ReAct Agent that has access to this tool.


from langchain import hub
from langchain.agents import create_react_agent
from langchain.agents import AgentExecutor

llm = ChatOpenAI()
prompt = hub.pull("hwchase17/react")
agent = create_react_agent(llm, [retriever_tool], prompt)
agent_executor = AgentExecutor(agent=agent, tools=[retriever_tool])

Now that we have that, we’re ready to integrate with promptquality, the exact same way as above.


import promptquality as pq

# Create callback handler
prompt_handler = pq.GalileoPromptCallback(
    scorers=[pq.Scorers.latency, pq.Scorers.groundedness, pq.Scorers.factuality]
)

# Run your chain experiments across multiple inputs with the galileo callback
inputs = [
    {"input": "What are hallucinations?"},
    {"input": "What are intrinsic hallucinations?"},
    {"input": "What are extrinsic hallucinations?"}
]
agent_executor.batch(inputs, config=dict(callbacks=[prompt_handler]))

# publish the results of your run
prompt_handler.finish()

Was this page helpful?

Evaluate and Debug ChainsProgrammatically fetch logged data