OpenAI Agent Integration

Follow this step-by-step guide to build a “Homework Assistant” AI agent pipeline using Galileo’s OpenAI integrations.

OpenAI Agent walkthrough

Create Project Folder

Create a new project folder and navigate to it in your terminal.

mkdir homework-assistant
cd homework-assistant

Install Dependencies

Install the Galileo SDK and other necessary dependencies using the following command in your terminal.

pip install galileo openai-agents python-dotenv

Create Project Files

In your project folder, create a new blank application file and .env file.

homework-assistant/
│── app.py
│── .env

Set Environment Variables

In your .env file, set your environment variables by filling in your API keys, Project name, and Log stream name.By using these exact variable names, Galileo will automatically use them in your application.

GALILEO_PROJECT="Homework Assistant"
GALILEO_LOG_STREAM="batch-1"
GALILEO_API_KEY="your-Galileo-api-key-here"

# Provide the console url below if you are using a
# custom deployment, and not using the free tier, or app.galileo.ai.
# This will look something like “console.galileo.yourcompany.com”.
# GALILEO_CONSOLE_URL=your-galileo-console-url

OPENAI_API_KEY="your-OpenAI-api-key-here"

NOTE: The Project name and Log stream name are customizable. Change them as needed for your own experiments. You can view all your Projects and Log streams in the Galileo Console.

Import Libraries

In your application file, add the following code to import all required libraries.

from galileo.handlers.openai_agents import GalileoTracingProcessor
from agents import (
    set_trace_processors,
    Agent,
    GuardrailFunctionOutput,
    InputGuardrail,
    Runner
)
from pydantic import BaseModel
import asyncio
from dotenv import load_dotenv
load_dotenv()

Define Output Structure

Add the code below to your application file to define an output structure.The Guardrail agent uses this structure to reject invalid outputs by type. For example, an int would be an invalid output in response to the question “Who was the first president of the United States?”This is achieved in Python using BaseModel.

BaseModel: A Python class from the pydantic library which automatically validates whether groups of values are the correct types.

# Create a class with BaseModel to validate output types
class HomeworkOutput(BaseModel):
    is_homework: bool
    reasoning: str

Create Tutor Agents

Add the code below to your application file to create two specialized “tutor agents”. One handles math questions, and the other handles history.

Agent: An AI module that receives inputs and provides specialized outputs based on predefined instructions.

# Create math tutor agent to answer math questions
math_tutor_agent = Agent(
    name="Math Tutor",
    handoff_description="Specialist agent for math questions",
    instructions="""
    You provide help with math problems.
    Explain your reasoning at each step and include examples
    """,
)

# Create history tutor agent to answer history questions
history_tutor_agent = Agent(
    name="History Tutor",
    handoff_description="Specialist agent for historical questions",
    instructions="""
    You provide assistance with historical queries.
    Explain important events and context clearly.
    """,
)

Create Guardrail Agent

Add the code below to your application file to set up a Guardrail Agent that will filter out non-homework questions.

Guardrail Agent: A specialized agent for evaluating inputs and determining whether they meet specific criteria.

# Create Guardrail agent to determine if input question
# is homework-related
guardrail_agent = Agent(
    name="Guardrail check",
    instructions="Check if the user is asking about homework.",
    output_type=HomeworkOutput
)

Define Guardrail Function

Add the code below to your application file to define the Guardrail Agent’s logic for accepting valid inputs (in this case, homework questions) and rejecting invalid ones.

Tripwire: A condition that triggers if the guardrail criteria are not met, preventing further processing.

# Set tripwire to filter out non-homework questions with Guardrail agent
async def homework_guardrail(ctx, agent, input_data):
    result = await Runner.run(guardrail_agent,
                              input_data,
                              context=ctx.context)
    final_output = result.final_output_as(HomeworkOutput)
    return GuardrailFunctionOutput(output_info=final_output,
        tripwire_triggered=not final_output.is_homework)

Create Triage Agent

Add the code below to your application file to set up a Triage Agent to pass the input question to the appropriate tutor agent.

Triage Agent: An agent that analyzes the input and determines which specialized agent should handle the request.

# Create Triage agent to determine which tutor agent to use
triage_agent = Agent(
    name="Triage Agent",
    instructions="""
    You determine which agent to use based on
    the user's homework question
    """,
    handoffs=[history_tutor_agent, math_tutor_agent],
    input_guardrails=[
        InputGuardrail(guardrail_function=homework_guardrail)
    ],
)

Run the Agents

Add the code below to your application file. It runs the complete system by sending sample inputs and observing which agents handles the questions.This code also sets a custom OpenAI trace processor. This call replaces the built in OpenAI trace processor with a Galileo one that logs traces to Galileo.To learn more, check out the Python GalileoTracingProcessor SDK docs.

Runner: A utility that executes the agents with provided input and context.

# Use the Runner to run the agents and get the answers
async def main():
    # Set the trace processor to the Galileo trace processor
    set_trace_processors([GalileoTracingProcessor()])

    result = await Runner.run(triage_agent,
        "who was the first president of the united states?")
    print(result.final_output)

    result = await Runner.run(triage_agent, "what is life?")
    print(result.final_output)

if __name__ == "__main__":
    asyncio.run(main())

Complete Application Code

Below is the final combined code for the “Homework Assistant” AI agent application. Review it and compare it with your code.

from galileo.handlers.openai_agents import GalileoTracingProcessor
from agents import (
    set_trace_processors,
    Agent,
    GuardrailFunctionOutput,
    InputGuardrail,
    Runner
)
from pydantic import BaseModel
import asyncio
from dotenv import load_dotenv
load_dotenv()

# Create a class with BaseModel to validate output types
class HomeworkOutput(BaseModel):
    is_homework: bool
    reasoning: str

# Create math tutor agent to answer math questions
math_tutor_agent = Agent(
    name="Math Tutor",
    handoff_description="Specialist agent for math questions",
    instructions="""
    You provide help with math problems.
    Explain your reasoning at each step and include examples
    """,
)

# Create history tutor agent to answer history questions
history_tutor_agent = Agent(
    name="History Tutor",
    handoff_description="Specialist agent for historical questions",
    instructions="""
    You provide assistance with historical queries.
    Explain important events and context clearly.
    """,
)

# Create Guardrail agent to determine if input question
# is homework-related
guardrail_agent = Agent(
    name="Guardrail check",
    instructions="Check if the user is asking about homework.",
    output_type=HomeworkOutput
)

# Set tripwire to filter out non-homework questions with Guardrail agent
async def homework_guardrail(ctx, agent, input_data):
    result = await Runner.run(guardrail_agent,
                              input_data,
                              context=ctx.context)
    final_output = result.final_output_as(HomeworkOutput)
    return GuardrailFunctionOutput(output_info=final_output,
        tripwire_triggered=not final_output.is_homework)

# Create Triage agent to determine which tutor agent to use
triage_agent = Agent(
    name="Triage Agent",
    instructions="""
    You determine which agent to use based on the user's homework question
    """,
    handoffs=[history_tutor_agent, math_tutor_agent],
    input_guardrails=[
        InputGuardrail(guardrail_function=homework_guardrail)
    ],
)

# Use the Runner to run the agents and get the answers
async def main():
    # Set the trace processor to the Galileo trace processor
    set_trace_processors([GalileoTracingProcessor()])

    result = await Runner.run(triage_agent,
        "who was the first president of the united states?")
    print(result.final_output)

    result = await Runner.run(triage_agent, "what is life?")
    print(result.final_output)

if __name__ == "__main__":
    asyncio.run(main())

Open project & Log stream

In your browser, open the Galileo Console. Then, select the Project and Log stream whose names you used in your .env file.You will see new Traces, each containing data logged from running your AI Agent pipeline.

NOTE: Learn more about using Projects and Log streams in the Getting Started Guide.

View Results

In the Galileo Console, click on one of the new Trace entries to see all of the data and steps executed by running your “Homework Assistant” application.You should see:

Each agent involved and when it was used
All agent inputs, outputs, and handoffs
Whether Guardrail Tripwires were passed or triggered
The time of execution, Project ID, Run ID, Trace ID, and Parent ID (viewable in the “Parameters” tab in the top-right)

OPTIONAL: Test the Guardrail Tripwire

To see the Tripwire get triggered, modify one of the inputs to be a question that is not about homework.

NOTE: This will cause an error because the Guardrail Agent rejects questions that trigger the Tripwire.

# Input is changed to non-homework question
async def main():
    result = await Runner.run(triage_agent, "What is the meaning of life?")
    print(result.final_output)

Congratulations!

Your OpenAI Agent Pipeline is complete and ready to use.

Next steps

Create your own project with new instructions to define your own specialized agents.
Include Metadata and Tags in your logs to track results and add automations.
Add Metrics to your experiment to evaluate results.
Create Datasets to improve evaluation accuracy and compare performance improvements.

Overview

Get Started

Observability

Evaluation Metrics

Signals

AI Assistant

Luna Studio

Experiments

Agent Control

Annotations

Integrations

Security

References

OpenAI Agent Integration

OpenAI Agent walkthrough

Next steps

​OpenAI Agent walkthrough

​Next steps

OpenAI Agent walkthrough

Next steps