> ## Documentation Index
> Fetch the complete documentation index at: https://docs.galileo.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Create a Local Metric
> Learn how to create a local metric in Python to use in your experiments
{/**/}
## Overview
This guide shows you how to create a custom [local metric](/concepts/metrics/custom-metrics/custom-metrics-ui-code#local-metrics) in Python to use in an experiment.
In this example, you will be creating a metric to rate the brevity (shortness) of an LLM's response based on word count. The sample code to run the experiment will use OpenAI as an LLM.
In this guide you will:
1. [Set up a project with Galileo](#install-dependencies)
{/**/}
2. [Create your local metric](#create-your-local-metric)
3. [Prepare the experiment](#prepare-the-experiment)
4. [Run the experiment](#run-the-experiment)
## Before you start
To complete this how-to, you will need:
* An [OpenAI API key](https://openai.com/api/)
* A [Galileo project](/concepts/projects)
* Your [Galileo API key](https://app.galileo.ai/settings/api-keys)
{/**/}
## Install dependencies
To use Galileo, you need to install some package dependencies, and configure environment variables.
Install the required dependencies for your app. Create a virtual environment using your preferred method, then install dependencies inside that environment:
```bash Python theme={null}
pip install "galileo[openai]" python-dotenv
```
```ini .env theme={null}
# Your Galileo API key
GALILEO_API_KEY="your-galileo-api-key"
# Your Galileo project name
GALILEO_PROJECT="your-galileo-project-name"
# The name of the Log stream you want to use for logging
GALILEO_LOG_STREAM="your-galileo-log-stream"
# Provide the console url below if you are using a
# custom deployment, and not using the free tier, or app.galileo.ai.
# This will look something like “console.galileo.yourcompany.com”.
# GALILEO_CONSOLE_URL="your-galileo-console-url"
# OpenAI properties
OPENAI_API_KEY="your-openai-api-key"
# Optional. The base URL of your OpenAI deployment.
# Leave this commented out if you are using the default OpenAI API.
# OPENAI_BASE_URL="your-openai-base-url-here"
# Optional. Your OpenAI organization.
# OPENAI_ORGANIZATION="your-openai-organization-here"
```
This assumes you are using a free Galileo account. If you are using a custom deployment, then you will also need to add the URL of your Galileo Console:
```ini .env theme={null}
GALILEO_CONSOLE_URL=your-Galileo-console-URL
```
## Create your local metric
The Scorer Function assigns one of three ranks — `"Terse"`, `"Temperate"`, or `"Talkative"`, depending on how many words the model outputs. Add this code to your `experiment.py` file.
```python Python theme={null}
from galileo import Trace, Span
def brevity_rank(step: Span | Trace) -> str:
"""
Rank response brevity based on word count.
"""
word_count = len(step.output.content.split(" "))
if word_count <= 3:
return "Terse"
if word_count <= 5:
return "Temperate"
return "Talkative"
```
Here, we tell Galileo that our custom metric returns a `str`. We give it a name ("Terseness"), then assign the Scorer. Add this code to your `experiment.py` file.
```python Python theme={null}
from galileo.schema.metrics import LocalMetricConfig
terseness = LocalMetricConfig[str](
# Metric name (shown as a column in Galileo)
name="Terseness",
# Scorer Function defined above
scorer_fn=brevity_rank
)
```
The metric has been created. Next, we can use it in an experiment.
## Prepare the experiment
For this example, we'll ask the LLM to specify the continent of four countries, encouraging it to be succinct.
Create a dataset of inputs to the experiment by adding this code to your `experiment.py` file.
```python Python theme={null}
# Simple dataset with four countries:
countries_dataset = [
{"input": "Indonesia"},
{"input": "New Zealand"},
{"input": "Greenland"},
{"input": "China"},
]
```
Next you need a [custom function](/sdk-api/experiments/running-experiments#run-experiments-with-custom-functions) to be called by your experiment. Add this code to your `experiment.py` file.
```python Python theme={null}
from galileo.openai import openai
# The function that calls the LLM for each input:
def llm_call(input):
client = openai.OpenAI(api_key=os.environ["OPENAI_API_KEY"])
return (
client.chat.completions.create(
model="gpt-4o",
messages=[
{
"role": "system",
"content": """
You are a geography expert.
Always answer as succinctly as possible.
"""
},
{
"role": "user",
"content": f"""
Which continent does the following
country belong to: {input}
"""
},
],
)
.choices[0]
.message.content
)
```
Finally, add code to run the experiment using your dataset and custom local metric.
```python Python theme={null}
import os
from galileo.experiments import run_experiment
# Run the Experiment!
results = run_experiment(
"terseness-local-metric",
dataset=countries_dataset,
function=llm_call,
metrics=[terseness],
project=os.environ["GALILEO_PROJECT"],
)
```
## Run the experiment
Now your experiment is set up, you can run it to see the results of your local metric.
```bash Terminal theme={null}
python experiment.py
```
When the experiment runs, it will output a link to view the results in the terminal.
```bash Terminal wrap theme={null}
(.venv) ➜ python experiment.py
Experiment terseness-local-metric has completed and results are available at https://console.galileo.ai//project/xxx/experiments/xxx
```
Follow the link in your terminal to view the results of the experiment. This experiment has 4 rows - one per item in the dataset.
The new Terseness metric is available in both the **Traces** table, and from the metrics pane when selecting a row.
You have successfully created a local metric and used it in an experiment.
## See also
* [Custom metrics in code](/concepts/metrics/custom-metrics/custom-metrics-ui-code)
* [Custom metrics using an LLM](/concepts/metrics/custom-metrics/custom-metrics-ui-llm)