Skip to main content
Galileo provides a comprehensive suite of out-of-the-box metrics designed to evaluate various aspects of AI system performance without requiring custom implementation. Most out-of-the-box metrics are LLM-based. With these metrics, you can quickly spot problems, track improvements, and make your AI work better for your users. These metrics apply to different node types (such as session, trace, or different span types), depending on the metric. You can then expand these metrics with custom metrics: After you set up metrics, you can use them in experiments and Log streams to monitor AI system performance.

​
Which metric should I use?

Metrics can use decision models, LLMs, Luna small language models (SLMs), or code to evaluate inputs and outputs. Choose a metric type depending on the reasoning and specialization you need: The following table describes the differences between decision model and LLM-as-a-judge metrics:

​
How do I use metrics effectively?

To get the most value from Galileo metrics:
  1. Start with key metrics: Focus on metrics most relevant to your use case.
  2. Establish baselines: Understand your current performance before making changes.
  3. Track trends over time: Monitor how metrics change as you iterate on your system.
  4. Combine multiple metrics: Look at related metrics together for a more complete picture.
  5. Set thresholds: Define acceptable ranges for critical metrics.
  6. Improve the metrics: Use CLHF to continuously improve metrics.

​
Set up an LLM or decision model integration

This step is required to use LLM-based out-of-the-box, LLM-as-a-judge, and decision model metrics.
1

Navigate to the Integrations page

In the Galileo console UI, navigate to the LLM integrations page by opening the user menu on the bottom-left corner, and then selecting Integrations.Integrations user menu
2

Add an integration

Locate the provider you are using or specify a custom integration. Select the +Add Integration button.LLM provider options
3

Add settings

Specify settings for your integration (such as an API key), then select Save changes.OpenAI integration input modal

​
Next steps

Out-of-the-box metrics

Learn about the pre-built metrics you can use to monitor AI system performance.

Custom decision model metrics

Learn how to create decision model metrics for fast, structured judgments.

Custom LLM-as-a-judge metrics

Learn how to create LLM-as-a-judge metrics to assess the quality of responses.

Custom code-based metrics

Learn how to create custom code-based metrics to evaluate your LLM applications.

Improve LLM-as-a-judge metrics with Autotune

Learn how to improve your LLM-as-a-judge metrics using expert feedback with Autotune.