Out-of-the-box Metric Categories

Out-of-the-box metrics can be broken down into the following key categories, each addressing a specific aspect of AI system performance. You may benefit from using metrics from more than one category. Galileo also supports custom metrics that you can implement in addition to out-of-the-box metrics. The Metrics Comparison provides a full list of the out-of-the-box metrics for each category.

1. Agentic performance metrics

Agentic Performance Metrics evaluate how effectively AI agents perform tasks, use tools, and progress toward goals.

When to Use	Example Use Cases
When building and optimizing AI systems that take actions, make decisions, or use tools to accomplish tasks.	Evaluating a travel planning agent’s ability to book complete itineraries Assessing a coding assistant’s appropriate use of APIs and libraries Measuring a data analysis agent’s tool selection effectiveness

2. Expression and readability metrics

Expression And Readability Metrics assess the style, tone, clarity, and overall presentation of AI-generated content.

When to Use	Example Use Cases
When the format, tone, and presentation of AI outputs are important for user experience or brand consistency.	Ensuring a luxury brand chatbot maintains a sophisticated tone Verifying educational content is presented at the appropriate reading level Measuring clarity and conciseness in technical documentation generation

3. Multimodal quality metrics

Multimodal Quality Metrics evaluate whether multimodal inputs and outputs (such as images and audio conversations) are usable and compliant for the task at hand.

When to Use	Example Use Cases
When your AI workflow includes images or audio conversations and you need quality/compliance signals.	Gating blurry document photos before extraction Enforcing brand rules on generated marketing images Detecting turn-taking issues in voice agents (overlap/barge-in)

4. Response quality metrics

Response Quality Metrics help you evaluate how correctly, consistently, and in line with ground truth your AI follows instructions and answers user queries in any setting — with or without RAG (correctness, instruction adherence, ground truth adherence).

5. RAG metrics

RAG Metrics help you evaluate retrieval and generation quality in RAG pipelines, including retrieval quality (chunk relevance, context relevance, context precision, Precision @ K) and generation quality (context adherence, completeness).

When to Use	Example Use Cases
When evaluating how well retrieval systems find relevant context and how well models use that context to produce accurate, complete, and well-grounded responses.	Measuring factual accuracy in a medical information system Evaluating how well a RAG system uses retrieved information Assessing if customer service responses address all parts of a query

6. Safety and compliance metrics

Safety And Compliance Metrics identify potential risks, harmful content, bias, or privacy concerns in AI interactions.

When to Use	Example Use Cases
When ensuring AI systems meet regulatory requirements, protect user privacy, and avoid generating harmful or biased content.	Detecting PII in healthcare chatbot conversations Identifying potential prompt injection attacks in public-facing systems Measuring bias in hiring or loan approval recommendation systems

7. Text-to-SQL metrics

Text-to-SQL metrics evaluate the accuracy and effectiveness of SQL queries generated by AI models from natural language inputs.

When to Use	Example Use Cases
Use these metrics to evaluate the quality of SQL queries generated from natural language inputs. They’re essential when building Text-to-SQL systems that need to produce syntactically correct queries grounded in your database schema.	A data analytics assistant where users ask questions in natural language and expect accurate query results A customer-facing data analytics chatbot that must prevent injection attacks from user inputs A business intelligence platform where ad-hoc queries must not impact database availability

Overview

Get Started

Observability

Evaluation Metrics

Signals

AI Assistant

Luna Studio

Experiments

Agent Control

Annotations

Integrations

Security

References

Out-of-the-box Metric Categories

1. Agentic performance metrics

2. Expression and readability metrics

3. Multimodal quality metrics

4. Response quality metrics

5. RAG metrics

6. Safety and compliance metrics

7. Text-to-SQL metrics

​1. Agentic performance metrics

​2. Expression and readability metrics

​3. Multimodal quality metrics

​4. Response quality metrics

​5. RAG metrics

​6. Safety and compliance metrics

​7. Text-to-SQL metrics

1. Agentic performance metrics

2. Expression and readability metrics

3. Multimodal quality metrics

4. Response quality metrics

5. RAG metrics

6. Safety and compliance metrics

7. Text-to-SQL metrics