Overview of Galileo Guardrail Metrics

​

Output Quality Metrics

• Correctness (Open Domain Hallucinations)
• Instruction Adherence: Scorers.instruction_adherence_plus
• Uncertainty
• Ground Truth Adherence: Scorers.ground_truth_adherence_plus
• Completeness
  • Completeness Luna: Scorers.completeness_luna
  • Completeness Plus: Scorers.completeness_plus
• BLEU and ROUGE

​

Agent Quality Metrics

• Action Completion: Scorers.action_completion_plus
• Action Advancement: Scorers.action_advancement_plus
• Tool Selection Quality: Scorers.tool_selection_quality_plus
• Tool Error: Scorers.tool_errors_plus

​

RAG Quality Metrics

• Context Adherence (Closed Domain Hallucinations)
  • Context Adherence Luna: Scorers.context_adherence_luna
  • Context Adherence Plus: Scorers.context_adherence_plus
• Chunk Attribution
  • Chunk Attribution Luna: Scorers.chunk_attribution_utilization_luna
  • Chunk Attribution Plus: Scorers.chunk_attribution_utilization_plus
• Chunk Utilization
  • Chunk Utilization Luna: Scorers.chunk_attribution_utilization_luna
  • Chunk Utilization Plus: Scorers.chunk_attribution_utilization_plus

​

Input Quality Metrics

• Prompt Perplexity

​

Safety Metrics

• Input & Output PII
• Input & Output Tone
• Input & Output Toxicity
• Input & Output Sexism
• Prompt Injection