Skip to main content
Use this when you already have a trained Luna Metric and want to evaluate it on a separate dataset. 
from galileo_luna_ft.training import run_evaluation

metrics = run_evaluation(
    model_output_dir="./luna_models/enter-your-lora-weights-directory-here",
    csv_path="./eval.csv",
    metric_type="boolean",  # or "multi_class"
    label_column="label",
    class_names=None,       # required for multi_class
    # base_model_path="/path/to/base/model",  # optional, in case your base model is also stored in a directory, otherwise we'll try to pull from HF
)

​
Required artifact files

  • prompt_template.txt (generated by this SDK)
  • Access to HF to pull the base model (Llama 3B/8B etc) or have the base model in another dir (base_model_path)
  • Luna metric adapter weight files in a local directory (generated by this SDK)

​
Dataset requirements

  • CSV must contain:
    • all prompt variables used by prompt_template.txt
    • the configured label_column