You can define custom metrics for your Observe projects by registering them using our promptquality library. For more information on registering scorers, see the Register Custom Metrics guide.

Using Your Registered Scorer

All your Registered Scorers will be shown under the Custom Metrics section of your Project Settings. The On/Off switch turns them on and off.

When your metrics are on, your registered scorer will be executed on new samples that get logged to Galileo Observe (Note: scorers don’t run retroactively, so past samples will not be scored). For each added Scorer, you’ll see a new column in your Data view.