Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.galileo.ai/llms.txt

Use this file to discover all available pages before exploring further.

​
Scorers

​
get_scorer_version

def get_scorer_version(self,
                       scorer_id: UUID,
                       version: int) -> Unset | BaseScorerVersionResponse
Arguments
  • name (str): Name of the scorer
  • version (int): Version of the scorer.
Returns
  • Scorer response if found, otherwise None.:

​
list

def list(self,
         name: str | None=None,
         types: list[ScorerTypes] | None=None) -> list[ScorerResponse]
Lists scorers, optionally filtering by name and/or type. The filters are combined with an AND condition, while the values within the types list are combined with an OR condition. For example, calling list(name="my_scorer", types=[ScorerTypes.llm, ScorerTypes.code]) will return scorers where the name is “my_scorer” AND the type is either “llm” OR “code”. Arguments
  • name: Name of the scorer to filter by.
  • types: List of scorer types to filter by. Defaults to all scorers.
Returns
  • A list of scorers that match the filter criteria.:

​
list_by_ids

def list_by_ids(self,
                scorer_ids: builtins.list[str]) -> builtins.list[ScorerResponse]
List scorers by their UUIDs. Arguments
  • scorer_ids: UUID strings of scorers to retrieve.
Returns
  • A list of scorers matching the given IDs.:

​
list_by_labels

def list_by_labels(self,
                   labels: builtins.list[str],
                   strict: bool=False) -> builtins.list[ScorerResponse]
List scorers by label. Arguments
  • labels: Label values to search for.
  • strict: When False (default), also matches by scorer name as a fallback for custom scorers that have no label set.
Returns
  • A list of scorers matching the label (or name) criteria.:

​
ScorerSettings

​
create

def create(self,
           project_id: str,
           run_id: str,
           scorers: list[ScorerConfig]) -> HTTPValidationError | RunScorerSettingsResponse | None
Arguments
  • project_id: ID of the project
  • run_id: ID of the run
  • scorers: List of scorer configurations
Returns
  • Upserted scorer settings.: