Skip to main content
PATCH
/
v2
/
scorers
/
{scorer_id}
Update
curl --request PATCH \
  --url https://api.galileo.ai/v2/scorers/{scorer_id} \
  --header 'Content-Type: application/json' \
  --header 'Galileo-API-Key: <api-key>' \
  --data '
{
  "name": "<string>",
  "description": "<string>",
  "tags": [
    "<string>"
  ],
  "defaults": {
    "model_name": "<string>",
    "num_judges": 123,
    "filters": [
      {
        "operator": "eq",
        "value": "<string>",
        "name": "node_name",
        "case_sensitive": true
      }
    ],
    "scoreable_node_types": [
      "<string>"
    ],
    "cot_enabled": true,
    "output_type": "boolean",
    "input_type": "basic"
  },
  "model_type": "slm",
  "ground_truth": true,
  "default_version_id": "<string>",
  "user_prompt": "<string>",
  "scoreable_node_types": [
    "<string>"
  ],
  "output_type": "boolean",
  "input_type": "basic",
  "multimodal_capabilities": [
    "vision"
  ],
  "roll_up_method": "average",
  "metric_color_picker_config": {
    "constraints": [
      {
        "color": "red",
        "operator": "eq",
        "value": 123
      }
    ],
    "type": "numeric"
  }
}
'
{
  "id": "<string>",
  "name": "<string>",
  "scorer_type": "llm",
  "tags": [
    "<string>"
  ],
  "defaults": {
    "model_name": "<string>",
    "num_judges": 123,
    "filters": [
      {
        "operator": "eq",
        "value": "<string>",
        "name": "node_name",
        "case_sensitive": true
      }
    ],
    "scoreable_node_types": [
      "<string>"
    ],
    "cot_enabled": true,
    "output_type": "boolean",
    "input_type": "basic"
  },
  "latest_version": {
    "id": "<string>",
    "version": 123,
    "scorer_id": "<string>",
    "generated_scorer": {
      "id": "<string>",
      "name": "<string>",
      "chain_poll_template": {
        "template": "<string>",
        "metric_system_prompt": "<string>",
        "metric_description": "<string>",
        "value_field_name": "rating",
        "explanation_field_name": "explanation",
        "metric_few_shot_examples": [
          {
            "generation_prompt_and_response": "<string>",
            "evaluating_response": "<string>"
          }
        ],
        "response_schema": {}
      },
      "instructions": "<string>",
      "user_prompt": "<string>"
    },
    "registered_scorer": {
      "id": "<string>",
      "name": "<string>",
      "score_type": "<string>"
    },
    "finetuned_scorer": {
      "id": "<string>",
      "name": "<string>",
      "lora_task_id": 123,
      "prompt": "<string>",
      "lora_weights_path": "<string>",
      "luna_input_type": "span",
      "luna_output_type": "float",
      "class_name_to_vocab_ix": {},
      "executor": "action_completion_luna"
    },
    "model_name": "<string>",
    "num_judges": 123,
    "scoreable_node_types": [
      "<string>"
    ],
    "cot_enabled": true,
    "output_type": "boolean",
    "input_type": "basic"
  },
  "model_type": "slm",
  "ground_truth": true,
  "default_version_id": "<string>",
  "default_version": {
    "id": "<string>",
    "version": 123,
    "scorer_id": "<string>",
    "generated_scorer": {
      "id": "<string>",
      "name": "<string>",
      "chain_poll_template": {
        "template": "<string>",
        "metric_system_prompt": "<string>",
        "metric_description": "<string>",
        "value_field_name": "rating",
        "explanation_field_name": "explanation",
        "metric_few_shot_examples": [
          {
            "generation_prompt_and_response": "<string>",
            "evaluating_response": "<string>"
          }
        ],
        "response_schema": {}
      },
      "instructions": "<string>",
      "user_prompt": "<string>"
    },
    "registered_scorer": {
      "id": "<string>",
      "name": "<string>",
      "score_type": "<string>"
    },
    "finetuned_scorer": {
      "id": "<string>",
      "name": "<string>",
      "lora_task_id": 123,
      "prompt": "<string>",
      "lora_weights_path": "<string>",
      "luna_input_type": "span",
      "luna_output_type": "float",
      "class_name_to_vocab_ix": {},
      "executor": "action_completion_luna"
    },
    "model_name": "<string>",
    "num_judges": 123,
    "scoreable_node_types": [
      "<string>"
    ],
    "cot_enabled": true,
    "output_type": "boolean",
    "input_type": "basic"
  },
  "user_prompt": "<string>",
  "scoreable_node_types": [
    "<string>"
  ],
  "output_type": "boolean",
  "input_type": "basic",
  "multimodal_capabilities": [
    "vision"
  ],
  "required_scorers": [
    "<string>"
  ],
  "deprecated": true,
  "roll_up_config": {
    "roll_up_methods": "<unknown>"
  },
  "label": "",
  "included_fields": [
    "<string>"
  ],
  "description": "<string>",
  "created_by": "<string>",
  "created_at": "2023-11-07T05:31:56Z",
  "updated_at": "2023-11-07T05:31:56Z",
  "roll_up_method": "average",
  "metric_color_picker_config": {
    "constraints": [
      {
        "color": "red",
        "operator": "eq",
        "value": 123
      }
    ],
    "type": "numeric"
  },
  "metric_name": "<string>"
}

Authorizations

​
Galileo-API-Key
string
header
required

Path Parameters

​
scorer_id
string<uuid4>
required

Body

application/json
​
name
string | null
​
description
string | null
​
tags
string[] | null
​
defaults
ScorerDefaults · object
​
model_type
enum<string> | null
Available options:
slm,
llm,
code
​
ground_truth
boolean | null
​
default_version_id
string<uuid4> | null
​
user_prompt
string | null
​
scoreable_node_types
string[] | null
​
output_type
enum<string> | null

Enumeration of output types.

Available options:
boolean,
categorical,
count,
discrete,
freeform,
percentage,
multilabel,
retrieved_chunk_list_boolean,
boolean_multilabel
​
input_type
enum<string> | null

Enumeration of input types.

Available options:
basic,
llm_spans,
retriever_spans,
sessions_normalized,
sessions_trace_io_only,
tool_spans,
trace_input_only,
trace_io_only,
trace_normalized,
trace_output_only,
agent_spans,
workflow_spans
​
multimodal_capabilities
enum<string>[] | null
Available options:
vision,
audio
​
roll_up_method
enum<string> | null

Display options for roll up methods when showing rolled up metrics in the UI.

Separates display intent from computation methods. The computation methods (NumericRollUpMethod, CategoricalRollUpMethod) control what aggregations are available. This enum controls how the UI displays the selected roll-up value for a scorer.

Available options:
average,
sum,
max,
min,
category_count,
percentage_true,
percentage_false
​
metric_color_picker_config
MetricColorPickerNumeric · object

Color picker configuration for numeric metrics.

Each constraint maps a numeric condition to a color. The UI uses these constraints to color-code metric values (e.g. green for high scores, red for low scores).

Example: { "type": "numeric", "constraints": [ {"color": "green", "operator": "gte", "value": 0.8}, {"color": "yellow", "operator": "between", "value": [0.3, 0.8]}, {"color": "red", "operator": "lt", "value": 0.3} ] }

Response

Successful Response

​
id
string<uuid4>
required
​
name
string
required
​
scorer_type
enum<string>
required
Available options:
llm,
code,
luna,
preset
​
tags
string[]
required
​
defaults
ScorerDefaults · object
​
latest_version
BaseScorerVersionDB · object

Scorer version from the scorer_versions table.

​
model_type
enum<string> | null
Available options:
slm,
llm,
code
​
ground_truth
boolean | null
​
default_version_id
string<uuid4> | null
​
default_version
BaseScorerVersionDB · object

Scorer version from the scorer_versions table.

​
user_prompt
string | null
​
scoreable_node_types
string[] | null
​
output_type
enum<string> | null

Enumeration of output types.

Available options:
boolean,
categorical,
count,
discrete,
freeform,
percentage,
multilabel,
retrieved_chunk_list_boolean,
boolean_multilabel
​
input_type
enum<string> | null

Enumeration of input types.

Available options:
basic,
llm_spans,
retriever_spans,
sessions_normalized,
sessions_trace_io_only,
tool_spans,
trace_input_only,
trace_io_only,
trace_normalized,
trace_output_only,
agent_spans,
workflow_spans
​
multimodal_capabilities
enum<string>[] | null
Available options:
vision,
audio
​
required_scorers
string[] | null
​
deprecated
boolean | null
​
roll_up_config
BaseMetricRollUpConfigDB · object

Configuration for rolling up metrics to parent/trace/session.

​
label
string | null
default:""
​
included_fields
string[]

Fields that can be used in the scorer to configure it. i.e. model, num_judges, etc. This enables the ui to know which fields a user can configure when they're setting a scorer

​
description
string | null
​
created_by
string<uuid4> | null
​
created_at
string<date-time> | null
​
updated_at
string<date-time> | null
​
roll_up_method
enum<string> | null

Display options for roll up methods when showing rolled up metrics in the UI.

Separates display intent from computation methods. The computation methods (NumericRollUpMethod, CategoricalRollUpMethod) control what aggregations are available. This enum controls how the UI displays the selected roll-up value for a scorer.

Available options:
average,
sum,
max,
min,
category_count,
percentage_true,
percentage_false
​
metric_color_picker_config
MetricColorPickerNumeric · object

Color picker configuration for numeric metrics.

Each constraint maps a numeric condition to a color. The UI uses these constraints to color-code metric values (e.g. green for high scores, red for low scores).

Example: { "type": "numeric", "constraints": [ {"color": "green", "operator": "gte", "value": 0.8}, {"color": "yellow", "operator": "between", "value": [0.3, 0.8]}, {"color": "red", "operator": "lt", "value": 0.3} ] }

​
metric_name
string | null