Skip to main content
POST
/
v2
/
scorers
/
{scorer_id}
/
version
/
preset
Create Preset Scorer Version
curl --request POST \
  --url https://api.galileo.ai/v2/scorers/{scorer_id}/version/preset \
  --header 'Content-Type: application/json' \
  --header 'Galileo-API-Key: <api-key>' \
  --data '
{
  "model_name": "<string>",
  "num_judges": 123,
  "scoreable_node_types": [
    "<string>"
  ],
  "cot_enabled": true,
  "output_type": "boolean",
  "input_type": "basic"
}
'
{
  "id": "<string>",
  "version": 123,
  "scorer_id": "<string>",
  "created_at": "2023-11-07T05:31:56Z",
  "updated_at": "2023-11-07T05:31:56Z",
  "generated_scorer": {
    "id": "<string>",
    "name": "<string>",
    "chain_poll_template": {
      "template": "<string>",
      "metric_system_prompt": "<string>",
      "metric_description": "<string>",
      "value_field_name": "rating",
      "explanation_field_name": "explanation",
      "metric_few_shot_examples": [
        {
          "generation_prompt_and_response": "<string>",
          "evaluating_response": "<string>"
        }
      ],
      "response_schema": {}
    },
    "created_by": "<string>",
    "created_at": "2023-11-07T05:31:56Z",
    "updated_at": "2023-11-07T05:31:56Z",
    "scoreable_node_types": [
      "chain"
    ],
    "scorer_configuration": {
      "model_alias": "gpt-4.1-mini",
      "num_judges": 3,
      "output_type": "boolean",
      "scoreable_node_types": [
        "<string>"
      ],
      "cot_enabled": false,
      "ground_truth": false,
      "multimodal_capabilities": [
        "vision"
      ]
    },
    "instructions": "<string>",
    "user_prompt": "<string>"
  },
  "registered_scorer": {
    "id": "<string>",
    "name": "<string>",
    "score_type": "<string>",
    "created_at": "2023-11-07T05:31:56Z",
    "updated_at": "2023-11-07T05:31:56Z",
    "created_by": "<string>",
    "data_type": "unknown",
    "scoreable_node_types": [
      "<string>"
    ]
  },
  "finetuned_scorer": {
    "id": "<string>",
    "name": "<string>",
    "lora_task_id": 123,
    "prompt": "<string>",
    "created_at": "2023-11-07T05:31:56Z",
    "updated_at": "2023-11-07T05:31:56Z",
    "created_by": "<string>",
    "lora_weights_path": "<string>",
    "luna_input_type": "span",
    "luna_output_type": "float",
    "class_name_to_vocab_ix": {},
    "executor": "action_completion_luna"
  },
  "model_name": "<string>",
  "num_judges": 123,
  "scoreable_node_types": [
    "<string>"
  ],
  "cot_enabled": true,
  "output_type": "boolean",
  "input_type": "basic",
  "chain_poll_template": {
    "template": "<string>",
    "metric_system_prompt": "<string>",
    "metric_description": "<string>",
    "value_field_name": "rating",
    "explanation_field_name": "explanation",
    "metric_few_shot_examples": [
      {
        "generation_prompt_and_response": "<string>",
        "evaluating_response": "<string>"
      }
    ],
    "response_schema": {}
  },
  "allowed_model": true
}

Documentation Index

Fetch the complete documentation index at: https://docs.galileo.ai/llms.txt

Use this file to discover all available pages before exploring further.

Authorizations

​
Galileo-API-Key
string
header
required

Path Parameters

​
scorer_id
string<uuid4>
required

Body

application/json
​
model_name
string | null
​
num_judges
integer | null
​
scoreable_node_types
string[] | null
​
cot_enabled
boolean | null
​
output_type
enum<string> | null

Enumeration of output types.

Available options:
boolean,
categorical,
count,
discrete,
freeform,
percentage,
multilabel,
retrieved_chunk_list_boolean,
boolean_multilabel
​
input_type
enum<string> | null

Enumeration of input types.

Available options:
basic,
llm_spans,
retriever_spans,
sessions_normalized,
sessions_trace_io_only,
tool_spans,
trace_input_only,
trace_io_only,
trace_normalized,
trace_output_only,
agent_spans,
workflow_spans

Response

Successful Response

​
id
string<uuid4>
required
​
version
integer
required
​
scorer_id
string<uuid4>
required
​
created_at
string<date-time>
required
​
updated_at
string<date-time>
required
​
generated_scorer
GeneratedScorerResponse · object
​
registered_scorer
CreateUpdateRegisteredScorerResponse · object
​
finetuned_scorer
FineTunedScorerResponse · object
​
model_name
string | null
​
num_judges
integer | null
​
scoreable_node_types
string[] | null
​
cot_enabled
boolean | null
​
output_type
enum<string> | null

Enumeration of output types.

Available options:
boolean,
categorical,
count,
discrete,
freeform,
percentage,
multilabel,
retrieved_chunk_list_boolean,
boolean_multilabel
​
input_type
enum<string> | null

What type of input to use for model-based scorers (sessions_normalized, trace_io_only, etc.).

Available options:
basic,
llm_spans,
retriever_spans,
sessions_normalized,
sessions_trace_io_only,
tool_spans,
trace_input_only,
trace_io_only,
trace_normalized,
trace_output_only,
agent_spans,
workflow_spans
​
chain_poll_template
ChainPollTemplate · object

Template for a chainpoll metric prompt, containing all the info necessary to send a chainpoll prompt.

​
allowed_model
boolean | null