Skip to main content
POST
/
v2
/
scorers
/
llm
/
validate
/
dataset
Validate Llm Scorer Dataset
curl --request POST \
  --url https://api.galileo.ai/v2/scorers/llm/validate/dataset \
  --header 'Content-Type: application/json' \
  --header 'Galileo-API-Key: <api-key>' \
  --data '
{
  "query": "<string>",
  "response": "<string>",
  "chain_poll_template": {
    "template": "<string>",
    "metric_system_prompt": "<string>",
    "metric_description": "<string>",
    "value_field_name": "rating",
    "explanation_field_name": "explanation",
    "metric_few_shot_examples": [
      {
        "generation_prompt_and_response": "<string>",
        "evaluating_response": "<string>"
      }
    ],
    "response_schema": {}
  },
  "scorer_configuration": {
    "model_alias": "gpt-4.1-mini",
    "num_judges": 3,
    "output_type": "boolean",
    "scoreable_node_types": [
      "<string>"
    ],
    "cot_enabled": false,
    "ground_truth": false,
    "multimodal_capabilities": []
  },
  "user_prompt": "<string>",
  "dataset_id": "<string>",
  "normalized_input": [
    {
      "text": "<string>",
      "type": "text"
    }
  ],
  "dataset_version_index": 123,
  "limit": 100,
  "starting_token": 123,
  "sort": {}
}
'
{
  "metrics_experiment_id": "<string>",
  "project_id": "<string>"
}

Authorizations

​
Galileo-API-Key
string
header
required

Body

application/json

Request to validate a new LLM scorer against a dataset.

​
query
string
required
​
response
string
required
​
chain_poll_template
ChainPollTemplate · object
required

Template for a chainpoll metric prompt, containing all the info necessary to send a chainpoll prompt.

​
scorer_configuration
GeneratedScorerConfiguration · object
required
​
user_prompt
string
required
​
dataset_id
string<uuid4>
required
​
normalized_input
(TextContentPart · object | FileContentPart · object)[] | null

Optional multimodal content parts. When set, replaces the text-only query/response formatting in the validation job so that file content is passed through to the LLM.

A text segment within a message.

​
dataset_version_index
integer | null
​
limit
integer
default:100

Maximum number of dataset rows to process.

​
starting_token
integer | null

Pagination offset into dataset rows.

​
sort
Sort · object

Optional sort configuration for dataset rows.

Response

Successful Response

​
metrics_experiment_id
string<uuid4>
required
​
project_id
string<uuid4>
required