Skip to main content
POST
/
v1
/
evaluate
/
runs
Create Workflows Run
curl --request POST \
  --url https://api.acme.rungalileo.io/v1/evaluate/runs \
  --header 'Content-Type: application/json' \
  --header 'Galileo-API-Key: <api-key>' \
  --data '
{
  "workflows": [
    {
      "input": "<string>",
      "type": "workflow",
      "redacted_input": "<string>",
      "output": "",
      "redacted_output": "<string>",
      "name": "",
      "created_at_ns": 123,
      "duration_ns": 0,
      "metadata": {},
      "status_code": 123,
      "ground_truth": "<string>",
      "steps": [
        "<unknown>"
      ],
      "parent": {
        "input": "<string>",
        "type": "workflow",
        "redacted_input": "<string>",
        "output": "",
        "redacted_output": "<string>",
        "name": "",
        "created_at_ns": 123,
        "duration_ns": 0,
        "metadata": {},
        "status_code": 123,
        "ground_truth": "<string>",
        "steps": "<array>",
        "parent": "<unknown>"
      }
    }
  ],
  "scorers": [
    {
      "name": "agentic_workflow_success",
      "filters": [
        {
          "value": "<string>",
          "operator": "eq",
          "name": "node_name",
          "filter_type": "string",
          "case_sensitive": true
        }
      ],
      "type": "plus",
      "model_name": "<string>",
      "num_judges": 5
    }
  ],
  "registered_scorers": [
    {
      "name": "<string>",
      "filters": [
        {
          "value": "<string>",
          "operator": "eq",
          "name": "node_name",
          "filter_type": "string",
          "case_sensitive": true
        }
      ]
    }
  ],
  "generated_scorers": [
    {
      "name": "<string>",
      "filters": [
        {
          "value": "<string>",
          "operator": "eq",
          "name": "node_name",
          "filter_type": "string",
          "case_sensitive": true
        }
      ]
    }
  ],
  "finetuned_scorers": [
    {
      "name": "<string>",
      "filters": [
        {
          "value": "<string>",
          "operator": "eq",
          "name": "node_name",
          "filter_type": "string",
          "case_sensitive": true
        }
      ]
    }
  ],
  "project_id": "<string>",
  "project_name": "<string>",
  "run_name": "<string>"
}
'
{
  "message": "<string>",
  "project_id": "<string>",
  "project_name": "<string>",
  "run_id": "<string>",
  "run_name": "<string>",
  "workflows_count": 123,
  "records_count": 123
}

Authorizations

​
Galileo-API-Key
string
header
required

Body

application/json
​
workflows
(WorkflowStep · object | ChainStep · object | LlmStep · object | RetrieverStep · object | ToolStep · object | AgentStep · object)[]
required

List of workflows to include in the run.

Minimum array length: 1
  • WorkflowStep
  • ChainStep
  • LlmStep
  • RetrieverStep
  • ToolStep
  • AgentStep
​
scorers
(AgenticWorkflowSuccessScorer · object | AgenticSessionSuccessScorer · object | BleuScorer · object | ChunkAttributionUtilizationScorer · object | CompletenessScorer · object | ContextAdherenceScorer · object | ContextRelevanceScorer · object | CorrectnessScorer · object | GroundTruthAdherenceScorer · object | InputPIIScorer · object | InputSexistScorer · object | InputToneScorer · object | InputToxicityScorer · object | InstructionAdherenceScorer · object | OutputPIIScorer · object | OutputSexistScorer · object | OutputToneScorer · object | OutputToxicityScorer · object | PromptInjectionScorer · object | PromptPerplexityScorer · object | RougeScorer · object | ToolErrorRateScorer · object | ToolSelectionQualityScorer · object | UncertaintyScorer · object)[]

List of Galileo scorers to enable.

  • AgenticWorkflowSuccessScorer
  • AgenticSessionSuccessScorer
  • BleuScorer
  • ChunkAttributionUtilizationScorer
  • CompletenessScorer
  • ContextAdherenceScorer
  • ContextRelevanceScorer
  • CorrectnessScorer
  • GroundTruthAdherenceScorer
  • InputPIIScorer
  • InputSexistScorer
  • InputToneScorer
  • InputToxicityScorer
  • InstructionAdherenceScorer
  • OutputPIIScorer
  • OutputSexistScorer
  • OutputToneScorer
  • OutputToxicityScorer
  • PromptInjectionScorer
  • PromptPerplexityScorer
  • RougeScorer
  • ToolErrorRateScorer
  • ToolSelectionQualityScorer
  • UncertaintyScorer
​
registered_scorers
RegisteredScorerConfig · object[]

List of registered scorers to enable.

​
generated_scorers
GeneratedScorerConfig · object[]

List of generated scorers to enable.

​
finetuned_scorers
FinetunedScorerConfig · object[]

List of finetuned scorers to enable.

​
project_id
string<uuid4> | null

Evaluate Project ID to which the run should be associated.

​
project_name
string | null

Evaluate Project name to which the run should be associated. If the project does not exist, it will be created.

​
run_name
string | null

Name of the run. If no name is provided, a timestamp-based name will be generated.

Response

Successful Response

​
message
string
required
​
project_id
string<uuid4>
required
​
project_name
string
required
​
run_id
string<uuid4>
required
​
run_name
string
required
​
workflows_count
integer
required
​
records_count
integer
required