evaluate

Create Workflows Run

Create a new Evaluate run with workflows.

Use this endpoint to create a new Evaluate run with workflows. The request body should contain the workflows to be ingested and evaluated.

Additionally, specify the project_id or project_name to which the workflows should be ingested. If the project does not exist, it will be created. If the project exists, the workflows will be logged to it. If both project_id and project_name are provided, project_id will take precedence. The run_name is optional and will be auto-generated (timestamp-based) if not provided.

The body is also expected to include the configuration for the scorers to be used in the evaluation. This configuration will be used to evaluate the workflows and generate the results.

POST

evaluate

runs

curl --request POST \
  --url https://api.acme.rungalileo.io/v1/evaluate/runs \
  --header 'Content-Type: application/json' \
  --header 'Galileo-API-Key: <api-key>' \
  --data '{
  "project_name": "my-evaluate-project",
  "run_name": "my-evaluate-run",
  "scorers": [
    {
      "name": "correctness"
    },
    {
      "name": "output_pii"
    }
  ],
  "workflows": [
    {
      "created_at_ns": 1744827344427819800,
      "duration_ns": 0,
      "input": "who is a smart LLM?",
      "metadata": {},
      "name": "llm",
      "output": "I am!",
      "type": "llm"
    }
  ]
}'

{
  "message": "<string>",
  "project_id": "<string>",
  "project_name": "<string>",
  "run_id": "<string>",
  "run_name": "<string>",
  "workflows_count": 123,
  "records_count": 123
}

Authorizations

Galileo-API-Key

string

header

required

Body

application/json

workflows

object[]

required

List of workflows to include in the run.

workflows.input

required

Input to the step.

workflows.type

string

default:workflow

Type of the step. By default, it is set to workflow.

Allowed value: "workflow"

workflows.output

default:

Output of the step.

workflows.name

string

default:

Name of the step.

workflows.created_at_ns

integer

Timestamp of the step's creation, as nanoseconds since epoch.

workflows.duration_ns

integer

default:0

Duration of the step in nanoseconds.

workflows.metadata

object

Metadata associated with this step.

workflows.status_code

integer | null

Status code of the step. Used for logging failed/errored steps.

workflows.ground_truth

string | null

Ground truth expected output for the step.

workflows.steps

object[]

Steps in the workflow.

workflows.steps.input

required

Input to the step.

workflows.steps.type

string

default:workflow

Type of the step. By default, it is set to workflow.

Allowed value: "workflow"

workflows.steps.output

default:

Output of the step.

workflows.steps.name

string

default:

Name of the step.

workflows.steps.created_at_ns

integer

Timestamp of the step's creation, as nanoseconds since epoch.

workflows.steps.duration_ns

integer

default:0

Duration of the step in nanoseconds.

workflows.steps.metadata

object

Metadata associated with this step.

workflows.steps.status_code

integer | null

Status code of the step. Used for logging failed/errored steps.

workflows.steps.ground_truth

string | null

Ground truth expected output for the step.

workflows.steps.steps

object[]

Steps in the workflow.

workflows.steps.steps.input

required

Input to the step.

workflows.steps.steps.type

string

default:workflow

Type of the step. By default, it is set to workflow.

Allowed value: "workflow"

workflows.steps.steps.output

default:

Output of the step.

workflows.steps.steps.name

string

default:

Name of the step.

workflows.steps.steps.created_at_ns

integer

Timestamp of the step's creation, as nanoseconds since epoch.

workflows.steps.steps.duration_ns

integer

default:0

Duration of the step in nanoseconds.

workflows.steps.steps.metadata

object

Metadata associated with this step.

workflows.steps.steps.status_code

integer | null

Status code of the step. Used for logging failed/errored steps.

workflows.steps.steps.ground_truth

string | null

Ground truth expected output for the step.

workflows.steps.steps.steps

object[]

Steps in the workflow.

workflows.steps.steps.parent

object | null

Parent node of the current node. For internal use only.

workflows.steps.parent

object | null

Parent node of the current node. For internal use only.

workflows.steps.parent.input

required

Input to the step.

workflows.steps.parent.type

enum<string>

Type of the step. By default, it is set to workflow.

Available options:

chain,

chat,

llm,

retriever,

tool,

agent,

workflow,

trace

workflows.steps.parent.output

default:

Output of the step.

workflows.steps.parent.name

string

default:

Name of the step.

workflows.steps.parent.created_at_ns

integer

Timestamp of the step's creation, as nanoseconds since epoch.

workflows.steps.parent.duration_ns

integer

default:0

Duration of the step in nanoseconds.

workflows.steps.parent.metadata

object

Metadata associated with this step.

workflows.steps.parent.status_code

integer | null

Status code of the step. Used for logging failed/errored steps.

workflows.steps.parent.ground_truth

string | null

Ground truth expected output for the step.

workflows.steps.parent.steps

object[]

Steps in the workflow.

parent

object

workflows.parent

object | null

Parent node of the current node. For internal use only.

workflows.parent.input

required

Input to the step.

workflows.parent.type

enum<string>

Type of the step. By default, it is set to workflow.

Available options:

chain,

chat,

llm,

retriever,

tool,

agent,

workflow,

trace

workflows.parent.output

default:

Output of the step.

workflows.parent.name

string

default:

Name of the step.

workflows.parent.created_at_ns

integer

Timestamp of the step's creation, as nanoseconds since epoch.

workflows.parent.duration_ns

integer

default:0

Duration of the step in nanoseconds.

workflows.parent.metadata

object

Metadata associated with this step.

workflows.parent.status_code

integer | null

Status code of the step. Used for logging failed/errored steps.

workflows.parent.ground_truth

string | null

Ground truth expected output for the step.

workflows.parent.steps

object[]

Steps in the workflow.

workflows.parent.steps.input

required

Input to the step.

workflows.parent.steps.type

string

default:workflow

Type of the step. By default, it is set to workflow.

Allowed value: "workflow"

workflows.parent.steps.output

default:

Output of the step.

workflows.parent.steps.name

string

default:

Name of the step.

workflows.parent.steps.created_at_ns

integer

Timestamp of the step's creation, as nanoseconds since epoch.

workflows.parent.steps.duration_ns

integer

default:0

Duration of the step in nanoseconds.

workflows.parent.steps.metadata

object

Metadata associated with this step.

workflows.parent.steps.status_code

integer | null

Status code of the step. Used for logging failed/errored steps.

workflows.parent.steps.ground_truth

string | null

Ground truth expected output for the step.

workflows.parent.steps.steps

object[]

Steps in the workflow.

workflows.parent.steps.parent

object | null

Parent node of the current node. For internal use only.

parent

object

scorers

object[]

List of Galileo scorers to enable.

registered_scorers

object[]

List of registered scorers to enable.

generated_scorers

object[]

List of generated scorers to enable.

project_id

string | null

Evaluate Project ID to which the run should be associated.

project_name

string | null

Evaluate Project name to which the run should be associated. If the project does not exist, it will be created.

run_name

string | null

Name of the run. If no name is provided, a timestamp-based name will be generated.

Response

200

application/json

Successful Response

message

string

required

project_id

string

required

project_name

string

required

run_id

string

required

run_name

string

required

workflows_count

integer

required

records_count

integer

required

Was this page helpful?

Suggest edits Raise issue

Get Token Get Evaluate Run Results

curl --request POST \
  --url https://api.acme.rungalileo.io/v1/evaluate/runs \
  --header 'Content-Type: application/json' \
  --header 'Galileo-API-Key: <api-key>' \
  --data '{
  "project_name": "my-evaluate-project",
  "run_name": "my-evaluate-run",
  "scorers": [
    {
      "name": "correctness"
    },
    {
      "name": "output_pii"
    }
  ],
  "workflows": [
    {
      "created_at_ns": 1744827344427819800,
      "duration_ns": 0,
      "input": "who is a smart LLM?",
      "metadata": {},
      "name": "llm",
      "output": "I am!",
      "type": "llm"
    }
  ]
}'

{
  "message": "<string>",
  "project_id": "<string>",
  "project_name": "<string>",
  "run_id": "<string>",
  "run_name": "<string>",
  "workflows_count": 123,
  "records_count": 123
}

API Reference

health

auth

evaluate

observe

protect

trace

log_stream

experiment

Webhooks

Create Workflows Run

Authorizations

Body

Response