evaluate

Create Workflows Run

Create a new Evaluate run with workflows.

Use this endpoint to create a new Evaluate run with workflows. The request body should contain the workflows to be ingested and evaluated.

Additionally, specify the project_id or project_name to which the workflows should be ingested. If the project does not exist, it will be created. If the project exists, the workflows will be logged to it. If both project_id and project_name are provided, project_id will take precedence. The run_name is optional and will be auto-generated (timestamp-based) if not provided.

The body is also expected to include the configuration for the scorers to be used in the evaluation. This configuration will be used to evaluate the workflows and generate the results.

POST

evaluate

runs

Authorizations

Galileo-API-Key

string

header

required

Body

application/json

workflows

object[]

required

List of workflows to include in the run.

workflows.input

required

Input to the step.

workflows.created_at_ns

integer

Timestamp of the step's creation, as nanoseconds since epoch.

workflows.duration_ns

integer

default:

Duration of the step in nanoseconds.

workflows.ground_truth

string | null

Ground truth expected output for the step.

workflows.metadata

object

Metadata associated with this step.

workflows.name

string

default:

Name of the step.

workflows.output

default:

Output of the step.

workflows.parent

object | null

Parent node of the current node. For internal use only.

workflows.parent.input

required

Input to the step.

workflows.parent.created_at_ns

integer

Timestamp of the step's creation, as nanoseconds since epoch.

workflows.parent.duration_ns

integer

default:

Duration of the step in nanoseconds.

workflows.parent.ground_truth

string | null

Ground truth expected output for the step.

workflows.parent.metadata

object

Metadata associated with this step.

workflows.parent.name

string

default:

Name of the step.

workflows.parent.output

default:

Output of the step.

workflows.parent.parent

object | null

Parent node of the current node. For internal use only.

workflows.parent.parent.input

required

Input to the step.

workflows.parent.parent.created_at_ns

integer

Timestamp of the step's creation, as nanoseconds since epoch.

workflows.parent.parent.duration_ns

integer

default:

Duration of the step in nanoseconds.

workflows.parent.parent.ground_truth

string | null

Ground truth expected output for the step.

workflows.parent.parent.metadata

object

Metadata associated with this step.

workflows.parent.parent.name

string

default:

Name of the step.

workflows.parent.parent.output

default:

Output of the step.

workflows.parent.parent.parent

object | null

Parent node of the current node. For internal use only.

workflows.parent.parent.status_code

integer | null

Status code of the step. Used for logging failed/errored steps.

workflows.parent.parent.steps

object[]

Steps in the workflow.

workflows.parent.parent.type

enum<string>

Type of the step. By default, it is set to workflow.

Available options:

chain,

chat,

llm,

retriever,

tool,

agent,

workflow

workflows.parent.status_code

integer | null

Status code of the step. Used for logging failed/errored steps.

workflows.parent.steps

object[]

Steps in the workflow.

workflows.parent.steps.input

required

Input to the step.

workflows.parent.steps.created_at_ns

integer

Timestamp of the step's creation, as nanoseconds since epoch.

workflows.parent.steps.duration_ns

integer

default:

Duration of the step in nanoseconds.

workflows.parent.steps.ground_truth

string | null

Ground truth expected output for the step.

workflows.parent.steps.metadata

object

Metadata associated with this step.

workflows.parent.steps.name

string

default:

Name of the step.

workflows.parent.steps.output

default:

Output of the step.

workflows.parent.steps.parent

object | null

Parent node of the current node. For internal use only.

workflows.parent.steps.status_code

integer | null

Status code of the step. Used for logging failed/errored steps.

workflows.parent.steps.steps

object[]

Steps in the workflow.

workflows.parent.steps.type

string

default:

workflow

Type of the step. By default, it is set to workflow.

Allowed value: "workflow"

workflows.parent.type

enum<string>

Type of the step. By default, it is set to workflow.

Available options:

chain,

chat,

llm,

retriever,

tool,

agent,

workflow

workflows.status_code

integer | null

Status code of the step. Used for logging failed/errored steps.

workflows.steps

object[]

Steps in the workflow.

workflows.steps.input

required

Input to the step.

workflows.steps.created_at_ns

integer

Timestamp of the step's creation, as nanoseconds since epoch.

workflows.steps.duration_ns

integer

default:

Duration of the step in nanoseconds.

workflows.steps.ground_truth

string | null

Ground truth expected output for the step.

workflows.steps.metadata

object

Metadata associated with this step.

workflows.steps.name

string

default:

Name of the step.

workflows.steps.output

default:

Output of the step.

workflows.steps.parent

object | null

Parent node of the current node. For internal use only.

workflows.steps.parent.input

required

Input to the step.

workflows.steps.parent.created_at_ns

integer

Timestamp of the step's creation, as nanoseconds since epoch.

workflows.steps.parent.duration_ns

integer

default:

Duration of the step in nanoseconds.

workflows.steps.parent.ground_truth

string | null

Ground truth expected output for the step.

workflows.steps.parent.metadata

object

Metadata associated with this step.

workflows.steps.parent.name

string

default:

Name of the step.

workflows.steps.parent.output

default:

Output of the step.

workflows.steps.parent.parent

object | null

Parent node of the current node. For internal use only.

workflows.steps.parent.status_code

integer | null

Status code of the step. Used for logging failed/errored steps.

workflows.steps.parent.steps

object[]

Steps in the workflow.

workflows.steps.parent.type

enum<string>

Type of the step. By default, it is set to workflow.

Available options:

chain,

chat,

llm,

retriever,

tool,

agent,

workflow

workflows.steps.status_code

integer | null

Status code of the step. Used for logging failed/errored steps.

workflows.steps.steps

object[]

Steps in the workflow.

workflows.steps.steps.input

required

Input to the step.

workflows.steps.steps.created_at_ns

integer

Timestamp of the step's creation, as nanoseconds since epoch.

workflows.steps.steps.duration_ns

integer

default:

Duration of the step in nanoseconds.

workflows.steps.steps.ground_truth

string | null

Ground truth expected output for the step.

workflows.steps.steps.metadata

object

Metadata associated with this step.

workflows.steps.steps.name

string

default:

Name of the step.

workflows.steps.steps.output

default:

Output of the step.

workflows.steps.steps.parent

object | null

Parent node of the current node. For internal use only.

workflows.steps.steps.status_code

integer | null

Status code of the step. Used for logging failed/errored steps.

workflows.steps.steps.steps

object[]

Steps in the workflow.

workflows.steps.steps.type

string

default:

workflow

Type of the step. By default, it is set to workflow.

Allowed value: "workflow"

workflows.steps.type

string

default:

workflow

Type of the step. By default, it is set to workflow.

Allowed value: "workflow"

workflows.type

string

default:

workflow

Type of the step. By default, it is set to workflow.

Allowed value: "workflow"

generated_scorers

object[]

List of generated scorers to enable.

project_id

string | null

Evaluate Project ID to which the run should be associated.

project_name

string | null

Evaluate Project name to which the run should be associated. If the project does not exist, it will be created.

registered_scorers

object[]

List of registered scorers to enable.

run_name

string | null

Name of the run. If no name is provided, a timestamp-based name will be generated.

scorers

object[]

List of Galileo scorers to enable.

Response

200 - application/json

message

string

required

project_id

string

required

project_name

string

required

records_count

integer

required

run_id

string

required

run_name

string

required

workflows_count

integer

required

Was this page helpful?

Suggest edits Raise issue

Get Token Get Evaluate Run Results

API Reference

health

auth

evaluate

observe

protect

Create Workflows Run

Authorizations

Body

Response