Skip to main content
POST
/
v1
/
evaluate
/
runs
Create Workflows Run
curl --request POST \
  --url https://api.acme.rungalileo.io/v1/evaluate/runs \
  --header 'Content-Type: application/json' \
  --header 'Galileo-API-Key: <api-key>' \
  --data '{
  "project_name": "my-evaluate-project",
  "run_name": "my-evaluate-run",
  "scorers": [
    {
      "name": "correctness"
    },
    {
      "name": "output_pii"
    }
  ],
  "workflows": [
    {
      "created_at_ns": 1755290434782459100,
      "duration_ns": 0,
      "input": "who is a smart LLM?",
      "metadata": {},
      "name": "llm",
      "output": "I am!",
      "type": "llm"
    }
  ]
}'
{
  "message": "<string>",
  "project_id": "<string>",
  "project_name": "<string>",
  "run_id": "<string>",
  "run_name": "<string>",
  "workflows_count": 123,
  "records_count": 123
}

​
WorkflowStep

A workflow step is the atomic unit of logging to Galileo. They represent a single execution of a workflow, such as a chain, agent, or a RAG execution. Workflows can have multiple steps, each of which can be a different type of node, such as an LLM, Retriever, or Tool. You can log multiple workflows in a single request. Each workflow step must have the following fields:
  • type: The type of the workflow.
  • input: The input to the workflow.
  • output: The output of the workflow.

​
Examples

​
LLM Step

{
  "type": "llm",
  "input": "What is the capital of France?",
  "output": "Paris"
}

​
Retriever Step

{
  "type": "retriever",
  "input": "What is the capital of France?",
  "output": [{ "content": "Paris is the capital and largest city of France." }]
}

​
Multi-Step

Workflow steps of type workflow, agent or chain can have sub-steps with children. A workflow with a retriver and an LLM step would look like this: 
{
  "type": "workflow",
  "input": "What is the capital of France?",
  "output": "Paris",
  "steps": [
    {
      "type": "retriever",
      "input": "What is the capital of France?",
      "output": [{ "content": "Paris is the capital and largest city of France." }]
    },
    {
      "type": "llm",
      "input": "What is the capital of France?",
      "output": "Paris"
    }
  ]
}

Authorizations

​
Galileo-API-Key
string
header
required

Body

application/json
​
workflows
Workflows · array
required

List of workflows to include in the run.

Minimum length: 1
  • WorkflowStep
  • ChainStep
  • LlmStep
  • RetrieverStep
  • ToolStep
  • AgentStep
​
scorers
Scorers · array

List of Galileo scorers to enable.

  • AgenticWorkflowSuccessScorer
  • AgenticSessionSuccessScorer
  • BleuScorer
  • ChunkAttributionUtilizationScorer
  • CompletenessScorer
  • ContextAdherenceScorer
  • ContextRelevanceScorer
  • CorrectnessScorer
  • GroundTruthAdherenceScorer
  • InputPIIScorer
  • InputSexistScorer
  • InputToneScorer
  • InputToxicityScorer
  • InstructionAdherenceScorer
  • OutputPIIScorer
  • OutputSexistScorer
  • OutputToneScorer
  • OutputToxicityScorer
  • PromptInjectionScorer
  • PromptPerplexityScorer
  • RougeScorer
  • ToolErrorRateScorer
  • ToolSelectionQualityScorer
  • UncertaintyScorer
​
registered_scorers
RegisteredScorerConfig · object[]

List of registered scorers to enable.

​
generated_scorers
GeneratedScorerConfig · object[]

List of generated scorers to enable.

​
finetuned_scorers
FinetunedScorerConfig · object[]

List of finetuned scorers to enable.

​
project_id
string<uuid4> | null

Evaluate Project ID to which the run should be associated.

​
project_name
string | null

Evaluate Project name to which the run should be associated. If the project does not exist, it will be created.

​
run_name
string | null

Name of the run. If no name is provided, a timestamp-based name will be generated.

Response

Successful Response

​
message
string
required
​
project_id
string<uuid4>
required
​
project_name
string
required
​
run_id
string<uuid4>
required
​
run_name
string
required
​
workflows_count
integer
required
​
records_count
integer
required
⌘I