> ## Documentation Index > Fetch the complete documentation index at: https://docs.galileo.ai/llms.txt > Use this file to discover all available pages before exploring further. # Create Experiment > Create a new experiment for a project. ## OpenAPI ````yaml https://api.galileo.ai/public/v2/openapi.json post /v2/projects/{project_id}/experiments openapi: 3.1.0 info: title: FastAPI version: 0.1.0 servers: - url: https://api.galileo.ai description: Galileo Public APIs - galileo-v2 security: [] paths: /v2/projects/{project_id}/experiments: post: tags: - experiment summary: Create Experiment description: Create a new experiment for a project. operationId: create_experiment_v2_projects__project_id__experiments_post parameters: - name: project_id in: path required: true schema: type: string format: uuid4 title: Project Id requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/ExperimentCreateRequest' examples: - name: my_first_experiment responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/ExperimentResponse' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' security: - APIKeyHeader: [] - OAuth2PasswordBearer: [] components: schemas: ExperimentCreateRequest: properties: name: type: string minLength: 1 title: Name task_type: anyOf: - type: integer const: 16 - type: integer const: 17 title: Task Type default: 16 playground_id: anyOf: - type: string format: uuid4 - type: 'null' title: Playground Id prompt_template_version_id: anyOf: - type: string format: uuid4 - type: 'null' title: Prompt Template Version Id dataset: anyOf: - $ref: '#/components/schemas/ExperimentDatasetRequest' - type: 'null' playground_prompt_id: anyOf: - type: string format: uuid4 - type: 'null' title: Playground Prompt Id prompt_settings: anyOf: - $ref: '#/components/schemas/PromptRunSettings-Input' - type: 'null' scorers: items: $ref: '#/components/schemas/ScorerConfig' type: array title: Scorers trigger: type: boolean title: Trigger default: false experiment_group_id: anyOf: - type: string format: uuid4 - type: 'null' title: Experiment Group Id experiment_group_name: anyOf: - type: string maxLength: 255 minLength: 1 - type: 'null' title: Experiment Group Name type: object required: - name title: ExperimentCreateRequest ExperimentResponse: properties: id: type: string format: uuid4 title: ID description: Galileo ID of the experiment created_at: type: string format: date-time title: Created description: Timestamp of the experiment's creation updated_at: anyOf: - type: string format: date-time - type: 'null' title: Last Updated description: Timestamp of the trace or span's last update name: type: string title: Name description: Name of the experiment default: '' project_id: type: string format: uuid4 title: Project ID description: Galileo ID of the project associated with this experiment created_by: anyOf: - type: string format: uuid4 - type: 'null' title: Created By created_by_user: anyOf: - $ref: '#/components/schemas/UserInfo' - type: 'null' num_spans: anyOf: - type: integer - type: 'null' title: Num Spans num_traces: anyOf: - type: integer - type: 'null' title: Num Traces task_type: $ref: '#/components/schemas/TaskType' dataset: anyOf: - $ref: '#/components/schemas/ExperimentDataset' - type: 'null' aggregate_metrics: additionalProperties: true type: object title: Aggregate Metrics structured_aggregate_metrics: anyOf: - additionalProperties: $ref: '#/components/schemas/MetricAggregates' type: object - type: 'null' title: Structured Aggregate Metrics description: >- Structured aggregate metrics with full statistical aggregates (avg, min, max, sum, count). Keys are scorer UUIDs for scorer-backed metrics (matching available_columns column IDs after stripping the 'metrics/' prefix) and raw strings for system metrics (e.g. 'duration_ns', 'cost'). Present only when use_clickhouse_run_aggregates flag is enabled. aggregate_feedback: additionalProperties: $ref: '#/components/schemas/FeedbackAggregate' type: object title: Aggregate Feedback description: >- Aggregate feedback information related to the experiment (traces only) deprecated: true rating_aggregates: additionalProperties: additionalProperties: $ref: '#/components/schemas/FeedbackAggregate' propertyNames: $ref: '#/components/schemas/RootType' type: object propertyNames: format: uuid4 type: object title: Rating Aggregates description: Annotation aggregates keyed by template ID and root type ranking_score: anyOf: - type: number - type: 'null' title: Ranking Score rank: anyOf: - type: integer - type: 'null' title: Rank winner: anyOf: - type: boolean - type: 'null' title: Winner playground_id: anyOf: - type: string format: uuid4 - type: 'null' title: Playground Id playground: anyOf: - $ref: '#/components/schemas/ExperimentPlayground' - type: 'null' prompt_run_settings: anyOf: - $ref: '#/components/schemas/PromptRunSettings-Output' - type: 'null' prompt_model: anyOf: - type: string - type: 'null' title: Prompt Model prompt: anyOf: - $ref: '#/components/schemas/ExperimentPrompt' - type: 'null' tags: additionalProperties: items: $ref: '#/components/schemas/RunTagDB' type: array type: object title: Tags status: $ref: '#/components/schemas/ExperimentStatus' experiment_group_id: anyOf: - type: string format: uuid4 - type: 'null' title: Experiment Group Id experiment_group_name: anyOf: - type: string - type: 'null' title: Experiment Group Name experiment_group_is_system: anyOf: - type: boolean - type: 'null' title: Experiment Group Is System type: object required: - id - project_id - task_type title: ExperimentResponse HTTPValidationError: properties: detail: items: $ref: '#/components/schemas/ValidationError' type: array title: Detail type: object title: HTTPValidationError ExperimentDatasetRequest: properties: dataset_id: type: string format: uuid4 title: Dataset Id version_index: type: integer title: Version Index type: object required: - dataset_id - version_index title: ExperimentDatasetRequest PromptRunSettings-Input: properties: logprobs: type: boolean title: Logprobs default: true top_logprobs: type: integer title: Top Logprobs default: 5 echo: type: boolean title: Echo default: false 'n': type: integer title: 'N' default: 1 reasoning_effort: type: string title: Reasoning Effort default: medium verbosity: type: string title: Verbosity default: medium deployment_name: anyOf: - type: string - type: 'null' title: Deployment Name model_alias: type: string title: Model Alias default: gpt-5.1 temperature: anyOf: - type: number - type: 'null' title: Temperature max_tokens: type: integer title: Max Tokens default: 4096 stop_sequences: anyOf: - items: type: string type: array - type: 'null' title: Stop Sequences top_p: type: number title: Top P default: 1 top_k: type: integer title: Top K default: 40 frequency_penalty: type: number title: Frequency Penalty default: 0 presence_penalty: type: number title: Presence Penalty default: 0 tools: anyOf: - items: additionalProperties: true type: object type: array - type: 'null' title: Tools tool_choice: anyOf: - type: string - $ref: '#/components/schemas/OpenAIToolChoice' - type: 'null' title: Tool Choice response_format: anyOf: - additionalProperties: type: string type: object - type: 'null' title: Response Format known_models: items: $ref: '#/components/schemas/Model' type: array title: Known Models type: object title: PromptRunSettings description: Prompt run settings. ScorerConfig: properties: model_name: anyOf: - type: string - type: 'null' title: Model Name num_judges: anyOf: - type: integer - type: 'null' title: Num Judges filters: anyOf: - items: oneOf: - $ref: '#/components/schemas/NodeNameFilter' - $ref: '#/components/schemas/MetadataFilter' - $ref: '#/components/schemas/ModalityFilter' discriminator: propertyName: name mapping: metadata: $ref: '#/components/schemas/MetadataFilter' modality: $ref: '#/components/schemas/ModalityFilter' node_name: $ref: '#/components/schemas/NodeNameFilter' type: array - type: 'null' title: Filters description: List of filters to apply to the scorer. scoreable_node_types: anyOf: - items: type: string type: array - type: 'null' title: Scoreable Node Types description: >- List of node types that can be scored by this scorer. Defaults to llm/chat. cot_enabled: anyOf: - type: boolean - type: 'null' title: Cot Enabled description: >- Whether to enable chain of thought for this scorer. Defaults to False for llm scorers. output_type: anyOf: - $ref: '#/components/schemas/OutputTypeEnum' - type: 'null' description: >- What type of output to use for model-based scorers (boolean, categorical, etc.). input_type: anyOf: - $ref: '#/components/schemas/InputTypeEnum' - type: 'null' description: >- What type of input to use for model-based scorers (sessions_normalized, trace_io_only, etc..). id: type: string format: uuid4 title: Id name: anyOf: - type: string - type: 'null' title: Name scorer_type: $ref: '#/components/schemas/ScorerTypes' model_type: anyOf: - $ref: '#/components/schemas/ModelType' - type: 'null' description: >- Type of model to use for this scorer. slm maps to luna, and llm maps to plus scorer_version: anyOf: - $ref: '#/components/schemas/BaseScorerVersionDB' - type: 'null' description: >- ScorerVersion to use for this scorer. If not provided, the latest version will be used. multimodal_capabilities: anyOf: - items: $ref: '#/components/schemas/MultimodalCapability' type: array - type: 'null' title: Multimodal Capabilities description: >- Multimodal capabilities which this scorer can utilize in its evaluation. roll_up_method: anyOf: - $ref: '#/components/schemas/RollUpMethodDisplayOptions' - type: 'null' score_type: anyOf: - type: string - type: 'null' title: Score Type description: Return type of code scorers (e.g., 'bool', 'int', 'float', 'str'). type: object required: - id - scorer_type title: ScorerConfig description: Used for configuring a scorer for a scorer job. UserInfo: properties: id: type: string format: uuid4 title: Id email: type: string title: Email first_name: anyOf: - type: string - type: 'null' title: First Name last_name: anyOf: - type: string - type: 'null' title: Last Name type: object required: - id - email title: UserInfo description: A user's basic information, used for display purposes. TaskType: type: integer enum: - 0 - 1 - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 - 10 - 11 - 12 - 13 - 14 - 15 - 16 - 17 - 18 title: TaskType description: >- Valid task types for modeling. We store these as ints instead of strings because we will be looking this up in the database frequently. ExperimentDataset: properties: dataset_id: anyOf: - type: string format: uuid4 - type: 'null' title: Dataset Id version_index: anyOf: - type: integer - type: 'null' title: Version Index name: anyOf: - type: string - type: 'null' title: Name type: object title: ExperimentDataset MetricAggregates: properties: avg: anyOf: - type: number - type: 'null' title: Avg sum: anyOf: - type: number - type: 'null' title: Sum min: anyOf: - type: number - type: 'null' title: Min max: anyOf: - type: number - type: 'null' title: Max count: anyOf: - type: integer - type: 'null' title: Count pct: anyOf: - type: number - type: 'null' title: Pct p50: anyOf: - type: number - type: 'null' title: P50 p90: anyOf: - type: number - type: 'null' title: P90 p95: anyOf: - type: number - type: 'null' title: P95 p99: anyOf: - type: number - type: 'null' title: P99 value_distribution: anyOf: - additionalProperties: type: integer type: object - type: 'null' title: Value Distribution description: >- Distribution of discrete values as {value: count}. For boolean metrics: {'0': 2, '1': 8}. For categorical metrics: {'low': 5, 'medium': 3, 'high': 2}. type: object title: MetricAggregates description: >- Structured aggregate values for a single metric, computed from ClickHouse row-level data. FeedbackAggregate: properties: aggregate: oneOf: - $ref: '#/components/schemas/LikeDislikeAggregate' - $ref: '#/components/schemas/StarAggregate' - $ref: '#/components/schemas/ScoreAggregate' - $ref: '#/components/schemas/TagsAggregate' - $ref: '#/components/schemas/TextAggregate' title: Aggregate discriminator: propertyName: feedback_type mapping: like_dislike: $ref: '#/components/schemas/LikeDislikeAggregate' score: $ref: '#/components/schemas/ScoreAggregate' star: $ref: '#/components/schemas/StarAggregate' tags: $ref: '#/components/schemas/TagsAggregate' text: $ref: '#/components/schemas/TextAggregate' type: object required: - aggregate title: FeedbackAggregate ExperimentPlayground: properties: playground_id: anyOf: - type: string format: uuid4 - type: 'null' title: Playground Id name: anyOf: - type: string - type: 'null' title: Name type: object title: ExperimentPlayground PromptRunSettings-Output: properties: logprobs: type: boolean title: Logprobs default: true top_logprobs: type: integer title: Top Logprobs default: 5 echo: type: boolean title: Echo default: false 'n': type: integer title: 'N' default: 1 reasoning_effort: type: string title: Reasoning Effort default: medium verbosity: type: string title: Verbosity default: medium deployment_name: anyOf: - type: string - type: 'null' title: Deployment Name model_alias: type: string title: Model Alias default: gpt-5.1 temperature: anyOf: - type: number - type: 'null' title: Temperature max_tokens: type: integer title: Max Tokens default: 4096 stop_sequences: anyOf: - items: type: string type: array - type: 'null' title: Stop Sequences top_p: type: number title: Top P default: 1 top_k: type: integer title: Top K default: 40 frequency_penalty: type: number title: Frequency Penalty default: 0 presence_penalty: type: number title: Presence Penalty default: 0 tools: type: string title: Tools tool_choice: anyOf: - type: string - $ref: '#/components/schemas/OpenAIToolChoice' - type: 'null' title: Tool Choice response_format: anyOf: - additionalProperties: type: string type: object - type: 'null' title: Response Format type: object title: PromptRunSettings description: Prompt run settings. ExperimentPrompt: properties: prompt_template_id: anyOf: - type: string format: uuid4 - type: 'null' title: Prompt Template Id version_index: anyOf: - type: integer - type: 'null' title: Version Index name: anyOf: - type: string - type: 'null' title: Name content: anyOf: - type: string - type: 'null' title: Content type: object title: ExperimentPrompt RunTagDB: properties: key: type: string maxLength: 256 title: Key value: type: string maxLength: 256 title: Value tag_type: type: string title: Tag Type project_id: type: string format: uuid4 title: Project Id run_id: type: string format: uuid4 title: Run Id created_by: type: string format: uuid4 title: Created By id: type: string format: uuid4 title: Id created_at: type: string format: date-time title: Created At updated_at: type: string format: date-time title: Updated At type: object required: - key - value - tag_type - project_id - run_id - created_by - id - created_at - updated_at title: RunTagDB ExperimentStatus: properties: log_generation: $ref: '#/components/schemas/ExperimentPhaseStatus' type: object title: ExperimentStatus ValidationError: properties: loc: items: anyOf: - type: string - type: integer type: array title: Location msg: type: string title: Message type: type: string title: Error Type type: object required: - loc - msg - type title: ValidationError OpenAIToolChoice: properties: type: type: string title: Type default: function function: $ref: '#/components/schemas/OpenAIFunction' type: object required: - function title: OpenAIToolChoice Model: properties: name: type: string title: Name alias: type: string title: Alias integration: $ref: '#/components/schemas/LLMIntegration' default: openai user_role: anyOf: - type: string - type: 'null' title: User Role assistant_role: anyOf: - type: string - type: 'null' title: Assistant Role system_supported: type: boolean title: System Supported default: false input_modalities: items: $ref: '#/components/schemas/ContentModality' type: array title: Input Modalities description: Input modalities that the model can accept. alternative_names: items: type: string type: array title: Alternative Names description: >- Alternative names for the model, used for matching with various current, versioned or legacy names. input_token_limit: anyOf: - type: integer - type: 'null' title: Input Token Limit output_token_limit: anyOf: - type: integer - type: 'null' title: Output Token Limit token_limit: anyOf: - type: integer - type: 'null' title: Token Limit output_price: type: number title: Output Price default: 0 input_price: type: number title: Input Price default: 0 cost_by: $ref: '#/components/schemas/ModelCostBy' default: tokens is_chat: type: boolean title: Is Chat default: false provides_log_probs: type: boolean title: Provides Log Probs default: false formatting_tokens: type: integer title: Formatting Tokens default: 0 response_prefix_tokens: type: integer title: Response Prefix Tokens default: 0 api_version: anyOf: - type: string - type: 'null' title: Api Version legacy_mistral_prompt_format: type: boolean title: Legacy Mistral Prompt Format default: false requires_max_tokens: type: boolean title: Requires Max Tokens default: false max_top_p: anyOf: - type: number - type: 'null' title: Max Top P params_map: $ref: '#/components/schemas/RunParamsMap' output_map: anyOf: - $ref: '#/components/schemas/OutputMap' - type: 'null' input_map: anyOf: - $ref: '#/components/schemas/InputMap' - type: 'null' type: object required: - name - alias title: Model NodeNameFilter: properties: name: type: string const: node_name title: Name default: node_name operator: type: string enum: - eq - ne - contains - one_of - not_in title: Operator value: anyOf: - type: string - items: type: string type: array title: Value case_sensitive: type: boolean title: Case Sensitive default: true type: object required: - operator - value title: NodeNameFilter description: Filters on node names in scorer jobs. MetadataFilter: properties: name: type: string const: metadata title: Name default: metadata operator: type: string enum: - one_of - not_in - eq - ne title: Operator key: type: string title: Key value: anyOf: - type: string - items: type: string type: array title: Value type: object required: - operator - key - value title: MetadataFilter description: Filters on metadata key-value pairs in scorer jobs. ModalityFilter: properties: name: type: string const: modality title: Name default: modality operator: type: string enum: - eq - ne - one_of - not_in title: Operator value: anyOf: - type: string description: >- Single enum value - specific options depend on the concrete enum type used example: ENUM_VALUE - items: type: string example: ENUM_VALUE type: array description: Array of enum values example: - ENUM_VALUE_1 - ENUM_VALUE_2 title: Value type: object required: - operator - value title: ModalityFilter description: |- Filters on content modalities in scorer jobs. Matches if at least one of the specified modalities is present. OutputTypeEnum: type: string enum: - boolean - categorical - count - discrete - freeform - percentage - multilabel - retrieved_chunk_list_boolean - boolean_multilabel title: OutputTypeEnum description: Enumeration of output types. InputTypeEnum: type: string enum: - basic - llm_spans - retriever_spans - sessions_normalized - sessions_trace_io_only - tool_spans - trace_input_only - trace_io_only - trace_normalized - trace_output_only - agent_spans - workflow_spans title: InputTypeEnum description: Enumeration of input types. ScorerTypes: type: string enum: - llm - code - luna - preset title: ScorerTypes ModelType: type: string enum: - slm - llm - code title: ModelType BaseScorerVersionDB: properties: id: type: string format: uuid4 title: Id version: type: integer title: Version scorer_id: type: string format: uuid4 title: Scorer Id generated_scorer: anyOf: - $ref: '#/components/schemas/BaseGeneratedScorerDB' - type: 'null' registered_scorer: anyOf: - $ref: '#/components/schemas/BaseRegisteredScorerDB' - type: 'null' finetuned_scorer: anyOf: - $ref: '#/components/schemas/BaseFinetunedScorerDB' - type: 'null' model_name: anyOf: - type: string - type: 'null' title: Model Name num_judges: anyOf: - type: integer - type: 'null' title: Num Judges scoreable_node_types: anyOf: - items: type: string type: array - type: 'null' title: Scoreable Node Types description: >- List of node types that can be scored by this scorer. Defaults to llm/chat. cot_enabled: anyOf: - type: boolean - type: 'null' title: Cot Enabled description: >- Whether to enable chain of thought for this scorer. Defaults to False for llm scorers. output_type: anyOf: - $ref: '#/components/schemas/OutputTypeEnum' - type: 'null' description: >- What type of output to use for model-based scorers (sessions_normalized, trace_io_only, etc.). input_type: anyOf: - $ref: '#/components/schemas/InputTypeEnum' - type: 'null' description: >- What type of input to use for model-based scorers (sessions_normalized, trace_io_only, etc.). type: object required: - id - version - scorer_id title: BaseScorerVersionDB description: Scorer version from the scorer_versions table. MultimodalCapability: type: string enum: - vision - audio title: MultimodalCapability RollUpMethodDisplayOptions: type: string enum: - average - sum - max - min - category_count - percentage_true - percentage_false title: RollUpMethodDisplayOptions description: >- Display options for roll up methods when showing rolled up metrics in the UI. Separates display intent from computation methods. The computation methods (NumericRollUpMethod, CategoricalRollUpMethod) control what aggregations are available. This enum controls how the UI displays the selected roll-up value for a scorer. LikeDislikeAggregate: properties: feedback_type: type: string const: like_dislike title: Feedback Type default: like_dislike like_count: type: integer title: Like Count dislike_count: type: integer title: Dislike Count unrated_count: type: integer title: Unrated Count type: object required: - like_count - dislike_count - unrated_count title: LikeDislikeAggregate StarAggregate: properties: feedback_type: type: string const: star title: Feedback Type default: star average: type: number title: Average counts: additionalProperties: type: integer type: object title: Counts unrated_count: type: integer title: Unrated Count type: object required: - average - counts - unrated_count title: StarAggregate ScoreAggregate: properties: feedback_type: type: string const: score title: Feedback Type default: score average: type: number title: Average unrated_count: type: integer title: Unrated Count type: object required: - average - unrated_count title: ScoreAggregate TagsAggregate: properties: feedback_type: type: string const: tags title: Feedback Type default: tags counts: additionalProperties: type: integer type: object title: Counts unrated_count: type: integer title: Unrated Count type: object required: - counts - unrated_count title: TagsAggregate TextAggregate: properties: feedback_type: type: string const: text title: Feedback Type default: text count: type: integer title: Count unrated_count: type: integer title: Unrated Count type: object required: - count - unrated_count title: TextAggregate ExperimentPhaseStatus: properties: progress_percent: type: number maximum: 1 minimum: 0 title: Progress Percent description: Progress percentage from 0.0 to 1.0 default: 0 type: object title: ExperimentPhaseStatus OpenAIFunction: properties: name: type: string title: Name type: object required: - name title: OpenAIFunction LLMIntegration: type: string enum: - anthropic - aws_bedrock - aws_sagemaker - azure - custom - databricks - mistral - nvidia - openai - vegas_gateway - vertex_ai - writer title: LLMIntegration ContentModality: type: string enum: - text - document - image - audio - video title: ContentModality description: Classification of content modality ModelCostBy: type: string enum: - tokens - characters title: ModelCostBy RunParamsMap: properties: model: anyOf: - type: string - type: 'null' title: Model temperature: anyOf: - type: string - type: 'null' title: Temperature max_tokens: anyOf: - type: string - type: 'null' title: Max Tokens stop_sequences: anyOf: - type: string - type: 'null' title: Stop Sequences top_p: anyOf: - type: string - type: 'null' title: Top P top_k: anyOf: - type: string - type: 'null' title: Top K frequency_penalty: anyOf: - type: string - type: 'null' title: Frequency Penalty presence_penalty: anyOf: - type: string - type: 'null' title: Presence Penalty echo: anyOf: - type: string - type: 'null' title: Echo logprobs: anyOf: - type: string - type: 'null' title: Logprobs top_logprobs: anyOf: - type: string - type: 'null' title: Top Logprobs 'n': anyOf: - type: string - type: 'null' title: 'N' api_version: anyOf: - type: string - type: 'null' title: Api Version tools: anyOf: - type: string - type: 'null' title: Tools tool_choice: anyOf: - type: string - type: 'null' title: Tool Choice response_format: anyOf: - type: string - type: 'null' title: Response Format reasoning_effort: anyOf: - type: string - type: 'null' title: Reasoning Effort verbosity: anyOf: - type: string - type: 'null' title: Verbosity deployment_name: anyOf: - type: string - type: 'null' title: Deployment Name type: object title: RunParamsMap description: >- Maps the internal settings parameters (left) to the serialized parameters (right) we want to send in the API requests. OutputMap: properties: response: type: string title: Response token_count: anyOf: - type: string - type: 'null' title: Token Count input_token_count: anyOf: - type: string - type: 'null' title: Input Token Count output_token_count: anyOf: - type: string - type: 'null' title: Output Token Count completion_reason: anyOf: - type: string - type: 'null' title: Completion Reason type: object required: - response title: OutputMap InputMap: properties: prompt: type: string title: Prompt prefix: type: string title: Prefix default: '' suffix: type: string title: Suffix default: '' type: object required: - prompt title: InputMap BaseGeneratedScorerDB: properties: id: type: string format: uuid4 title: Id name: type: string title: Name instructions: anyOf: - type: string - type: 'null' title: Instructions chain_poll_template: $ref: '#/components/schemas/ChainPollTemplate' user_prompt: anyOf: - type: string - type: 'null' title: User Prompt type: object required: - id - name - chain_poll_template title: BaseGeneratedScorerDB BaseRegisteredScorerDB: properties: id: type: string format: uuid4 title: Id name: type: string title: Name score_type: anyOf: - type: string - type: 'null' title: Score Type type: object required: - id - name title: BaseRegisteredScorerDB BaseFinetunedScorerDB: properties: id: type: string format: uuid4 title: Id name: type: string title: Name lora_task_id: type: integer title: Lora Task Id lora_weights_path: anyOf: - type: string - type: 'null' title: Lora Weights Path prompt: type: string title: Prompt luna_input_type: anyOf: - $ref: '#/components/schemas/LunaInputTypeEnum' - type: 'null' luna_output_type: anyOf: - $ref: '#/components/schemas/LunaOutputTypeEnum' - type: 'null' class_name_to_vocab_ix: anyOf: - additionalProperties: items: type: integer type: array uniqueItems: true type: object - additionalProperties: type: integer type: object - type: 'null' title: Class Name To Vocab Ix executor: anyOf: - $ref: '#/components/schemas/ScorerName' - type: 'null' description: >- Executor pipeline. Defaults to finetuned scorer pipeline but can run custom galileo score pipelines. type: object required: - id - name - lora_task_id - prompt title: BaseFinetunedScorerDB ChainPollTemplate: properties: metric_system_prompt: anyOf: - type: string - type: 'null' title: Metric System Prompt description: System prompt for the metric. metric_description: anyOf: - type: string - type: 'null' title: Metric Description description: Description of what the metric should do. value_field_name: type: string title: Value Field Name description: Field name to look for in the chainpoll response, for the rating. default: rating explanation_field_name: type: string title: Explanation Field Name description: >- Field name to look for in the chainpoll response, for the explanation. default: explanation template: type: string title: Template description: Chainpoll prompt template. metric_few_shot_examples: items: $ref: '#/components/schemas/FewShotExample' type: array title: Metric Few Shot Examples description: Few-shot examples for the metric. response_schema: anyOf: - additionalProperties: true type: object - type: 'null' title: Response Schema description: Response schema for the output type: object required: - template title: ChainPollTemplate description: |- Template for a chainpoll metric prompt, containing all the info necessary to send a chainpoll prompt. LunaInputTypeEnum: type: string enum: - span - trace_object - trace_input_output_only title: LunaInputTypeEnum LunaOutputTypeEnum: type: string enum: - float - string - string_list - bool_list title: LunaOutputTypeEnum ScorerName: type: string enum: - action_completion_luna - action_advancement_luna - agentic_session_success - agentic_session_success - agentic_workflow_success - agentic_workflow_success - agent_efficiency - agent_flow - bleu - chunk_attribution_utilization_luna - chunk_attribution_utilization - chunk_relevance - chunk_relevance_luna - context_precision - precision_at_k - completeness_luna - completeness - context_adherence - context_adherence_luna - context_relevance - context_relevance_luna - conversation_quality - correctness - ground_truth_adherence - visual_fidelity - visual_quality - input_pii - input_pii_gpt - input_sexist - input_sexist - input_sexist_luna - input_sexist_luna - input_tone - input_tone_gpt - input_toxicity - input_toxicity_luna - instruction_adherence - output_pii - output_pii_gpt - output_sexist - output_sexist - output_sexist_luna - output_sexist_luna - output_tone - output_tone_gpt - output_toxicity - output_toxicity_luna - prompt_injection - prompt_injection_luna - prompt_perplexity - rouge - reasoning_coherence - sql_efficiency - sql_adherence - sql_injection - sql_correctness - tool_error_rate - tool_error_rate_luna - tool_selection_quality - tool_selection_quality_luna - uncertainty - user_intent_change - interruption_detection title: ScorerName FewShotExample: properties: generation_prompt_and_response: type: string title: Generation Prompt And Response evaluating_response: type: string title: Evaluating Response type: object required: - generation_prompt_and_response - evaluating_response title: FewShotExample description: Few-shot example for a chainpoll metric prompt. securitySchemes: APIKeyHeader: type: apiKey in: header name: Galileo-API-Key OAuth2PasswordBearer: type: oauth2 flows: password: scopes: {} tokenUrl: https://api.galileo.ai/login ````