Skip to main content
POST
Validate Decision Scorer Log Record

Authorizations

Splunk-AO-API-Key
string
header
required

Body

application/json

Test a decision evaluator on log records, before or after it is saved.

model_name, question and answer_options are at the top level, in place of the LLM prompt fields (ERD S6). The other evaluator settings are in scorer_configuration, as in the LLM request. The request also holds the query that selects the records.

model_name
string
required
Required string length: 1 - 255
question
string
required
Required string length: 1 - 2000
scorer_configuration
DecisionScorerTestConfiguration · object
required

The evaluator settings of a decision Test request, the same place as the LLM scorer_configuration.

scoreable_node_types holds exactly one node type. input_type defaults to the decision default for it (get_decision_input_type).

starting_token
integer
default:0
limit
integer
default:100
previous_last_row_id
string<uuid4> | null
log_stream_id
string<uuid4> | null

Log stream id associated with the traces.

experiment_id
string<uuid4> | null

Experiment id associated with the traces.

metrics_testing_id
string<uuid4> | null

Metrics testing id associated with the traces.

filters
(LogRecordsIDFilter · object | LogRecordsDateFilter · object | LogRecordsNumberFilter · object | LogRecordsBooleanFilter · object | LogRecordsCollectionFilter · object | LogRecordsTextFilter · object | LogRecordsFullyAnnotatedFilter · object)[]
filter_tree
FilterLeaf[Annotated[Union[LogRecordsIDFilter, LogRecordsDateFilter, LogRecordsNumberFilter, LogRecordsBooleanFilter, LogRecordsCollectionFilter, LogRecordsTextFilter, LogRecordsFullyAnnotatedFilter], FieldInfo(annotation=NoneType, required=True, discriminator='type')]] · object
sort
LogRecordsSortClause · object | null

Sort for the query. Defaults to native sort (created_at, id descending).

truncate_fields
boolean
default:false
include_counts
boolean
default:false

If True, include computed child counts (e.g., num_traces for sessions, num_spans for traces).

include_code_metric_metadata
boolean
default:false

If True, include per-row scorer metadata (the dict returned alongside the score by code-based scorers via the (score, metadata) tuple-return contract) on each MetricSuccess in the response. Off by default to keep payloads small for callers that don't need it.

answer_options

Response

Successful Response

Response model for validating a scorer based on log records.

Returns the uuid of the experiment created with the copied log records to store the metric testing results. Also returns the project_id so callers can poll /projects/{project_id}/traces/search.

metrics_experiment_id
string<uuid4>
required
project_id
string<uuid4>
required