Skip to main content

Type Alias: ScorerConfigOpenAPI

Defined in: src/types/openapi.types.ts ScorerConfig Used for configuring a scorer for a scorer job.

Properties

cot_enabled?

Defined in: src/types/openapi.types.ts Cot Enabled Whether to enable chain of thought for this scorer. Defaults to False for llm scorers.

filters?

Defined in: src/types/openapi.types.ts Filters List of filters to apply to the scorer.

id

Defined in: src/types/openapi.types.ts Id

input_type?

Defined in: src/types/openapi.types.ts What type of input to use for model-based scorers (sessions_normalized, trace_io_only, etc..).

model_name?

Defined in: src/types/openapi.types.ts Model Name

model_type?

Defined in: src/types/openapi.types.ts Type of model to use for this scorer. slm maps to luna, and llm maps to plus

multimodal_capabilities?

Defined in: src/types/openapi.types.ts Multimodal Capabilities Multimodal capabilities which this scorer can utilize in its evaluation.

name?

Defined in: src/types/openapi.types.ts Name

num_judges?

Defined in: src/types/openapi.types.ts Num Judges

output_type?

Defined in: src/types/openapi.types.ts What type of output to use for model-based scorers (boolean, categorical, etc.).

roll_up_method?

Defined in: src/types/openapi.types.ts

score_type?

Defined in: src/types/openapi.types.ts Score Type Return type of code scorers (e.g., ‘bool’, ‘int’, ‘float’, ‘str’).

scoreable_node_types?

Defined in: src/types/openapi.types.ts Scoreable Node Types List of node types that can be scored by this scorer. Defaults to llm/chat.

scorer_type

Defined in: src/types/openapi.types.ts

scorer_version?

Defined in: src/types/openapi.types.ts ScorerVersion to use for this scorer. If not provided, the latest version will be used.