Documentation Index
Fetch the complete documentation index at: https://docs.galileo.ai/llms.txt
Use this file to discover all available pages before exploring further.
Function: runExperiment()
function runExperiment<T>(
params: RunExperimentParams<T>,
): Promise<RunExperimentOutput>;
Defined in: src/utils/experiments.ts
Runs an experiment by processing each row of a dataset through a specified function.
If metrics are provided, they will be used to evaluate the experiment.
Usage:
// Run an experiment with a runner function
const results = await runExperiment({
name: 'my-experiment',
dataset: [{ country: 'France'}],
function: async (input) => {
const response = await openai.chat.completions.create({
model: 'gpt-4o-mini',
messages: [
{
role: 'user',
content: `What is the capital of ${input['country']}?`
}
]
});
return response.choices[0].message.content;
},
metrics: ['accuracy'],
projectName: 'my-project'
});
// Run an experiment with a prompt template
const promptTemplate = await createPromptTemplate({
template: [{ role: 'user', content: 'What is the capital of {{ country }}?' }],
name: 'my-prompt-template',
projectName: 'my-project'
});
const results = await runExperiment({
name: 'my-experiment',
dataset: [{ country: 'France' }],
promptTemplate
metrics: ['accuracy'],
projectName: 'my-project'
});
The project can be specified by providing exactly one of the project name
(via the ‘project’ parameter or the GALILEO_PROJECT environment variable)
or the project ID (via the ‘project_id’ parameter or the GALILEO_PROJECT_ID environment variable).
Type Parameters
T extends Record<string, unknown>
Parameters
params
RunExperimentParams<T>
Returns
Promise<RunExperimentOutput>
Array of outputs from the processing function