- OpenAI:
- Any Evaluate runs created from the Galileo Playground or with
pq.run(...)
, using the chosen model. - Any Evaluate workflow runs using
davinci-001
. - Any Observe worfklows using
davinci-001
.
- Any Evaluate runs created from the Galileo Playground or with
- Azure OpenAI:
- Any Evaluate runs created from the Galileo Playground or with
pq.run(...)
, using the chosen model. - Any Evaluate workflow runs
text-davinci-003
ortext-curie-001
, if they’re available in your Azure deployment. - Any Observe worfklows using
text-davinci-003
ortext-curie-001
, if they’re available in your Azure deployment.
- Any Evaluate runs created from the Galileo Playground or with
To calculate the Uncertainty metric, we require having
text-curie-001
or
text-davinci-003
models available in your Azure environment. This is required
in order to fetch log probabilities. For Galileo’s Guardrail metrics that rely
on GPT calls (Factuality and Groundedness), we require using 0613
or
above versions of gpt-35-turbo
(Azure docs).