promptquality
, the Python client to interact with Galileo’s GenAI Studio: Evaluate.
executor
and aggregator
function. The executor
is a row-level calculator, while the aggregator
consolidates all of the calculated row values.
In this example, I want to publish both the top topic and its score, so my executor
will serialize the JSON into a string. Then, my aggregator
will evaluate that string to parse the numeric label score for aggregation.
When we invoke our run, the executor
and aggregator
will be computed within your Python runtime / notebook / application.
executor
and aggregator
as inputs).
We will also define our prompt template with placeholders {context}
and {question}.
These will be replaced by values with the same keys in our dataset
.