Quickstart

​

How to get started with Galileo Evaluate

​

Create a Galileo Account

1. Go to your Galileo console page (a link that looks something like “console.galileo.yourcompany.com”). Speak to your Galileo Admin, or send a Slack on your shared Galileo slack channel if you don’t know the URL.

2. Create an account by asking your Admin to send you an invite or directly via the Console homepage.

​

Install the Galileo Client

• Python
• TypeScript

import promptquality as pq

MY_GALILEO_URL = # e.g. "console.galileo.yourcompany.com"
pq.login(MY_GALILEO_URL)

template = "Explain {topic} to me like I'm a 5 year old"

data = {"topic": ["Quantum Physics", "Politics", "Large Language Models"]}

pq.run(project_name='my_first_project',
      template=template,
      dataset=data,
      settings=pq.Settings(model_alias='ChatGPT (16K context)',
                            temperature=0.8,
                            max_tokens=400))

import os

os.environ['GALILEO_API_KEY']="Your Galileo API key"

MY_GALILEO_URL = # e.g. "console.galileo.yourcompany.com"
pq.login(MY_GALILEO_URL)

template = ...

​

Getting an API Key

To create an API key:

​

Running your first eval

First, create an eval set. Once you have your eval set, you’re ready to start your first evaluation run:

• If you have not written any code yet and are looking to evaluate a model and template for your use case, check out Creating Prompt Runs.

  • If you want to try multiple templates or model combinations in one go, check out Prompt Sweeps

• If you have an application or prototype you’d like to evaluate, check out Integrating Evaluate into my existing application.