​
How to get started with Galileo Evaluate

​
Create a Galileo Account

  1. Go to your Galileo console page (a link that looks something like “console.galileo.yourcompany.com”). Speak to your Galileo Admin, or send a Slack on your shared Galileo slack channel if you don’t know the URL.
  2. Create an account by asking your Admin to send you an invite or directly via the Console homepage.

​
Install the Galileo Client

  1. Open a Python notebook or any python environment where you want to install Galileo
  2. Install the python client via pip install promptquality
  3. Next, run the following code to create your first run. Replace YOUR_GALILEO_URL with your Galileo console page URL (looks something like “console.galileo.yourcompany.com”). 
import promptquality as pq

MY_GALILEO_URL = # e.g. "console.galileo.yourcompany.com"
pq.login(MY_GALILEO_URL)

template = "Explain {{topic}} to me like I'm a 5 year old"

data = {"topic": ["Quantum Physics", "Politics", "Large Language Models"]}

pq.run(project_name='my_first_project',
      template=template,
      dataset=data,
      settings=pq.Settings(model_alias='ChatGPT (16K context)',
                            temperature=0.8,
                            max_tokens=400))

​
Authentication

By default, pq.login() will take the user to the Galileo console to copy and paste a short term token.Alternatively, it is possible to programmatically authenticate a user by setting the GALILEO_API_KEY environment variable.
import os

os.environ['GALILEO_API_KEY']="Your Galileo API key"

MY_GALILEO_URL = # e.g. "console.galileo.yourcompany.com"
pq.login(MY_GALILEO_URL)

template = ...

​
Getting an API Key

To create an API key:
1

Go to your Galileo Console settings and select API Keys

2

Select Create a new key

3

Give your key a distinct name and hit Create

​
Running your first eval

First, create an eval set. Once you have your eval set, you’re ready to start your first evaluation run: