Quickstart Guide | Galileo Evaluate

How to get started with Galileo Evaluate

Create a Galileo Account

Go to your Galileo console page (a link that looks something like “console.galileo.yourcompany.com”). Speak to your Galileo Admin, or send a Slack on your shared Galileo slack channel if you don’t know the URL.
Create an account by asking your Admin to send you an invite or directly via the Console homepage.

Install the Galileo Client

Open a Python notebook or any python environment where you want to install Galileo
Install the python client via pip install promptquality
Next, run the following code to create your first run. Replace YOUR_GALILEO_URL with your Galileo console page URL (looks something like “console.galileo.yourcompany.com”).

import promptquality as pq

MY_GALILEO_URL = # e.g. "console.galileo.yourcompany.com"
pq.login(MY_GALILEO_URL)

template = "Explain {{topic}} to me like I'm a 5 year old"

data = {"topic": ["Quantum Physics", "Politics", "Large Language Models"]}

pq.run(project_name='my_first_project',
      template=template,
      dataset=data,
      settings=pq.Settings(model_alias='ChatGPT (16K context)',
                            temperature=0.8,
                            max_tokens=400))

Authentication

By default, pq.login() will take the user to the Galileo console to copy and paste a short term token.

Alternatively, it is possible to programmatically authenticate a user by setting the GALILEO_API_KEY environment variable.

import os

os.environ['GALILEO_API_KEY']="Your Galileo API key"

MY_GALILEO_URL = # e.g. "console.galileo.yourcompany.com"
pq.login(MY_GALILEO_URL)

template = ...

Open a Python notebook or any python environment where you want to install Galileo
Install the python client via pip install promptquality
Next, run the following code to create your first run. Replace YOUR_GALILEO_URL with your Galileo console page URL (looks something like “console.galileo.yourcompany.com”).

import promptquality as pq

MY_GALILEO_URL = # e.g. "console.galileo.yourcompany.com"
pq.login(MY_GALILEO_URL)

template = "Explain {{topic}} to me like I'm a 5 year old"

data = {"topic": ["Quantum Physics", "Politics", "Large Language Models"]}

pq.run(project_name='my_first_project',
      template=template,
      dataset=data,
      settings=pq.Settings(model_alias='ChatGPT (16K context)',
                            temperature=0.8,
                            max_tokens=400))

Authentication

By default, pq.login() will take the user to the Galileo console to copy and paste a short term token.

Alternatively, it is possible to programmatically authenticate a user by setting the GALILEO_API_KEY environment variable.

import os

os.environ['GALILEO_API_KEY']="Your Galileo API key"

MY_GALILEO_URL = # e.g. "console.galileo.yourcompany.com"
pq.login(MY_GALILEO_URL)

template = ...

Open a TypeScript project where you want to install Galileo
Install the client via npm with npm install @rungalileo/galileo

If you are not using Observe Callback features you can use the --no-optional flag to avoid extraneous dependencies.

Add your console URL (GALILEO_CONSOLE_URL) and API key (GALILEO_API_KEY) to your environment variables in your .env file.

GALILEO_CONSOLE_URL="https://console.galileo.yourcompany.com"
GALILEO_API_KEY="Your API Key"

# Alternatively, you can also use username/password.
GALILEO_USERNAME="Your Username"
GALILEO_PASSWORD="Your Password"

import { GalileoEvaluateWorkflow } from "@rungalileo/galileo";

// Initialize and create project
const evaluateWorkflow = new GalileoEvaluateWorkflow("Evaluate Project"); // Project Name
await evaluateWorkflow.init();

Getting an API Key

To create an API key:

Go to your Galileo Console settings and select API Keys

Select Create a new key

Give your key a distinct name and hit Create

Running your first eval

First, create an eval set. Once you have your eval set, you’re ready to start your first evaluation run:

If you have not written any code yet and are looking to evaluate a model and template for your use case, check out Creating Prompt Runs.
- If you want to try multiple templates or model combinations in one go, check out Prompt Sweeps
If you have an application or prototype you’d like to evaluate, check out Integrating Evaluate into my existing application.

Introduction

Evaluate

Observe

Protect

Galileo Guardrail Metrics

Fine Tune

Galileo NLP Studio

Quickstart Guide | Galileo Evaluate

How to get started with Galileo Evaluate

Create a Galileo Account

Install the Galileo Client

Authentication

Authentication

Getting an API Key

Running your first eval

Introduction

Evaluate

Observe

Protect

Galileo Guardrail Metrics

Fine Tune

Galileo NLP Studio

​How to get started with Galileo Evaluate

​Create a Galileo Account

​Install the Galileo Client

​Authentication

​Authentication

​Getting an API Key

​Running your first eval

How to get started with Galileo Evaluate

Create a Galileo Account

Install the Galileo Client

Authentication

Authentication

Getting an API Key

Running your first eval