> ## Documentation Index
> Fetch the complete documentation index at: https://docs.galileo.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Datasets
> Learn how to create and manage datasets for use in your experiments with our SDKs
{/* */}
Datasets allow you to store and reuse well-defined data for use in experiments. Datasets can be stored and versioned in Galileo, and available for experiments running both in the console as well as in code.
Dataset fields can be sent to a function that is being tested by your application, or used as input variables to [prompts](/sdk-api/experiments/prompts).
## Work with datasets
Datasets can be used in two ways:
1. [**Using the Galileo Console**](#create-and-manage-datasets-in-the-galileo-console)
* Create and manage datasets directly through the Galileo Console
* Visually organize and track test cases
* No coding required
2. [**Using the Galileo SDK**](#create-and-manage-datasets-in-code)
* Programmatically create and manage datasets using Python
* Integrate dataset management into your existing workflows
* Automate dataset operations
Choose the approach that best fits your workflow and team's needs. Many users combine both approaches, using code for bulk operations and the console for visualization and quick edits.
## Dataset fields
Each record in a Galileo dataset can have 4 top-level, standard fields:
1. **`input`** - Input data that can be passed to your LLM application or prompt to set up a test case.
2. **`generated_output`** - Output generated by your LLM application or prompt when a test case is run.
3. **`ground_truth`** - Expected output for manual reference, or authoritative data for the [Ground Truth Adherence](/concepts/metrics/response-quality/ground-truth-adherence) metric.
4. **`metadata`** - Additional attributes for grouping, filtering, or categorizing test cases.
Notes:
* In the [Galileo console UI](#create-and-manage-datasets-in-the-galileo-console), the standard fields are displayed as Input, Generated Output, Ground Truth, and Metadata.
* In the [Galileo SDK](#create-and-manage-datasets-in-code), custom fields (e.g. `your_custom_variable`) can be represented in JSON format and put into the `input` or other standard fields.
## Create and manage datasets in the Galileo console
### Create a new dataset
The dataset creation button, is your starting point for organizing test cases in Galileo's interface.
From the **Datasets** page of the Galileo console, click the **Create Dataset** button.
You can also create a dataset from a **Playgrounds** page. Click the **Add Dataset** button, then select **+ Create new dataset**.
You can choose to create a dataset by:
* [Uploading a dataset file](#dataset-file-uploads)
* [Auto-generating a synthetic dataset](#synthetic-data-generation)
* [Creating a dataset manually](#manual-dataset-creation)
#### Dataset file uploads
* An uploaded file can be in CSV, JSON/JSONC, or Feather format.
* The file needs to have at least one column that maps to "Input" values.
* Your file columns can have any name.
* Once you have uploaded the file, you can name the dataset.
#### Configure dataset columns
As an option, you can map columns (e.g. in your file uploads) to the dataset's Input, Generated Output, Ground Truth, and Metadata. Simply drag selected columns from "Original Dataset" to "New Dataset" (left to right). Click the **Save Dataset** button when you are done.
You can re-map columns from an existing dataset using the **Configure** menu option.
In addition, you can choose to enable the "Configure dataset columns" checkbox in the **Copy to dataset** button action.
#### Synthetic data generation
You can utilize Large Language Models (LLMs) to generate datasets that you can use to test your AI applications. These test datasets can be used before and after your app is deployed to production.
This feature requires an integration with a supported LLM provider (for example, OpenAI, Azure, Mistral). To configure an integration, visit the LLM provider's platform to obtain an API key, then add the key from the model selection dialog, or from Galileo's [integrations page](https://app.galileo.ai/settings/integrations).
To generate data, provide **Input Examples** for the AI model. At least one example is required, though more examples can help improve the synthetic data.
After data generation is completed, select **Save Dataset** to continue working with the data (including editing, exporting, and sharing the data).
You can also customize the generated data by setting:
* The number of rows that you ask the LLM to generate.
* The LLM model that you're utilizing.
* **Your AI app's use case** (Optional): What task is your AI app doing? For example, chatbot to answer customer service questions.
* **Special instructions** (Optional): Additional guidance to further refine the generated output.
* **The generated data types** (Optional): Customize data types that the generated data should follow.
> Data types can be used for testing specific scenarios. For example, testing your app's resilience to prompt injection scenarios where attackers try to get your app to produce harmful output.
Synthetically generated data can be used in many scenarios -- expanding upon your existing datasets to increase test coverage and help you more quickly improve your AI applications.
#### Manual dataset creation
The console allows you to manually add and edit data rows. Select the **Save dataset** button when you are done.
### Add rows to your dataset
You can manually add new rows to your dataset through the console, allowing you to capture problematic inputs or edge cases as you discover them.
After making changes to your dataset, select the **Save changes** button to create a new version that preserves your modifications while maintaining the history of previous versions.
### View version history
A dataset's **Version History** tab allows you to track changes over time, see when modifications were made, and access previous versions for comparison or regression testing.
## Create and manage datasets in code
### Create datasets
When you create a dataset, it is uploaded to Galileo and available to future experiments. Datasets need to have unique names, and are available to all projects across your organization.
```python Python theme={null}
from galileo.datasets import create_dataset
# Create a dataset with test data
test_data = [
{
"input": "Which continent is Spain in?",
"ground_truth": "Europe",
},
{
"input": "Which continent is Japan in?",
"ground_truth": "Asia",
},
]
dataset = create_dataset(
name="countries",
content=test_data
)
```
```python Python (Beta) theme={null}
from galileo import Dataset
# Create a dataset with test data
test_data = [
{
"input": "Which continent is Spain in?",
"ground_truth": "Europe",
},
{
"input": "Which continent is Japan in?",
"ground_truth": "Asia",
},
]
dataset = Dataset(
name="countries",
content=test_data
)
dataset.create()
```
```typescript TypeScript theme={null}
import { createDataset } from "galileo";
const testData = [
{
input: "Which continent is Spain in?",
output: "Europe"
},
{
input: "Which continent is Japan in?",
output: "Asia"
},
];
const dataset = await createDataset(
testData,
"countries",
);
```
See the [`create_dataset` Python SDK docs](/sdk-api/python/reference/datasets#create-dataset) or [`createDataset` TypeScript SDK docs](/sdk-api/typescript/reference/README/functions/createDataset) for more details.
### Get existing datasets
Once a dataset has been created in Galileo, you can retrieve it to use in your experiments by name or ID.
```python Python theme={null}
from galileo.datasets import get_dataset
# Get a dataset by name
dataset = get_dataset(
name="countries"
)
# Get a dataset by ID
dataset = get_dataset(
id="dataset-id"
)
# Get its content
dataset.get_content()
```
```python Python (Beta) theme={null}
from galileo import Dataset
# Get a dataset by name
dataset = Dataset.get(name="countries")
if dataset is None:
raise ValueError("Dataset 'countries' not found")
# Get a dataset by ID
dataset = Dataset.get(id="dataset-id")
if dataset is None:
raise ValueError("Dataset with given ID not found")
# Get its content
content = dataset.get_content()
```
```typescript TypeScript theme={null}
import { getDataset, getDatasetContent } from "galileo";
// Get a dataset by name
const datasetByName = await getDataset({
name: "countries"
});
// Get a dataset by ID
const datasetById = await getDataset({
id: "dataset-id"
});
// Get its content
const content = await getDatasetContent({ datasetId: datasetByName.id });
```
See the [`get_dataset` Python SDK docs](/sdk-api/python/reference/datasets#get-dataset) or [`getDataset` TypeScript SDK docs](/sdk-api/typescript/reference/README/functions/getDataset) for more details.
### Add rows to existing datasets
Once a dataset has been created, you can manually add rows to it.
```python Python theme={null}
from galileo.datasets import get_dataset
# Get an existing dataset
dataset = get_dataset(
name="countries"
)
# Add new rows to the dataset
dataset.add_rows([
{
"input": "Which continent is Morocco in?",
"ground_truth": "Africa",
},
{
"input": "Which continent is Australia in?",
"ground_truth": "Oceania",
},
])
```
```python Python (Beta) theme={null}
from galileo import Dataset
# Get an existing dataset
dataset = Dataset.get(name="countries")
if dataset is None:
raise ValueError("Dataset 'countries' not found")
# Add new rows to the dataset
dataset.add_rows([
{
"input": "Which continent is Morocco in?",
"ground_truth": "Africa",
},
{
"input": "Which continent is Australia in?",
"ground_truth": "Oceania",
},
])
```
```typescript TypeScript theme={null}
import { addRowsToDataset, getDataset } from "galileo";
// Get an existing dataset
const dataset = await getDataset({
name: "countries"
});
// Add new rows to the dataset
await addRowsToDataset({
datasetId: dataset.id,
rows: newRows
});
```
See the [`add_rows` Python SDK docs](/sdk-api/python/reference/datasets#add-rows) or [`addRowsToDataset` TypeScript SDK docs](/sdk-api/typescript/reference/README/functions/addRowsToDataset) for more details.
### Generate synthetic data to extend a dataset
Galileo can use an LLM integration to generate rows of synthetic data that you can then add to a dataset. This synthetic data is generated using a mixture of prompts, instructions, few-shot examples, and data types.
Once these rows have been generated, they can be added to a new or existing dataset.
```python Python theme={null}
from galileo.datasets import extend_dataset
# Generate synthetic data
dataset = extend_dataset(
prompt_settings={'model_alias': 'GPT-4o'},
prompt="Nutrition and health chatbot",
instructions="Questions that an average-health person would be interested in",
examples=[
"Is cereal for breakfast healthy?",
"How many cups of coffee is unhealthy?",
],
data_types=['General Query'],
count=10,
)
# Print the generated dataset contents
for row in dataset:
print(row.values_dict.additional_properties["input"])
```
```python Python (Beta) theme={null}
from galileo import Dataset
# Get an existing dataset
dataset = Dataset.get(name="countries")
if dataset is None:
raise ValueError("Dataset 'countries' not found")
# Extend with synthetic data.
# Note: unlike the non-beta extend_dataset() which is a pure generator,
# dataset.extend() automatically persists the generated rows to the server
# before returning — no separate save step is needed.
new_rows = dataset.extend(
prompt="Nutrition and health chatbot",
instructions="Questions that an average-health person would be interested in",
examples=[
"Is cereal for breakfast healthy?",
"How many cups of coffee is unhealthy?",
],
count=10,
)
# Print the generated rows (already saved to the dataset)
for row in new_rows:
print(row)
```
```typescript TypeScript theme={null}
import { extendDataset } from "galileo";
// Generate synthetic data
const datasetContents = await extendDataset({
promptSettings: { modelAlias: 'GPT-4.1-mini' },
prompt: "Nutrition and health chatbot",
instructions: "Questions that an average-health person would be interested in",
examples: [
"Is cereal for breakfast healthy?",
"How many cups of coffee is unhealthy?",
],
dataTypes: ['General Query'],
count: 10,
});
// Print the generated dataset contents
datasetContents.forEach(row => {
console.log(row.valuesDict.input);
});
```
See the [`extend_dataset` Python SDK docs](/sdk-api/python/reference/datasets#extend-dataset) or [`extendDataset` TypeScript SDK docs](/sdk-api/typescript/reference/README/functions/extendDataset) for more details.
### List datasets
You can retrieve all the datasets for a project.
```python Python theme={null}
from galileo.datasets import list_datasets
# List all datasets in a project
datasets = list_datasets()
# List datasets with a custom limit
datasets = list_datasets(
limit=50,
)
```
```python Python (Beta) theme={null}
from galileo import Dataset
# List all datasets
datasets = Dataset.list()
# List datasets with a custom limit
datasets = Dataset.list(limit=50)
```
```typescript TypeScript theme={null}
import { getDatasets } from "galileo";
// Get all the datasets in a project
const datasets = await getDatasets();
```
See the [`list_datasets` Python SDK docs](/sdk-api/python/reference/datasets#list-datasets) or [`getDatasets` TypeScript SDK docs](/sdk-api/typescript/reference/README/functions/getDatasets) for more details.
### Delete datasets
If a dataset is no longer needed, you can delete it by name or ID.
```python Python theme={null}
from galileo.datasets import delete_dataset
# Delete a dataset by name
delete_dataset(name="countries")
# Delete a dataset by ID
delete_dataset(id="dataset-id")
```
```python Python (Beta) theme={null}
from galileo import Dataset
# Get and delete a dataset by name
dataset = Dataset.get(name="countries")
if dataset is None:
raise ValueError("Dataset 'countries' not found")
dataset.delete()
```
```typescript TypeScript theme={null}
import { deleteDataset } from "galileo";
// Delete a dataset by name
deleteDataset({name: "countries"})
// Delete a dataset by ID
deleteDataset({id: "dataset-id"});
```
See the [`delete_dataset` Python SDK docs](/sdk-api/python/reference/datasets#delete-dataset) or [`deleteDataset` TypeScript SDK docs](/sdk-api/typescript/reference/README/functions/deleteDataset) for more details.
### Work with dataset versions
Galileo automatically creates new versions of datasets when they are modified. You can access different versions by getting the dataset history.
```python Python theme={null}
from galileo.datasets import get_dataset_version_history
# Get the version history
datasets = get_dataset_version_history(
dataset_name="countries"
)
# List out the rows added with each version
for dataset in datasets.versions:
print(f"""
Version index: {dataset.version_index},
rows added: {dataset.rows_added}
""")
```
```python Python (Beta) theme={null}
from galileo import Dataset
# Get an existing dataset
dataset = Dataset.get(name="countries")
if dataset is None:
raise ValueError("Dataset 'countries' not found")
# Get the version history
version_history = dataset.get_versions()
# List out the rows added with each version
for version in version_history.versions:
print(f"""
Version index: {version.version_index},
rows added: {version.rows_added}
""")
# Get specific version of the dataset
specific_version = dataset.get_version_content(index=1)
```
```typescript TypeScript theme={null}
// Currently not supported in TypeScript
```
See the [`get_dataset_version_history` Python SDK docs](/sdk-api/python/reference/datasets#aget-dataset-version-history) for more details.
### Use datasets in experiments
Datasets are primarily used for running experiments to evaluate the performance of your LLM applications:
```python Python theme={null}
from galileo.datasets import get_dataset
from galileo.experiments import run_experiment
from galileo.prompts import get_prompt
from galileo import GalileoMetrics
# Get an existing dataset
dataset = get_dataset(
name="countries"
)
# Get an existing prompt
prompt = get_prompt(
name="geography-prompt"
)
# Run an experiment with the dataset and prompt
results = run_experiment(
"geography-experiment",
dataset=dataset,
prompt_template=prompt,
metrics=[GalileoMetrics.completeness],
project="my-project",
)
```
```python Python (Beta) theme={null}
from galileo import Dataset, Experiment, Metric, Prompt
# Get an existing dataset
dataset = Dataset.get(name="countries")
if dataset is None:
raise ValueError("Dataset 'countries' not found")
# Get an existing prompt
prompt = Prompt.get(name="geography-prompt")
if prompt is None:
raise ValueError("Prompt 'geography-prompt' not found")
# Create and run the experiment in one step
# (create() triggers the run automatically)
experiment = Experiment(
name="geography-experiment",
dataset=dataset,
prompt=prompt,
metrics=[Metric.metrics.completeness],
project_name="my-project",
)
experiment.create()
```
```typescript TypeScript theme={null}
import {
GalileoMetrics, getDataset,
getPrompt, runExperiment
} from "galileo";
// Get an existing dataset
const dataset = await getDataset({
name: "countries"
});
// Get an existing prompt
const prompt = await getPrompt({
name: "geography-prompt",
});
// Run an experiment with the dataset and prompt
await runExperiment({
name: "geography-experiment",
dataset: dataset,
promptTemplate: prompt,
metrics: [GalileoMetrics.correctness],
projectName: "my-project",
});
```
## Best practices for dataset management
When working with datasets consider these tips:
1. Start small and representative: begin with a handful of diverse examples to validate quickly.
2. Grow incrementally: add cases as you find bugs, edge cases, or new scenarios.
3. Version thoughtfully: create new versions for significant changes and compare results over time.
4. Document changes: record the rationale behind additions and versions in comments or changelogs.
5. Organize by purpose: separate datasets for basics, edge cases, and regressions.
6. Choose the right approach: use the console for quick edits/visualization and the SDK for automation/bulk.
7. Track progress: monitor metrics/dashboards or review results to catch regressions.
8. Keep history: archive old cases and maintain version history—don’t delete.
9. Keep your dataset schema consistent: ensure every row includes all fields referenced by prompts.
10. Use nested access for dictionaries: reference nested fields with dot notation (e.g., `input.metadata.days`).
11. Test your prompt templates: render with sample rows to verify variable substitution.
12. Document your prompt templates: note required fields and assumptions near the template.
## Related resources
Learn about more datasets, the data driving your experiments.
Learn how to use datasets and experiments to improve your application.
Learn how to create and use prompt templates in experiments