Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.galileo.ai/llms.txt

Use this file to discover all available pages before exploring further.

Galileo supports AWS Bedrock integration via Inference Profiles. This enables the mapping of Galileo-supported model identifiers to an AWS Bedrock Inference Profile ARN, providing greater flexibility and alignment with existing Bedrock configurations. This page explains what inference profiles are, how Galileo integrates with them, and how to configure the integration using a simple setup script.

​
What are AWS Bedrock Inference Profiles?

An AWS Bedrock Inference Profile is an AWS resource that represents a way to invoke a foundation model (such as Anthropic Claude, Meta Llama, or Mistral) while tracking usage and cost under a named profile in your AWS account.

​
How Galileo works with Inference Profiles

When you use inference profiles with Galileo:
  • You create an Inference Profile in AWS Bedrock.
  • You create an IAM role in your AWS account that Galileo can assume.
  • You register that role and your Inference Profile ARN with Galileo.
When you run evaluations or prompts in Galileo, Galileo:
  • Invokes Bedrock using your Inference Profile
  • Logs results and metrics back to Galileo
  • Your models, data, and billing remain fully in your AWS account.

​
Prerequisites

Before running the setup script, make sure you have:
  • A Galileo API key. The key is tied to a specific Galileo user, and the integration will be created or updated under that user.
  • An AWS IAM role that Galileo can assume, with:
    • bedrock:InvokeModel permission on the models or inference profiles you intend to use.
    • A trust policy that allows Galileo to call sts:AssumeRole.
  • One or more Inference Profile ARNs already created in AWS Bedrock.

​
Setting up the AWS Bedrock Inference Profile integration

The script below configures the AWS Bedrock integration in Galileo. It does not create AWS resources. 

#!/bin/bash
#
# This is an example script for how to set AWS Bedrock integration in Galileo.
# Customize inference_profiles to your needs. The values below are just examples.
# You can use either a model alias or a model name, and map it to an
# inference profile ARN.
#

if [ -z "$GALILEO_API_KEY" ]; then
  echo "Error: GALILEO_API_KEY environment variable is not set"
  exit 1
fi

if [ -z "$AWS_ROLE_ARN" ]; then
  echo "Error: AWS_ROLE_ARN environment variable is not set"
  exit 1
fi

if [ -z "$GALILEO_API_URL" ]; then
  echo "Error: GALILEO_API_URL environment variable is not set"
  exit 1
fi

curl "${GALILEO_API_URL}/integrations/aws_bedrock" \
  -X PUT \
  -H "Galileo-API-Key: ${GALILEO_API_KEY}" \
  -H "content-type: application/json" \
  --data-raw "$(cat <<EOF
{
  "credential_type": "assumed_role",
  "region": "us-east-1",
  "token": {
    "aws_role_arn": "${AWS_ROLE_ARN}"
  },
  "inference_profiles": {
    "anthropic.claude-3-sonnet-20240229-v1:0":
      "arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/my-sonnet-profile"
  }
}
EOF
)"

​
Verifying the integration was updated

A successful PUT returns a JSON response like this: 
{
  "id": "a0ce9eff-1290-41ae-b78d-a01118a1122c",
  "permissions": [],
  "name": "aws_bedrock",
  "created_at": "2026-03-20T22:36:57.225940Z",
  "updated_at": "2026-03-20T22:36:57.225946Z",
  "created_by": "e19dd8ef-c881-4c17-bc2a-fa549f691a5c",
  "is_selected": false
}
A few fields are worth checking:
  • created_by is the Galileo user that owns this integration. If you have multiple users in your organization, this confirms which user’s integration was just updated.
  • updated_at confirms the change was persisted just now.
If the request returns a 4xx or 5xx error instead, the integration was not updated. Resolve the error and re-run the script before testing inference again.

​
Sharing integrations across users

Each AWS Bedrock integration belongs to the Galileo user who created it. Multiple users in the same organization can each create their own AWS Bedrock integration, and each one stays associated only with the user who set it up. A user can also share their integration with other users or with user groups. Shared integrations appear in the recipient’s Galileo UI labeled as Shared. To start using a shared integration, the recipient must select it in the UI. Once selected, they can use it for their inference runs.

​
Supported models

Galileo supports the following AWS Bedrock model aliases. Use any of these as a key in the inference_profiles map of the setup script.
  • AI21 - Jamba 1.5 Large (Bedrock)
  • AI21 - Jamba 1.5 Mini (Bedrock)
  • Amazon - Nova 2 Lite (Bedrock)
  • Amazon - Nova Lite (Bedrock)
  • Amazon - Nova Micro (Bedrock)
  • Amazon - Nova Premier (Bedrock)
  • Amazon - Nova Pro (Bedrock)
  • Anthropic - Claude 3 Haiku (Bedrock)
  • Anthropic - Claude 3.5 Sonnet (Bedrock)
  • Anthropic - Claude 3.5 Sonnet v2 (Bedrock)
  • Anthropic - Claude 3.7 Sonnet (Bedrock)
  • Anthropic - Claude 4 Opus (Bedrock)
  • Anthropic - Claude 4 Sonnet (Bedrock)
  • Anthropic - Claude Haiku 4.5 (Bedrock)
  • Anthropic - Claude Opus 4.1 (Bedrock)
  • Anthropic - Claude Opus 4.5 (Bedrock)
  • Anthropic - Claude Opus 4.6 (Bedrock)
  • Anthropic - Claude Opus 4.7 (Bedrock)
  • Anthropic - Claude Sonnet 4.5 (Bedrock)
  • Anthropic - Claude Sonnet 4.6 (Bedrock)
  • Cohere - Command R v1 (Bedrock)
  • Cohere - Command R+ v1 (Bedrock)
  • DeepSeek - R1 (Bedrock)
  • Google - Gemma 3 12B (Bedrock)
  • Google - Gemma 3 27B (Bedrock)
  • Google - Gemma 3 4B (Bedrock)
  • Meta - Llama 3 70B Instruct v1 (Bedrock)
  • Meta - Llama 3 8B Instruct v1 (Bedrock)
  • Meta - Llama 3.1 70B Instruct v1 (Bedrock)
  • Meta - Llama 3.1 8B Instruct v1 (Bedrock)
  • Meta - Llama 3.2 11B Instruct (Bedrock)
  • Meta - Llama 3.2 1B Instruct (Bedrock)
  • Meta - Llama 3.2 3B Instruct (Bedrock)
  • Meta - Llama 3.2 90B Instruct (Bedrock)
  • Meta - Llama 3.3 70B Instruct (Bedrock)
  • Meta - Llama 4 Maverick 17B Instruct (Bedrock)
  • Meta - Llama 4 Scout 17B Instruct (Bedrock)
  • MiniMax - M2 (Bedrock)
  • Mistral - 7B Instruct (Bedrock)
  • Mistral - Large (Bedrock)
  • Mistral - Large 3 (Bedrock)
  • Mistral - Magistral Small (Bedrock)
  • Mistral - Ministral 14B (Bedrock)
  • Mistral - Ministral 3B (Bedrock)
  • Mistral - Ministral 8B (Bedrock)
  • Mistral - Pixtral Large 25.02 (Bedrock)
  • Mistral - Small 24.02 (Bedrock)
  • Mixtral - 8x7B Instruct (Bedrock)
  • Moonshot - Kimi K2 Thinking (Bedrock)
  • NVIDIA - Nemotron Nano 12B (Bedrock)
  • NVIDIA - Nemotron Nano 9B (Bedrock)
  • OpenAI - GPT OSS 120B (Bedrock)
  • OpenAI - GPT OSS 20B (Bedrock)
  • Qwen - Qwen3 32B (Bedrock)
  • Qwen - Qwen3 Coder 30B (Bedrock)
  • Qwen - Qwen3 Next 80B (Bedrock)
  • Qwen - Qwen3 VL 235B A22B (Bedrock)
  • Writer - Palmyra X4 (Bedrock)
  • Writer - Palmyra X5 (Bedrock)

​
Troubleshooting

If you’ve updated the integration but inference is still using the old configuration, walk through this checklist:
  • Confirm the PUT request returned a success response (see Verifying the integration was updated), not a 4xx or 5xx error.
  • Check which Galileo API key was used in the script — it corresponds to a specific Galileo user.
  • Check which AWS Role ARN was used, and verify it has bedrock:InvokeModel and sts:AssumeRole permissions.
  • Check the model aliases in the request body. Each one must be a Bedrock model alias supported by Galileo (see Supported models).
  • Check the inference profile ARNs in the request body. Each ARN must point to an inference profile that exists in your AWS account and that the provided IAM role has permission to invoke (bedrock:InvokeModel on the profile, plus permission to invoke its underlying foundation model).
  • Confirm the integration belonging to the API key’s user is the one being used for inference.
  • If the integration is shared with other users, confirm those users have selected the shared integration in the Galileo UI.