Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.galileo.ai/llms.txt

Use this file to discover all available pages before exploring further.

Some models struggle to confidently generate responses, leading to hesitation, incomplete answers, or repeated disclaimers. For example, consider this prompt and response: 
Prompt: "Tell me about climate change."

Model Response: “Well, there are many aspects to climate change. Some people think it’s caused by humans, and others think it’s just natural. It’s hard to say exactly.”

​
What went wrong?

  • The prompt did not provide enough context for confident decision-making
  • The model allowed too much randomness in token selection
  • The prompt was ambiguous in the response it expected

​
How it showed up in metrics

  • High Uncertainty: The model hesitated in its response
  • High Prompt Perplexity: The model struggled with predicting the next token
  • Mid-range Instruction Adherence: The model understood the instructions but lacked decisiveness

​
Improvements and solutions

For the following improvements, we will be showing how we could change a simple prompt script like the below example: 
import os
from galileo.openai import openai
from dotenv import load_dotenv
load_dotenv()

client = openai.OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))

prompt = "Tell me about climate change."
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "system", "content": prompt}],
)
print(response.choices[0].message.content.strip())
1

Provide Stronger Context in Prompts

Include explicit guiding statements, for example:
  prompt = """Tell me about climate change.
    Assume the following is true:
    1. Climate change is driven by human activity.
    2. Provide an explanation based on scientific evidence."""
This should reduce the uncertaincy and perplexity in your metrics on Galileo.
2

Adjust Model Sampling Parameters

Lower temperature to make the model more deterministic, for example:
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Tell me a joke."}],
    temperature=0.3  # Reduced temperature for more predictable responses
)
Use top-k sampling to limit options and prevent hesitation, for example:
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Tell me about climate change."}],
    # This is often how top k would be set.
    # In practice, ChatGPT does not allow modifying top-k, but other models do
    top_k=50  # Limits token selection to the top 50 choices
)
Lowering the temperature and decreasing top_k both generally increase the prompt adherence.
3

Modify Prompt Structure

Use direct phrasing to force a single, clear response, for example:
python example.py prompt = "Provide a single, definitive explanation of climate change in two sentences." typescript example.ts const prompt = "Provide a single, definitive explanation of climate change in two sentences.";
Avoid prompts that allow multiple equally valid answers, for example avoiding confusing by adding the source of the opinion we care about:
prompt = "What are the primary causes of climate change according to scientists?"
4

Apply Uncertainty-Based Filtering

Automatically reject responses with an Uncertainty score above a set threshold