Reducing Hesitation and Uncertainty

Some models struggle to confidently generate responses, leading to hesitation, incomplete answers, or repeated disclaimers. For example, consider this prompt and response:

Prompt: "Tell me about climate change."

Model Response: “Well, there are many aspects to climate change. Some people think it’s caused by humans, and others think it’s just natural. It’s hard to say exactly.”

What went wrong?

The prompt did not provide enough context for confident decision-making
The model allowed too much randomness in token selection
The prompt was ambiguous in the response it expected

How it showed up in metrics

Mid-range Instruction Adherence: The model understood the instructions but lacked decisiveness

Improvements and solutions

For the following improvements, we will be showing how we could change a simple prompt script like the below example:

import os
from galileo.openai import openai
from dotenv import load_dotenv
load_dotenv()

client = openai.OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))

prompt = "Tell me about climate change."
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "system", "content": prompt}],
)
print(response.choices[0].message.content.strip())

import { OpenAI } from "openai";
import { observe } from "galileo";
import { wrapOpenAI } from "galileo/wrappers";

const openai = wrapOpenAI(new OpenAI({ apiKey: "" }));

const callOpenAI = async (country: string) => {
  const response = await openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: `What is the capital of ${country}?` }],
  });
  return response.choices[0].message.content;
};

const prompt = "Tell me about climate change.";
await callOpenAI(prompt);

Provide Stronger Context in Prompts

Include explicit guiding statements, for example:

  prompt = """Tell me about climate change.
    Assume the following is true:
    1. Climate change is driven by human activity.
    2. Provide an explanation based on scientific evidence."""

const prompt = `Tell me about climate change.
  Assume the following is true:
  1. Climate change is driven by human activity.
  2. Provide an explanation based on scientific evidence.`;

This should reduce the uncertainty and perplexity in your metrics on Galileo.

Adjust Model Sampling Parameters

Lower temperature to make the model more deterministic, for example:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Tell me a joke."}],
    temperature=0.3  # Reduced temperature for more predictable responses
)

const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Tell me a joke." }],
  temperature: 0.3, // Reduced temperature for more predictable responses
});

Use top-k sampling to limit options and prevent hesitation, for example:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Tell me about climate change."}],
    # This is often how top k would be set.
    # In practice, ChatGPT does not allow modifying top-k, but other models do
    top_k=50  # Limits token selection to the top 50 choices
)

const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Tell me about climate change." }],
  // This is often how top k would be set.
  // In practice, ChatGPT does not allow modifying top-k, but other models do
  top_k: 50, // Limits token selection to the top 50 choices.
});

Lowering the temperature and decreasing top_k both generally increase the prompt adherence.

Modify Prompt Structure

Use direct phrasing to force a single, clear response, for example:

python example.py prompt = "Provide a single, definitive explanation of climate change in two sentences."  typescript example.ts const prompt = "Provide a single, definitive explanation of climate change in two sentences.";

Avoid prompts that allow multiple equally valid answers, for example avoiding confusing by adding the source of the opinion we care about:

prompt = "What are the primary causes of climate change according to scientists?"

const prompt = "What are the primary causes of climate change according to scientists?";

Agentic AI

Conversational AI

Retrieval-Augmented Generation

Reducing Hesitation and Uncertainty

What went wrong?

How it showed up in metrics

Improvements and solutions

​What went wrong?

​How it showed up in metrics

​Improvements and solutions

What went wrong?

How it showed up in metrics

Improvements and solutions