> ## Documentation Index
> Fetch the complete documentation index at: https://docs.galileo.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Rules
> Learn about defining rules for runtime protection
Rules are conditions that you never want your application to break. A rule is composed of three parts:
* A metric
* An operator
* A target value
These rules will take the value from an evaluated metric, then use the operator to compare with the target value. If this comparison returns `true`, then the rule has been broken and is triggered. You can then use this in your code to change the response from your application.
## Create rules
To create a rule, create an instance of the `Rule`, setting the metric and operator. Set the target values as needed (for example, for the `empty` and `not_empty` operators, there is no need to set a target value).
The metric uses the [`GalileoMetrics`](/sdk-api/metrics/metrics) enum, and the operator uses the `RuleOperator` enum.
```python Python theme={null}
from galileo import GalileoMetrics
from galileo_core.schemas.protect.rule import Rule, RuleOperator
rule = Rule(
metric=GalileoMetrics.input_toxicity,
operator=RuleOperator.gt,
target_value=0.10
)
```
Valid values for the operator are:
| Value | Type | Description | Notes |
| :----------------------- | :----------------------- | :-------------------- | :----------------------------------------------------------- |
| `RuleOperator.gt` | Numerical | Greater than | |
| `RuleOperator.lt` | Numerical | Less than | |
| `RuleOperator.gte` | Numerical | Greater than or equal | |
| `RuleOperator.lte` | Numerical | Less than or equal | |
| `RuleOperator.eq` | Numerical or Categorical | Equals | |
| `RuleOperator.neq` | Numerical or Categorical | Not equals | |
| `RuleOperator.contains` | Categorical | Contains | Does the list of categories contain the single target values |
| `RuleOperator.all` | Categorical | All | Does the list of categories contain all the target values |
| `RuleOperator.any` | Categorical | Any | Does the list of categories contain any of the target values |
| `RuleOperator.empty` | Categorical | Empty | Is the list of categories empty |
| `RuleOperator.not_empty` | Categorical | Not empty | Is the list of categories not empty |
## Metrics and Operators supported
Rules support the following Luna-2 metrics:
* [Action Advancement](#action-advancement)
* [Action Completion](#action-completion)
* [Completeness](#completeness)
* [Context Adherence](#context-adherence)
* [PII](#pii-personal-identifiable-information)
* [Prompt Injection](#prompt-injection)
* [Sexism](#sexism)
* [Tone](#tone)
* [Tool Errors](#tool-errors)
* [Tool Selection Quality](#tool-selection-quality)
* [Toxicity](#toxicity)
You can also use [custom code-based metrics](#custom-code-based-metrics)
Metrics can have different output values (e.g. numerical, categorical), therefore the available operators and target values differ depending on the metric.
### Action advancement
This rule measures if your agent accomplishes, or is making progress towards a goal.
Read more about [action advancement](/concepts/metrics/agentic/action-advancement).
| Setting | Value |
| :---------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Metric | `GalileoMetrics.action_advancement` |
| Required Payload Fields | Both `input` and `output` must be included |
| Values | 0.0 - 1.0 |
| Operators supported | Greater than (`RuleOperator.gt`)
Less than (`RuleOperator.lt`)
Greater than or equal (`RuleOperator.gte`)
Less than or equal (`RuleOperator.lte`) |
```python Python theme={null}
from galileo import GalileoMetrics
from galileo_core.schemas.protect.rule import Rule, RuleOperator
rule = Rule(
metric=GalileoMetrics.action_advancement,
operator=RuleOperator.lt,
target_value=0.90
)
```
### Action completion
This rule measures if your agent successfully accomplished all of the user's goals.
Read more about [action completion](/concepts/metrics/agentic/action-completion).
| Setting | Value |
| :---------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Metric | `GalileoMetrics.action_completion` |
| Required Payload Fields | Both `input` and `output` must be included |
| Values | 0.0 - 1.0 |
| Operators supported | Greater than (`RuleOperator.gt`)
Less than (`RuleOperator.lt`)
Greater than or equal (`RuleOperator.gte`)
Less than or equal (`RuleOperator.lte`) |
```python Python theme={null}
from galileo import GalileoMetrics
from galileo_core.schemas.protect.rule import Rule, RuleOperator
rule = Rule(
metric=GalileoMetrics.action_completion,
operator=RuleOperator.lt,
target_value=0.90
)
```
### Completeness
This rule measures how thoroughly your model's response covered the relevant information available in the context provided.
Read more about [completeness](/concepts/metrics/rag/generation-quality/completeness).
| Setting | Value |
| :---------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Metric | `GalileoMetrics.completeness` |
| Required Payload Fields | Both `input` and `output` must be included |
| Values | 0.0 - 1.0 |
| Operators supported | Greater than (`RuleOperator.gt`)
Less than (`RuleOperator.lt`)
Greater than or equal (`RuleOperator.gte`)
Less than or equal (`RuleOperator.lte`) |
```python Python theme={null}
from galileo import GalileoMetrics
from galileo_core.schemas.protect.rule import Rule, RuleOperator
rule = Rule(
metric=GalileoMetrics.completeness,
operator=RuleOperator.lt,
target_value=0.90
)
```
### Context Adherence
This rule measures whether your model's response was purely based on the context provided. It can be used to stop hallucinations from reaching your end users.
Read more about [context adherence](/concepts/metrics/rag/generation-quality/context-adherence).
| Setting | Value |
| :---------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Metric | `GalileoMetrics.context_adherence` |
| Required Payload Fields | Both `input` and `output` must be included |
| Values | 0.0 - 1.0 |
| Operators supported | Greater than (`RuleOperator.gt`)
Less than (`RuleOperator.lt`)
Greater than or equal (`RuleOperator.gte`)
Less than or equal (`RuleOperator.lte`) |
Generally, `0.1` is a good threshold below which the response is not adhering to the context. Creating a rule for less than 0.1 will trigger the rule when the response does not adhere to the provided context.
```python Python theme={null}
from galileo import GalileoMetrics
from galileo_core.schemas.protect.rule import Rule, RuleOperator
rule = Rule(
metric=GalileoMetrics.context_adherence,
operator=RuleOperator.lt,
target_value=0.10
)
```
### PII (Personal Identifiable Information)
This rule is used to detect and stop Personal Identifiable Information (PII). When applied on the input, it can be used to stop the user or company PII from being included in API calls to external services. When applied on the output, it can be used to prevent data leakage or PII being shown back to the user.
Read more about [PII categories and their definitions](/concepts/metrics/safety-and-compliance/pii).
| Setting | Value |
| :---------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Metric | `GalileoMetrics.input_pii` (for detecting PII in the input)
`GalileoMetrics.output_pii` (for detecting PII in the output) |
| Required Payload Fields | `input` for input PII, `output` for output PII |
| Values | Depending on the operator, one or more of:
`account_info`
`address`
`credit_card_info`
`date_of_birth`
`email`
`name`
`network_info`
`password`
`phone_number`
`ssn`
`username` |
| Operators supported | Any (`RuleOperator.any`) - A list of categories
All (`RuleOperator.all`) - A list of categories
Contains (`RuleOperator.contains`) - A single category
Equal (`RuleOperator.eq`) - A single category
Not equal (`RuleOperator.neq`) - A single category
Empty (`RuleOperator.empty`)
Not empty (`RuleOperator.not_empty`) |
```python Python theme={null}
from galileo import GalileoMetrics
from galileo_core.schemas.protect.rule import Rule, RuleOperator
rule = Rule(
metric=GalileoMetrics.output_pii,
operator=RuleOperator.any,
target_value=["ssn", "address"]
)
```
### Prompt Injection
This rule is used to detect and stop prompt injections in the input.
Prompt Injection Protect rules now compare a float score instead of categorical attack labels. Existing rules that used operators like `any`, `contains`, or `eq` against label values should be migrated to numeric thresholds such as `gte
0.5`.
Read more about [Prompt Injection](/concepts/metrics/safety-and-compliance/prompt-injection).
| Setting | Value |
| :---------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Metric | `GalileoMetrics.prompt_injection` |
| Required Payload Fields | `input` |
| Values | 0.0 - 1.0. Higher values indicate a higher probability of prompt injection. |
| Operators supported | Greater than (`RuleOperator.gt`)
Less than (`RuleOperator.lt`)
Greater than or equal (`RuleOperator.gte`)
Less than or equal (`RuleOperator.lte`) |
Generally, `0.5` is a good threshold above which the input should be treated as likely prompt injection. Creating a rule for greater than or equal to `0.5` will trigger the rule when prompt injection risk is elevated.
```python Python theme={null}
from galileo import GalileoMetrics
from galileo_core.schemas.protect.rule import Rule, RuleOperator
rule = Rule(
metric=GalileoMetrics.prompt_injection,
operator=RuleOperator.gte,
target_value=0.5
)
```
### Sexism
This rule is used to detect sexist or biased language. When applied on the input, it can be used to detect sexist remarks in user queries. When applied on the output, it can be used to prevent your application from using an making biased or sexist comments in its responses.
Read more about [sexism](/concepts/metrics/safety-and-compliance/sexism).
| Setting | Value |
| :---------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Metric | `GalileoMetrics.input_sexism` (for detecting sexism in the input)
`GalileoMetrics.output_sexism` (for detecting sexism in the output) |
| Required Payload Fields | `input` for input sexism, `output` for output sexism |
| Values | 0.0 - 1.0. Higher values indicate higher sexism. |
| Operators supported | Greater than (`RuleOperator.gt`)
Less than (`RuleOperator.lt`)
Greater than or equal (`RuleOperator.gte`)
Less than or equal (`RuleOperator.lte`) |
```python Python theme={null}
from galileo import GalileoMetrics
from galileo_core.schemas.protect.rule import Rule, RuleOperator
rule = Rule(
metric=GalileoMetrics.input_sexism,
operator=RuleOperator.gt,
target_value=0.95
)
```
### Tone
This rule is used to detect the primary tone from the text. When applied on the input, it can be used to detect negative tones in user queries. When applied on the output, it can be used to prevent your application from using an undesired tone in its responses.
Read more about [tone](/concepts/metrics/expression-and-readability/tone).
| Setting | Value |
| :---------------------- | :-------------------------------------------------------------------------------------------------------------------------------------- |
| Metric | `GalileoMetrics.input_tone` (for detecting tone in the input)
`GalileoMetrics.output_tone` (for detecting tone in the output) |
| Required Payload Fields | `input` for input tone, `output` for output tone |
| Values | One of:
`anger`
`annoyance`
`confusion`
`fear`
`joy`
`love`
`sadness`
`surprise`
`neutral` |
| Operators supported | Equal (`RuleOperator.eq`) - A single category
Not equal (`RuleOperator.neq`) - A single category |
```python Python theme={null}
from galileo import GalileoMetrics
from galileo_core.schemas.protect.rule import Rule, RuleOperator
rule = Rule(
metric=GalileoMetrics.input_tone,
operator=RuleOperator.neq,
target_value="neutral"
)
```
### Tool errors
This rule measures any errors when executing tools.
Read more about [tool errors](/concepts/metrics/agentic/tool-error).
| Setting | Value |
| :---------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Metric | `GalileoMetrics.tool_error_rate` |
| Required Payload Fields | Both `input` and `output` must be included |
| Values | 0.0 - 1.0 |
| Operators supported | Greater than (`RuleOperator.gt`)
Less than (`RuleOperator.lt`)
Greater than or equal (`RuleOperator.gte`)
Less than or equal (`RuleOperator.lte`) |
```python Python theme={null}
from galileo import GalileoMetrics
from galileo_core.schemas.protect.rule import Rule, RuleOperator
rule = Rule(
metric=GalileoMetrics.tool_error_rate,
operator=RuleOperator.lt,
target_value=0.90
)
```
### Tool selection quality
This rule measures whether the agent selected the correct tool, and for each tool passed the correct arguments.
Read more about [tool selection quality](/concepts/metrics/agentic/tool-selection-quality).
| Setting | Value |
| :---------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Metric | `GalileoMetrics.tool_selection_quality` |
| Required Payload Fields | Both `input` and `output` must be included |
| Values | 0.0 - 1.0 |
| Operators supported | Greater than (`RuleOperator.gt`)
Less than (`RuleOperator.lt`)
Greater than or equal (`RuleOperator.gte`)
Less than or equal (`RuleOperator.lte`) |
```python Python theme={null}
from galileo import GalileoMetrics
from galileo_core.schemas.protect.rule import Rule, RuleOperator
rule = Rule(
metric=GalileoMetrics.tool_selection_quality,
operator=RuleOperator.lt,
target_value=0.90
)
```
### Toxicity
This rule is used to detect and stop toxic or foul language in the input (user query) or output (response shown to the user).
Read more about [toxicity](/concepts/metrics/safety-and-compliance/toxicity).
| Setting | Value |
| :---------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Metric | `GalileoMetrics.input_toxicity` (for detecting toxicity in the input)
`GalileoMetrics.output_toxicity` (for detecting toxicity in the output) |
| Required Payload Fields | `input` for input toxicity, `output` for output toxicity |
| Values | 0.0 - 1.0. Higher values indicate higher toxicity. |
| Operators supported | Greater than (`RuleOperator.gt`)
Less than (`RuleOperator.lt`)
Greater than or equal (`RuleOperator.gte`)
Less than or equal (`RuleOperator.lte`) |
```python Python theme={null}
from galileo import GalileoMetrics
from galileo_core.schemas.protect.rule import Rule, RuleOperator
rule = Rule(
metric=GalileoMetrics.input_toxicity,
operator=RuleOperator.gt,
target_value=0.10
)
```
### Custom code-based metrics
You can use [custom code-based metrics](/concepts/metrics/custom-metrics/custom-metrics-ui-code) in your runtime protection rulesets.
```python Python theme={null}
from galileo_core.schemas.protect.rule import Rule, RuleOperator
rule = Rule(
metric="your-metric-name",
operator=RuleOperator.gt,
target_value=0.95
)
```
The operators and target values here should match the type of data that the registered scorer is expected to produce.
## Related resources
Learn the basics of running runtime protection.
Learn about defining rulesets for runtime protection.
Learn about defining stages for runtime protection to be used during different stages in your application workflow.
Learn how to invoke runtime protection in code using the Galileo SDK.