This page describes the rich error types offered by Galileo for Object Detection
Update your annotations in a few clicks from the console.
The inner prediction has higher confidence than the larger box, and is thus selected as a TP. The duplicated outer prediction is however a better bounding box than both the TP prediction and the annotation..
Typical classification error where the annotation is mislabeled.
Localization error exhibiting an inaccurate annotation.
The annotation does not represent any object.
There are multiple annotations for the same object.
[0,1]
interval appearing below the image indicates the range (in orange) for the IoU between the predicted box (in red) and an annotated box (in yellow). Note that it contains two thresholds: the background threshold t_b
and the foreground threshold t_f
. Galileo sets the background threshold t_b
at 0.1
and the foreground threshold t_f
at the mAP threshold
used to compute the mAP score. As an example, a predicted box overlapping with an annotation with IoU >= t_f
will be given the classification error type if the class of the annotation doesn’t match that of the prediction.
With the above ambiguous definition, there are cases where a predicted box could be part of multiple error types. To avoid ambiguity, Galileo classifies the errors in the following order:
[t_b, t_f]
[t_f, 1]
[t_f, 1]
< t_b
with all annotations.
[t_b, t_f]
with a box of different label).
Finally, the Missed error type is given to any annotation that is already considered a FN, and that was not used in the above definition by either a Classification Error or a Localization Error. Note that Missed annotations can overlap with predictions, for example, they can overlap < t_b
with a classification and localization error.
The predicted box is a localization error. Without the context of the associated annotation, this would be confusing since the prediction looks correct. With the context, one can see that the annotation is inaccurate and should be updated.