DRISEScoring

class xaitk_saliency.impls.gen_detector_prop_sal.drise_scoring.DRISEScoring

This D-RISE implementation transforms black-box object detector predictions into visual saliency heatmaps. Specifically, we make use of perturbed detections generated using the RISEGrid image perturbation class and a similarity metric that captures both the localization and categorization aspects of object detection.

Object detection representations used here would need to encapsulate localization information (i.e. bounding box regions), class scores, and objectness scores (if applicable to the detector, such as YOLOv3). Object detections are converted into (4+1+nClasses) vectors (4 indices for bounding box locations, 1 index for objectness, and nClasses indices for different object classes).

If your detections consist of a single class prediction and confidence score instead of scores for each class, it is best practice to replace the objectness score with the confidence score and use a one-hot encoding of the prediction as the class scores.

Based on Petsiuk et al: https://arxiv.org/abs/2006.03204

Methods

from_config	Instantiate a new instance of this class given the configuration JSON-compliant dictionary encapsulating initialization arguments.
generate	Generate visual saliency heatmaps from black-box object detector predictions
get_config	Get the configuration dictionary of the DRISEScoring instance.
get_default_config	Generate and return a default configuration dictionary for this class.
get_impls	Discover and return a set of classes that implement the calling class.
iou	Compute the intersection over union (IoU) of two sets of boxes.
is_usable	Check whether this class is available for use.

generate(ref_dets: ndarray, perturbed_dets: ndarray, perturbed_masks: ndarray) → ndarray

Generate visual saliency heatmaps from black-box object detector predictions

Parameters:

• ref_dets – np.ndarray Reference detections from the reference image

• perturbed_dets – np.ndarray Pertured detections generated from the reference image

• perturbed_masks – np.ndarray Perturbation masks numpy.ndarray over the reference image.

Returns:

np.ndarray Generated visual saliency heatmap.

get_config() → dict

Get the configuration dictionary of the DRISEScoring instance.

Returns:

dict[str, Any]: Configuration dictionary.

iou(box_a: ndarray, box_b: ndarray) → ndarray

Compute the intersection over union (IoU) of two sets of boxes.

E.g.:

A ∩ B / A ∪ B = A ∩ B / (area(A) + area(B) - A ∩ B)

Parameters:

• box_a – (np.array) bounding boxes, Shape: [A,4]

• box_b – (np.array) bounding boxes, Shape: [B,4]

Returns:

iou(np.array), Shape: [A,B].