deepdisc.astrodet.visualizer

Classes

ColorMode	Enum of different color modes to use for instance visualizations.
VisImage
Visualizer	Visualizer that draws data about detection/segmentation on images.

Module Contents

class ColorMode

Bases: enum.Enum

Enum of different color modes to use for instance visualizations.

IMAGE = 0

Picks a random color for every instance and overlay segmentations with low opacity.

SEGMENTATION = 1

Let instances of the same category have similar colors (from metadata.thing_colors), and overlay them with high opacity. This provides more attention on the quality of segmentation.

IMAGE_BW = 2

Same as IMAGE, but convert all areas without masks to gray-scale. Only available for drawing per-instance mask predictions.

class VisImage(img, scale=1.0)

img

scale = 1.0

_setup_figure(img)

:param Same as in __init__().:

Returns:

top level container for all the image plot elements. ax (matplotlib.pyplot.Axes): contains figure elements and sets the coordinate system.

Return type:

fig (matplotlib.pyplot.figure)

reset_image(img)

Parameters:

img – same as in __init__

save(filepath)

Parameters:

filepath (str) – a string that contains the absolute path, including the file name, where the visualized image will be saved.

get_image()

Returns:

the visualized image of shape (H, W, 3) (RGB) in uint8 type. The shape is scaled w.r.t the input image using the given scale argument.

Return type:

ndarray

class Visualizer(img_rgb, metadata=None, scale=1.0, instance_mode=ColorMode.IMAGE, enable_color_jitter=True, keypoint_threshold=0.05)

Visualizer that draws data about detection/segmentation on images.

It contains methods like draw_{text,box,circle,line,binary_mask,polygon} that draw primitive objects to images, as well as high-level wrappers like draw_{instance_predictions,sem_seg,panoptic_seg_predictions,dataset_dict} that draw composite data in some pre-defined style.

Note that the exact visualization style for the high-level wrappers are subject to change. Style such as color, opacity, label contents, visibility of labels, or even the visibility of objects themselves (e.g. when the object is too small) may change according to different heuristics, as long as the results still look visually reasonable.

To obtain a consistent style, you can implement custom drawing functions with the abovementioned primitive methods instead. If you need more customized visualization styles, you can process the data yourself following their format documented in tutorials (/tutorials/models, /tutorials/datasets). This class does not intend to satisfy everyone’s preference on drawing styles.

This visualizer focuses on high rendering quality rather than performance. It is not designed to be used for real-time applications.

img

metadata = None

output

cpu_device

_default_font_size

_instance_mode

keypoint_threshold = 0.05

enable_color_jitter = True

draw_instance_predictions(predictions, alpha=0.5, lf=True, ls='-', boxf=False, draw_masks=True, custom_labels=None)

Draw instance-level prediction results on an image.

Parameters:

predictions (Instances) – the output of an instance detection/segmentation model. Following fields will be used to draw: “pred_boxes”, “pred_classes”, “scores”, “pred_masks” (or “pred_masks_rle”).

Returns:

image object with visualizations.

Return type:

output (VisImage)

draw_sem_seg(sem_seg, area_threshold=None, alpha=0.8)

Draw semantic segmentation predictions/labels.

Parameters:

• sem_seg (Tensor or ndarray) – the segmentation of shape (H, W). Each value is the integer label of the pixel.

• area_threshold (int) – segments with less than area_threshold are not drawn.

• alpha (float) – the larger it is, the more opaque the segmentations are.

Returns:

image object with visualizations.

Return type:

output (VisImage)

draw_panoptic_seg(panoptic_seg, segments_info, area_threshold=None, alpha=0.7)

Draw panoptic prediction annotations or results.

Parameters:

• panoptic_seg (Tensor) – of shape (height, width) where the values are ids for each segment.

• segments_info (list[dict] or None) – Describe each segment in panoptic_seg. If it is a list[dict], each dict contains keys “id”, “category_id”. If None, category id of each pixel is computed by pixel // metadata.label_divisor.

• area_threshold (int) – stuff segments with less than area_threshold are not drawn.

Returns:

image object with visualizations.

Return type:

output (VisImage)

draw_panoptic_seg_predictions

draw_dataset_dict(dic, lf=True, boxf=True, alpha=0.5, ls='-', draw_masks=True)

Draw annotations/segmentaions in Detectron2 Dataset format.

Parameters:

dic (dict) – annotation/segmentation data of one image, in Detectron2 Dataset format.

Returns:

image object with visualizations.

Return type:

output (VisImage)

overlay_instances(*, boxes=None, labels=None, masks=None, keypoints=None, assigned_colors=None, alpha=0.5, ls='-')

Parameters:

• boxes (Boxes, RotatedBoxes or ndarray) – either a Boxes, or an Nx4 numpy array of XYXY_ABS format for the N objects in a single image, or a RotatedBoxes, or an Nx5 numpy array of (x_center, y_center, width, height, angle_degrees) format for the N objects in a single image,

• labels (list[str]) – the text to be displayed for each instance.

• masks (masks-like object) –

  Supported types are:

  • detectron2.structures.PolygonMasks, detectron2.structures.BitMasks.

  • list[list[ndarray]]: contains the segmentation masks for all objects in one image. The first level of the list corresponds to individual instances. The second level to all the polygon that compose the instance, and the third level to the polygon coordinates. The third level should have the format of [x0, y0, x1, y1, …, xn, yn] (n >= 3).

  • list[ndarray]: each ndarray is a binary mask of shape (H, W).

  • list[dict]: each dict is a COCO-style RLE.

• keypoints (Keypoint or array like) – an array-like object of shape (N, K, 3), where the N is the number of instances and K is the number of keypoints. The last dimension corresponds to (x, y, visibility or score).

• assigned_colors (list[matplotlib.colors]) – a list of colors, where each color corresponds to each mask or box in the image. Refer to ‘matplotlib.colors’ for full list of formats that the colors are accepted in.

Returns:

image object with visualizations.

Return type:

output (VisImage)

overlay_rotated_instances(boxes=None, labels=None, assigned_colors=None)

Parameters:

• boxes (ndarray) – an Nx5 numpy array of (x_center, y_center, width, height, angle_degrees) format for the N objects in a single image.

• labels (list[str]) – the text to be displayed for each instance.

• assigned_colors (list[matplotlib.colors]) – a list of colors, where each color corresponds to each mask or box in the image. Refer to ‘matplotlib.colors’ for full list of formats that the colors are accepted in.

Returns:

image object with visualizations.

Return type:

output (VisImage)

draw_and_connect_keypoints(keypoints)

Draws keypoints of an instance and follows the rules for keypoint connections to draw lines between appropriate keypoints. This follows color heuristics for line color.

Parameters:

keypoints (Tensor) – a tensor of shape (K, 3), where K is the number of keypoints and the last dimension corresponds to (x, y, probability).

Returns:

image object with visualizations.

Return type:

output (VisImage)

draw_text(text, position, *, font_size=None, color='g', horizontal_alignment='center', rotation=0)

Parameters:

• text (str) – class label

• position (tuple) – a tuple of the x and y coordinates to place text on image.

• font_size (int, optional) – font of the text. If not provided, a font size proportional to the image width is calculated and used.

• color – color of the text. Refer to matplotlib.colors for full list of formats that are accepted.

• horizontal_alignment (str) – see matplotlib.text.Text

• rotation – rotation angle in degrees CCW

Returns:

image object with text drawn.

Return type:

output (VisImage)

draw_box(box_coord, alpha=0.5, edge_color='g', line_style='-')

Parameters:

• box_coord (tuple) – a tuple containing x0, y0, x1, y1 coordinates, where x0 and y0 are the coordinates of the image’s top left corner. x1 and y1 are the coordinates of the image’s bottom right corner.

• alpha (float) – blending efficient. Smaller values lead to more transparent masks.

• edge_color – color of the outline of the box. Refer to matplotlib.colors for full list of formats that are accepted.

• line_style (string) – the string to use to create the outline of the boxes.

Returns:

image object with box drawn.

Return type:

output (VisImage)

draw_rotated_box_with_label(rotated_box, alpha=0.5, edge_color='g', line_style='-', label=None)

Draw a rotated box with label on its top-left corner.

Parameters:

• rotated_box (tuple) – a tuple containing (cnt_x, cnt_y, w, h, angle), where cnt_x and cnt_y are the center coordinates of the box. w and h are the width and height of the box. angle represents how many degrees the box is rotated CCW with regard to the 0-degree box.

• alpha (float) – blending efficient. Smaller values lead to more transparent masks.

• edge_color – color of the outline of the box. Refer to matplotlib.colors for full list of formats that are accepted.

• line_style (string) – the string to use to create the outline of the boxes.

• label (string) – label for rotated box. It will not be rendered when set to None.

Returns:

image object with box drawn.

Return type:

output (VisImage)

draw_circle(circle_coord, color, radius=3)

Parameters:

• circle_coord (list(int) or tuple(int)) – contains the x and y coordinates of the center of the circle.

• color – color of the polygon. Refer to matplotlib.colors for a full list of formats that are accepted.

• radius (int) – radius of the circle.

Returns:

image object with box drawn.

Return type:

output (VisImage)

draw_line(x_data, y_data, color, linestyle='-', linewidth=None)

Parameters:

• x_data (list[int]) – a list containing x values of all the points being drawn. Length of list should match the length of y_data.

• y_data (list[int]) – a list containing y values of all the points being drawn. Length of list should match the length of x_data.

• color – color of the line. Refer to matplotlib.colors for a full list of formats that are accepted.

• linestyle – style of the line. Refer to matplotlib.lines.Line2D for a full list of formats that are accepted.

• linewidth (float or None) – width of the line. When it’s None, a default value will be computed and used.

Returns:

image object with line drawn.

Return type:

output (VisImage)

draw_binary_mask(binary_mask, color=None, *, edge_color=None, text=None, alpha=0.5, area_threshold=10)

Parameters:

• binary_mask (ndarray) – numpy array of shape (H, W), where H is the image height and W is the image width. Each value in the array is either a 0 or 1 value of uint8 type.

• color – color of the mask. Refer to matplotlib.colors for a full list of formats that are accepted. If None, will pick a random color.

• edge_color – color of the polygon edges. Refer to matplotlib.colors for a full list of formats that are accepted.

• text (str) – if None, will be drawn on the object

• alpha (float) – blending efficient. Smaller values lead to more transparent masks.

• area_threshold (float) – a connected component smaller than this area will not be shown.

Returns:

image object with mask drawn.

Return type:

output (VisImage)

draw_soft_mask(soft_mask, color=None, *, text=None, alpha=0.5)

Parameters:

• soft_mask (ndarray) – float array of shape (H, W), each value in [0, 1].

• color – color of the mask. Refer to matplotlib.colors for a full list of formats that are accepted. If None, will pick a random color.

• text (str) – if None, will be drawn on the object

• alpha (float) – blending efficient. Smaller values lead to more transparent masks.

Returns:

image object with mask drawn.

Return type:

output (VisImage)

draw_polygon(segment, color, edge_color=None, alpha=0.5)

Parameters:

• segment – numpy array of shape Nx2, containing all the points in the polygon.

• color – color of the polygon. Refer to matplotlib.colors for a full list of formats that are accepted.

• edge_color – color of the polygon edges. Refer to matplotlib.colors for a full list of formats that are accepted. If not provided, a darker shade of the polygon color will be used instead.

• alpha (float) – blending efficient. Smaller values lead to more transparent masks.

Returns:

image object with polygon drawn.

Return type:

output (VisImage)

_jitter(color)

Randomly modifies given color to produce a slightly different color than the color given.

Parameters:

color (tuple[double]) – a tuple of 3 elements, containing the RGB values of the color picked. The values in the list are in the [0.0, 1.0] range.

Returns:

a tuple of 3 elements, containing the RGB values of the

color after being jittered. The values in the list are in the [0.0, 1.0] range.

Return type:

jittered_color (tuple[double])

_create_grayscale_image(mask=None)

Create a grayscale version of the original image. The colors in masked area, if given, will be kept.

_change_color_brightness(color, brightness_factor)

Depending on the brightness_factor, gives a lighter or darker color i.e. a color with less or more saturation than the original color.

Parameters:

• color – color of the polygon. Refer to matplotlib.colors for a full list of formats that are accepted.

• brightness_factor (float) – a value in [-1.0, 1.0] range. A lightness factor of 0 will correspond to no change, a factor in [-1.0, 0) range will result in a darker color and a factor in (0, 1.0] range will result in a lighter color.

Returns:

a tuple containing the RGB values of the

modified color. Each value in the tuple is in the [0.0, 1.0] range.

Return type:

modified_color (tuple[double])

_convert_boxes(boxes)

Convert different format of boxes to an NxB array, where B = 4 or 5 is the box dimension.

_convert_masks(masks_or_polygons)

Convert different format of masks or polygons to a tuple of masks and polygons.

Return type:

list[GenericMask]

_draw_text_in_mask(binary_mask, text, color)

Find proper places to draw text given a binary mask.

_convert_keypoints(keypoints)

get_output()

Returns:

the image output containing the visualizations added to the image.

Return type:

output (VisImage)