pipert.utils.visualizer

Package Contents

Classes

Visualizer

VideoVisualizer

class pipert.utils.visualizer.Visualizer(img_rgb, metadata, scale=1.0, instance_mode=ColorMode.IMAGE)[source]
draw_instance_predictions(self, predictions)

Draw instance-level prediction results on an image.

Args:
predictions (Instances): the output of an instance detection/segmentation

model. Following fields will be used to draw: “pred_boxes”, “pred_classes”, “scores”, “pred_masks” (or “pred_masks_rle”).

Returns:

output (VisImage): image object with visualizations.

draw_sem_seg(self, sem_seg, area_threshold=None, alpha=0.8)

Draw semantic segmentation predictions/labels.

Args:

sem_seg (Tensor or ndarray): the segmentation of shape (H, W). area_threshold (int): segments with less than area_threshold are not drawn. alpha (float): the larger it is, the more opaque the segmentations are.

Returns:

output (VisImage): image object with visualizations.

draw_panoptic_seg_predictions(self, panoptic_seg, segments_info, area_threshold=None, alpha=0.7)

Draw panoptic prediction results on an image.

Args:
panoptic_seg (Tensor): of shape (height, width) where the values are ids for each

segment.

segments_info (list[dict]): Describe each segment in panoptic_seg.

Each dict contains keys “id”, “category_id”, “isthing”.

area_threshold (int): stuff segments with less than area_threshold are not drawn.

Returns:

output (VisImage): image object with visualizations.

draw_dataset_dict(self, dic)

Draw annotations/segmentaions in Detectron2 Dataset format.

Args:

dic (dict): annotation/segmentation data of one image, in Detectron2 Dataset format.

Returns:

output (VisImage): image object with visualizations.

overlay_instances(self, *, boxes=None, labels=None, masks=None, keypoints=None, assigned_colors=None, alpha=0.5)
Args:
boxes (Boxes, RotatedBoxes or ndarray): either a Boxes,

or an Nx4 numpy array of XYXY_ABS format for the N objects in a single image, or a RotatedBoxes, or an Nx5 numpy array of (x_center, y_center, width, height, angle_degrees) format for the N objects in a single image,

labels (list[str]): the text to be displayed for each instance. masks (masks-like object): Supported types are:

  • structures.masks.PolygonMasks, structures.masks.BitMasks.

  • list[list[ndarray]]: contains the segmentation masks for all objects in one image.

    The first level of the list corresponds to individual instances. The second level to all the polygon that compose the instance, and the third level to the polygon coordinates. The third level should have the format of [x0, y0, x1, y1, …, xn, yn] (n >= 3).

  • list[ndarray]: each ndarray is a binary mask of shape (H, W).

  • list[dict]: each dict is a COCO-style RLE.

keypoints (Keypoint or array like): an array-like object of shape (N, K, 3),

where the N is the number of instances and K is the number of keypoints. The last dimension corresponds to (x, y, visibility or score).

assigned_colors (list[matplotlib.colors]): a list of colors, where each color

corresponds to each mask or box in the image. Refer to ‘matplotlib.colors’ for full list of formats that the colors are accepted in.

Returns:

output (VisImage): image object with visualizations.

overlay_rotated_instances(self, boxes=None, labels=None, assigned_colors=None)
Args:
boxes (ndarray): an Nx5 numpy array of

(x_center, y_center, width, height, angle_degrees) format for the N objects in a single image.

labels (list[str]): the text to be displayed for each instance. assigned_colors (list[matplotlib.colors]): a list of colors, where each color

corresponds to each mask or box in the image. Refer to ‘matplotlib.colors’ for full list of formats that the colors are accepted in.

Returns:

output (VisImage): image object with visualizations.

draw_and_connect_keypoints(self, keypoints)

Draws keypoints of an instance and follows the rules for keypoint connections to draw lines between appropriate keypoints. This follows color heuristics for line color.

Args:
keypoints (Tensor): a tensor of shape (K, 3), where K is the number of keypoints

and the last dimension corresponds to (x, y, probability).

Returns:

output (VisImage): image object with visualizations.

draw_text(self, text, position, *, font_size=None, color='g', horizontal_alignment='center', rotation=0)
Args:

text (str): class label position (tuple): a tuple of the x and y coordinates to place text on image. font_size (int, optional): font of the text. If not provided, a font size

proportional to the image width is calculated and used.

color: color of the text. Refer to matplotlib.colors for full list

of formats that are accepted.

horizontal_alignment (str): see matplotlib.text.Text rotation: rotation angle in degrees CCW

Returns:

output (VisImage): image object with text drawn.

draw_box(self, box_coord, alpha=0.5, edge_color='g', line_style='-')
Args:
box_coord (tuple): a tuple containing x0, y0, x1, y1 coordinates, where x0 and y0

are the coordinates of the image’s top left corner. x1 and y1 are the coordinates of the image’s bottom right corner.

alpha (float): blending efficient. Smaller values lead to more transparent masks. edge_color: color of the outline of the box. Refer to matplotlib.colors

for full list of formats that are accepted.

line_style (string): the string to use to create the outline of the boxes.

Returns:

output (VisImage): image object with box drawn.

draw_rotated_box_with_label(self, rotated_box, alpha=0.5, edge_color='g', line_style='-', label=None)
Args:
rotated_box (tuple): a tuple containing (cnt_x, cnt_y, w, h, angle),

where cnt_x and cnt_y are the center coordinates of the box. w and h are the width and height of the box. angle represents how many degrees the box is rotated CCW with regard to the 0-degree box.

alpha (float): blending efficient. Smaller values lead to more transparent masks. edge_color: color of the outline of the box. Refer to matplotlib.colors

for full list of formats that are accepted.

line_style (string): the string to use to create the outline of the boxes. label (string): label for rotated box. It will not be rendered when set to None.

Returns:

output (VisImage): image object with box drawn.

draw_circle(self, circle_coord, color, radius=3)
Args:
circle_coord (list(int) or tuple(int)): contains the x and y coordinates

of the center of the circle.

color: color of the polygon. Refer to matplotlib.colors for a full list of

formats that are accepted.

radius (int): radius of the circle.

Returns:

output (VisImage): image object with box drawn.

draw_line(self, x_data, y_data, color, linestyle='-', linewidth=None)
Args:
x_data (list[int]): a list containing x values of all the points being drawn.

Length of list should match the length of y_data.

y_data (list[int]): a list containing y values of all the points being drawn.

Length of list should match the length of x_data.

color: color of the line. Refer to matplotlib.colors for a full list of

formats that are accepted.

linestyle: style of the line. Refer to matplotlib.lines.Line2D

for a full list of formats that are accepted.

linewidth (float or None): width of the line. When it’s None,

a default value will be computed and used.

Returns:

output (VisImage): image object with line drawn.

draw_binary_mask(self, binary_mask, color=None, *, edge_color=None, text=None, alpha=0.5, area_threshold=4096)
Args:
binary_mask (ndarray): numpy array of shape (H, W), where H is the image height and

W is the image width. Each value in the array is either a 0 or 1 value of uint8 type.

color: color of the mask. Refer to matplotlib.colors for a full list of

formats that are accepted. If None, will pick a random color.

edge_color: color of the polygon edges. Refer to matplotlib.colors for a

full list of formats that are accepted.

text (str): if None, will be drawn in the object’s center of mass. alpha (float): blending efficient. Smaller values lead to more transparent masks. area_threshold (float): a connected component small than this will not be shown.

Returns:

output (VisImage): image object with mask drawn.

draw_polygon(self, segment, color, edge_color=None, alpha=0.5)
Args:

segment: numpy array of shape Nx2, containing all the points in the polygon. color: color of the polygon. Refer to matplotlib.colors for a full list of

formats that are accepted.

edge_color: color of the polygon edges. Refer to matplotlib.colors for a

full list of formats that are accepted. If not provided, a darker shade of the polygon color will be used instead.

alpha (float): blending efficient. Smaller values lead to more transparent masks.

Returns:

output (VisImage): image object with polygon drawn.

_jitter(self, color)

Randomly modifies given color to produce a slightly different color than the color given.

Args:
color (tuple[double]): a tuple of 3 elements, containing the RGB values of the color

picked. The values in the list are in the [0.0, 1.0] range.

Returns:
jittered_color (tuple[double]): a tuple of 3 elements, containing the RGB values of the

color after being jittered. The values in the list are in the [0.0, 1.0] range.

_create_grayscale_image(self, mask=None)

Create a grayscale version of the original image. The colors in masked area, if given, will be kept.

_change_color_brightness(self, color, brightness_factor)

Depending on the brightness_factor, gives a lighter or darker color i.e. a color with less or more saturation than the original color.

Args:
color: color of the polygon. Refer to matplotlib.colors for a full list of

formats that are accepted.

brightness_factor (float): a value in [-1.0, 1.0] range. A lightness factor of

0 will correspond to no change, a factor in [-1.0, 0) range will result in a darker color and a factor in (0, 1.0] range will result in a lighter color.

Returns:
modified_color (tuple[double]): a tuple containing the RGB values of the

modified color. Each value in the tuple is in the [0.0, 1.0] range.

_convert_boxes(self, boxes)

Convert different format of boxes to an NxB array, where B = 4 or 5 is the box dimension.

_convert_masks(self, masks_or_polygons)

Convert different format of masks or polygons to a tuple of masks and polygons.

Returns:

list[GenericMask]:

_convert_keypoints(self, keypoints)
get_output(self)

Returns: output (VisImage): the image output containing the visualizations added to the image.

class pipert.utils.visualizer.VideoVisualizer(metadata, instance_mode=ColorMode.IMAGE)[source]
draw_instance_predictions(self, frame, predictions, names)

Draw instance-level prediction results on an image.

Args:

frame (ndarray): an RGB image of shape (H, W, C), in the range [0, 255]. predictions (Instances): the output of an instance detection/segmentation

model. Following fields will be used to draw: “pred_boxes”, “pred_classes”, “scores”, “pred_masks” (or “pred_masks_rle”).

Returns:

output (VisImage): image object with visualizations.

draw_sem_seg(self, frame, sem_seg, area_threshold=None)

Args: sem_seg (ndarray or Tensor): semantic segmentation of shape (H, W),

each value is the integer label.

area_threshold (Optional[int]): only draw segmentations larger than the threshold

_assign_colors(self, instances)

Naive tracking heuristics to assign same color to the same instance, will update the internal state of tracked instances.

Returns:

list[tuple[float]]: list of colors.