deepdisc.model.meta_arch

Classes

GeneralizedRCNNWCS	Generalized R-CNN. Any models that contains the following three components:
Backbone	Backbone model meant for pre-training.

Module Contents

class GeneralizedRCNNWCS(*, backbone: Backbone, proposal_generator: torch.nn.Module, roi_heads: torch.nn.Module, pixel_mean: Tuple[float], pixel_std: Tuple[float], input_format: str | None = None, vis_period: int = 0)

Bases: torch.nn.Module

Generalized R-CNN. Any models that contains the following three components: 1. Per-image feature extraction (aka backbone) 2. Region proposal generation 3. Per-region feature extraction and prediction

backbone

proposal_generator

roi_heads

input_format = None

vis_period = 0

classmethod from_config(cfg)

property device

_move_to_current_device(x)

visualize_training(batched_inputs, proposals)

A function used to visualize images and proposals. It shows ground truth bounding boxes on the original image and up to 20 top-scoring predicted object proposals on the original image. Users can implement different visualization functions for different models.

Parameters:

• batched_inputs (list) – a list that contains input to the model.

• proposals (list) – a list that contains predicted proposals. Both batched_inputs and proposals should have the same length.

forward(batched_inputs: List[Dict[str, torch.Tensor]])

Parameters:

batched_inputs –

a list, batched outputs of DatasetMapper . Each item in the list contains the inputs for one image. For now, each item in the list is a dict that contains:

• image: Tensor, image in (C, H, W) format.

• instances (optional): groundtruth Instances

• proposals (optional): Instances, precomputed proposals.

Other information that’s included in the original dicts, such as:

• ”height”, “width” (int): the output resolution of the model, used in inference. See postprocess() for details.

Returns:

Each dict is the output for one input image. The dict contains one key “instances” whose value is a Instances. The Instances object has the following keys: “pred_boxes”, “pred_classes”, “scores”, “pred_masks”, “pred_keypoints”

Return type:

list[dict]

inference(batched_inputs: List[Dict[str, torch.Tensor]], detected_instances: List[detectron2.structures.Instances] | None = None, do_postprocess: bool = True)

Run inference on the given inputs.

Parameters:

• batched_inputs (list[dict]) – same as in forward()

• detected_instances (None or list[Instances]) – if not None, it contains an Instances object per image. The Instances object contains “pred_boxes” and “pred_classes” which are known boxes in the image. The inference will then skip the detection of bounding boxes, and only predict other per-ROI outputs.

• do_postprocess (bool) – whether to apply post-processing on the outputs.

Returns:

When do_postprocess=True, same as in forward(). Otherwise, a list[Instances] containing raw network outputs.

preprocess_image(batched_inputs: List[Dict[str, torch.Tensor]])

Normalize, pad and batch the input images.

static _postprocess(instances, batched_inputs: List[Dict[str, torch.Tensor]], image_sizes)

Rescale the output instances to the target size.

class Backbone(*, backbone: torch.nn.Module, feature_level: str = None, projection_head: torch.nn.Module, pixel_mean: Tuple[float], pixel_std: Tuple[float], input_format: str | None = None)

Bases: torch.nn.Module

Backbone model meant for pre-training.

backbone

projection_head

feature_level = None

input_format = None

property device

_move_to_current_device(x)

forward(batched_inputs: List[Tuple[torch.Tensor, torch.Tensor]])

Parameters:

batched_inputs –

a list, batched outputs of DatasetMapper . Each item in the list contains the image and corresponding label. For now, each item in the list is a tuple that contains:

• image: Tensor, image in (C, H, W) format.

• label (optional):`

Returns:

Each item is the predicted label for one input image.

Return type:

list[labels]

inference(batched_inputs: List[Tuple[torch.Tensor, torch.Tensor]])

Run inference on the given inputs.

Parameters:

• batched_inputs (list[dict]) – same as in forward()

• detected_instances (None or list[Instances]) – if not None, it contains an Instances object per image. The Instances object contains “pred_boxes” and “pred_classes” which are known boxes in the image. The inference will then skip the detection of bounding boxes, and only predict other per-ROI outputs.

• do_postprocess (bool) – whether to apply post-processing on the outputs.

Returns:

When do_postprocess=True, same as in forward(). Otherwise, a list[Instances] containing raw network outputs.

preprocess_image(batched_inputs: List[torch.Tensor])

Normalize, pad and batch the input images.

static _postprocess(instances, batched_inputs: List[Dict[str, torch.Tensor]], image_sizes)

Rescale the output instances to the target size.