deepdisc.astrodet.astrodet

Classes

`NewAstroTrainer`	Use this for models that use yacs cfg files
`AstroPredictor`	Create a simple end-to-end predictor with the given config that runs on
`COCOeval_opt_custom`
`COCOEvaluatorRecall`	Override this class in order to call the custom function above
`train_mapper_cls`	This class is used to load in and augment data during training.
`test_mapper_cls`	This class is used to load in and augment data during validation.

Functions

`set_mpl_style`()	Function to set MPL style
`_evaluate_predictions_on_coco`(coco_gt, coco_results, ...)
`convert_to_coco_dict`(dataset_name, mbins, mind, logger)	Convert an instance detection/segmentation or keypoint detection dataset
`convert_to_coco_json`(dataset_name, output_file[, ...])	Converts dataset into COCO format and saves it to a json file.
`read_image_hsc`(filenames[, normalize, stretch, Q, m, ...])	Read in a formatted HSC image
`read_image_decam`(filename[, normalize, stretch, Q, m, ...])	Read in a formatted simulated DECam image
`gaussblur`(image)
`addelementwise16`(image)
`addelementwise8`(image)
`addelementwise`(image)
`get_astro_dicts`(img_dir)	This function reads in the scarlet model files and formats annotations for detectron2

Module Contents

set_mpl_style()[source]: Function to set MPL style

class NewAstroTrainer(model, data_loader, optimizer, cfg)[source]

Bases: detectron2.engine.SimpleTrainer

Use this for models that use yacs cfg files

Parameters:

model (torch nn.module) – The model being trained
data_loader (detectron2 DataLoader) – The data loader that loads the training set
optimizer – The learning optimizer
cfg (config file) – The model config

checkpointer[source]

lossList = [][source]

vallossList = [][source]

period = 20[source]

iterCount = 0[source]

scheduler[source]

valloss = 0[source]

set_period(p)[source]

run_step()[source]

classmethod build_lr_scheduler(cfg, optimizer)[source]: It now calls detectron2.solver.build_lr_scheduler(). Overwrite it if you’d like a different scheduler.

add_val_loss(val_loss)[source]: It now calls detectron2.solver.build_lr_scheduler(). Overwrite it if you’d like a different scheduler.

class AstroPredictor(cfg, lazy=False, cfglazy=None, checkpoint=None)[source]

Create a simple end-to-end predictor with the given config that runs on single device for a single input image. Compared to using the model directly, this class does the following additions: 1. Load checkpoint from cfg.MODEL.WEIGHTS. 2. Always take BGR image as the input and apply conversion defined by cfg.INPUT.FORMAT. 3. Apply resizing defined by cfg.INPUT.{MIN,MAX}_SIZE_TEST. 4. Take one input image and produce a single output, instead of a batch. This is meant for simple demo purposes, so it does the above steps automatically. This is not meant for benchmarks or running complicated inference logic. If you’d like to do anything more complicated, please refer to its source code as examples to build and use the model manually. .. attribute:: metadata

the metadata of the underlying dataset, obtained from cfg.DATASETS.TEST.

type:

Metadata

Examples:

pred = DefaultPredictor(cfg)
inputs = cv2.imread("input.jpg")
outputs = pred(inputs)

cfg[source]

aug[source]

input_format[source]

__call__(original_image)[source]

Parameters:: original_image (np.ndarray) – an image of shape (H, W, C) (in BGR order).
Returns:: the output of the model for one image only. See /tutorials/models for details about the format.
Return type:: predictions (dict)

class COCOeval_opt_custom[source]

Bases: detectron2.evaluation.fast_eval_api.COCOeval_opt

evaluate_custom()[source]: Run per image evaluation on given images and store results (a list of dict) in self.evalImgs :return: None

accumulate_custom(p=None)[source]

YL: Override in order to put in some output commands

Accumulate per image evaluation results and store the result in self.eval :param p: input params for evaluation :return: None

summarize_custom()[source]: Compute and display summary metrics for evaluation results. Note this functin can only be applied on the default parameter setting

_evaluate_predictions_on_coco(coco_gt, coco_results, iou_type, kpt_oks_sigmas=None, use_fast_impl=True, img_ids=None, max_dets_per_image=None, areaRng=None)[source]

convert_to_coco_dict(dataset_name, mbins, mind, logger)[source]

Convert an instance detection/segmentation or keypoint detection dataset in detectron2’s standard format into COCO json format.

Generic dataset description can be found here: https://detectron2.readthedocs.io/tutorials/datasets.html#register-a-dataset

COCO data format description can be found here: http://cocodataset.org/#format-data

Parameters:: dataset_name (str) – name of the source dataset Must be registered in DatastCatalog and in detectron2’s standard format. Must have corresponding metadata “thing_classes”
Returns:: serializable dict in COCO json format
Return type:: coco_dict

convert_to_coco_json(dataset_name, output_file, mbins=[0, 1], mind=-1, allow_cached=True)[source]

Converts dataset into COCO format and saves it to a json file. dataset_name must be registered in DatasetCatalog and in detectron2’s standard format.

Parameters:

dataset_name – reference from the config file to the catalogs must be registered in DatasetCatalog and in detectron2’s standard format
output_file – path of json file that will be saved to
allow_cached – if json file is already present then skip conversion

class COCOEvaluatorRecall(dataset_name, tasks=None, distributed=True, output_dir=None, *, max_dets_per_image=None, areaRng=None, use_fast_impl=True, kpt_oks_sigmas=(), allow_cached_coco=True)[source]

Bases: detectron2.evaluation.coco_evaluation.COCOEvaluator

Override this class in order to call the custom function above

Evaluate AR for object proposals, AP for instance detection/segmentation, AP for keypoint detection outputs using COCO’s metrics. See http://cocodataset.org/#detection-eval and http://cocodataset.org/#keypoints-eval to understand its metrics. The metrics range from 0 to 100 (instead of 0 to 1), where a -1 or NaN means the metric cannot be computed (e.g. due to no predictions made).

In addition to COCO, this evaluator is able to support any bounding box detection, instance segmentation, or keypoint detection dataset.

_logger[source]

_distributed[source]

_output_dir[source]

_use_fast_impl[source]

_max_dets_per_image[source]

_areaRng[source]

_cpu_device[source]

_metadata[source]

_do_evaluation[source]

_eval_predictions(predictions, img_ids=None)[source]

_derive_coco_results(coco_eval, iou_type, class_names=None)[source]

Parameters:

coco_eval (None or COCOEval) – None represents no predictions from model.
iou_type (str)
class_names (None or list[str]) – if provided, will use it to predict per-category AP.

Returns:

score}

Return type:

a dict of {metric name

read_image_hsc(filenames, normalize='lupton', stretch=0.5, Q=10, m=0, ceil_percentile=99.995, dtype=np.uint8, A=10000.0, do_norm=False)[source]

Read in a formatted HSC image

Parameters:

filenames (list) – The list of g,r,i band files
normalize (str) – The key word for the normalization scheme
stretch (float, int, float) – Parameters for lupton normalization
Q (float, int, float) – Parameters for lupton normalization
m (float, int, float) – Parameters for lupton normalization
ceil_percentile – If do_norm is true, cuts data off at this percentile
dtype (numpy datatype) – data type of the output array
A (float) – scaling factor for zscoring
do_norm (boolean) – For normalizing top fit dtype range

Return type:

Scaled image

read_image_decam(filename, normalize='lupton', stretch=0.5, Q=10, m=0, ceil_percentile=99.995, dtype=np.uint8, A=10000.0, do_norm=False)[source]

Read in a formatted simulated DECam image

Parameters:

filenames (list) – The list of g,r,i band files
normalize (str) – The key word for the normalization scheme
stretch (float, int, float) – Parameters for lupton normalization
Q (float, int, float) – Parameters for lupton normalization
m (float, int, float) – Parameters for lupton normalization
ceil_percentile – If do_norm is true, cuts data off at this percentile
dtype (numpy datatype) – data type of the output array
A (float) – scaling factor for zscoring
do_norm (boolean) – For normalizing top fit dtype range

Return type:

Scaled image

gaussblur(image)[source]

Parameters:: image (ndarray)
Return type:: augmented image

addelementwise16(image)[source]

Parameters:: image (ndarray)
Return type:: augmented image

addelementwise8(image)[source]

Parameters:: image (ndarray)
Return type:: augmented image

addelementwise(image)[source]

Parameters:: image (ndarray)
Return type:: augmented image

get_astro_dicts(img_dir)[source]

This function reads in the scarlet model files and formats annotations for detectron2

Parameters:: img_dir (str) – Directory where the scarlet outputs are stored
Returns:: dataset_dicts – A list of dictionaries that contain annotations for train, test, val sets
Return type:: list

class train_mapper_cls(**read_image_args)[source]

This class is used to load in and augment data during training. It is initialized and then called during training where it augments images and returns them along with annotations

Parameters:

read_image_args (keyword args)
function (These are the necessary arguments for the read_image_hsc)

ria[source]

__call__(dataset_dict)[source]

class test_mapper_cls(**read_image_args)[source]

This class is used to load in and augment data during validation. It is initialized and then called during validation where it augments images and returns them along with annotations

Parameters:

read_image_args (keyword args)
function (These are the necessary arguments for the read_image_hsc)

ria[source]

__call__(dataset_dict)[source]