Training

For training, LitPanopticDetr implements a Pytorch Lightning Module that uses as default the detr_r50 module coupled with detr_panoptic. For this reason, alonet.detr.criterion and alonet.detr.matcher are used in the training. However, the alonet.detr_panoptic.callbacks are adapted to the predictions of the masks in the inference process.

Training

Pytorch Lightning Module to train models based on detr_panoptic module

class alonet.detr_panoptic.train.LitPanopticDetr(args=None, model=None, **kwargs)

Bases: alonet.detr.train.LitDetr

Parameters

argsNamespace, optional

Attributes stored in specific Namespace, by default None

weightsstr, optional

Weights name to load, by default None

gradient_clip_valfloat, optional

pytorch_lightning.trainer.trainer parameter. 0 means don’t clip, by default 0.1

accumulate_grad_batchesint, optional

Accumulates grads every k batches or as set up in the dict, by default 4

model_namestr, optional

Name use to define the model, by default “detr-r50-panoptic”

modeltorch.nn, optional

Custom model to train

Notes

Arguments entered by the user (kwargs) will replace those stored in args attribute

static add_argparse_args(parent_parser, parser=None)

Add arguments to parent parser with default values

Parameters

parent_parserArgumentParser

Object to append new arguments

parserArgumentParser.argument_group, optional

Argument group to append the parameters, by default None

Returns

ArgumentParser

Object with new arguments concatenated

build_criterion(matcher=None, loss_dice_weight=2, loss_focal_weight=2, loss_ce_weight=1, loss_boxes_weight=5, loss_giou_weight=2, eos_coef=0.1, losses=['masks', 'boxes', 'labels'], aux_loss_stage=6)

Build the default criterion

Parameters

matchertorch.nn, optional

One specfic matcher to use in criterion process, by default the output of build_matcher()

loss_ce_weightfloat, optional

Weight of cross entropy loss in total loss, by default 1

loss_boxes_weightfloat, optional

Weight of boxes loss in total loss, by default 5

loss_giou_weightfloat, optional

Weight of GIoU loss in total loss, by default 2

loss_dice_weightfloat, optional

Weight of DICE/F-1 loss in total loss, by default 2

loss_focal_weightfloat, optional

Weight of sigmoid focal loss in total loss, by default 2

eos_coeffloat, optional

Background/End of the Sequence (EOS) coefficient, by default 0.1

losseslist, optional

List of losses to take into account in total loss, by default [“labels”, “boxes”, “masks”]. Possible values: [“labels”, “boxes”, “masks”] (use the latest in segmentation tasks)

aux_loss_stageint, optional

Size of stages from aux_outputs key in forward ouputs, by default 6

Returns

DetrCriterion

Criterion use to train the model

build_model(num_classes=250, aux_loss=True, weights=None)

Build the default model

Parameters

num_classesint, optional

Number of classes in embed layer, by default 250

aux_lossbool, optional

Return auxiliar outputs in forward output, by default True

weightsstr, optional

Path or id to load weights, by default None

Returns

detr_panoptic

Pytorch model

Raises

Exception

Only detr-r50-panoptic and deformable-detr-r50-panoptic models are supported yet.

callbacks(data_loader)

Given a data loader, this method will return the default callbacks of the training loop.

Parameters

data_loadertorch.utils.data.DataLoader

Dataloader to get a sample to use on object_detector_callback

Returns

List[Callbacks]

Callbacks use in train process

precision: int

run_train(data_loader, args, project='panoptic-detr', expe_name=None, callbacks=None)

Train the model using pytorch lightning

Parameters

data_loadertorch.utils.data.DataLoader

Dataloader use in callbacks() function

projectstr, optional

Project name using to save checkpoints, by default “detr”

expe_namestr, optional

Specific experiment name to save checkpoints, by default “detr_50”

callbackslist, optional

List of callbacks to use, by default callbacks() output

argsNamespace, optional

Additional arguments use in training process, by default None

training: bool

training_step(frames, batch_idx)

Train the model for one step

Parameters

framesUnion[list, Frames]

List of frame without batch dimension or a Frame with the batch dimension

batch_idxint

Batch id given by Lightning

Returns

dict

Dictionary with the loss to optimize, m_outputs forward outputs and metrics to log.

use_amp: bool

validation_step(frames, batch_idx)

Run one step of validation

Parameters

framesUnion[list, Frames]

List of frame without batch dimension or a Frame with the batch dimension

batch_idxint

Batch id given by Lightning

Returns

dict

Dictionary with the loss to optimize, m_outputs forward outputs and metrics to log.

Callbacks

Detr callbacks adapted to use in training.

class alonet.detr_panoptic.callbacks.PanopticApMetricsCallbacks(*args, **kwargs)

Bases: alonet.callbacks.map_metrics_callback.ApMetricsCallback

Callback that stores samples in each step to calculate the AP for one IoU and one class

add_sample(base_metric, pred_boxes, gt_boxes, pred_masks=None, gt_masks=None)

Add a sample to some Metrics. One might want to inhert this method to edit the pred_boxes and gt_boxes boxes before to add them.

Parameters

base_metricMetrics

Metric intance.

pred_boxesBoundingBoxes2D

Predicted boxes2D.

gt_boxesBoundingBoxes2D

GT boxes2d.

pred_masksMask

Predicted Masks for segmentation task

gt_masksMask

GT masks in segmentation task.

class alonet.detr_panoptic.callbacks.PanopticObjectDetectorCallback(val_frames)

Bases: alonet.callbacks.object_detector_callback.ObjectDetectorCallback

The callback load frames every x training step as well as once every validation step on the given val_frames and log the different objects predicted

Parameters

val_framesUnion[list, Frames]

List of sample from the validation set to use to load the validation progress

on_train_batch_end(trainer, pl_module, outputs, batch, batch_idx, dataloader_idx)

Called when the train batch ends.

on_validation_epoch_end(trainer, pl_module)

Called when the val epoch ends.