Architecture

Backbone

Backbone modules for Deformable DETR

class alonet.deformable_detr.backbone.Backbone(name, train_backbone, return_interm_layers, dilation)

Bases: alonet.deformable_detr.backbone.BackboneBase

ResNet backbone with frozen BatchNorm.

training: bool

class alonet.deformable_detr.backbone.BackboneBase(backbone, train_backbone, return_interm_layers)

Bases: alonet.detr.backbone.BackboneBase

Base class to define behavior of backbone

training: bool

class alonet.deformable_detr.backbone.Joiner(backbone, position_embedding)

Bases: alonet.detr.backbone.Joiner

A sequential wrapper for backbone and position embedding.

self.forward returns a tuple:

• list of feature maps from backbone

• list of position encoded feature maps

training: bool

Transformer

class alonet.deformable_detr.deformable_transformer.DeformableTransformer(d_model=256, nhead=8, num_encoder_layers=6, num_decoder_layers=6, dim_feedforward=1024, encoder=None, decoder=None, decoder_layer=None, encoder_layer=None, dropout=0.1, activation='relu', return_intermediate_dec=False, num_feature_levels=4, dec_n_points=4, enc_n_points=4, two_stage=False, two_stage_num_proposals=300)

Bases: torch.nn.modules.module.Module

Transformer with Multiscale Deformable Attention

For more details: Deformable DETR https://arxiv.org/abs/2010.04159

Notes

This class can only be used with “cuda” device.

forward(srcs, masks, pos_embeds, query_embed=None, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

gen_encoder_output_proposals(memory, memory_padding_mask, spatial_shapes)

get_proposal_pos_embed(proposals)

get_valid_ratio(mask)

Get the ratio of (active mask shape / mask shape). This ratio will be use to create refenrence points for sampling to ensure those points in active zone.

Parameters

mask: (b, H, W)

Returns

valid_ratio: (b, 2)

training: bool

class alonet.deformable_detr.deformable_transformer.DeformableTransformerDecoder(decoder_layer, num_layers, return_intermediate=False)

Bases: torch.nn.modules.module.Module

decoder_forward(tgt, reference_points, src, src_spatial_shapes, src_level_start_index, src_valid_ratios, query_pos=None, src_padding_mask=None, tgt_key_padding_mask=None, **kwargs)

forward(tgt, reference_points, src, src_spatial_shapes, src_level_start_index, src_valid_ratios, query_pos=None, src_padding_mask=None, tgt_key_padding_mask=None, decoder_outputs=None, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

pre_process_tgt(tgt, query_pos, tgt_key_padding_mask, reference_points, **kwargs)

Pre process decoder inputs

training: bool

class alonet.deformable_detr.deformable_transformer.DeformableTransformerDecoderLayer(d_model=256, dim_feedforward=1024, dropout=0.1, activation='relu', n_levels=4, n_heads=8, n_points=4)

Bases: torch.nn.modules.module.Module

decoder_layer_forward(tgt, query_pos, reference_points, src, src_spatial_shapes, level_start_index, tgt_key_padding_mask=None, src_padding_mask=None, **kwargs)

Decoder forward layer

forward(tgt, query_pos, reference_points, src, src_spatial_shapes, level_start_index, tgt_key_padding_mask=None, src_padding_mask=None, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

forward_ffn(tgt)

pre_process_tgt(tgt, query_pos, tgt_key_padding_mask, **kwargs)

Pre process decoder inputs

training: bool

static with_pos_embed(tensor, pos)

class alonet.deformable_detr.deformable_transformer.DeformableTransformerEncoder(encoder_layer, num_layers)

Bases: torch.nn.modules.module.Module

forward(src, spatial_shapes, level_start_index, valid_ratios, pos=None, padding_mask=None, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

static get_reference_points(spatial_shapes, valid_ratios, device)

Get the reference points used in sampling

Parameters

spatial_shapes: (num_levels, 2)

height and width of each feature level

valid_ratios: (b, num_levels, 2)

ratio: [unpadded height / mask height, unpadded width / mask width]

Returns

reference_points: (b, num_points, num_levels, 2)

num_points = sum of height*width for each feature levels

training: bool

class alonet.deformable_detr.deformable_transformer.DeformableTransformerEncoderLayer(d_model=256, d_ffn=1024, dropout=0.1, activation='relu', n_levels=4, n_heads=8, n_points=4)

Bases: torch.nn.modules.module.Module

forward(src, pos, reference_points, spatial_shapes, level_start_index, padding_mask=None, **kwargs)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

forward_ffn(src)

training: bool

static with_pos_embed(tensor, pos)

alonet.deformable_detr.deformable_transformer.build_deforamble_transformer(args)