Frame¶

class aloscene.frame.Frame(x, boxes2d: Optional[Union[dict, aloscene.bounding_boxes_2d.BoundingBoxes2D]] = None, boxes3d: Optional[Union[dict, aloscene.bounding_boxes_3d.BoundingBoxes3D]] = None, labels: Optional[Union[dict, aloscene.labels.Labels]] = None, flow: Optional[Union[dict, aloscene.flow.Flow]] = None, segmentation: Optional[Union[dict, aloscene.mask.Mask]] = None, disparity: Optional[Union[dict, aloscene.disparity.Disparity]] = None, points2d: Optional[Union[dict, aloscene.points_2d.Points2D]] = None, normalization='255', mean_std=None, names=('C', 'H', 'W'), *args, **kwargs)¶

Bases: aloscene.tensors.spatial_augmented_tensor.SpatialAugmentedTensor

Augmented Frame tensor. The Frame cam be created using the path to an image. Othwewise, if the frame is created from a existing tensor or numpy array, the frame dimensions are expected to be (“C”, “H”, “W”). If this is not the case, the names must be passed to the tensor.

If your data is more than 3 dimensional you might need to set the names to (“B”, “C”, “H”, “W”) for batch dimension or (“T”, “C”, “H”, “W”) for the temporal dimension, or even (“B”, “T”, “C”, “H”, “W”) for batch and temporal dimension. Checkout the example below for an example.

Parameters

boxes2d: dict | aloscene.BoundingBoxes2D: Dict of boxes2d or an instance of aloscene.BoundingBoxes2D
boxes3d: dict | aloscene.BoundingBoxes3D: Dict of boxes3d or an instance of aloscene.BoundingBoxes3D
labels: dict | aloscene.Labels: Dict of labels or an instance of aloscene.Labels
flow: dict | aloscene.Flow: Dict of flow or an instance of aloscene.Flow
segmentation: dict | aloscene.Mask: Dict of segmentation (aloscene.Mask) or an instance of aloscene.Mask
segmentation: dict | aloscene.Disparity: Dict of Disparity or an instance of aloscene.Disparity
normalization: str: One of [“255”, “01”, “minmax_sym”]
mean_std: tuple: Tuple with the mean and std of the tensor. (mean, std). Example: ((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)))

Notes

Note on dimension:

C refers to the channel dimension
N refers to a dimension with a dynamic number of elements.
H refers to the height of a SpatialAugmentedTensor
W refers to the width of a SpatialAugmentedTensor
B refers to the batch dimension
T refers to the temporal dimension

Examples

>>> # Creating a frame from a given path
>>> frane = aloscene.Frame("path/to/frame.jpg")
>>> frame.get_view().render()
>>>
>>> # Creating a frame from a numpy array or tensor
>>> data = np.zeros((3, 256, 512))
>>> frame = aloscene.Frame(data, normalization="01")
>>>
>>> # Creating a frame from a numpy array or tensor
>>> data = np.zeros((1, 3, 256, 512))
>>> frame = aloscene.Frame(data, normalization="01", names=("B", "C", "H", "W"))

append_boxes2d(boxes, name=None)¶

Attach a set of BoundingBoxes2D to the frame.

Parameters

boxes: aloscene.BoundingBoxes2D: Boxes to attached to the Frame
name: str: If none, the boxes will be attached without name (if possible). Otherwise if no other unnamed boxes are attached to the frame, the boxes will be added to the set of boxes.

Examples

>>> # Adding one set of unnamed boxes
>>> frane = aloscene.Frame("path/to/frame.jpg")
>>> boxes2d = aloscene.BoundingBoxes2D([[0.5, 0.5, 0.5, 0.5]], boxes_format="xcyc", absolute=False)
>>> frame.append_boxes2d(boxes2d)
>>>
>>> # Adding one set of named boxes
>>> frane = aloscene.Frame("path/to/frame.jpg")
>>> boxes2d = aloscene.BoundingBoxes2D([[0.5, 0.5, 0.5, 0.5]], boxes_format="xcyc", absolute=False)
>>> frame.append_boxes2d(boxes2d, "boxes_set")

append_boxes3d(boxes_3d, name=None)¶

Attach BoundingBoxes3D to the frame

Parameters

boxes: BoundingBoxes3D: Boxes to attached to the Frame
name: str: If none, the boxes will be attached without name (if possible). Otherwise if no other unnamed boxes are attached to the frame, the boxes will be added to the set of boxes.

Examples

>>> # To be render and transform properly, boxes3D required cam_extrinsic parameters.
>>> # To make things easier to get started, let's use the waymo dataset sample
>>> # and let's add a new set of boxes 3D
>>> frame = alodataset.WaymoDataset(sample=True).getitem(0)["front"]
>>> # [xc, yc, zc, Dx, Dy, Dz, heading]
>>> boxes3d = aloscene.BoundingBoxes3D([[-6.2120, -1.0164, 22.5095, 1.9325, 1.4111, 4.0868, -0.2779]])
>>> We append to boxes3D twice because the frame has a temporal dimension (T=2)
>>> frame.append_boxes3d([boxes3d, boxes3d], "my_set")

append_disparity(disparity, name=None)¶

Attach a disparity map to the frame.

Parameters

disparity: aloscene.Disparity: Disparity to attach to the Frame
name: str: If none, the disparity will be attached without name (if possible). Otherwise if no other unnamed disparity are attached to the frame, the disparity will be added to the set of flow.

Examples

>>> frame = aloscene.Frame("/path/to/image.jpeg")
>>> disparity = aloscene.Disparity(np.zeros((1, frame.H, frame.W)))
>>> frame.append_disparity(disparity)

append_flow(flow, name=None)¶

Attach a flow to the frame.

Parameters

flow: aloscene.Flow: Flow to attach to the Frame
name: str: If none, the flow will be attached without name (if possible). Otherwise if no other unnamed flow are attached to the frame, the flow will be added to the set of flow.

Examples

>>> frame = aloscene.Frame("/path/to/image.jpeg")
>>> flow = aloscene.Flow(np.zeros((2, frame.H, frame.W)))
>>> frame.append_flow(flow)

append_labels(labels, name=None)¶

Attach a set of labels to the frame. This can be usefull for classification or multi label classification. The rank of the label must be >= 1

Parameters

labels: aloscene.Labels: Set of labels to attached to the frame
name: str: If none, the label will be attached without name (if possible). Otherwise if no other unnamed labels are attached to the frame, the labels will be added to the set of labels.

Examples

>> frame = aloscene.Frame(“/path/to/image.jpeg”) >> labels = aloscene.Labels([42]) >> frame.append_labels(labels)

append_points2d(points, name=None)¶

Attach a set of points to the frame.

Parameters

boxes: Points2D: Points to attach to the Frame
name: str: If None, the points will be attached without name (if possible). Otherwise if no other unnamed points are attached to the frame, the points will be added to the set of points.

append_segmentation(segmentation, name=None)¶

Attach a segmentation to the frame.

Parameters

segmentation: aloscene.Mask: Mask with size (N,H,W), where N is the features maps, each one for one object. Each feature map must be a binary mask. For that, is a type of aloscene.Mask
name: str: If none, the mask will be attached without name (if possible). Otherwise if no other unnamed mask are attached to the frame, the mask will be added to the set of mask.

mean_std_norm(mean, std, name)¶

Normnalize the tensor from the current tensor normalization to the expected resnet normalization (x - mean) / std with x normalized with value between 0 and 1.

Examples

>>> mean = (0.485, 0.456, 0.406)
>>> std = (0.229, 0.224, 0.225)
>>> normalized_frame = frame.mean_std_norm(mean, std, "my_norm")

Return type: ~Frame

norm01()¶

Normnalize the tensor from the current tensor normalization to values between 0 and 1.

Examples

>>> frame_01 = frame.norm01()

Return type: ~Frame

norm255()¶

Normnalize the tensor from the current tensor normalization to values between 0 and 255

Examples

>>> frame_255 = frame.norm255()

Return type: ~Frame

norm_as(target_frame)¶

Normalize the tensor as the given target_frame.

Parameters

target_frame: aloscene.Frame: Will change the frame normalization as this target_frame

Examples

>>> new_frame = frame.norm_as(target_frame)

Return type: ~Frame

norm_minmax_sym()¶

Normalize the tensor to values between -1 and 1

Examples

>>> frame_minmax_sym = frame.norm_minmax_sym()

norm_resnet()¶

Normalized the current frame based on the normalized use on resnet on pytorch. This method will simply call frame.mean_std_norm() with the resnet mean/std and the name resnet.

Examples

>>> frame_resnet = frame.norm_resnet()

Return type: ~Frame

save(tgt_path)¶

Save the frame into a given location

Parameters

tgt_path: str: target path to save the frame. Example: /path/to/image.jpeg

Notes

The attached labels will not be saved.