Disparity

The aloscene Disparity object represents a Disparity Map.

Basic Use

Creating a Disparity object

A Disparity object can be initialized from a path to a disparity file:

from aloscene import Disparity
disp = Disparity("path/to/disparity.pfm")

or from an existing tensor:

import torch
disp_tensor = torch.zeros((1,400,600))
disp = Disparity(disp_tensor, names=("C","H","W"))

An occlusion mask can be attached to the Disparity during its creation:

from aloscene import Mask
occlusion = Mask(torch.zeros(1,400, 600), names=("C","H","W"))
disp = Disparity("path/to/disparity.pfm", occlusion)

or later:

disp.append_occlusion(occlusion)

Sign convention

Warning

The disparity between left and right image can be expressed in two different conventions:

  • “unsigned”: absolute difference of pixels between left and right image

  • “signed”: relative difference of pixels (negative for left image)

If camera_side is known, it is possible to switch from one convention to the other:

disp = Disparity(torch.ones((1, 5, 5)), disp_format="unsigned", names=("C", "H", "W"), camera_side="left")
disp_signed = disp.signed()
print("unsigned:", disp.as_tensor().unique())
print("signed": disp_signed.as_tensor().unique())

In this example, we switch from “unsigned” to “signed” for left camera, therefore the disparity become negative:

unsigned: tensor([1.])
signed: tensor([-1.])

Easy visualization

With aloscene API, it is really straigthforward to visualize data. In one line of code, you can get a RGB view of a Disparity object and its occlusion map and display it:

from alodataset import SintelDisparityDataset
# read disparity data
dataset = SintelDisparityDataset()
frame = next(iter(dataset.stream_loader()))
disparity = frame["left"].disparity
# visualize it
view = disparity.get_view() # view containing a numpy array with RGB data
view.render() # render with matplotlib or cv2 and show data in a new window
../_images/disp_view.jpg

Visualization of a disparity sample from MPI Sintel Disparity dataset.

Easy transformations

With aloscene API, it is easy to apply transformations to visual data, even when it necessitate extra care.

For example, horizontally flipping a frame with disparity data is usually tedious. The image, the disparity and the occlusion should be flipped. But the disparity necessitate a specific transformation: the sign of disparity values must be inverted only if the disparity is expressed in relative units.

With aloscene API, this is done automatically in one function call:

frame_flipped = frame["left"].hflip() # flip frame and every attached labels
disp_flipped = frame_flipped.disparity
disp_flipped.get_view().render() # display flipped flow and occlusion
../_images/disp_hflip.jpg

Visualization of the previous disparity sample, after horizontal flip. The colors are different, because disparity values have been correctly modified. The occlusion has also been flipped.

Warning

When horizontally flipping frames with disparity labels, the “left” and “right” frames should be swapped. This is necessary to keep the disparity values consistent with the images content.

Disparity API

class aloscene.disparity.Disparity(x, occlusion: Optional[aloscene.mask.Mask] = None, disp_format='unsigned', camera_side=None, png_negate=None, *args, names=('C', 'H', 'W'), **kwargs)

Bases: aloscene.tensors.spatial_augmented_tensor.SpatialAugmentedTensor

Disparity map

Parameters
x: tensor or str

loaded disparity tensor or path to the disparity file from which disparity will be loaded.

occlusionaloscene.Mask

Occlusion mask for this disparity map. Default value : None.

disp_format{‘signed’|’unsigned’}

If unsigned, disparity is interpreted as a distance (positive value) in pixels. If signed, disparity is interpreted as a relative offset (values can be negative).

camera_side{‘left’, ‘right’, None}

side of the camera. Default None. Necessary to switch from unsigned to signed format.

png_negate: bool

if true, the sign of disparity is reversed when loaded from file. this parameter should be explicitely set every time a .png file is used.

append_occlusion(occlusion, name=None)

Attach an occlusion mask to the frame.

Parameters
occlusion: Mask

Occlusion mask to attach to the Frame

name: str

If none, the occlusion mask will be attached without name (if possible). Otherwise if no other unnamed occlusion mask are attached to the frame, the mask will be added to the set of mask.

signed()

Returns a copy of disparity map in signed disparity format

unsigned()

Returns a copy of disparity map in unsigned disparity format