Tensors and Arrays for Imaging

Views

We refer to an image’s view as the order of the dimensions for its channels-width-height, e.g. chw or hwc, along with the ordering of the channel dimension, e.g. RGB or BGR.

The following commonly used packages have their own default view for images:

Package	Data Type	View
PyTorch	Torch Tensors/Variables	chw-RGB
OpenCV [1]	Numpy Arrays	hwc-BGR
PyPlot	Numpy Arrays	hwc-RGB

[1]	OpenCV provides its own functionality for changing the channel ordering.

Type and View Conversions

dlt.util.change_view() can be used to convert between views. Please see the documentation for available views.

 # Load image using OpenCV
 img = cv2.imread('kodim16.png')
 # Change the view and display with pyplot
 plt_img = change_view(img, 'cv', 'plt')
 plt.imshow(plt_img)
 plt.show()

Functions such as dlt.util.cv2torch(), dlt.util.torch2cv() and dlt.util.torch2plt(), change the view as well as the data type. They also accept Variables or GPU Tensors as inputs and always return a Tensor or Array (copy) in main memory.

 # Load image using OpenCV
 img = cv2.imread('kodim16.png').astype('float32')/255.0
 # Model to processes the image
 net = nn.MaxPool2d(5,stride=2).cuda()
 # Create input using cv2torch and get result
 result = net(Variable(util.cv2torch(img).unsqueeze(0).cuda()))

Displaying

imshow

dlt.viz.imshow() can be used to display images. Accepts Arrays and Tensors as well as different views. The view must be provided as an argument.

 # Load image using OpenCV
 img = cv2.imread('kodim16.png')
 # Display image for 5 seconds
 dlt.viz.imshow(img, view='cv')

make_grid

dlt.util.make_grid() can be used to create grids. It can accept mixed lists of Arrays, Tensors and Variables, as well as different sized images (as long as they have the same view).

 # Load image using OpenCV
 img = cv2.imread('kodim16.png').astype('float32')/255.0
 # Model to processes the image
 net = nn.MaxPool2d(5,stride=2).cuda()
 result = net(Variable(util.cv2torch(img).unsqueeze(0).cuda()))
 # Make a grid with the images
 input_img = dlt.util.change_view(img, 'cv', 'torch') # all must have the same view
 viz.imshow(dlt.util.make_grid([input_img, result, net(result), net(net(result))]))

HDR

The functions dlt.hdr.imread() and dlt.hdr.imwrite() support all the OpenCV formats (including ‘.hdr’ and ‘.exr’) as well as ‘.pfm’.