馃殌 The feature
Make to_image reading return tensors in CHW
Motivation, pitch
|
@torch.jit.unused |
|
def to_image(inpt: Union[torch.Tensor, PIL.Image.Image, np.ndarray]) -> tv_tensors.Image: |
|
"""See :class:`~torchvision.transforms.v2.ToImage` for details.""" |
|
if isinstance(inpt, np.ndarray): |
|
output = torch.from_numpy(np.atleast_3d(inpt)).permute((2, 0, 1)).contiguous() |
|
elif isinstance(inpt, PIL.Image.Image): |
|
output = pil_to_tensor(inpt) |
|
elif isinstance(inpt, torch.Tensor): |
|
output = inpt |
|
else: |
|
raise TypeError( |
|
f"Input can either be a pure Tensor, a numpy array, or a PIL image, but got {type(inpt)} instead." |
|
) |
|
return tv_tensors.Image(output) |
|
|
|
|
import pyvips
import torchvision,torchvision.transforms.v2
img = pyvips.Image.new_from_file(r"F:\test.jpg")
tensor = torchvision.transforms.v2.functional.to_image(img.numpy())
tensor.shape
#torch.Size([3, 1830, 1370])
tensor.shape, tensor.stride(), tensor.is_contiguous(memory_format=torch.channels_last), tensor.unsqueeze(0).is_contiguous(memory_format=torch.channels_last)
# (torch.Size([3, 1830, 1370]), (2507100, 1370, 1), False, False) # Not memory_format=torch.channels_last
new_tensor = torch.from_numpy(tensor.numpy()).permute((2,0,1))
new_tensor.shape, new_tensor.stride(), new_tensor.is_contiguous(memory_format=torch.channels_last), new_tensor.unsqueeze(0).is_contiguous(memory_format=torch.channels_last)
# (torch.Size([3, 1830, 1370]), (1, 4110, 3), False, True) # This is memory_format=torch.channels_last
https://docs.pytorch.org/vision/main/transforms.html#range-and-dtype
...
Transforms tend to be sensitive to the input strides / memory format. Some transforms will be faster with channels-first images while others prefer channels-last. Like torch operators, most transforms will preserve the memory format of the input, but this may not always be respected due to implementation details. You may want to experiment a bit if you鈥檙e chasing the very best performance. Using torch.compile() on individual transforms may also help factoring out the memory format variable (e.g. on Normalize). Note that we鈥檙e talking about memory format, not tensor shape.
Note that resize transforms like Resize and RandomResizedCrop typically prefer channels-last input and tend not to benefit from torch.compile() at this time.
...
As torchvision.transforms.v2.functional.to_image usually being the starting point of the processing pipeline, the latter process of it would always be float().div_(255) or to_dtype then resize and etc., these all prefer channels_last memory_format for better performance.
Therefore the starting point of the processing pipeline should keep its NHWC memory format. And which make the memcopy by contiguous() avoid that always happen (since image is always HWC when read in numpy)
Thanks for your reading.
I could make a PR
馃殌 The feature
Make to_image reading return tensors in CHW
Motivation, pitch
vision/torchvision/transforms/v2/functional/_type_conversion.py
Lines 10 to 25 in da44627
As
torchvision.transforms.v2.functional.to_imageusually being the starting point of the processing pipeline, the latter process of it would always befloat().div_(255)orto_dtypethenresizeand etc., these all prefer channels_last memory_format for better performance.Therefore the starting point of the processing pipeline should keep its NHWC memory format. And which make the memcopy by
contiguous()avoid that always happen (since image is always HWC when read in numpy)Thanks for your reading.
I could make a PR