PyTorch

PyTorch is one of the most popular deep learning frameworks. CV-CUDA allows seamless interoperability with PyTorch, allowing you to use CV-CUDA’s optimized computer vision operations within your existing PyTorch workflows.

Key Points:

  • PyTorch tensors must be on GPU (.cuda()) to convert to CV-CUDA

  • Use cvcuda.as_tensor() to convert PyTorch tensors to CV-CUDA

  • Use torch.as_tensor() to convert CV-CUDA tensors back to PyTorch

  • You can use .clone() when converting from CV-CUDA to avoid shared memory issues, but this will incur a memcpy operation and potential performance degradation. Both torch and cvcuda support zero-copy intepability through their as_tensor functions.

Required Imports:

import torch
import cvcuda

PyTorch to CV-CUDA:

torch_tensor = torch.randn(10, 10)
torch_tensor = torch_tensor.cuda()  # move to GPU
cvcuda_tensor = cvcuda.as_tensor(torch_tensor)

The PyTorch tensor must be moved to GPU using .cuda() before conversion. The cvcuda.as_tensor() function creates a CV-CUDA tensor that shares the same GPU memory as the PyTorch tensor using the __cuda_array_interface__ protocol.

CV-CUDA to PyTorch:

# Clone so all tensors aren't sharing same GPU buffer
new_torch_tensor = torch.as_tensor(cvcuda_tensor.cuda())
cloned_tensor = (
    new_torch_tensor.clone()
)  # clone so that all tensors don't share same GPU buffer
assert cloned_tensor.data_ptr() != new_torch_tensor.data_ptr()

The .clone() call is important to avoid multiple tensors sharing the same GPU buffer, which can lead to unexpected behavior if one tensor is modified or deallocated.

Complete Example: See samples/interoperability/pytorch_interop.py