PyTorch

PyTorch is one of the most popular deep learning frameworks. CV-CUDA allows seamless interoperability with PyTorch, allowing you to use CV-CUDA’s optimized computer vision operations within your existing PyTorch workflows.

Key Points:

  • PyTorch tensors must be on GPU (.cuda()) to convert to CV-CUDA

  • Use cvcuda.as_tensor() to convert PyTorch tensors to CV-CUDA

  • Use torch.as_tensor() to convert CV-CUDA tensors back to PyTorch

  • You can use .clone() when converting from CV-CUDA to avoid shared memory issues, but this will incur a memcpy operation and potential performance degradation. Both torch and cvcuda support zero-copy intepability through their as_tensor functions.

Required Imports:

import torch
import cvcuda

PyTorch to CV-CUDA:

torch_tensor = torch.randn(10, 10)
torch_tensor = torch_tensor.cuda()  # move to GPU
cvcuda_tensor = cvcuda.as_tensor(torch_tensor)

The PyTorch tensor must be moved to GPU using .cuda() before conversion. The cvcuda.as_tensor() function creates a CV-CUDA tensor that shares the same GPU memory as the PyTorch tensor using the __cuda_array_interface__ protocol.

CV-CUDA to PyTorch:

# Clone so all tensors aren't sharing same GPU buffer
new_torch_tensor = torch.as_tensor(cvcuda_tensor.cuda())
cloned_tensor = (
    new_torch_tensor.clone()
)  # clone so that all tensors don't share same GPU buffer
assert cloned_tensor.data_ptr() != new_torch_tensor.data_ptr()

The .clone() call is important to avoid multiple tensors sharing the same GPU buffer, which can lead to unexpected behavior if one tensor is modified or deallocated.

Stream synchronization:

CV-CUDA streams are non-blocking, so they do not implicitly synchronize with PyTorch’s current stream. CV-CUDA inserts the necessary cross-stream barrier automatically when wrapping a PyTorch tensor via cvcuda.as_tensor(); no explicit torch.cuda.synchronize() is required:

src = torch.randint(0, 256, (2, 64, 64, 3), dtype=torch.uint8).cuda()
src_nv = cvcuda.as_tensor(src, "NHWC")
target = cvcuda.Stream()
with target:
    out_nv = cvcuda.flip(src_nv, -1, stream=target)
target.sync()

To opt out (e.g., when you’ve manually synchronized), export your buffer with stream: -1 in its CAI dict.

Note

Defensive synchronization for PyTorch buffers. PyTorch’s __cuda_array_interface__ is CAI v2 — it carries no stream field at all. Per the CAI v3 spec, the stream field uses sentinel integers (0 = “no stream associated, consumer must synchronize”, 1 = legacy default stream, 2 = per-thread default stream, other positive integers = real stream handles). Because PyTorch advertises v2, CV-CUDA has no producer-stream information and would race against any work PyTorch had queued on a non-default (non-blocking) stream.

To handle this defensively, CV-CUDA falls back to cudaDeviceSynchronize on the first use of any externally-wrapped buffer whose producer stream is unknown or is a default-stream sentinel. PyTorch tensors fall under “unknown” (CAI v2, no stream field) and so always pay one host-side device sync at cvcuda.as_tensor time; subsequent CV-CUDA ops on the same wrapper take the event-based fast path. CV-CUDA → CV-CUDA chains advertise the actual writer stream in CAI v3 and stay on the fast path end-to-end.

This was verified on PyTorch 2.9.0+cu128. If a future PyTorch release adopts CAI v3 and reports a real writer stream, CV-CUDA will pick that up automatically and skip the defensive sync.

Complete Example: See samples/interoperability/pytorch_interop.py