There are many deep learning frameworks such as Tensorflow, PyTorch, and Keras. Amongst the three, PyTorch seems to be a very popular framework that academics (and industries) use to implement and conduct deep learning research as they provide maximum flexibility and speed. In this blog post, I will document my learning journey of the deep learning framework PyTorch, with the objective to use it to build NLP applications using deep learning.

What are tensors?

Tensors are similar to Numpy’s darrays, with the addition that Tensors can be used on a GPU. Below are few ways to initialise a tensor object. You can also create a tensor based on an existing tensor.

  • torch.empty(matrix shape)
    • To create an uninitialised matrix but it does not contain definite known values

  • torch.rand(matrix shape)
    • To construct a randomly initialised matrix
  • torch.zeros(matrix shape, dtype)
    • Construct a matrix filled zeros and of dtype long
  • torch.tensor([data])
    • Construct a tensor directly from data
  • torch.view(new shape)
    • To resize/reshape tensor
  • torch.from_numpy(numpy array)
    • Convert NumPy to tensor

Tensor object

  • .size()
    • Matrix shape
  • .item()
    • To get the value of a one element tensor
  • .numpy()
    • Convert tensor to numpy
  • .to()
    • Use this method to move the tensors to any device, CPU or GPU

Operations

  • Addition through ‘+’ or .add(x, y, [out = output tensor])
    • You can also do addition in-place
      • y.add_(x)
      • The underscore post operation indicates in-place operation

NumPy Bridge

Both the Torch Tensor and NumPy array share their underlying memory locations. This means that changing one of them will change the other too!

CUDA Tensors

To check if CUDA is available on your device:

  • torch.cuda.is_available()
    • Return true or false
  • To initialise a CUDA device object
    • device = torch.device(“cuda”)
Ryan

Ryan

Data Scientist

Leave a Reply