Orangele's Blog.

Differences of dot(), matmul(), multiply(), outer(), *, @ in numpy

Numpy
Word count: 854Reading time: 5 min
2019/12/07 Share

Numpy’s dot(), matmul(), multiply(), outer() functions and *, @ operators in python can all indicate mutiplication operations between matrices and are often easily-confused. Here I want to make a summary on their uses based on the offical documents and some examples.

1. numpy.matmul() and @

According to offical docs, numpy.matmul() has four patterns depending on the arguments:

  1. If both arguments are 2-D they are multiplied like conventional matrices.
  2. if either argument is N-D, N > 2, it is treated as a stack of matrices residing in the last two indexes and broadcast accordingly.
  3. If the first argument is 1-D, it is promoted to a matrix by prepending a 1 to its dimensions. After matrix multiplication the prepended 1 is removed.
  4. If the second argument is 1-D, it is promoted to a matrix by appending a 1 to its dimensions. After matrix multiplication the appended 1 is removed.

The second points means that the last two demensions of the two matrices must obey the conventional rule $(n,k)\cdot(k,m)\rightarrow(n,m)$, but the demensions except the last two must have the same length.

1
2
3
4
5
6
7
8
9
10
>>> a = np.ones([9, 5, 7, 4])
>>> b = np.ones([9, 5, 4, 3])
>>> np.matmul(a, b).shape
(9, 5, 7, 3)
>>> c = np.ones([9, 6, 4, 3])
>>> np.matmul(a, c).shape
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (9,5,7,4)->(9,5,newaxis,newaxis) (9,6,4,3)->(9,6,newaxis,newaxis) and requested 
shape (7,3)

Point 3 and 4 make us not have to worry about whether the 1d-array shoud be a row or a column in a matrix multiplication. If both of the arguments are 1-D, it provides the inner product of them.

Also, we should notice that multiplication by scalars is not allowed.

As for @, frome here we can know that the @ operator calls the array’s __matmul__ method which is actually the same as np.multiply().

2. numpy.multiply() and *

Both of numpy.multiply() and * multiply numpy.ndarray arguments element-wise.

1
2
3
4
5
6
7
8
>>> a= np.array([[1, 2, 3], [4, 5, 6]])
>>> b= np.array([1, 2, 3])
>>> a*b
array([[ 1,  4,  9],
       [ 4, 10, 18]])
>>> np.multiply(a,b)
array([[ 1,  4,  9],
       [ 4, 10, 18]])

However, in the case of numpy.matrix objects, numpy.multiply() does element-wise multiplication but * does conventional matrix multiplication.

1
2
3
4
5
6
7
8
>>> a = np.mat('1 2;3 4')
>>> b = np.mat('1 2;3 4')
>>> np.multiply(a,b)
matrix([[ 1,  4],
        [ 9, 16]])
>>> a*b
matrix([[ 7, 10],
        [15, 22]])

3. numpy.dot()

According to offical docs, numpy.dot() has five patterns depending on the arguments:

  1. If both a and b are 1-D arrays, it is inner product of vectors (without complex conjugation).
  2. If both a and b are 2-D arrays, it is matrix multiplication, but using matmul or a @ b is preferred.
  3. If either a or b is 0-D (scalar), it is equivalent to multiply and using numpy.multiply(a, b) or a * b is preferred.
  4. If a is an N-D array and b is a 1-D array, it is a sum product over the last axis of a and b.
  5. If a is an N-D array and b is an M-D array (where M>=2), it is a sum product over the last axis of a and the second-to-last axis of b:
    dot(a, b)[i,j,k,m] = sum(a[i,j,:] * b[k,:,m])

We can see that numpy.dot() can be used as numpy.matmul() or numpy.multiply() under specific situations. But the reason why such use is not recommanded is that using numpy.matmul() or numpy.multiply() should be faster.

numpy.matmuldiffers from numpy.dot() in two ways:

  1. numpy.matmul() supports multiplication by scalars but numpy.dot() doesn’t.
  2. they treat N-D array in different ways:
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    >>> a = np.arange(3*4*5*6).reshape((3,4,5,6))
    >>> b = np.arange(3*4*5*6).reshape((5,4,6,3))
    >>> np.dot(a, b)[2,3,2,1,2,2]
    242925
    >>> sum(a[2,3,2,:] * b[1,2,:,2])
    242925
    >>> c = np.arange(3*4*5*6).reshape((3,4,6,5))
    >>> np.dot(a,b).shape
    (3, 4, 5, 5, 4, 3)
    >>> np.matmul(a,c).shape
    (3, 4, 5, 5)

4. numpy.outer()

The numpy.outer() computes the outer product of two vectors.
Given two vectors, a = [a0, a1, ..., aM] and b = [b0, b1, ..., bN], the outer product is:

1
2
3
4
[[a0*b0  a0*b1 ... a0*bN ]
 [a1*b0    .
 [ ...          .
 [aM*b0            aM*bN ]]

Author：Orangele

原文链接：https://orangelle.github.io/2019/12/07/Differences-of-dot-matmul-multiply-outer-in-numpy/

发表日期：December 7th 2019, 2:20:07 am

更新日期：December 7th 2019, 2:24:36 am

版权声明：This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License

CATALOG
  1. 1. 1. numpy.matmul() and @
  2. 2. 2. numpy.multiply() and *
  3. 3. 3. numpy.dot()
  4. 4. 4. numpy.outer()
Total : 4
2019
Numpy C++ Machine Learning Statistics Tensorflow
Coding Theory