Numpy is very slow for basic array operations - arrays

Question

I have a code where I do a lot of basic arithmetic calculations with a bunch of numerical data that is is multiple arrays. I have realized that in most concievable operations, numpy classes are always slower than the default python ones. Why is this?
For example I have a simple snippet where all I do is just update 1 numpy array element with another one retrieved from another numpy array, or I update it with the mathematical product of 2 other numpy array elements. It should be a basic operation, yet it will always be at least 2-3x slower than if I do it with list.
First I thought that it's because I haven't harmonized the data structures and the compiler has to do a lot of unecessary transformations. So then I recoded the whole thing and replaced every float with numpy.float64 and every list with numpy.ndarray, and the entire data is numpy.float64 all across the code so that it doesn't have to do any unecessary transformations.
The code is still 2-3 times slower than if I just use list and float.
For example:
ALPHA = [[random.uniform(*a_param) for k in range(l2)] for l in range(l1)]
COEFF = [[random.uniform(*c_param) for k in range(l2)] for l in range(l1)]
summa=0.0
for l in range(l1):
for k in range(l2):
summa+=COEFF[l][k] * ALPHA[l][k]
will always be 2-3x faster than:
ALPHA = numpy.random.uniform(*a_param, (l1,l2))
COEFF = numpy.random.uniform(*c_param, (l1,l2))
summa=0.0
for l in range(l1):
for k in range(l2):
summa+=COEFF[l][k] * ALPHA[l][k]
How is this possible, am I doing something wrong , since numpy is supposed to speed up things.
For the record I am using Python 3.5.3 and numpy (1.12.1), should I update?

Answer 1

Modifying a single element of a NumPy array is not expected to be faster than modifying a single element of a Python list. The speedup from using NumPy comes when you perform "vectorized" operations on entire arrays (or subsets of arrays). Try assigning the first 10000 elements of a NumPy array to be equal to the first 10000 elements of another, and compare that with using lists.
If your data and/or operations are very small (one or just a few elements), you are probably better off not using NumPy.

Answer 2

I tried two things:
Running your two blocks of code. For me, they were about the same speed.
Writing a new function that exploits numpy's vectorized math. This is several times faster than the other methods.
Here are my functions:
import numpy as np
def with_lists(l1, l2):
ALPHA = [[random.uniform(0, 1) for k in range(l2)] for l in range(l1)]
COEFF = [[random.uniform(0, 1) for k in range(l2)] for l in range(l1)]
summa=0.0
for l in range(l1):
for k in range(l2):
summa+=COEFF[l][k] * ALPHA[l][k]
return summa
def with_arrays(l1, l2):
ALPHA = np.random.uniform(size=(l1,l2))
COEFF = np.random.uniform(size=(l1,l2))
summa=0.0
for l in range(l1):
for k in range(l2):
summa+=COEFF[l][k] * ALPHA[l][k]
return summa
def with_ufunc(l1, l2):
"""Avoid the loop completely by exploitng numpy's
elementwise math."""
ALPHA = np.random.uniform(size=(l1,l2))
COEFF = np.random.uniform(size=(l1,l2))
return np.sum(COEFF * ALPHA)
When I compare the speed (I'm using the %timeit magic in IPython), I get the following:
>>> %timeit with_lists(10, 10)
107 µs ± 4.7 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
>>> %timeit with_arrays(10, 10)
91.9 µs ± 10.5 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
>>> %timeit with_ufunc(10, 10)
12.6 µs ± 589 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
The third function, without loops, about 10 to 30 times faster on my machine, depending on the values of l1 and l2.