Related
How do I extract unique values from an ndarray?
import numpy as np
arr = np.array([[1, 2, 2], [2, 2, 3]], np.int32)
set(arr)
Traceback (most recent call last):
File "/home/me/PycharmProjects/project/example.py", line 6, in <module>
print(set(arr))
TypeError: unhashable type: 'numpy.ndarray'
import numpy as np
arr = np.array([[1, 2, 2], [2, 2, 3]], np.int32)
print(arr)
print(np.unique(arr))
[[1 2 2]
[2 2 3]]
[1 2 3]
Documentation
I previously asked a question for merging every 4 images of size 64x64 to 128x128 and I edited the answer as below:
How to merge multiple images from CNN prediction into one image?
# Initializing counters
i = 0 # Old image number
j = 0 # New image number
# Pre-allocate new images array
pred_128 = np.zeros((32, 128, 128, 1))
# Loop over new images
while j < 32:
pred_128 [j, :64, :64, 0] = pred_64[0+i, :, :, 0] # Upper left
pred_128 [j, 64:, :64, 0] = pred_64[2+i, :, :, 0] # Lower left
pred_128 [j, :64, 64:, 0] = pred_64[1+i, :, :, 0] # Upper right
pred_128 [j, 64:, 64:, 0] = pred_64[3+i, :, :, 0] # Lower right
# Add to counters
i += 4
j += 1
I want now to reuse this code to generate (32, 128, 128, 1) from different image size and:
1- (512, 32, 32, 1)
2- (2048, 16, 16, 1)
For the first case (512, 32, 32, 1), I used the following code and it returns error:
# Initializing counters
i = 0 # Old image number
j = 0 # New image number
# Pre-allocate new images array
pred_128 = np.zeros((32, 128, 128, 1))
# Loop over new images
while j < 32:
pred_128 [j, :32, :32, 0] = pred_32[0+i, :, :, 0] # Upper left
pred_128 [j, 32:, :32, 0] = pred_32[2+i, :, :, 0] # Lower left
pred_128 [j, :32, 32:, 0] = pred_32[1+i, :, :, 0] # Upper right
pred_128 [j, 32:, 32:, 0] = pred_32[3+i, :, :, 0] # Lower right
# Add to counters
i += 8
j += 1
ValueError Traceback (most recent call last)
<ipython-input-48-b4a45801c652> in <module>()
9 while j < 32:
10 pred_128 [j, :32, :32, 0] = pred_32[0+i, :, :, 0] # Upper left
---> 11 pred_128 [j, 32:, :32, 0] = pred_32[2+i, :, :, 0] # Lower left
12 pred_128 [j, :32, 32:, 0] = pred_32[1+i, :, :, 0] # Upper right
13 pred_128 [j, 32:, 32:, 0] = pred_32[3+i, :, :, 0] # Lower right
ValueError: could not broadcast input array from shape (32,32) into shape (96,32)
Can anyone help for reproducing the codes and solve the issue for the two different cases:
1- (512, 32, 32, 1) #merging every 16 images
2- (2048, 16, 16, 1) #merging every 64 images
error after using the proposed code:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-79-b71bf1e0ef80> in <module>()
12 # Loop over new images
13 for i in range(0, out_shape[0]):
---> 14 for x in range(0, out_shape[1]/dx):
15 for y in range(0, out_shape[2]/dy):
16 pred_128[i, 0+dx*x:dx*(x+1), 0+dy*y:dy*(y+1), 0] = pred_32[input_im_no, :, :, 0]
TypeError: 'float' object cannot be interpreted as an integer
-order of every 16 32x32 image in the original data
You will need to add more tiles to get the complete image. For the pred_32case you will need 16 input images for 1 output image and for the pred_16 case 64 input images for 1 output image. It is probably easier in this case to write a loop that 'shifts' over the desired output image and inputs one image at a time. Assuming your images are filling the greater image from left to right, I think the following code might help you out:
# Pre-allocate new images array
out_shape = (32, 128, 128, 1))
pred_128 = np.zeros(out_shape)
# Input sizes
dx = 32 # 16 for the pred_16
dy = 32 # 16 for the pred_16
# Input images counter
input_im_no = 0
# Loop over new images
for i in range(0, out_shape[0]):
for y in range(0, int(out_shape[1]/dy)):
for x in range(0, int(out_shape[2]/dx)):
pred_128[i, 0+dx*x:dx*(x+1), 0+dy*y:dy*(y+1), 0] = pred_32[input_im_no, :, :, 0]
# Select next image
input_im_no += 1
EDIT: x and y order after question update.
If I declare an array "v" whose shape is (3,100) when I want to change its values column by column making use a "for" python changes the dimension of "v[:,i]" for (3,) this is annoying and I can't make the change because at the left member it has a (3,) array and in the right, it has an (3,1) array.
I would like to know, why does this happen? and which are my options to cope with this?
Thanks.
v = np.ones( (3, 100) );
for i in range( 0 , 100 ):
v[:,i] = np.array([[1],
[2],
[3]])
ValueError: could not broadcast input array from shape (3,1) into shape (3)
In [379]: M = np.arange(12).reshape(3,4)
Indexing with a scalar reduced the dimension by one. That's a basic rule of indexing - in numpy and python.
In [380]: M[0,:]
Out[380]: array([0, 1, 2, 3])
In [381]: M[:,0]
Out[381]: array([0, 4, 8])
Same for a list:
In [383]: M.tolist()
Out[383]: [[0, 1, 2, 3], [4, 5, 6, 7], [8, 9, 10, 11]]
In [384]: M.tolist()[0]
Out[384]: [0, 1, 2, 3]
Index with a list/array or slice, does preserve the dimension:
In [385]: M[:,[0]]
Out[385]:
array([[0],
[4],
[8]])
So assigning a (3,) to the (3,) slot is fine:
In [386]: M[:,0] = [10,20,30]
Assigning a (3,1) to that slot produces an error:
In [387]: M[:,0] = [[10],[20],[30]]
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'list'
The above exception was the direct cause of the following exception:
ValueError Traceback (most recent call last)
<ipython-input-387-1bbfa6dfa93c> in <module>
----> 1 M[:,0] = [[10],[20],[30]]
ValueError: setting an array element with a sequence.
In [388]: M[:,0] = np.array([[10],[20],[30]]) # or with an array
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-388-6e511ffdc44e> in <module>
----> 1 M[:,0] = np.array([[10],[20],[30]])
ValueError: could not broadcast input array from shape (3,1) into shape (3)
By broadcasting (3,) can go into (1,3), but not (3,1) into (3,). One solution is to flatten the RHS:
In [389]: M[:,0] = np.array([[10],[20],[30]]).ravel()
Assigning to a (3,1) slot also works:
In [390]: M[:,[0]] = np.array([[10],[20],[30]])
In [391]: M[:,0:1] = np.array([[10],[20],[30]])
We could also transpose the (3,1) to (1,3). Or assign to M[:,0][:,None] or M[:,0,None] (both of which create a (3,1)).
What I think you are asking is: how to set them column wise.
v = np.ones( (3,100) )
for i in range( 0 , 100 ):
v[:,i] = np.array([1,
3,
2])
The change is to remove extra brackets in your assignment.
If you are trying to do something else, you can try doing it for rows, and flip array sideways:
v = np.ones((100,3))
for i in range(0,100):
v[i] = np.array([1,3,2])
EDIT: changed the whitespace to be the same as the author
I'm unclear on how the outer function provided to walk should work.
The example in documentation referenced here:
https://docs.hylang.org/en/stable/contrib/walk.html
Suggests the outer function can be first which would return the first element of the collection produced by mapping over with inner.
However when I try to aggregate the result in outer (eg sum or first) I get an error as per below - complaining that int is not iterable - looking at the source code I suspect this is because of (type form) in the macro definition:
((type form) (outer (HyExpression (map inner form))))
Can anyone confirm, and advise if there is a way of having outer return a different type to the input form? i.e. can (walk inc sum [1 2 3 4 5]) be made to provide the sum of the list [2 3 4 5 6] as I'd expect?
=> (walk inc identity [1 2 3 4 5])
[2, 3, 4, 5, 6]
=> (walk inc accumulate [1 2 3 4 5])
[2, 5, 9, 14, 20]
=> (walk inc sum [1 2 3 4 5])
Traceback (most recent call last):
File "stdin-75eb4a20707c49e8c921e986e7d6164d36b7e4b2", line 1, in <module>
(walk inc sum [1 2 3 4 5])
File "/home/phil/.local/lib/python3.6/site-packages/hy/contrib/walk.hy", line 22, in walk
((type form) (outer (HyExpression (map inner form))))]
TypeError: 'int' object is not iterable
=> (walk inc first [1 2 3 4 5])
Traceback (most recent call last):
File "stdin-710dcc990bf071fe1a9a4c5501831c41867f1879", line 1, in <module>
(walk inc first [1 2 3 4 5])
File "/home/phil/.local/lib/python3.6/site-packages/hy/contrib/walk.hy", line 22, in walk
((type form) (outer (HyExpression (map inner form))))]
TypeError: 'int' object is not iterable
=>
It's a bug. Say (sum (map inc [1 2 3 4 5])) instead.
Let's say I have a 1D numpy array x and another one y = x ** 2.
I am looking for an easier alternative to
for i in range(x.size):
print(x[i], y[i])
With one array one can do print(*x, sep = '\n') which is easier than a for loop. I'm thinking of something like converting x and y to arrays of strings and then adding them up into an array z and then using print(*z, sep = '\n'). However, I tried to do that but numpy gives an error when the add operation is performed.
Edit: This is the function I use for this
def to_str(*args):
return '\n'.join([' '.join([str(ls[i]) for ls in args]) for i in range(len(args[0]))]) + '\n'
>>> x = np.arange(10)
>>> y = x ** 2
>>> print(to_str(x,y))
0 0
1 1
2 4
3 9
4 16
5 25
6 36
7 49
8 64
9 81
>>>
or if something quick and dirty is enough:
print(np.array((x,y)).T)
You could do something along these lines -
# Input arrays
In [238]: x
Out[238]: array([14, 85, 79, 89, 41])
In [239]: y
Out[239]: array([13, 79, 13, 79, 11])
# Join arrays with " "
In [240]: z = [" ".join(item) for item in np.column_stack((x,y)).astype(str)]
# Finally print it
In [241]: print(*z, sep='\n')
14 13
85 79
79 13
89 79
41 11
# Original approach for printing
In [242]: for i in range(x.size):
...: print(x[i], y[i])
...:
14 13
85 79
79 13
89 79
41 11
To make things a bit more compact, np.column_stack((x,y)) could be replaced by np.vstack((x,y)).T.
There are few other methods to create z as listed below -
z = [str(i)[1:-1] for i in zip(x,y)] # Prints commas between elems
z = [str(i)[1:-1] for i in np.column_stack((x,y))]
z = [str(i)[1:-1] for i in np.vstack((x,y)).T]
Here is one way without loop:
print(np.array2string(np.column_stack((x, y)),separator=',').replace(' [ ','').replace('],', '').strip('[ ]'))
Demo:
In [86]: x
Out[86]: array([0, 1, 2, 3, 4])
In [87]: y
Out[87]: array([ 0, 1, 4, 9, 16])
In [85]: print(np.array2string(np.column_stack((x, y)),separator=',').replace(' [ ','').replace('],', '').strip('[ ]'))
0, 0
1, 1
2, 4
3, 9
4,16
There are 2 issues - combining the 2 arrays, and printing the result
In [1]: a = np.arange(4)
In [2]: b = a**2
In [3]: ab = [a,b] # join arrays in a simple list
In [4]: ab
Out[4]: [array([0, 1, 2, 3]), array([0, 1, 4, 9])]
In [6]: list(zip(*ab)) # 'transpose' that list
Out[6]: [(0, 0), (1, 1), (2, 4), (3, 9)]
That zip(*) is a useful tool or idiom.
I could use your print(*a, sep...) method with this
In [11]: print(*list(zip(*ab)), sep='\n')
(0, 0)
(1, 1)
(2, 4)
(3, 9)
Using sep is a neat py3 trick, but is rarely used. I'm not even sure how to do the equivalent with the older py2 print statement.
But if we convert the list of arrays into a 2d array we have more options.
In [12]: arr = np.array(ab)
In [13]: arr
Out[13]:
array([[0, 1, 2, 3],
[0, 1, 4, 9]])
In [14]: np.vstack(ab) # does the same thing
Out[14]:
array([[0, 1, 2, 3],
[0, 1, 4, 9]])
For simply looking at the 2 arrays together this arr is quite useful. And if the lines get too long, transpose it:
In [15]: arr.T
Out[15]:
array([[0, 0],
[1, 1],
[2, 4],
[3, 9]])
In [16]: print(arr.T)
[[0 0]
[1 1]
[2 4]
[3 9]]
Note that array print format is different that for nested lists. That's intentional.
The brackets seldom get in the way of understanding the display. They even help with the array becomes 3d and higher.
For printing a file that can be read by other programs, np.savetxt is quite useful. It lets me specify the delimiter, and the format for each column or line.
In [17]: np.savetxt('test.csv', arr.T, delimiter=',',fmt='%10d')
In ipython I can look at the file with a simple system call:
In [18]: cat test.csv
0, 0
1, 1
2, 4
3, 9
I can omit the delimiter parameter.
I can reload it with loadtxt
In [20]: np.loadtxt('test.csv',delimiter=',',dtype=int)
Out[20]:
array([[0, 0],
[1, 1],
[2, 4],
[3, 9]])
In Py3 it is hard to write savetxt to the screen. It operates on a byte string file, and sys.stdout is unicode. In Py2 np.savetxt(sys.stdout, ...) might work.
savetxt is not sophisticated. In this example, it is essentially doing a fwrite equivalent of:
In [21]: for row in arr.T:
...: print('%10d,%10d'%tuple(row))
...:
0, 0
1, 1
2, 4
3, 9