numpy shape inconsistent with array structure - arrays

I am going mad over this thing.
I have 2 lists
A = [ [[1,2,3],[1,2,3],[1,2,3]], [[1,2,3],[1,2,3],[1,2,3]]]
B = [ [[1,2,3],[1,2,3],[1,2,3]], [[1,2,3],[1,2,3]]]
When I call the shape of A and B as numpy array I get this:
In [33]: np.asarray(A).shape
Out[33]: (2, 3, 3)
In [31]: np.asarray(B).shape
Out[31]: (2,)
How do I shape A in the same way as B, that is (2,)?
I think I understand why it's happening but I don't know how to prevent this to happening.
Anyone any help/idea please?
thanks!

Your 2 lists:
In [232]: A
Out[232]: [[[1, 2, 3], [1, 2, 3], [1, 2, 3]], [[1, 2, 3], [1, 2, 3], [1, 2, 3]]]
In [233]: B
Out[233]: [[[1, 2, 3], [1, 2, 3], [1, 2, 3]], [[1, 2, 3], [1, 2, 3]]]
Now, explain why the B result is better than the A one?
In [234]: np.array(A)
Out[234]:
array([[[1, 2, 3],
[1, 2, 3],
[1, 2, 3]],
[[1, 2, 3],
[1, 2, 3],
[1, 2, 3]]])
In [235]: np.array(B)
<ipython-input-235-c938532b77c1>:1: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
np.array(B)
Out[235]:
array([list([[1, 2, 3], [1, 2, 3], [1, 2, 3]]),
list([[1, 2, 3], [1, 2, 3]])], dtype=object)

Related

Fill an empty array (Vector of vectors) in Julia using for loop

Given
b=[ [1,[2,3,7]],
[2,[7,2,7]],
[3,[2,3,9]] ]
a=[ [2,3,7],[7,2,7],[2,3,9] ]
I want to get b from empty vector of vectors using for loop.
b=[ [ ] ]
for i in 1:3
b[i][1]=i
b[i][2]=a[i]
end
julia> a = [[2,3,7],[7,2,7],[2,3,9]];
julia> b = enumerate(a) |> collect
3-element Vector{Tuple{Int64, Vector{Int64}}}:
(1, [2, 3, 7])
(2, [7, 2, 7])
(3, [2, 3, 9])
As the output says, this returns a Vector of Tuples. If you do need exactly a vector of vectors for some reason, you can do:
julia> b = enumerate(a) |> collect .|> collect
3-element Vector{Vector{Any}}:
[1, [2, 3, 7]]
[2, [7, 2, 7]]
[3, [2, 3, 9]]
For a tuple, just use tuple.(1:3,a), if a vector is required do vec.(1:3,a). Note that a tuple is an immutable container, so you might gain some performance benefits, and you can still change the elements of the vectors inside since vectors are mutable.
b = tuple.(1:3,a)
3-element Vector{Tuple{Int64, Vector{Int64}}}:
(1, [2, 3, 7])
(2, [7, 2, 7])
(3, [2, 3, 9])
b = vec.(1:3,a)
3-element Vector{Vector{Any}}:
[1, [2, 3, 7]]
[2, [7, 2, 7]]
[3, [2, 3, 9]]

Confirm all columns in a pandas dataframe are 1-D

It is not good practise to include multi-arrays/lists as columns in a pandas dataframe. In the event that I want to raise a value error whenever any column in a dataframe is not 1-D.
Given a dataset
dfA = pd.DataFrame(
np.array(
[
[1, (0,2), 0, 3],
[1, (0,0), 1, 2],
[0, (5,1),6, 1],
[4, (3,0), 3, 4],
[1, (1,1), 0, 2],
[2, (0,1), 3, 5],
[1, (3,3), 1, 2],
[6, (4,3), 5, 3],
[3, (0,2), 1, 2],
[2, (0,0), 2, 1],
]
),
columns=['A', 'B', 'C', 'D'])
I want to do something similar to
if columns in dfA are not all 1-D:
raise ValueError("Dataframe must only have 1-D columns")
In your case you can slice the 1st row , then np.shape
dfA.iloc[0].map(lambda x :np.shape(x))!=()
Out[413]:
A False
B True
C False
D False
Name: 0, dtype: bool

numpy splitting arrays column wise and stacking to each row

I have a 2D array of dimension 12*80.
I want to split each row into subarrays of size 4 (total of 20 arrays) and stack the resulting arrays to the rows vertically.
Let us say that my array is
>>> A
array([[1, 2, 2, 2, 2, 2],
[3, 3, 1, 3, 1, 3],
[3, 1, 2, 1, 1, 3]])
>>>
and I want to split each row into 3 subarrays and stack vertically. My expected output is
>>> A
array([[1, 2],
[2, 2],
[2, 2],
[3, 3],
[1, 3],
[1, 3],
[3, 1],
[2, 1],
[1, 3]])
>>>
Is there any other way other than iterating over and splitting each row one by one ? A better efficient implementation ?

How to get all sub matrices of 2D array without numpy?

I need to get all submatrices of the 2D array and to do the manipulation for each submatrix. So I created example matrix:
M3 = [list(range(5)) for i in range(6)]
[[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4],
[0, 1, 2, 3, 4]]
I need to capture 3 rows and 3 columns and then shift this "window" till I get all submatrices. The first submatrix would be:
[[0, 1, 2],
[0, 1, 2],
[0, 1, 2]]
and the last one is:
[[2, 3, 4],
[2, 3, 4],
[2, 3, 4]]
For this matrix I need 12 submatrices. However, I become more using code with which I tried to solve the problem:
for j in range(len(M3[0])-3):
for i in range(len(M3)-3):
for row in M3[0+j:3+j]:
X_i_j = [row[0+i:3+i] for row in M3[0+j:3+j]]
print(X_i_j)
I get 18 but not 12 (with two duplicates of each submatrix):
[[0, 1, 2], [0, 1, 2], [0, 1, 2]]
[[0, 1, 2], [0, 1, 2], [0, 1, 2]]
[[0, 1, 2], [0, 1, 2], [0, 1, 2]]
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
...
[[2, 3, 4], [2, 3, 4], [2, 3, 4]]
[[2, 3, 4], [2, 3, 4], [2, 3, 4]]
And with this sample of code I get 6 submatrices with 1 duplicate for each:
for i in range(len(M3)-3):
for j in range(len(M3[0])-3):
X_i_j = [row[0+i:3+i] for row in M3[0+j:3+j]]
print(X_i_j)
I do not see what is wrong with it and why I get the duplicates. How can I get all sub matrices of 2D array without numpy for this case?
Your code is working ( with change of order of vars and constants ):
for j in range(len(M3)-2):
for i in range(len(M3[0])-2):
X_i_j = [row[0+i:3+i] for row in M3[0+j:3+j]]
print('=======')
for x in X_i_j:
print(x)
I would solve it slightly different.
a function to read y-number-of-rows
then a function to read x-number-of-columns from those rows, which then is your sub.
This would work for any (2D) array / sub-array
Sample:
def read_y_rows(array, rows, offset):
return array[offset:rows + offset]
def read_x_cols(array, cols, offset):
return list(row[offset:cols + offset] for row in array)
def get_sub_arrays(array, x_dim_cols, y_dim_rows):
"""
get 2D sub arrays by x_dim columns and y_dim rows
from 2D array (list of lists)
"""
result = []
for start_row in range(len(array) - y_dim_rows + 1):
y_rows = read_y_rows(array, y_dim_rows, start_row)
for start_col in range(len(max(array, key=len)) - x_dim_cols + 1):
x_columns = read_x_cols(y_rows, x_dim_cols, start_col)
result.append(x_columns)
return result
to use it you could do:
M3 = [list(range(5)) for i in range(6)]
sub_arrays = get_sub_arrays(M3, 3, 3) ## this would also work for 2x2 arrays
the sub_arrays is again a list of lists, containing all found subarrays, you could print them like this:
for sub_array in sub_arrays:
print()
for row in sub_array:
print(row)
I know it is a lot more code than above, just wanted to share this code.

a repeated permutation with limitations

I am trying to generate all possible combinations of certain values in an array of 15 which add up to 50.
$a = [3, 4, 1, 2, 5]
print $a.repeated_permutation(15).to_a
In this case,
[2,2,2,2,4,4,4,4,4,4,4,4,4,3,3]
[2,2,2,4,2,4,4,4,4,4,4,4,4,3,3]
[2,2,4,2,2,4,4,4,4,4,4,4,4,3,3]
are all possible answers.
After some investigation I realize the code to do this is a bit over my head, but I will leave the question up if it might help someone else.
For some reference as to what I am working on, Project Euler, problem 114. It's pretty difficult, and so I am attempting to solve only a single case where my 50-space-long grid is filled only with 3-unit-long blocks. The blocks must be separated by at least one blank, so I am counting the blocks as 4. This (with some tweaking, which I have left out as this is confusing enough already) allows for twelve blocks plus three single blanks, or a maximum of fifteen elements.
Approach
I think recursion is the way to go here, where your recursive method looks like this:
def recurse(n,t)
where
n is the number of elements required; and
t is the required total.
If we let #arr be the array of integers you are given, recurse(n,t) returns an array of all permutations of n elements from #arr that sum to t.
Assumption
I have assumed that the elements of #arr are non-negative integers, sorted by size, but the method can be easily modified if it includes negative integers (though performance will suffer). Without loss of generality, we can assume the elements of #arr are unique, sorted by increasing magnitude.
Code
def recurse(n,t)
if n == 1
#arr.include?(t) ? [[t]] : nil
else
#arr.each_with_object([]) do |i,a|
break if i > t # as elements of #arr are non-decreasing
if (ret = recurse(n-1,t-i))
ret.each { |b| a << [i,*b] }
end
end
end
end
Examples
#arr = [3, 4, 1, 2, 5].sort
#=> [1, 2, 3, 4, 5]
recurse(1,4)
#=> [[4]]
recurse(2,6)
#=> [[1, 5], [2, 4], [3, 3], [4, 2], [5, 1]]
recurse(3,10)
#=> [[1, 4, 5], [1, 5, 4], [2, 3, 5], [2, 4, 4], [2, 5, 3],
# [3, 2, 5], [3, 3, 4], [3, 4, 3], [3, 5, 2], [4, 1, 5],
# [4, 2, 4], [4, 3, 3], [4, 4, 2], [4, 5, 1], [5, 1, 4],
# [5, 2, 3], [5, 3, 2], [5, 4, 1]]
recurse(3,50)
#=> []
Improvement
We can do better, however, by first computing all combinations, and then computing the permutations of each of those combinations.
def combo_recurse(n,t,last=0)
ndx = #arr.index { |i| i >= last }
return nil if ndx.nil?
arr_above = #arr[ndx..-1]
if n == 1
arr_above.include?(t) ? [[t]] : nil
else
arr_above.each_with_object([]) do |i,a|
break if i > t # as elements of #arr are non-decreasing
if (ret = combo_recurse(n-1,t-i,i))
ret.each { |b| a << [i,*b] }
end
end
end
end
combo_recurse(1,4)
#=> [[4]]
combo_recurse(2,6)
#=> [[1, 5], [2, 4], [3, 3]]
combo_recurse(3,10)
#=> [[1, 4, 5], [2, 3, 5], [2, 4, 4], [3, 3, 4]]
combo_recurse(3,50)
#=> []
combo_recurse(15,50).size
#=> 132
combo_recurse(15,50).first(5)
#=> [[1, 1, 1, 1, 1, 1, 4, 5, 5, 5, 5, 5, 5, 5, 5],
# [1, 1, 1, 1, 1, 2, 3, 5, 5, 5, 5, 5, 5, 5, 5],
# [1, 1, 1, 1, 1, 2, 4, 4, 5, 5, 5, 5, 5, 5, 5],
# [1, 1, 1, 1, 1, 3, 3, 4, 5, 5, 5, 5, 5, 5, 5],
# [1, 1, 1, 1, 1, 3, 4, 4, 4, 5, 5, 5, 5, 5, 5]]
We can then compute the permutations from the combinations:
combo_recurse(2,6).flat_map { |a| a.permutation(a.size).to_a }.uniq
#=> [[1, 5], [5, 1], [2, 4], [4, 2], [3, 3]]
combo_recurse(3,10).flat_map { |a| a.permutation(a.size).to_a }.uniq
#=> [[1, 4, 5], [1, 5, 4], [4, 1, 5], [4, 5, 1], [5, 1, 4],
# [5, 4, 1], [2, 3, 5], [2, 5, 3], [3, 2, 5], [3, 5, 2],
# [5, 2, 3], [5, 3, 2], [2, 4, 4], [4, 2, 4], [4, 4, 2],
# [3, 3, 4], [3, 4, 3], [4, 3, 3]]
We can approximate the number of permutations for (15,50) (it will be somewhat high because uniq is not applied):
def factorial(n)
(1..n).reduce :*
end
Math.log10 combo_recurse(15,50).reduce(1) { |t,a| t*factorial(a.size) }
#=> 1599.3779486682888
That is, the result has about 1,600 digits. What platform will you be running this on?

Resources