I have the following problem. I need to change the shape of one Numpy array to match the shape of another Numpy array by adding rows and columns.
Let's say this is the array that needs to be changed:
change_array = np.random.rand(150, 120)
And this is the reference array:
reference_array = np.random.rand(200, 170)
To match the shapes I'm adding rows and columns containing zeros, using the following function:
def match_arrays(change_array, reference_array):
cols = np.zeros((change_array.shape[0], (reference_array.shape[1] - change_array.shape[1])), dtype=np.int8)
change_array = np.append(change_array, cols, axis=1)
rows = np.zeros(((reference_array.shape[0] - change_array.shape[0]), reference_array.shape[1]), dtype=np.int8)
change_array = np.append(change_array, rows, axis=0)
return change_array
Which perfectly works and changes the shape of change_array to the shape of reference_array. However, using this method, the array needs to be copied twice in memory. I understand how Numpy needs to make a copy of the array in memory in order to have space to append the rows and columns.
As my arrays can get very large I am looking for another, more memory efficient method, that can achieve the same result. Thanks!
Here are a couple ways. In the code examples, I'll use the following arrays:
In [190]: a
Out[190]:
array([[12, 11, 15],
[16, 15, 10],
[16, 12, 13],
[11, 19, 10],
[12, 12, 11]])
In [191]: b
Out[191]:
array([[70, 82, 83, 93, 97, 55],
[50, 86, 53, 75, 75, 69],
[60, 50, 76, 52, 72, 88],
[72, 79, 66, 93, 58, 58],
[57, 92, 71, 97, 91, 50],
[60, 77, 67, 91, 91, 63],
[60, 90, 91, 50, 86, 71]])
Use numpy.pad:
In [192]: np.pad(a, [(0, b.shape[0] - a.shape[0]), (0, b.shape[1] - a.shape[1])], 'constant')
Out[192]:
array([[12, 11, 15, 0, 0, 0],
[16, 15, 10, 0, 0, 0],
[16, 12, 13, 0, 0, 0],
[11, 19, 10, 0, 0, 0],
[12, 12, 11, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0]])
Or, use a more efficient version of your function, in which the result is preallocated as an array of zeros with the same shape as reference_array, and then the values in change_array are copied into the result:
In [193]: def match_arrays(change_array, reference_array):
...: result = np.zeros(reference_array.shape, dtype=change_array.dtype)
...: nrows, ncols = change_array.shape
...: result[:nrows, :ncols] = change_array
...: return result
...:
In [194]: match_arrays(a, b)
Out[194]:
array([[12, 11, 15, 0, 0, 0],
[16, 15, 10, 0, 0, 0],
[16, 12, 13, 0, 0, 0],
[11, 19, 10, 0, 0, 0],
[12, 12, 11, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0]])
Related
I’m doing Ruby task, which is “You have an array of numbers. Your task is to sort ascending odd numbers but even numbers must be on their places. Zero isn't an odd number and you don't need to move it. If you have an empty array, you need to return it”.
Decided to split initial array: odd nums pushed into another array, sorted it and even numbers added to hash, where the key is num and its initial index is value. After that, trying to insert the even nums from hash to odd array, at initial indexes of even nums. So, the code looks like this:
def sort_array(source_array)
even_nums = Hash.new
odd_nums = []
return source_array if source_array.length == 0
source_array.each_with_index {|n, ind| even_nums[n] = ind if n.even?}
source_array.select{|n| odd_nums.push(n) if n.odd?}
odd_nums.sort!
even_nums.each do |k, v|
odd_nums.insert(v, k)
end
odd_nums
end
With small array like [5, 3, 2, 8, 1, 4, 11] it works as expected but if I pass something bigger like [84, -64, 40, 53, 5, 88, 2, 14, 29, -79, -44, -23, 20, -67, -12, 28, -28, -37, -27, -62, -54, 93, -61, 50, 65, -63, -62, 77, -16, 49, -43, 26, -73, -27, 88, -88, -62, 84, 54, 25, 25, -2, -99, 69, -23, 47, -92, 7, -62, -62, -58, -30, -75, -31, 65, -63, 16, 64, -7, -22, -6, -82]
I’m getting nils at the end of the sorted array. Like this:
[-99, -64, 40, -79, -75, -73, 2, 14, -67, -63, -44, -63, 20, -61, -12, 28, -28, -43, -37, -31, -54, -27, -27, 50, -23, -23, -7, 5, -16, 7, 25, 26, 25, 29, 47, -88, 49, 53, 54, 65, 65, -2, 69, 77, 93, nil, -92, nil, nil, 88, -58, -30, nil, nil, nil, nil, 16, 64, nil, -22, -6, -82, 84, nil, -62]
Struggling to understand, why it isn’t working with bigger arrays?
There's a fairly easy way to do this if you think about it as two operations:
def sort_array(arr)
# Extract and sort the odd values
odd = arr.select(&:odd?).sort
# Merge the sorted odd values back in
arr.map do |v|
v.odd? ? odd.shift : v
end
end
Not much to it.
You had some of the right pieces, but I think you got stuck in the weeds when it started to get overly complicated.
That can be done as follows.
def sort_odds(arr)
odd_pos = arr.each_index.select { |i| arr[i].odd? }
odd_pos.zip(odd_pos.sort_by { |i| arr[i] }).
each_with_object(arr.dup) do |(old_pos,new_pos),a|
a[new_pos] = arr[old_pos]
end
end
sort_odds [5, 3, 2, 8, 1, 4, 11]
# o o e e o e o
#=> [1, 3, 2, 8, 5, 4, 11]
The steps are as follows.
arr = [5, 3, 2, 8, 1, 4, 11]
# o o e e o e o
odd_pos = arr.each_index.select { |i| arr[i].odd? }
#=> [0, 1, 4, 6]
new_pos = odd_pos.zip(odd_pos.sort_by { |i| arr[i] })
#=> [[0, 4], [1, 1], [4, 0], [6, 6]]
new_pos.each_with_object(arr.dup) do|(old_pos,new_pos),a|
a[new_pos] = arr[old_pos]
end
#=> [1, 3, 2, 8, 5, 4, 11]
I think this other answer is simpler and more elegant than mine, but this works too. Notably, this solution would allow you to validate the position of your even numbers (for example, in a spec) by looking up the indexes and values in evens. Unless you already know what the output array should look like, this may matter when it comes time to debug the interim results.
def odd_sorted array
odds = array.select { |e| e.odd? }.sort
evens = array.each_with_index.select { |e| e.first.even? }
arr2 = Array.new(array.size)
# put even numbers in at their previous index
evens.each do |e|
arr2.each_index { |i| arr2[i] = e[0] if e[1] == i }
end
arr2.each_with_index { |e, i| arr2[i] = odds.shift if e.nil? }
end
odd_sorted [5, 3, 2, 8, 1, 4, 11]
#=> [1, 3, 2, 8, 5, 4, 11]
odd_sorted [84, -64, 40, 53, 5, 88, 2, 14, 29, -79, -44, -23, 20]
#=> [84, -64, 40, -79, -23, 88, 2, 14, 5, 29, -44, 53, 20]
The Array#map solution is definitely more elegant, but this is (in my personal opinion) more debuggable. Your mileage in that regard will certainly vary.
I'm trying to use reqwest library to download a PNG file, but when I download it I see a strange behaviour respect other programming languages like: Python.
For instance:
let content = reqwest::get("https://www.google.es/images/searchbox/desktop_searchbox_sprites302_hr.png").await?;
If I print the result as a bytes array (println!("{:?}", content.text().await?.as_bytes());
[ 191, 189, 80, 78, 71, 13, 10, 26, 10, 0, 0, 0, 13, 73, 72, 68, 82, 0, 0, 0, 40, 0, 0, 0, 82, 8, 3, 0, 0, 0, 17, 191, 189, 102, 191, 189, 0, 0, 0, 108, 80, 76, 84, 69, 0, 0, 0, 191, 189, 191, 189, 191, 189,...]
However, the result using Python requests is:
[137, 80, 78, 71, 13, 10, 26, 10, 0, 0, 0, 13, 73, 72, 68, 82, 0, 0, 0, 40, 0, 0, 0, 82, 8, 3, 0, 0, 0, 17, 153, 102, 248, ...]
In the Rust version, I see a lot of 191, 189. This sequence repeats a lot throughout the array, but in Python doesn't appear at all.
What am I doing wrong in Rust?
I see a lot of 191, 189
It's better seen as EF, BF, BD, which is the Unicode replacement character encoded as UTF-8. Binary data is not text data. You should not use text for binary data, instead use bytes.
const URL: &str = "https://www.google.es/images/searchbox/desktop_searchbox_sprites302_hr.png";
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let content = reqwest::get(URL).await?;
let bytes = content.bytes().await?;
println!("{:x?}", &bytes[..]);
Ok(())
}
[89, 50, 4e, 47, d, a, 1a, a, 0, 0, 0, d, 49, 48, 44, 52, 0, 0, 0, 28, 0, 0, 0, 52, 8, 3, 0, 0, 0, 11, 99, 66, f8, 0, 0, 0, 6c, 50, 4c, 54, 45, 0, 0, 0, 9f, ...
I have an array of arrays that looks like
time
array([array([ 0, 1, 0, 10, 12, 2011], dtype=int16),
array([ 0, 1, 0, 10, 12, 2011], dtype=int16),
array([ 0, 1, 0, 10, 12, 2011], dtype=int16), ...,
array([ 0, 59, 23, 10, 12, 2011], dtype=int16),
array([ 0, 59, 23, 10, 12, 2011], dtype=int16),
array([ 0, 59, 23, 10, 12, 2011], dtype=int16)],
dtype=object)
and I would like to transform this into something like
time
array([0:1:0 10-12-2011,
etc
0:59:23 10-12-2011])
I feel like I should be able to do this for the whole structure without having to loop through each individual row/column.
I would say you cannot avoid loops, but you can get a pretty decent result by looping through the outer array and converting your data into datetime objects. Let's say a is your array:
from datetime import datetime
results = array([datetime(*row[::-1]) for row in a])
i have a numpy array r when i used to create another array r2 out of it and turning that new array r2 to zero it also changed the original array r
I have searched around the similar questions but did not turned around the any satisfying answer for this, so please consider suggesting an appropriate answer.
Original Array:
>>> r
array([[ 0, 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10, 11],
[12, 13, 14, 15, 16, 17],
[18, 19, 20, 21, 22, 23],
[24, 25, 26, 27, 28, 29],
[30, 31, 32, 33, 34, 35]])
another numpy array from original array r2 as follows:
>>> r2 = r[:3, :3]
>>> r2
array([[ 0, 1, 2],
[ 6, 7, 8],
[12, 13, 14]])
So, When i do set new array to r2 to zero
>>> r2[:] = 0
>>> r2
array([[0, 0, 0],
[0, 0, 0],
[0, 0, 0]])
So, when i see original array then it also been looked change:
Array Changed after chanin the new array:
>>> r
array([[ 0, 0, 0, 3, 4, 5],
[ 0, 0, 0, 9, 10, 11],
[ 0, 0, 0, 15, 16, 17],
[18, 19, 20, 21, 22, 23],
[24, 25, 26, 27, 28, 29],
[30, 30, 30, 30, 30, 30]])
Happy New Years in advanced, Guys!
Explanation
r2 = r[:3, :3]
Doesn't create a new array, but renames the current array. What you need to do is known as 'deep copy'. Use, numpy.copy() to do what you need.
x = np.array([1, 2, 3])
y = x
z = np.copy(x)
x[0] = 10
x[0] == y[0]
True
x[0] == z[0]
False
Read more from,
https://het.as.utexas.edu/HET/Software/Numpy/reference/generated/numpy.copy.html
If I have the following data:
A = np.random.random((3, 4, 5))
# np.all(indices < A.shape) is true
indices = np.array([
[0, 0, 0],
[1, 2, 4],
...
[2, 3, 4]
])
How can I use each row of indices as a set of axis indices into A to give the following?
B = np.array([
A[0, 0, 0],
A[1, 2, 4],
...
A[2, 3, 4]
])
Here's a 2d example:
In [1]: A=np.arange(10,22).reshape(3,4)
In [2]: A
Out[2]:
array([[10, 11, 12, 13],
[14, 15, 16, 17],
[18, 19, 20, 21]])
In [3]: ind=np.array([[0,1],[1,3],[2,0],[0,2]])
In [4]: ind
Out[4]:
array([[0, 1],
[1, 3],
[2, 0],
[0, 2]])
In [5]: A[ind[:,0],ind[:,1]]
Out[5]: array([11, 17, 18, 12])
or for your variables,
A[indices[:,0], indices[:,1], indices[:,2]]
Or more generally:
In [8]: tuple(ind.T)
Out[8]: (array([0, 1, 2, 0]), array([1, 3, 0, 2]))
In [9]: A[tuple(ind.T)]
Out[9]: array([11, 17, 18, 12])
This is based on the idea that A[a,b] is the same as A[(a,b)]. And when a and b are matching lists or arrays, it selects values by pairing them up, roughly the same as
[A[i,j] for i,j in zip(a,b)]
For a product like indexing, the index arrays need to have more dimensions. ix_ is a handy way of generating such arrays:
In [53]: np.ix_(ind[:,0],ind[:,1])
Out[53]:
(array([[0],
[1],
[2],
[0]]), array([[1, 3, 0, 2]]))
In [54]: A[np.ix_(ind[:,0],ind[:,1])]
Out[54]:
array([[11, 13, 10, 12],
[15, 17, 14, 16],
[19, 21, 18, 20],
[11, 13, 10, 12]])
In [56]: A[ind[:,[0]],ind[:,1]]
Out[56]:
array([[11, 13, 10, 12],
[15, 17, 14, 16],
[19, 21, 18, 20],
[11, 13, 10, 12]])
You could use np.ravel_multi_index to generate the linear indices and then extract the selective elements from A with linear-indexing using np.take like so -
np.take(A,np.ravel_multi_index(indices.T,A.shape))