Related
I have a 3d numpy array and want to generate a secondary array consisting of the minimum of each value and the values in the 10 rows directly above and 10 rows directly below (i.e each entry is the minimum value from 21 values) for each 2d array.
I've been trying to use 'numpy.clip' to deal with the edges of the array - here the range of values which the minimum is taken from should simply reduce to 10 at the values on the top/bottom of the array. I think something like 'scipy.signal.argrelmin' seems to be what I'm after.
Here's my code so far, definitely not the best way to go about it:
import numpy as np
array_3d = np.random.random_integers(50, 80, (3, 50, 18))
minimums = np.zeros(array_3d.shape)
for array_2d_index in range(len(array_3d)):
for row_index in range(len(array_3d[array_2d_index])):
for col_index in range(len(array_3d[array_2d_index][row_index])):
minimums[array_2d_index][row_index][col_index] = min(array_3d[array_2d_index][np.clip(row_index-10, 0, 49):np.clip(row_index+10, 0, 49)][col_index])
The main issue I think is that this is taking the minimum from the columns either side of each entry instead of the rows, which has been giving index errors.
Would appreciate any advice, thanks.
Approach #1
Here's one approach with np.lib.stride_tricks.as_strided -
def strided_3D_axis1(array_3d, L):
s0,s1,s2 = array_3d.strides
strided = np.lib.stride_tricks.as_strided
m,n,r = array_3d.shape
nL = n-L+1
return strided(array_3d, (m,nL,L,r),(s0,s1,s1,s2))
out = strided_3D_axis1(array_3d, L=21).min(axis=-2)
Sample run -
1) Input :
In [179]: array_3d
Out[179]:
array([[[73, 65, 51, 76, 59],
[74, 57, 75, 53, 70],
[60, 74, 52, 54, 60],
[54, 52, 62, 75, 50],
[68, 56, 68, 63, 77]],
[[62, 70, 60, 79, 74],
[70, 68, 50, 74, 57],
[63, 57, 69, 65, 54],
[63, 63, 68, 58, 60],
[70, 66, 65, 78, 78]]])
2) Strided view :
In [180]: strided_3D_axis1(array_3d, L=3)
Out[180]:
array([[[[73, 65, 51, 76, 59],
[74, 57, 75, 53, 70],
[60, 74, 52, 54, 60]],
[[74, 57, 75, 53, 70],
[60, 74, 52, 54, 60],
[54, 52, 62, 75, 50]],
[[60, 74, 52, 54, 60],
[54, 52, 62, 75, 50],
[68, 56, 68, 63, 77]]],
[[[62, 70, 60, 79, 74],
[70, 68, 50, 74, 57],
[63, 57, 69, 65, 54]],
[[70, 68, 50, 74, 57],
[63, 57, 69, 65, 54],
[63, 63, 68, 58, 60]],
[[63, 57, 69, 65, 54],
[63, 63, 68, 58, 60],
[70, 66, 65, 78, 78]]]])
3) Strided view based min :
In [181]: strided_3D_axis1(array_3d, L=3).min(axis=-2)
Out[181]:
array([[[60, 57, 51, 53, 59],
[54, 52, 52, 53, 50],
[54, 52, 52, 54, 50]],
[[62, 57, 50, 65, 54],
[63, 57, 50, 58, 54],
[63, 57, 65, 58, 54]]])
Approach #2
Here's another with broadcasting upon creating all sliding indices along the second axis -
array_3d[:,np.arange(array_3d.shape[1]-L+1)[:,None] + range(L)].min(-2)
Approach #3
Here's another using Scipy's 1D minimum filter -
from scipy.ndimage.filters import minimum_filter1d as minf
L = 21
hL = (L-1)//2
out = minf(array_3d,L,axis=1)[:,hL:-hL]
Runtime test -
In [231]: array_3d = np.random.randint(50, 80, (3, 50, 18))
In [232]: %timeit strided_3D_axis1(array_3d, L=21).min(axis=-2)
10000 loops, best of 3: 54.2 µs per loop
In [233]: %timeit array_3d[:,np.arange(array_3d.shape[1]-L+1)[:,None] + range(L)].min(-2)
10000 loops, best of 3: 81.3 µs per loop
In [234]: L = 21
...: hL = (L-1)//2
...:
In [235]: %timeit minf(array_3d,L,axis=1)[:,hL:-hL]
10000 loops, best of 3: 32 µs per loop
How can I return an array of palindrome numbers in a given array? Not palindromic numbers like 11, 22, 33, 44, &c., but the numbers that are palindromic to other number in the same array. Assume all elements are positive and the result should not return single digit numbers (example below)
Let's say I have array = [13, 31, 51, 79, 83, 97]. Since 13 & 31 and 79 & 97 are palindromes, I want it to return array_pali = [13, 31, 79, 97]
def pali_array(array)
array_reverse = array.map{|el| el.to_s.reverse.to_i}
array_pali = array & array_reverse
return array_pali
end
My initial plan is to come up with a reverse of that array, array_reverse = array.map{|el| el.to_s.reverse.to_i} and intersect them: array & array_reverse
A problem that occur is, if I want to return array of prime numbers from 2-100 as given:
array = [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97]
and I reversed it:
array_reverse = array.map{|el| el.to_s.reverse.to_i}
=> [2, 3, 5, 7, 11, 31, 71, 91, 32, 92, 13, 73, 14, 34, 74, 35, 95, 16, 76, 17, 37, 97, 38, 98, 79]
It returns:
array & array_reverse
=> [2, 3, 5, 7, 11, 13, 17, 31, 37, 71, 73, 79, 97]
The problem with this approach:
2, 3, 5, 7, and 11 are not palindromes to other elements. The reverse of single digit number is that number itself and it causes the code to return all single digit numbers, and all palindromic number (like 11, 22, 33). It should return only [13, 17, 31, 37, 71, 73, 79, 97]
How can I make it to return only elements that are palindromic to other element in the same array?
Here's a very naive and lazy implementation. Doesn't preserve original order of elements. Should be O(N).
I hope the code is self-explanatory.
array = [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97]
require 'set'
seen_numbers = Set.new
result = []
array.each do |elem|
next if elem < 10
normal_str = elem.to_s
rev_str = normal_str.reverse
if seen_numbers.include?(rev_str)
result << rev_str.to_i
result << elem
end
seen_numbers << normal_str
end
result # => [13, 31, 17, 71, 37, 73, 79, 97]
Think this works if you want an alternative:
array = [13, 31, 51, 79, 83, 97]
array.combination(2)
.select {|pair| pair.first.to_s == pair.last.to_s.reverse }
.flatten
#=> [13, 31, 79, 97]
Use Array#combination to get every pair combo then we select only those palindrome-pairs. Then just flatten your array.
arr = [7, 13, 31, 51, 31, 60, 70, 13, 79, 83, 79, 97]
Notice that there are various duplicate values in arr.
arr.reject { |n| n < 10 || (n%10).zero? }.
group_by { |n| n.to_s.each_char.sort }.
values.
reject { |arr| arr.size == 1 }.
flat_map { |arr| arr.group_by(&:itself).values.min_by(&:size) }
#=> [13, 13, 97]
The matching values are easy enough to calculate if required.
|| (n%10).zero? was added just to speed things up a bit.
The steps are as follows.
a = arr.reject { |n| n < 10 || (n%10).zero? }
#=> [13, 31, 51, 31, 13, 79, 83, 79, 97]
b = a.group_by { |n| n.to_s.each_char.sort }
#=> {["1", "3"]=>[13, 31, 31, 13], ["1", "5"]=>[51],
# ["7", "9"]=>[79, 79, 97], ["3", "8"]=>[83]}
c = b.values
#=> [[13, 31, 31, 13], [51], [79, 79, 97], [83]]
d = c.reject { |arr| arr.size == 1 }
#=> [[13, 31, 31, 13], [79, 79, 97]]
d.flat_map { |arr| arr.group_by(&:itself).values.min_by(&:size) }
#=> [13, 13, 97]
Consider the last step. flat_map passes the first element of d to its block and sets the block variable to that value:
arr = d[0]
#=> [13, 31, 31, 13]
and performs the block calculation:
e = arr.group_by(&:itself)
#=> {13=>[13, 13], 31=>[31, 31]}
f = e.values
#=> [[13, 13], [31, 31]]
f.min_by(&:size)
#=> [13, 13]
Next,
arr = d[1]
#=> [79, 79, 97]
e = arr.group_by(&:itself)
#=> {79=>[79, 79], 97=>[97]}
f = e.values
#=> [[79, 79], [97]]
f.min_by(&:size)
#=> [97]
flat_map therefore returns
[*[13, 13], *[97]]
#=> [13, 13, 97]
I have 100 3x3x3 matrices that I would like to multiply with another large matrix of size 3x5x5 (similar to convolving one image with multiple filters, but not quite).
For the sake of explanation, this is what my large matrix looks like:
>>> x = np.arange(75).reshape(3, 5, 5)
>>> x
array([[[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19],
[20, 21, 22, 23, 24]],
[[25, 26, 27, 28, 29],
[30, 31, 32, 33, 34],
[35, 36, 37, 38, 39],
[40, 41, 42, 43, 44],
[45, 46, 47, 48, 49]],
[[50, 51, 52, 53, 54],
[55, 56, 57, 58, 59],
[60, 61, 62, 63, 64],
[65, 66, 67, 68, 69],
[70, 71, 72, 73, 74]]])
In memory, I assume all sub matrices in the large matrix are stored in contiguous locations (please correct me if I'm wrong). What I want to do is, from this 3x5x5 matrix, I want to extract 3 5x3 columns from each sub-matrix of the large matrix and then join them horizontally to get a 5x9 matrix (I apologise if this part is not clear, I can explain in more detail if need be). If I were using numpy, I'd do:
>>> k = np.hstack(np.vstack(x)[:, 0:3].reshape(3, 5, 3))
>>> k
array([[ 0, 1, 2, 25, 26, 27, 50, 51, 52],
[ 5, 6, 7, 30, 31, 32, 55, 56, 57],
[10, 11, 12, 35, 36, 37, 60, 61, 62],
[15, 16, 17, 40, 41, 42, 65, 66, 67],
[20, 21, 22, 45, 46, 47, 70, 71, 72]])
However, I'm not using python so I do not have any access to the numpy functions that I need in order to reshape the data blocks into a form I want to carry out multiplication... I can only directly call the cblas_sgemm function (from the BLAS library) in C, where k corresponds to input B.
Here's my call to cblas_sgemm:
cblas_sgemm( CblasRowMajor, CblasNoTrans, CblasTrans,
100, 5, 9,
1.0,
A, 9,
B, 9, // this is actually wrong, since I don't know how to specify the right parameter
0.0,
result, 5);
Basically, the ldb attribute is the offender here, because my data is not blocked the way I need it to be. I have tried different things, but I am not able to get cblas_sgemm to understand how I want it to read and understand my data.
In short, I don't know how to tell cblas_sgemm to read x like k.Is there a way I can smartly reshape my data in python before sending it to C, so that cblas_sgemm can work the way I want it to?
I will transpose k by setting CblasTrans, so during multiplication, B is 9x5. My matrix A is of shape 100x9. Hope that helps.
Any help would be appreciated. Thanks!
In short, I don't know how to tell cblas_sgemm to read x like k.
You can't. You'll have to make a copy.
Consider k:
In [20]: k
Out[20]:
array([[ 0, 1, 2, 25, 26, 27, 50, 51, 52],
[ 5, 6, 7, 30, 31, 32, 55, 56, 57],
[10, 11, 12, 35, 36, 37, 60, 61, 62],
[15, 16, 17, 40, 41, 42, 65, 66, 67],
[20, 21, 22, 45, 46, 47, 70, 71, 72]])
In a two-dimensional array, the spacing of the elements in memory must be the same in each axis. You know from how x was created that the consecutive elements in memory are 0, 1, 2, 3, 4, ..., but your first row of k contains 0, 1, 2, 25, 26, ..... The is no spacing between 1 and 2 (i.e. the memory address increases by the size of one element of the array), but there is a large jump in memory between 2 and 25. So you'll have to make a copy to create k.
Having said that, there is an alternative method to efficiently achieve your desired final result using a bit of reshaping (without copying) and numpy's einsum function.
Here's an example. First define x and A:
In [52]: x = np.arange(75).reshape(3, 5, 5)
In [53]: A = np.arange(90).reshape(10, 9)
Here's my understanding of what you want to achieve; A.dot(k.T) is the desired result:
In [54]: k = np.hstack(np.vstack(x)[:, 0:3].reshape(3, 5, 3))
In [55]: A.dot(k.T)
Out[55]:
array([[ 1392, 1572, 1752, 1932, 2112],
[ 3498, 4083, 4668, 5253, 5838],
[ 5604, 6594, 7584, 8574, 9564],
[ 7710, 9105, 10500, 11895, 13290],
[ 9816, 11616, 13416, 15216, 17016],
[11922, 14127, 16332, 18537, 20742],
[14028, 16638, 19248, 21858, 24468],
[16134, 19149, 22164, 25179, 28194],
[18240, 21660, 25080, 28500, 31920],
[20346, 24171, 27996, 31821, 35646]])
Here's how you can get the same result by slicing x and reshaping A:
In [56]: x2 = x[:,:,:3]
In [57]: A2 = A.reshape(-1, 3, 3)
In [58]: einsum('ijk,jlk', A2, x2)
Out[58]:
array([[ 1392, 1572, 1752, 1932, 2112],
[ 3498, 4083, 4668, 5253, 5838],
[ 5604, 6594, 7584, 8574, 9564],
[ 7710, 9105, 10500, 11895, 13290],
[ 9816, 11616, 13416, 15216, 17016],
[11922, 14127, 16332, 18537, 20742],
[14028, 16638, 19248, 21858, 24468],
[16134, 19149, 22164, 25179, 28194],
[18240, 21660, 25080, 28500, 31920],
[20346, 24171, 27996, 31821, 35646]])
I have a string of digits:
s = "12345678910"
As you can see it is the numbers 1 through 10 listed in increasing order. I want to convert it to an array of those numbers:
a = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
How can I do it?
How about this:
a = ["123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899"]
b = a.first.each_char.map {|n| n.to_i }
if b.size > 8
c = b[0..8]
c += b[9..b.size].each_slice(2).map(&:join).map(&:to_i)
end
# It would yield as follows:
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]
For later numbers beyond 99, modify existing predicate accordingly.
Assuming a monotonic sequence, here's my run at it.
input = a.first.chars
output = []
previous_int = 0
until input.empty?
temp = []
temp << input.shift until temp.join.to_i > previous_int
previous_int = temp.join.to_i
output << previous_int
end
puts output.to_s
#=> [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
Assumptions
the first (natural) number extracted from the string is the first character of the string converted to an integer;
if the number n is extracted from the string, the next number extracted, m, satisfies n <= m (i.e., the sequence is monotonically non-decreasing);
if n is extracted from the string, the next number extracted will have as few digits as possible (i.e., at most one greater than the number of digits in n); and
there is no need to check the validity of the string (e.g., "54632" is invalid).
Code
def split_it(str)
return [] if str.empty?
a = [str[0]]
offset = 1
while offset < str.size
sz = a.last.size
sz +=1 if str[offset,sz] < a.last
a << str[offset, sz]
offset += sz
end
a.map(&:to_i)
end
Examples
split_it("12345678910")
#=> [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
split_it("12343636412252891407189118901")
#=> [1, 2, 3, 4, 36, 36, 41, 225, 289, 1407, 1891, 18901]
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Why do I always get the same sequence of random numbers with rand()?
I've been experimenting with generating random numbers in C, and I've come across something weird. I don't know if it's only on my compiler but whenever I try to generate a pseudo-random number with the rand() function, it returns a very predictable number — the number generated with the parameter before plus 3.125 to be exact. It's hard to explain but here's an example.
srand(71);
int number = rand();
printf("%d", number);
This returns 270.
srand(72);
int number = rand();
printf("%d", number);
This returns 273.
srand(73);
int number = rand();
printf("%d", number);
This returns 277.
srand(74);
int number = rand();
printf("%d", number);
This returns 280.
Every eighth number is 4 higher. Otherwise it's 3.
This can't possibly be right. Is there something wrong with my compiler?
Edit: I figured it out — I created a function where I seed only once, then I loop the rand() and it generates random numbers. Thank you all!
The confusion here is about how pseudorandom number generators work.
Pseudorandom number generators like C's rand work by having a number representing the current 'state'. Every time the rand function is called, some deterministic computations are done on the 'state' number to produce the next 'state' number. Thus, if the generator is given the same input (the same 'state'), it will produce the same output.
So, when you seed the generator with srand(74), it will always generate the same string of numbers, every time. When you seed the generator with srand(75), it will generate a different string of numbers, etc.
The common way to ensure different output each time is to always provide a different seed, usually done by seeding the generator with the current time in seconds/milliseconds, e.g. srand(time(NULL)).
EDIT: Here is a Python session demonstrating this behavior. It is entirely expected.
>>> import random
If we seed the generator with the same number, it will always output the same sequence:
>>> random.seed(500)
>>> [random.randint(0, 100) for _ in xrange(20)]
[80, 95, 58, 25, 76, 37, 80, 34, 57, 79, 1, 33, 40, 29, 92, 6, 45, 31, 13, 11]
>>> random.seed(500)
>>> [random.randint(0, 100) for _ in xrange(20)]
[80, 95, 58, 25, 76, 37, 80, 34, 57, 79, 1, 33, 40, 29, 92, 6, 45, 31, 13, 11]
>>> random.seed(500)
>>> [random.randint(0, 100) for _ in xrange(20)]
[80, 95, 58, 25, 76, 37, 80, 34, 57, 79, 1, 33, 40, 29, 92, 6, 45, 31, 13, 11]
If we give it a different seed, even a slightly different one, the numbers will be totally different from the old seed, yet still the same if the same (new) seed is used:
>>> random.seed(501)
>>> [random.randint(0, 100) for _ in xrange(20)]
[64, 63, 24, 81, 33, 36, 72, 35, 95, 46, 37, 2, 76, 21, 46, 68, 47, 96, 39, 36]
>>> random.seed(501)
>>> [random.randint(0, 100) for _ in xrange(20)]
[64, 63, 24, 81, 33, 36, 72, 35, 95, 46, 37, 2, 76, 21, 46, 68, 47, 96, 39, 36]
>>> random.seed(501)
>>> [random.randint(0, 100) for _ in xrange(20)]
[64, 63, 24, 81, 33, 36, 72, 35, 95, 46, 37, 2, 76, 21, 46, 68, 47, 96, 39, 36]
How do we make our program have different behavior each time? If we supply the same seed, it will always behave the same. We can use the time.time() function, which will yield a different number each time we call it:
>>> import time
>>> time.time()
1347917648.783
>>> time.time()
1347917649.734
>>> time.time()
1347917650.835
So if we keep re-seeding it with a call to time.time(), we will get a different sequence of numbers each time, because the seed is different each time:
>>> random.seed(time.time())
>>> [random.randint(0, 100) for _ in xrange(20)]
[60, 75, 60, 26, 19, 70, 12, 87, 58, 2, 79, 74, 1, 79, 4, 39, 62, 20, 28, 19]
>>> random.seed(time.time())
>>> [random.randint(0, 100) for _ in xrange(20)]
[98, 45, 85, 1, 67, 25, 30, 88, 17, 93, 44, 17, 94, 23, 98, 32, 35, 90, 56, 35]
>>> random.seed(time.time())
>>> [random.randint(0, 100) for _ in xrange(20)]
[44, 17, 10, 98, 18, 6, 17, 15, 60, 83, 73, 67, 18, 2, 40, 76, 71, 63, 92, 5]
Of course, even better than constantly re-seeding it is to seed it once and keep going from there:
>>> random.seed(time.time())
>>> [random.randint(0, 100) for _ in xrange(20)]
[94, 80, 63, 66, 31, 94, 74, 15, 20, 29, 76, 90, 50, 84, 43, 79, 50, 18, 58, 15]
>>> [random.randint(0, 100) for _ in xrange(20)]
[30, 53, 75, 19, 35, 11, 73, 88, 3, 67, 55, 43, 37, 91, 66, 0, 9, 4, 41, 49]
>>> [random.randint(0, 100) for _ in xrange(20)]
[69, 7, 25, 68, 39, 57, 72, 51, 33, 93, 81, 89, 44, 61, 78, 77, 43, 10, 33, 8]
Every invocation of rand() returns the next number in a predefined sequence where the starting number is the seed supplied to srand(). That' why it's called a pseudo-random number generator, and not a random number generator.
rand() is implemented by a pseudo random number generator.
The distribution of numbers generated by consecutive calls to rand() have the properties of being random numbers, but the order is pre-determined.
The 'start' number is determined by the seed that you provide.
You should give a PRNG a single seed only. Providing it with multiple seeds can radically alter the randomness of the generator. In addition, providing it the same seed over and over removes all randomness.
Generating a "random" number regardless of the implementation is dependent on a divergent infinite sequence. The infinite sequence is generated using the seed of the random function and it is actually pseudo random because of its nature. This would explain to you why your number is actually very dependent on the seed that you give the function.
In some implementations the sequence is only one and the seed is the starting member of the sequence. In others there are difference sequences depending on the seed. If a seed is not provided then the seed is determined by the internal "clock".
The number is truncated when using an upper and lower bounds for your random number by respectively doing randValue % upperBound and randValue + lowerBound. Random implementation is very similar to Hash Functions. Depending on architecture the upper bound of the random value is set depending on what it the largest integer/double that it can carry out if not set lower by the user.