The main goal is to find a periodic sequence in an array with bash,for example :
{ 2, 5, 7, 8, 2, 6, 5, 3, 5, 4, 2, 5, 7, 8, 2, 6, 5, 3, 5, 4, 2, 5, 7, 8, 2, 6, 5, 3, 5, 4 }
or { 2, 5, 6, 3, 4, 2, 5, 6, 3, 4, 2, 5, 6, 3, 4, 2, 5, 6, 3, 4 }
which must return as identified sequence for the two example
{ 2, 5, 7, 8, 2, 6, 5, 3, 5, 4 } and { 2, 5, 6, 3, 4 }
I tried with a list and a sub-list made of two arrays but with no success.
I must be missing something in my loops . I think to the "tortoise and hare" algorithm as an alternative but i miss some knowledge in bash commands to implement it .
I prefer to post my second try with tortoise and hare as the first seem to be a useless try :
#!/bin/bash
declare -A array=( 1, 2, 3, 1, 2, 3, 1, 2, 3 )
declare -A found=()
loop="notfound"
tortoise=`echo ${array[0]}`
hare=`echo ${array[0]}`
found[0]=`echo ${array[0]}`
while ( $loop == "notfound" )
do
for ((i=1;i=`echo ${#array[#]}`;i++))
do
if (( `echo ${array[$#]}` == $hare ))
then
echo "no loop found"
exit 0
fi
hare=`echo ${array[$i]}`
if (( `echo ${array[$#]}` == $hare ))
then
echo "no loop found"
exit 0
fi
hare=`echo ${array[$(($i+1))]}`
tortoise=`echo ${array[$i]}`
found[$i]=`echo ${array[$i]}`
if (( $hare == $tortoise ))
then
loop="found"
printf "$found[#]}"
fi
done
done
I got errors on associative array needing indice
Given an array a of single decimal digits
a=(2 5 7 8 2 6 5 3 5 4 2 5 7 8 2 6 5 3 5 4 2 5 7 8 2 6 5 3 5 4)
then using regular expression backsubstitution, for example in perl
printf '%d' "${a[#]}" | perl -lne 'print $1 if /^(\d+)\1+/'
2578265354
Testing with an incomplete sequence
a=(1 2 3 1 2 3 1 2)
printf '%d' "${a[#]}" | perl -lne 'print $1 if /^(\d+)\1+/'
123
If you only want complete repeats, add a $ line anchor to the RE, /^(\d+)\1+$/
Now, if you want to identify the longest subsequence that is "most nearly" repeated, that's a little trickier. For example, in the case of your 250-digit sequence, there is a 118-digit subsequence, repeated 2 times (with 16 characters left over), whereas your expected output is a 13-digit subsequence (repeated 19 times, with 3 digits left over). So you want an algorithm that is "greedy but not too greedy".
One (hopefully not too inefficient) way to do that would be to successively remove trailing digits until an anchored match is obtained i.e. the entire remaining sequence s* may be represented as n x t for some subsequence t. In perl, we can write that as a simple loop
perl -lne 'while (! s/^(\d+)\1+$/$1/) {chop $_}; print'
Testing with your 250-digit sequence:
a=( 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 2 1 2 0 0 2 0 2 2 2 1 1 0 )
Then
printf '%d' "${a[#]}" | perl -lne 'while (! s/^(\d+)\1+$/$1/) {chop $_}; print'
1102120020222
NOTE: this will fail to terminate if the string is exhausted before a match is found; if that's a possibility, you will need to test for that and break out of the while loop.
I tested this only with the inputs you provided.
assumptions - pattern to match always starts at the beginning of the array and repeats there after.
#!/bin/bash
#arr=(2 5 7 8 2 6 5 3 5 4 2 5 7 8 2 6 5 3 5 4 2 5 7 8 2 6 5 3 5 4)
arr=(2 5 6 3 4 2 5 6 3 4 2 5 6 3 4 2 5 6 3 4)
echo ${arr[#]}
n=${#arr[*]}
match=0
in_pattern=false
print_array()
{
local first=$1
local last=$2
local i
for ((i=first; i<=last; i++));do
printf "%d " ${arr[i]}
done
printf "\n"
}
i=0
start=0
end=0
j=$((i+1))
while (( j < n )); do
#echo "arr[$i] ${arr[i]} arr[$j] ${arr[j]}"
if [[ ${arr[i]} -ne ${arr[j]} ]];then
if [[ $match -ge 1 ]];then
echo "arr[$i] != arr[$j]"
echo "pattern doesnt repeat after match # $match"
exit 1
fi
((j++))
i=0
in_pattern=false
continue
fi
if $in_pattern ; then
if [[ $i -eq $end ]];then
((match++))
end_match=$j
echo "match # $match matched from $start -> $end and $start_match -> $end_match"
print_array $start $end
print_array $start_match $end_match
((j++))
i=0
in_pattern=false
continue
fi
else
if [[ $match -eq 0 ]];then
end=$((j-1))
fi
start_match=$j
in_pattern=true
#echo "trying to match from start $start end $end to start_match $start_match"
fi
((i++))
((j++))
done
output with first array -
./sequence.sh
2 5 7 8 2 6 5 3 5 4 2 5 7 8 2 6 5 3 5 4 2 5 7 8 2 6 5 3 5 4
match # 1 matched from 0 -> 9 and 10 -> 19
2 5 7 8 2 6 5 3 5 4
2 5 7 8 2 6 5 3 5 4
match # 2 matched from 0 -> 9 and 20 -> 29
2 5 7 8 2 6 5 3 5 4
2nd array -
/sequence.sh
2 5 6 3 4 2 5 6 3 4 2 5 6 3 4 2 5 6 3 4
match # 1 matched from 0 -> 4 and 5 -> 9
2 5 6 3 4
2 5 6 3 4
match # 2 matched from 0 -> 4 and 10 -> 14
2 5 6 3 4
2 5 6 3 4
match # 3 matched from 0 -> 4 and 15 -> 19
2 5 6 3 4
2 5 6 3 4
Related
I have the following dataset about the choices of different car brands and their attributes. I would like to create a matrix based on each attribute of the cars.
RespNum Task Concept Make Exterior.Design Interior.design
1 100086500 1 1 3 2 3
2 100086500 1 2 1 3 2
3 100086500 1 3 4 1 1
4 100086500 1 4 0 0 0
5 100086500 2 1 1 3 2
6 100086500 2 2 5 1 3
Driving.performance Driving.attributes Comfort Practibility Safety
1 1 1 1 3 3
2 3 3 3 2 1
3 2 2 2 1 2
4 0 0 0 0 0
5 3 2 1 1 3
6 1 3 3 3 2
Quality Equipment Sustainability Economy Price Response
1 2 1 1 3 1 0
2 1 3 3 1 3 0
3 3 2 2 2 2 1
4 0 0 0 0 0 0
5 3 2 1 1 4 0
6 1 3 3 3 8 0
I am using the function:
Make = attribcoding(6,4,'Other')
The first input (6) is the number of levels, the second (4) is the column position in the dataset, and the last ('Other') is the name of the outside option. However, I get the following error message:
Error in dimnames(x) <- dn :
length of 'dimnames' [2] not equal to array extent
I am writing code in Forth that should create a 12x12 array of random numbers from 1 to 8.
create big_array 144 allocate drop
: reset_array big_array 144 0 fill ;
reset_array
variable rnd here rnd !
: random rnd # 31421 * 6927 + dup rnd ! ;
: choose random um* nip ;
: random_fill 144 1 do 8 choose big_array i + c! loop ;
random_fill
: Array_# 12 * + big_array swap + c# ;
: show_small_array cr 12 0 do 12 0 do i j Array_# 5 u.r loop cr loop ;
show_small_array
However, I notice that elements 128 to 131 of my array are always much larger than expected:
0 4 0 4 2 6 0 5 2 5 7 3
6 3 7 3 7 7 3 1 5 0 6 1
0 3 3 0 3 1 0 7 2 0 4 5
3 7 6 6 2 1 0 2 3 4 2 7
4 7 1 5 3 5 7 2 3 5 3 6
3 0 6 4 1 3 3 2 5 4 4 7
3 2 1 4 3 4 3 7 2 6 5 5
2 4 4 3 4 5 4 4 6 5 6 0
2 5 2 7 3 1 5 0 1 4 6 7
2 0 3 3 0 7 3 6 4 1 3 6
0 1 1 6 0 3 0 2 169 112 41 70
7 2 3 1 2 2 7 6 0 5 1 2
Moreover, when I try to change the value of these elements individually, this causes the other three elements to change value. For example, if I code:
9 choose big_array 128 + c!
then the array will become:
0 4 0 4 2 6 0 5 2 5 7 3
6 3 7 3 7 7 3 1 5 0 6 1
0 3 3 0 3 1 0 7 2 0 4 5
3 7 6 6 2 1 0 2 3 4 2 7
4 7 1 5 3 5 7 2 3 5 3 6
3 0 6 4 1 3 3 2 5 4 4 7
3 2 1 4 3 4 3 7 2 6 5 5
2 4 4 3 4 5 4 4 6 5 6 0
2 5 2 7 3 1 5 0 1 4 6 7
2 0 3 3 0 7 3 6 4 1 3 6
0 1 1 6 0 3 0 2 2 12 194 69
7 2 3 1 2 2 7 6 0 5 1 2
Do you have any idea why these specific elements are always impacted and if there is a way to prevent this?
Better readability and less error prone: 144 allocate ⇨ 144 chars allocate
A mistake: create big_array 144 allocate drop ⇨ create big_array 144 chars allot
A mistake: random um* nip ⇨ random swap mod
A mistake: 144 1 do ⇨ 144 0 do
An excessive operation: big_array swap + ⇨ big_array +
And add the stack comments, please. Especially, when you ask for help.
Do you have any idea why these specific elements are always impacted and if there is a way to prevent this?
Since you try to use memory in the dictionary space without reserving it. This memory is used by the Forth system.
Let's say we have array
0 1 2 3 4 5 8 7 8 9
There are two indexes that have value 8:
(i.10) ([#~8={) 0 1 2 3 4 5 8 7 8 9
6 8
Is there any shorter way to get this result? May be some built-in verb.
But more important. What about higher dimensions?
Let's say we have matrix 5x4
1 2 3 4 5
2 3 4 5 6
3 4 5 6 7
4 5 6 7 8
I want to find out what are coordinates with value 6.
I want to get result such (there are three coordinates):
4 1
3 2
2 3
It's pretty basic task and I think it should exist some simple solution.
The same in three dimensions?
Thank you
Using Sparse array functionality ($.) provides a very fast and lean solution that also works for multiple dimensions.
]a=: 5 ]\ 1 + i. 8
1 2 3 4 5
2 3 4 5 6
3 4 5 6 7
4 5 6 7 8
6 = a
0 0 0 0 0
0 0 0 0 1
0 0 0 1 0
0 0 1 0 0
4 $. $. 6 = a
1 4
2 3
3 2
Tacitly:
getCoords=: 4 $. $.
getCoords 6 = a ,: a
0 1 4
0 2 3
0 3 2
1 1 4
1 2 3
1 3 2
Verb indices I. almost does the job.
When you have a simple list, I.'s use is straightforward:
I. 8 = 0 1 2 3 4 5 8 7 8 9
6 8
For higher order matrices you can pair it with antibase #: to get the coordinates in base $ matrix. Eg:
]a =: 4 5 $ 1 2 3 4 5 2 3 4 5 6 3 4 5 6 7 4 5 6 7 8
1 2 3 4 5
2 3 4 5 6
3 4 5 6 7
4 5 6 7 8
I. 6 = ,a
9 13 17
($a) #: 9 13 17
1 4
2 3
3 2
Similarly, for any number of dimensions: flatten (,), compare (=), get indices (I.) and convert coordinates (($a)&#:):
]coords =: ($a) #: I. 5 = , a =: ? 5 6 7 $ 10
0 0 2
0 2 1
0 2 3
...
(<"1 coords) { a
5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5
By the way, you can write I. x = y as x (I.#:=) y for extra performance. It is special code for
indices where x f y
can any one help me to Vectorized this loop.
i have large Matrix and i want to replace all the pixel values whose length is less then some threshold Value For simplicity lets say
a = randi([1 5],10,10);
for i = 1:length(a)
someMat=a(a==i);
if length(someMat)<20
a(a==i)=0;
end
end
but its killing me.
Example:
a = randi([1 5],10,10)
a =
5 2 1 5 5 5 2 2 3 2
3 3 5 4 4 4 3 1 1 5
5 1 3 5 3 3 4 1 3 1
3 1 5 3 2 5 1 1 5 1
1 1 4 3 4 3 4 4 5 1
1 4 3 5 1 1 2 2 2 1
3 3 5 2 4 1 1 3 2 4
4 1 5 3 4 5 3 4 3 3
5 3 5 5 4 3 1 3 4 1
4 1 1 3 5 5 1 3 3 5
Result for Thresold 20
5 0 1 5 5 5 0 0 3 0
3 3 5 0 0 0 3 1 1 5
5 1 3 5 3 3 0 1 3 1
3 1 5 3 0 5 1 1 5 1
1 1 0 3 0 3 0 0 5 1
1 0 3 5 1 1 0 0 0 1
3 3 5 0 0 1 1 3 0 0
0 1 5 3 0 5 3 0 3 3
5 3 5 5 0 3 1 3 0 1
0 1 1 3 5 5 1 3 3 5
length of pixel 4 was 17
length of pixel 2 was 10
i try it by some thing like
[nVal Index] = histc(a(:),unique(a)); %
nVal(nVal>20) = 1; % just some threshold value and assigning by some Number may be zero as well
But I dont Know how to replace the Index Values of the corresponding Pixal and apply reshape to get it in original form. Here Even i am not sure that i will get the same Matrix With Reshape . Please Help me.....
thanks
I think this does what you want:
threshold_length = 20;
replace_value = 0;
u = unique(a); %// values of a
h = histc(a(:), u); %// count for each value
r = u(h<threshold_length); %// values to be removed
a(ismember(a,r)) = replace_value; %// remove those values
I see #LuisMendo arrived at mostly the same solution quicker than I did, but an alternative to using ismember is to use more of what unique gives you:
threshold = 20;
[vals, ~, ix] = unique(a); % capture the values and their indices
counts = histc(a(:), vals); % count the occurrences of each value
vals(counts<threshold) = 0; % zero the values that aren't common enough
a(:) = vals(ix); % recreate the matrix with updated values
I have this dataframe:
df <- data.frame(subject = c(rep("one", 20), c(rep("two", 20))),
score1 = sample(1:3, 40, replace=T),
score2 = sample(1:6, 40, replace=T),
score3 = sample(1:3, 40, replace=T),
score4 = sample(1:4, 40, replace=T))
subject score1 score2 score3 score4
1 one 2 4 2 2
2 one 3 3 1 2
3 one 1 2 1 3
4 one 3 4 1 2
5 one 1 2 2 3
6 one 1 5 2 4
7 one 2 5 3 2
8 one 1 5 1 3
9 one 3 5 2 2
10 one 2 3 3 4
11 one 3 2 1 3
12 one 2 5 2 1
13 one 2 4 1 4
14 one 2 2 1 3
15 one 1 3 1 4
16 one 1 6 1 3
17 one 3 4 2 2
18 one 3 2 1 3
19 one 2 5 3 1
20 one 3 6 2 1
21 two 1 6 3 4
22 two 1 2 1 2
23 two 3 2 1 2
24 two 1 2 2 1
25 two 2 3 1 3
26 two 1 5 3 3
27 two 2 4 1 4
28 two 2 6 2 4
29 two 1 6 2 2
30 two 1 5 1 4
31 two 2 1 2 4
32 two 3 6 1 1
33 two 1 1 3 1
34 two 2 4 2 3
35 two 2 1 3 2
36 two 2 3 1 3
37 two 1 2 3 4
38 two 3 5 2 2
39 two 2 1 3 4
40 two 2 1 1 3
Note that the scores have different ranges of values. Score 1 ranges from 1-3, score 2 from -6, score 3 from 1-3, score 4 from 1-4
I'm trying to reshape data like this:
library(reshape2)
dfMelt <- melt(df, id.vars="subject")
acast(dfMelt, subject ~ value ~ variable)
Aggregation function missing: defaulting to length
, , score1
1 2 3 4 5 6
one 6 7 7 0 0 0
two 8 9 3 0 0 0
, , score2
1 2 3 4 5 6
one 0 5 3 4 6 2
two 5 4 2 2 3 4
, , score3
1 2 3 4 5 6
one 10 7 3 0 0 0
two 8 6 6 0 0 0
, , score4
1 2 3 4 5 6
one 3 6 7 4 0 0
two 3 5 5 7 0 0
Note that the output array includes scores as "0" if they are missing. Is there any way to stop these missing scores being outputted by acast?
In this case, you might do better sticking to base R's table feature. I'm not sure that you can have an irregular array like you are looking for.
For example:
> lapply(df[-1], function(x) table(df[[1]], x))
$score1
x
1 2 3
one 9 6 5
two 11 4 5
$score2
x
1 2 3 4 5 6
one 2 5 4 3 3 3
two 4 2 2 3 4 5
$score3
x
1 2 3
one 9 5 6
two 4 11 5
$score4
x
1 2 3 4
one 4 4 8 4
two 2 6 5 7
Or, using your "long" data:
with(dfMelt, by(dfMelt, variable,
FUN = function(x) table(x[["subject"]], x[["value"]])))
Since each "score" subset is going to have a different shape, you will not be able to preserve the array structure. One option is to use lists of two-dim arrays or data.frames. eg:
# your original acast call
res <- acast(dfMelt, subject ~ value ~ variable)
# remove any columns that are all zero
apply(res, 3, function(x) x[, apply(x, 2, sum)!=0] )
Which gives:
$score1
1 2 3
one 7 8 5
two 6 8 6
$score2
1 2 3 4 5 6
one 4 2 6 4 1 3
two 2 5 3 4 3 3
$score3
1 2 3
one 5 10 5
two 5 11 4
$score4
1 2 3 4
one 5 4 4 7
two 4 6 6 4