I have an array ${timearray[*]} that contains a number of times in this format
20:56 20:57 20:59 21:01 21:04
There is a second array ${productarray[*]} that contains different times
20:54 20:56 20:58 21:00 21:02
I need to get a difference between the two by subtracting time minus product. To do this I believe I need to convert these times into epoch time before subtracting, I'll then divide by 60 and round to the nearest minute. I attempted using a for loop like this to do the conversion.
arraylength=`expr ${#timearray[#]} -1`
for ((l=0; l<=$arraylength; l++))
do
epochtimearray=(`date --date="${timearray[$l]}" +%s`)
done
However the resulting epochtimearray only contains the epoch value of the last time
echo ${epochtimearray[*]}
1472331840
Does anyone see what I'm missing here or is there a better way to subtract time times.
To add an element to an array, use the += operator:
epochtimearray+=(`date --date="${timearray[$l]}" +%s`)
or set the element at the given index:
epochtimearray[l]=(`date --date="${timearray[$l]}" +%s`)
This diffs a bunch of times - I don't recommend it for large numbers of values but it's definitely better than running date in a loop
# populate couple arrays
declare -a timearray=(20:56 20:57 20:59 21:01 21:04)
declare -a productarray=(20:54 20:56 20:58 21:00 21:02)
# convert multiple times for today into epoch seconds
IFS=$'\n'
timeepochs=($(echo "${timearray[*]}"|date -f- +%s))
prodepochs=($(echo "${productarray[*]}"|date -f- +%s))
unset IFS
for ((i=0; i < ${#timeepochs[*]}; ++i)); do
echo "$i: ${timearray[i]} - ${productarray[i]} = $((timeepochs[i] - prodepochs[i])) seconds"
done
It is usually much easier to loop over array elements than to try to construct the indices:
for d in "${timearray[#]}"; do
epochtimearray+=($(date "$d" +%s))
done
But if you are using Gnu grep (which apparently you are), you can use the -f option to process all the times with a single call:
epochtimearray=($(date +%s -f-<<<"$(IFS=$'\n'; echo "${timearray[*]}")"))
But you don't actually need to construct the temporary arrays; you can put the whole thing together using a couple of standard Unix utilities:
paste -d- <(date +%s -f-<<<"$(IFS=$'\n'; echo "${timearray[*]}")") \
<(date +%s -f-<<<"$(IFS=$'\n'; echo "${productarray[*]}")") |
bc
That uses paste to combine the two lists into two vertical columns separated by a dash (-d-) and then feeds the resulting lines (which look a lot like subtractions :) ) into the calculator bc, which calculates the value of each line.
Related
I have bash array (called tenantlist_array below) populated with elements with the following format:
{3 characters}-{3-5 characters}{3-5 digits}-{2 chars}{1-2 digits}.
Example:
abc-hac101-bb0
xyz-b2blo97250-aa99
abc-b2b9912-xy00
fff-hac101-g3
Array elements are unique. Please notice the hyphen, it is part of every array element.
I need to check if the supplied string (used in the below example as a variable tenant) produces a full match with any array element - because array elements are unique, the first match is sufficient.
I am iterating over array elements using the simple code:
tenant="$1"
for k in "${tenantlist_array[#]}"; do
result=$(grep -x -- "$tenant" <<<"$k")
if [[ $result ]]; then
break
fi
done
Please note - I need to have a full string match - if, for example, the string I am searching is hac101 it must not match any array element even if can be a substring if an array element.
In other words, only the full string abc-hac101-bb0 must produce the match with the first element. Strings abc, abc-hac, b2b, 99, - must not produce the match. That's why -x parameter is with the grep call.
Now, the above code works, but I find it quite slow. I've run it with the array having 193 elements and on an ordinary notebook it takes almost 90 seconds to iterate over the array elements:
real 1m2.541s
user 0m0.500s
sys 0m24.063s
And with the 385 elements in the array, time is following:
real 2m8.618s
user 0m0.906s
sys 0m48.094s
So my question - is there a faster way to do it?
Without running any loop you can do this using glob:
tenant="$1"
[[ $(printf '\3%s\3' "${tenantlist_array[#]}") == *$'\3'"$tenant"$'\3'* ]] &&
echo "ok" || echo "no"
In printf we place a control character \3 around each element and while comparing we make sure to place \3 before & after search key.
Thanks to #arco444, the solution is astonishingly simple:
tenant="$1"
for k in "${tenantlist_array[#]}"; do
if [[ $k = "$tenant" ]]; then
result="$k"
break
fi
done
And the seed difference for the 385 member array:
real 0m0.007s
user 0m0.000s
sys 0m0.000s
Thousand times faster.
This gives an idea of how wasteful is calling grep, which needs to be avoided, if possible.
This is an alternative way of using grep that actually uses grep at most of its power.
The code to "format" the array could be completely removed just appending a \n at the end of each uuid string when creating the array the first time.
This code would also degrade much slower with the length of the strings that are compared and with the length of the array.
tenant="$1"
formatted_array=""
for k in "${tenantlist_array[#]}"; do
formatted_array="$formatted_array $i\n"
done
result=$(echo -e "$formatted_array" | grep $tenant)
I'm writing a shell script, and I have created an array containing several strings:
array=('string1' 'string2' ... 'stringN')
Now, I have a string saved in a variable, say a:
a='stringM'
And this string is part of the array.
My question is: how do I find the position of the string in the array, without having to check the terms one by one with a for loop?
Thanks in advance
The basic question is: why do you want to avoid a for loop?
Syntactical convenience and expressiveness: you want a more elegant way to conduct your search.
Performance: you're looking for the fastest way to conduct your search.
tl;dr
For performance reasons, prefer external-utility solutions to pure shell approaches; fortunately, external-utility solutions are often also the more expressive solutions:
For large element counts, they will be much faster.
While they will be slower for small element counts, the absolute time spent executing will still be low overall.
The following snippet shows you how these two goals intersect (note that both commands return the 1-based index of the item found; assumes that the array elements have no embedded newlines):
# Sample input array - adjust the number to experiment
array=( {1..300} )
# Look for the next-to-last item
itmToFind=${array[#]: -1}
# Bash `for` loop
i=1
time for a in "${array[#]}"; do
[[ $a == "$itmToFind" ]] && { echo "$i"; break; }
(( ++i ))
done
# Alternative approach: use external utility `grep`
IFS=$'\n' # make sure that "${array[*]}" expands to \n-separated elements
time grep -m1 -Fxn "$itmToFind" <<<"${array[*]}" | cut -d: -f1
grep's -m1 option means that at most one match is searched for; -Fnx means that the search term should be treated as a literal (-F), match exactly (the full line, -x), and prefix each match with its line number (-n).
With the array size given - 300 on my machine - the above commands perform about the same:
300
real 0m0.005s
user 0m0.004s
sys 0m0.000s
300
real 0m0.004s
user 0m0.002s
sys 0m0.002s
The specific threshold will vary, but:
Generally speaking, the higher the element count, the faster a solution based on an external utility such as grep will be.
For low element counts, the absolute time spent will probably not matter much, even if the external utility solution is comparatively slower.
To show one end of the extreme, here are the timings for a 1,000,000-element array (1 million elements):
1000000
real 0m13.861s
user 0m13.180s
sys 0m0.357s
1000000
real 0m1.520s
user 0m1.411s
sys 0m0.005s
without any other information on array there is no other solution than check each element, if data is sorted a search by dichotomy can be done.
otherwise another structure can be used like a hash.
for example instead of elements appending to array since bash 4.
declare -A hash
i=0;
for str in string{A..Z}; do
hash[$str]=$((i++))
done
echo "${hash['stringI']}"
Not sure if this will work for you or if this is the best way to do it avoiding a for loop, but you can try:
$ array=('string1' 'string2' 'string3' 'string4')
$ a='string3'
$ printf "%s\n" "${array[#]}" | grep -m1 -Fxn "$a" | cut -d: -f1
3
$ i=$(( $(printf "%s\n" "${array[#]}" | grep -m1 -Fxn "$a" | cut -d: -f1) - 1 ))
$ echo $i
2
Breaking it down:
printf "%s\n" "${array[#]}"
prints every element of the array separated by a new line, then we pipe it to grep to get the matching line number for the $a variable and use cut to get only the line number wihtout the match:
printf "%s\n" "${array[#]}" | grep -m1 -Fxn "$a" | cut -d: -f1
Finally, substract 1 from the matching line number returned using arithmetic expansion and store it in $i:
i=$(( $(printf "%s\n" "${array[#]}" | grep -m1 -Fxn "$a" | cut -d: -f1) - 1 ))
As others have shown way based on current array, may I suggest you could also turn the array into an associative one and have your strings as the indexes pointing to numbers.
declare -A array=(['string1']=1
['string2']=2
...
['stringN']=N )
a='stringM'
echo ${array[$a]}
I have a document which contains several names of files over which I want to use grep to gather all files with the xsd extension. When I use grep with my regex pattern, I get the correct results, about 18 of them. Now I want to store these results in an array. I used the following bash code :
targets=($(grep -i "AppointmentManagementService[\.]" AppointmentManagementService\?wsdl))
Then I print the array size :
echo ${#targets[#]}
which turns out to be 80 instead of 18 since it stored only a part of one result into an array cell. How do I make sure only one result goes into one array cell?
The results probably get split over multiple cells because a character (most likely space) is interpreted as an internal field separator.
Try executing it like this:
IFS=$'\n' targets=($(grep -i "AppointmentManagementService[\.]" AppointmentManagementService\?wsdl))
After reading up on how to initialize arrays in Bash, and seeing some basic examples put forward in blogs, there remains some uncertainties on its practical use. An interesting example perhaps would be to sort in ascending order -- list countries from A to Z in random order, one for each letter.
But in the real world, how is a Bash array applied? What is it applied to? What is the common use case for arrays? This is one area I am hoping to be familiar with. Any champions in the use of bash arrays? Please provide your example.
There are a few cases where I like to use arrays in Bash.
When I need to store a collections of strings that may contain spaces or $IFS characters.
declare -a MYARRAY=(
"This is a sentence."
"I like turtles."
"This is a test."
)
for item in "${MYARRAY[#]}"; do
echo "$item" $(echo "$item" | wc -w) words.
done
This is a sentence. 4 words.
I like turtles. 3 words.
This is a test. 4 words.
When I want to store key/value pairs, for example, short names mapped to long descriptions.
declare -A NEWARRAY=(
["sentence"]="This is a sentence."
["turtles"]="I like turtles."
["test"]="This is a test."
)
echo ${NEWARRAY["turtles"]}
echo ${NEWARRAY["test"]}
I like turtles.
This is a test.
Even if we're just storing single "word" items or numbers, arrays make it easy to count and slice our data.
# Count items in array.
$ echo "${#MYARRAY[#]}"
3
# Show indexes of array.
$ echo "${!MYARRAY[#]}"
0 1 2
# Show indexes/keys of associative array.
$ echo "${!NEWARRAY[#]}"
turtles test sentence
# Show only the second through third elements in the array.
$ echo "${MYARRAY[#]:1:2}"
I like turtles. This is a test.
Read more about Bash arrays here. Note that only Bash 4.0+ supports every operation I've listed (associative arrays, for example), but the link shows which versions introduced what.
i have to task to write a bash script which can let user to choose which array element(s) to display.
for example, the array have these element
ARRAY=(zero one two three four five six)
i want the script to be able to print out the element after user enter the array index
let say the user entered(in one line) : 1 2 6 5
then the output will be : one two six five
the index entered can be single(input: 0 output: zero) or multiple(such as above) and the output will be correspond to the index(es) entered
any help is much appreciated.
thanks
ARRAY=(zero one two three four five six)
for i in $#; do
echo -n ${ARRAY[$i]}
done
echo
Then call this script like this:
script.sh 1 2 6 5
it will print:
one two six five
You would want to use associative arrays for a generic solution, i.e. assuming your keys aren't just integers:
# syntax for declaring an associative array
declare -A myArray
# to quickly assign key value store
myArray=([1]=one [2]=two [3]=three)
# or to assign them one by one
myArray[4]='four'
# you can use strings as key values
myArray[five]=5
# Using them is straightforward
echo ${myArray[3]} # three
# Assuming input in $#
for i in 1 2 3 4 five; do
echo -n ${myArray[$i]}
done
# one two three four 5
Also, make sure you're using Bash4 (version 3 doesn't support it). See this answer and comment for more details.