I'm trying to print a word from variable with multiple elements using array. Here is a script:
location=($(here mysql query to extract name of locations))
for i in "${location[#]}"; do
echo "$i"
done
But these location name contains spaces: MUMBAI - BORIVALI, DELHI - LAJPATNAGAR and so on. So it prints like: Output in debug mode:
+ for i in '"${LOCATION[#]}"'
+ echo DELHI
DELHI
+ for i in '"${LOCATION[#]}"'
+ echo -
-
+ for i in '"${LOCATION[#]}"'
+ echo LAJPATNAGAR
LAJPATNAGAR
+ for i in '"${LOCATION[#]}"'
+ echo MUMBAI
MUMBAI
+ for i in '"${LOCATION[#]}"'
+ echo -
-
+ for i in '"${LOCATION[#]}"'
+ echo BORIVLI
BORIVLI
I've tried with double quotes:
location=("$(here mysql query to extract name of locations)")
then output is
+ for i in '"${LOCATION[#]}"'
+ echo 'DELHI - LAJPATNAGAR
MUMBAI - BORIVLI'
together.
I want output:
+ echo 'DELHI - LAJPATNAGAR'
DELHI - LAJPATNAGAR
+ echo 'MUMBAI - BORIVLI'
MUMBAI - BORIVLI
If the locations don't contain newlines, you can try
while read -r location ; do
echo "$location"
done < <(mysql ...)
read reads line by line if given just one parameter. By default, it reads from stdin, but with process substitution, we redirected output of the mysql command to it.
Try setting IFS to newline-only:
IFS="$(echo)"
location=($(here mysql query to extract name of locations))
for i in "${location[#]}"; do
echo "$i"
done
Example with test.txt:
A B C
D E
F
Run the above code, replacing the MySQL query with cat test.txt. Output:
A B C
D E
F
If you drop the IFS="$(echo)" line, you will get each letter on a separate line.
Just count 3 fields:
location=($(here mysql query to extract name of locations))
for ((i=0; i <= ${#location}; i+=3))
do
echo "${location[i]} ${location[i+1]} ${location[i+2]}"
done
Related
I'm working on a file that contains info such as:
service.txt
service1 - info1
service2 - info2
service3 - info3
...
I added each line of the file to an array. Subsequently with the select command I want to query the elements of the array displaying only the "serviceN" and display the "info" once the element has been selected.
At the moment I can't cut the line to display only the "service"
`
#File example
#service.txt
#service1 - info1
#service2 - info2
#...
#serviceN - infoN
#!/bin/bash
file='service.txt'
line=()
while read -r line; do
line+=($line)
done < $file
echo "Select the service..."
select line in ${line[#]}; do # how can i use "cut" here?
echo $line
break
done
exit 0
You don't need the cut for this particular problem. In pure bash:
#!/bin/bash
readarray -t lines < service.txt
echo "Select the service..." >&2
select service in "${lines[#]%%[[:blank:]]*}"; do
echo "$service"
break
done
For the "${lines[#]%%[[:blank:]]*}", see Shell Parameter Expansion, paragraph starting with ${parameter%word}.
I have 2 files. I need to add the count of rows of the both and write it to 3rd file.
If the content of 3rd file is >25 , i need to print error.txt and if =<25 , i need to print success.txt
Scenario:
file 1(p.csv) , count: 20
file 2 (m.csv), count : 5
file 3 , count 25
--->should print error.txt
I have tried the below code, but file 3 is not getting generated with expected output.
file_1=$(cat p.csv | wc -l)
echo $file_1
file_2=$(cat m.csv | wc -l)
echo $file_2
file_3 = $(`expr $(file_1) + $(file_2)`)
echo $file_3 > file_3.txt
if [ $file_3 -gt 25 ]; then
touch error.txt
else
touch success.txt
fi
Error message:
20
5
count_test.sh: line 16: file_1: command not found
count_test.sh: line 16: file_2: command not found
expr: syntax error
count_test.sh: line 16: file_3: command not found
count_test.sh: line 20: [: -gt: unary operator expected
Some fixes are required, here is my version:
#!/bin/bash
file_1=$(wc -l p.csv | cut -d' ' -f1)
echo "file_1=$file_1"
file_2=$(wc -l m.csv | cut -d' ' -f1)
echo "file_2=$file_2"
file_3=$(( file_1 + file_2 ))
echo "file_3=$file_3"
if [[ $file_3 -gt 25 ]]
then
echo "ERROR"
touch error.txt
else
echo "Success"
touch success.txt
fi
The arithmetic line was modified to use the $(( )) syntax.
you do not need file_3.txt for the if. If you required it for some other reason, you can put bach the echo "$file_3" > file_3.txt" line.
I added a couple echo statements for debugging purposes.
Some errors you made:
echo $file_1
# Not really wrong, but use quotes
echo "$file_1"
Don't use spaces around = in assignment, don't use backtics and use {} around variables
file_3 = $(`expr $(file_1) + $(file_2)`)
# change this to
file_3=$(expr ${file_1} + ${file_2})
# and consider using (( a = b + c )) for calculations
When you only want the final results, consider
if (( $(cat [pm].csv | wc -l) > 25 )); then
touch error.txt
else
touch success.txt
fi
New to shell scripting..
I have a huge csv file, with a varying length f11, like
"000000aaad000000bhb200000uwwed..."
"000000aba200000bbrb2000000wwqr00000caba2000000bhbd000000qwew..."
.
.
After splitting the string in size of 10, I need 6-9 characters. then I have to join them back using delimiter '|' like
0aaa|0bhb|uwwe...
0aba|bbrb|0wwq|caba|0bhb|0qwe...
and join the processed f11 with other fields
this is the time taken for processing 10k records ->
real 4m43.506s
user 0m12.366s
sys 0m12.131s
20K records ->
real 5m20.244s
user 2m21.591s
sys 3m20.042s
80K records (around 3.7Million f11 split and merge with '|') ->
real 21m18.854s
user 9m41.944s
sys 13m29.019s
My expected time is 30mins for processing 650K records (around 56Million f11 split and merge). Any way to optimize ?
while read -r line1; do
f10=$( echo $line1 | cut -d',' -f1,2,3,4,5,7,9,10)
echo $f10 >> $path/other_fields
f11=$( echo $line1 | cut -d',' -f11 )
f11_trim=$(echo "$f11" | tr -d '"')
echo $f11_trim | fold -w10 > $path/f11_extract
cat $path/f11_extract | awk '{print $1}' | cut -c6-9 >> $path/str_list_trim
arr=($(cat $path/str_list_trim))
printf "%s|" ${arr[#]} >> $path/str_list_serialized
printf '\n' >> $path/str_list_serialized
arr=()
rm $path/f11_extract
rm $path/str_list_trim
done < $input
sed -i 's/.$//' $path/str_list_serialized
sed -i 's/\(.*\)/"\1"/g' $path/str_list_serialized
paste -d "," $path/other_fields $path/str_list_serialized > $path/final_out
Your code is not time-efficient due to:
invoking multiple commands including awk within the loop.
generating many intermediate temporal files.
You can do the job just with awk:
awk -F, -v OFS="," ' # assign input/output field separator to a comma
{
len = length($11) # length of the 11th field
s = ""; d = "" # clear output string and the delimiter
for (i = 1; i <= len / 10; i++) { # iterate over the 11th field
s = s d substr($11, (i - 1) * 10 + 6, 4) # concatenate 6-9th substring of 10 characters long chunks
d = "|" # set the delimiter to a pipe character
}
$11 = "\"" s "\"" # assign the 11th field to the generated string
} 1' "$input" # the final "1" tells awk to print all fields
Example of the input:
1,2,3,4,5,6,7,8,9,10,000000aaad000000bhb200000uwwed
1,2,3,4,5,6,7,8,9,10,000000aba200000bbrb2000000wwqr00000caba2000000bhbd000000qwew
Output:
1,2,3,4,5,6,7,8,9,10,"0aaa|0bhb|uwwe"
1,2,3,4,5,6,7,8,9,10,"0aba|bbrb|0wwq|caba|0bhb|0qwe"
I am trying to parse the output on svn info without resorting to an external shell command like sed or awk. This is purely academic as I know I could do this in a heartbeat with those tools.
Output I am parsing is:
Path: .
URL: svn://brantwinter#192.168.10.222/handbrake_batch/trunk/handbrake
Repository Root: svn://ilium007#192.168.10.222/handbrake_batch
Repository UUID: 99c2cca7-102b-e211-ab20-02060a000c0b
Revision: 6
Node Kind: directory
Schedule: normal
Last Changed Author: ilium007
Last Changed Rev: 6
Last Changed Date: 2012-11-10 19:00:35 +1000 (Sat, 10 Nov 2012)
Here is my code:
#!/bin/bash
#set -x
OLD_IFS="$IFS"
IFS=$'\r\n'
# Get the output from svn info into an array
SVN_INFO_ARR=(`svn info`)
COUNT=0
for i in ${SVN_INFO_ARR[#]}; do
echo $COUNT
echo "$i"
(( COUNT++ ))
done
# Get the element that says "Revision: 6"
REV_ARR=${SVN_INFO_ARR[4]}
# Testing the loop over what should be a two element array
COUNT=0
for i in ${REV_ARR[#]}; do
echo $COUNT
echo "$i"
(( COUNT++ ))
done
#This should give the number 6 (or string or something)
REV_NUMBER=${REV_ARR[1]}
echo ${REV_NUMBER}
### INCREMENT REVISION NUMBER FROM ARRAY ELEMENT ###
#NEW_REV_NUMBER= ????? + 1
IFS="$OLD_IFS"
I would like to be able to take the string:
Revision: 6
and pull out the 6 and increment by 1 so I can update a release txt file to be included in the SVN commit.
I have tried to make that 6 turn into a 7 for an hour now and feel like an idiot because I can't do it.
You need parenthesis: Change this:
# Get the element that says "Revision: 6"
REV_ARR=${SVN_INFO_ARR[4]}
to this:
# Get the element that says "Revision: 6"
REV_ARR=(${SVN_INFO_ARR[4]})
before
#This should give the number 6 (or string or something)
REV_NUMBER=${REV_ARR[1]}
so you'll be able to:
((REV_NUMBER++))
Edit:
As you wrote:
SVN_INFO_ARR=(`svn info`)
instead of just:
SVN_INFO_ARR=`svn info`
The parenthesis is used in bash to define an array. Have a look at:
man -Len -P'less +"/^ *Arrays"' bash
Instead of hardcoding the array indices, a better way would be to filter out the line you need and extract the number
Here's one way using regex (Bash 4)
while read -r line; do
if [[ $line =~ Revision:\ ([0-9]+) ]]; then
new_rev_num=$((BASH_REMATCH[1]+1))
echo $new_rev_num
break
fi
done < $(svn info)
Use grep to only select the line you need. Then, use Parameter expansion to remove "Revision: ". Finally, use let to do the arithmetics:
REVISION=$(svn info | grep '^Revision:')
REVISION=${REVISION#* }
let REVISION++
This code worked in the end:
#!/bin/bash
set -x
OLD_IFS="$IFS"
IFS=$'\r\n'
# Get the output from svn info into an array
SVN_INFO_ARR=(`svn info`)
IFS="$OLD_IFS"
# Get the element that says "Revision: 6"
REV_ARR=(${SVN_INFO_ARR[4]})
#This should give the number 6 (or string or something)
REV_NUMBER=${REV_ARR[1]}
echo $REV_NUMBER
echo $(( REV_NUMBER + 1 ))
The answer above had me stumped for a while because it was missing the $ in front of:
echo $(( REV_NUMBER + 1 ))
and the ((REV_NUMBER++)) notation did not work, I still got 6, not 7:
+ OLD_IFS='
'
+ IFS='
'
+ SVN_INFO_ARR=(`svn info`)
++ svn info
+ IFS='
'
+ REV_ARR=(${SVN_INFO_ARR[4]})
+ REV_NUMBER=6
+ echo 6
6
+ echo 6
6
lets say i used grep and cut to store data into variables. i need to get the first second and third values of each variable concatenated with each other. i think i need to use arrays to achieve this but i don't know how to go about doing that. for example if $one holds a b c and $two holds 1 2 3 and $three holds x y z i want to concatenate so that my output would look like a1x b2y c3z. like i said i think i need to store my grep/cut output into an array but i am not sure how to do that. Thanks.
In pure bash, you can do something like this:
v1="a b c"
v2="1 2 3"
v3="x y z"
for v in v1 v2 v3; do
read p1 p2 p3 <<< ${!v}
a1="$a1$p1"
a2="$a2$p2"
a3="$a3$p3"
done
echo $a1
echo $a2
echo $a3
The last three echoes output:
a1x
b2y
c3z
This might work for you:
v1="a b c"
v2="1 2 3"
v3="x y z"
parallel --xapply echo {1}{2}{3} ::: $v1 ::: $v2 ::: $v3
a1x
b2y
c3z
You can use sed,tr,etc to translate ' ' to '\n'.
Then use paste to concatenate them vertically.
$ v1="a b c"
$ v2="1 2 3"
$ v3="x y z"
$ paste <(tr ' ' '\n' <<<$v1) <(tr ' ' '\n' <<<$v2) <(tr ' ' '\n' <<<$v3) | tr -d '\t'
a1x
b2y
c3z
Or
$ paste <(echo "${v1// /$'\n'}") <(echo "${v2// /$'\n'}") <(echo "${v3// /$'\n'}") | tr -d '\t'
a1x
b2y
c3z
Note: If you save them in separated files, It will be much easier.
Another solution in pure bash using an array :
$ arr=( $v1 $v2 $v3 )
$ for ((i=0; i<3; i++)); do
for ((j=i; j<${#arr[#]}; j+=3)); do printf '%s' ${arr[j]}; done
echo
done
a1x
b2y
c3z
Pure bash using an array:
declare -a a=( $v1 $v2 $v3 )
echo "${a[0]}${a[3]}${a[6]}"
echo "${a[1]}${a[4]}${a[7]}"
echo "${a[2]}${a[5]}${a[8]}"