bash: store/update text data [closed] - arrays

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 2 years ago.
Improve this question
so for reading the list of file, I use this code here below:
IFS=$'\n' read -d '' -r -a data < ./somefolder/mytext.txt
for i in {0..9} #i know that i have 10 items, thats why i use 0..9
do
echo "${data[$i]}"
done
lets say i have 1-10 in the txt file, so it should print like below:
1
2
3
4
5
6
7
8
9
10
Questions:
is there any simpler way to read/write the text list than this?
how to save/update/overwrite data of mytext.txt? lets say change 4 to 88 for example.
Full example:
#!bin/bash
IFS=$'\n' read -d '' -r -a data < ./somefolder/mytext.txt
for i in {0..9} #i know that i have 10 items, thats why i use 0..9
do
echo "${data[$i]}"
done
echo "change 4 to anything"
read any
update(){
for n in {0..9}
do
if [[ n == 3 ]]; then
echo any
else
echo "${data[$n]}"
fi
done
}
update > ./somefolder/mytext.txt
#i dont know what i should do, it throws some errors saying syntax error
echo "saved"
exit 0

This is the code and output of the code, it is not the same as you describe in the comments.
printf '%s\n' {a..z} > file.txt
cat file.txt
a
b
c
d
e
f
g
h
i
j
k
l
m
n
o
p
q
r
s
t
u
v
w
x
y
z
A quick way of showing line numbers by using grep
grep -n . file.txt
A function to loop through an array.
func() {
n=1
for f; do
if (( n == 3 )); then
printf '%d %s\n' "$n" foo
else
printf '%d %s\n' "$n" "$f"
fi
((n++))
done
}
mapfile -t array < file.txt
func "${array[#]}"
Output
1 a
2 b
3 foo
4 d
5 e
6 f
7 g
8 h
9 i
10 j
11 k
12 l
13 m
14 n
15 o
16 p
17 q
18 r
19 s
20 t
21 u
22 v
23 w
24 x
25 y
26 z
On the other hand if you just want to replace everything with anything at a certain line and and ed is acceptable/available.
#!/usr/bin/env bash
printf '%s\n' ,n | ed -s file.txt
read -rp 'Change 4 to anything: ' input
printf '%s\n' "4c" "$input" . ,n w | ed -s file.txt
A more flexible version of the previous script.
#!/usr/bin/env bash
total=$(printf '%s\n' '$=' | ed -s file.txt)
printf '%s\n' ,n | ed -s file.txt
read -rp 'Enter the line number you want to change: ' int
if [[ $int == *[!0-9]* ]]; then
printf >&2 '%s is not an int\n' "$int"
exit 1
elif (( int > total )); then
printf >&2 '%s is out of range!' "$int"
exit 1
fi
read -rp "Enter the replacement at line $int: " input
printf '%s\n' "${int}c" "$input" . ,n w | ed -s file.txt
Caveat The file.txt name and path is still hard coded to the script, just add an additional read for the the file.

Related

Computing sum of specific field from array entries

I have an array trf. Would like to compute the sum of the second element in each array entry.
Example of array contents
trf=( "2 13 144" "3 21 256" "5 34 389" )
Here is the current implementation, but I do not find it robust enough. For instance, it fails with arbitrary number of elements (but considered constant from one array element to another) in each array entry.
cnt=0
m=${#trf[#]}
while (( cnt < m )); do
while read -r one two three
do
sum+="$two"+
done <<< $(echo ${array[$count]})
let count=$count+1
done
sum+=0
result=`echo "$sum" | /usr/bin/bc -l`
You're making it way too complicated. Something like
#!/usr/bin/env bash
trf=( "2 13 144" "3 21 256" "5 34 389" )
declare -i sum=0 # Integer attribute; arithmetic evaluation happens when assigned
for (( n = 0; n < ${#trf[#]}; n++)); do
read -r _ val _ <<<"${trf[n]}"
sum+=$val
done
printf "%d\n" "$sum"
in pure bash, or just use awk (This is handy if you have floating point numbers in your real data):
printf "%s\n" "${trf[#]}" | awk '{ sum += $2 } END { print sum }'
You can use printf to print the entire array, one entry per line. On such an input, one loop (while read) would be sufficient. You can even skip the loop entirely using cut and tr to build the bc command. The echo 0 is there so that bc can handle empty arrays and the trailing + inserted by tr.
{ printf %s\\n "${trf[#]}" | cut -d' ' -f2 | tr \\n +; echo 0; } | bc -l
For your examples this generates prints 68 (= 13+21+34+0).
Try this printf + awk combo:
$ printf '%s\n' "${trf[#]}" | awk '{print $2}{a+=$2}END{print "sum:", a}'
13
21
34
sum: 68
Oh, it's already suggested by Shawn. Then with loop:
$ for item in "${trf[#]}"; do
echo $item
done | awk '{print $2}{a+=$2}END{print "sum:", a}'
13
21
34
sum: 68
For relatively small arrays a for/while double loop should be ok re: performance; placing the final sum in the $result variable (as in OP's code):
result=0
for element in "${trf[#]}"
do
while read -r a b c
do
((result+=b))
done <<< "${element}"
done
echo "${result}"
This generates:
68
For larger data sets I'd probably opt for one of the awk-only solutions (for performance reasons).

Picking input record fields with AWK

Let's say we have a shell variable $x containing a space separated list of numbers from 1 to 30:
$ x=$(for i in {1..30}; do echo -n "$i "; done)
$ echo $x
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
We can print the first three input record fields with AWK like this:
$ echo $x | awk '{print $1 " " $2 " " $3}'
1 2 3
How can we print all the fields starting from the Nth field with AWK? E.g.
4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
EDIT: I can use cut, sed etc. to do the same but in this case I'd like to know how to do this with AWK.
Converting my comment to answer so that solution is easy to find for future visitors.
You may use this awk:
awk '{for (i=3; i<=NF; ++i) printf "%s", $i (i<NF?OFS:ORS)}' file
or pass start position as argument:
awk -v n=3 '{for (i=n; i<=NF; ++i) printf "%s", $i (i<NF?OFS:ORS)}' file
Version 4: Shortest is probably using sub to cut off the first three fields and their separators:
$ echo $x | awk 'sub(/^ *([^ ]+ +){3}/,"")'
Output:
4 5 6 7 8 9 ...
This will, however, preserve all space after $4:
$ echo "1 2 3 4 5" | awk 'sub(/^ *([^ ]+ +){3}/,"")'
4 5
so if you wanted the space squeezed, you'd need to, for example:
$ echo "1 2 3 4 5" | awk 'sub(/^ *([^ ]+ +){3}/,"") && $1=$1'
4 5
with the exception that if there are only 4 fields and the 4th field happens to be a 0:
$ echo "1 2 3 0" | awk 'sub(/^ *([^ ]+ +){3}/,"")&&$1=$1'
$ [no output]
in which case you'd need to:
$ echo "1 2 3 0" | awk 'sub(/^ *([^ ]+ +){3}/,"") && ($1=$1) || 1'
0
Version 1: cut is better suited for the job:
$ cut -d\ -f 4- <<<$x
Version 2: Using awk you could:
$ echo -n $x | awk -v RS=\ -v ORS=\ 'NR>=4;END{printf "\n"}'
Version 3: If you want to preserve those varying amounts of space, using GNU awk you could use split's fourth parameter seps:
$ echo "1 2 3 4 5 6 7" |
gawk '{
n=split($0,a,FS,seps) # actual separators goes to seps
for(i=4;i<=n;i++) # loop from 4th
printf "%s%s",a[i],(i==n?RS:seps[i]) # get fields from arrays
}'
Adding one more approach to add all value into a variable and once all fields values are done with reading just print the value of variable. Change the value of n= as per from which field onwards you want to get the data.
echo "$x" |
awk -v n=3 '{val="";for(i=n; i<=NF; i++){val=(val?val OFS:"")$i};print val}'
With GNU awk, you can use the join function which has been a built-in include since gawk 4.1:
x=$(seq 30 | tr '\n' ' ')
echo "$x" | gawk '#include "join"
{split($0, arr)
print join(arr, 4, length(arr), "|")}
'
4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30
(Shown here with a '|' instead of a ' ' for clarity...)
Alternative way of including join:
echo "$x" | gawk -i join '{split($0, arr); print join(arr, 4, length(arr), "|")}'
Using gnu awk and gensub:
echo $x | awk '{ print gensub(/^([[:digit:]]+[[:space:]]){3}(.*$)/,"\\2",$0)}'
Using gensub, split the string into two sections based on regular expressions and print the second section only.

Gnuplot: How to plot bash array without dumping it to a file

I am trying to plot a bash array using gnuplot without dumping the array to a temporary file.
Let's say:
myarray=$(seq 1 5)
I tried the following:
myarray=$(seq 1 5)
gnuplot -p <<< "plot $myarray"
I got the following error:
line 0: warning: Cannot find or open file "1"
line 0: No data in plot
gnuplot> 2
^
line 0: invalid command
gnuplot> 3
^
line 0: invalid command
gnuplot> 4
^
line 0: invalid command
gnuplot> 5''
^
line 0: invalid command
Why it doesn't interpret the array as a data block?
Any help is appreciated.
bash array
myarray=$(seq 1 5)
The myarray is not a bash array, it is a normal variable.
The easiest is to put the data to stdin and plot <cat.
seq 5 | gnuplot -p -e 'plot "<cat" w l'
Or with your variable and with using a here-string:
<<<"$myarray" gnuplot -p -e 'plot "<cat" w l'
Or with your variable with redirection with echo or printf:
printf "%s\n" "$myarray" | gnuplot -p -e 'plot "<cat" w l'
And if you want to plot an actual array, just print it on separate lines and then pipe to gnuplot
array=($(seq 5))
printf "%s\n" "${array[#]}" | gnuplot -p -e 'plot "<cat" w l'
Plot STDIN
gnuplot -p -e 'plot "/dev/stdin"'
Sample:
( seq 5 10; seq 7 12 ) | gnuplot -p -e 'plot "/dev/stdin"'
or
gnuplot -p -e 'plot "/dev/stdin" with steps' < <( seq 5 10; seq 7 12 )
More tunned plot
gnuplot -p -e "set terminal wxt 0 enhanced;set grid;
set label \"Test demo with random values\" at 0.5,0 center;
set yrange [ \"-1\" : \"80\" ] ; set timefmt \"%s\";
plot \"/dev/stdin\" using 1:2 title \"RND%30+40\" with impulse;" < <(
paste <(
seq 2300 2400
) <(
for ((i=101;i--;)){ echo $[RANDOM%30+40];}
)
)
Please note that this is still one line, you could Copy'n paste into any terminal console.

Output looped array data to separate columns in bash

I have three loops which process array data and print to the same log file. I would like to sort the output of each loop into columns which are separated by tabs using bash code:
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
Notice: 1 stands for the content of loop 1, 2 stands for the content of loop 2 and 3 stands for the content of loop 3.
declare -a Array1
declare -a Array2
declare -a Array3
for (( i = 0 ; i < 9 ; i++))
do
echo "${Array1[$i]}"
done | tee -a log.txt
for (( i = 0 ; i < 9 ; i++))
do
echo "(( ${Array1[$i]}-${Array2[$i]} ))" | bc
done | tee -a log.txt
for (( i = 0 ; i < 9 ; i++))
do
echo "${Array3[$i]}"
done | tee -a log.txt
I tried some stuff with the column command, but it doesn't work out as outlined above.
The simplest option may be to use a single loop.
An alternative is to take the output format that you've got already and convert it into columns. This is one way of doing it:
# Read the concatenated results into an array, $results
IFS=$'\n' read -d '' -r -a results < log.txt
# Print the concatenated results in columns
for (( i=0 ; i<9; i++ )) ; do
printf '%s\t%s\t%s\n' "${results[i]}" "${results[i+9]}" "${results[i+18]}"
done
If you don't need the log.txt file, you could just put the results into an array as you calculate them (using as many loops as you like) and print them afterwards.

Sorting an array in a BASH Script by columns and rows while keeping them intact

I have a test file that looks like this:
1 1 1 1 1
9 3 4 5 5
6 7 8 9 7
3 6 8 9 1
3 4 2 1 4
6 4 4 7 7
Each row is supposed to represent a students grades. So the user puts in either an 'r' or a 'c' into the command line to choose to sort by rows or columns, followed by the file name. Sorting by rows would represent getting a students average and sorting my columns would represent a particular assignments average.
I am not doing anything with the choice variable yet because I need to get the array sorted first so I can take the averages and then get the median for each column and row.
So im not sure how I can choose to sort by those specific options. Here is what I have so far:
#!/bin/bash
choice="$1"
filename="$2"
c=0
if [ -e "$filename" ]
then
while read line
do
myArray[$c]=$line
c=$(expr $c + 1)
done < "$filename"
else
echo "File does not exist"
fi
printf -- '%s\n' "${myArray[#]}"
FS=$'\n' sorted=($(sort -n -k 1,1<<<"${myArray[*]}"))
echo " "
printf '%s\n' "${sorted[#]}"
This is only sorting the first column though and im not sure why its even doing that. Any push in the right direction would be appreciated. Examples would help a ton, thanks!
UPDATE:
With the changes that were suggested I have this so far:
#!/bin/sh
IFS=$'\n';
choice="$1"
filename="$2"
if [ -e "$filename" ]
then
while read line
do
myArray[$c]=$line
c=$(expr $c + 1)
done < "$filename"
else
echo "File does not exist."
fi
printf -- '%s\n' "${myArray[#]}"
width=${myArray[0]// /}
width=${#width}
height=${#myArray[#]}
bar=()
for w in $(seq 0 1 $((${width}-1)))
do
tmp=($(sort -n <<<"${myArray[*]}"))
for h in $(seq 0 1 $((${height}-1)))
do
myArray[h]=${myArray[h]#* }
bar[h]="${bar[h]} ${tmp[h]%% *}"
bar[h]="${bar[h]# }"
done
done
printf -- '%s\n' "${bar[*]}"
But now I am getting some really strange output of way more numbers than i started with and in a seemingly random order.
actually it is sorting $line(s) which are strings. you need to initialize the column to sort correctly, so that it is an array
UPDATE:
the following code is really straight forward. no performance aspects are regarded. so for large datasets this will take a while to sort column wise. your datasets have to contain lines of numbers seperated by single spaces to make this work.
#!/bin/bash
IFS=$'\n';
# here you can place your read line function
ar[0]="5 3 2 8"
ar[1]="1 1 1 1"
ar[2]="3 2 4 5"
printf -- '%s\n' "${ar[*]}" # print the column wise unsorted ar
echo
# sorting
width=${ar[0]// /}
width=${#width}
height=${#ar[#]}
bar=()
for w in $(seq 0 1 $((${width}-1))); do # for each column
#sort -n <<<"${ar[*]}" # debug, see first column removal
tmp=($(sort -n <<<"${ar[*]}")) # this just sorts lexigraphically by "first column"
# rows are strings, see initial definition of ar
#echo
for h in $(seq 0 1 $((${height}-1))); do # update first column
ar[h]=${ar[h]#* } # strip first column
bar[h]="${bar[h]} ${tmp[h]%% *}" # add sorted column to new array
bar[h]="${bar[h]# }" # trim leading space
done
#printf -- '%s\n' "${bar[*]}" # debug, see growing bar
#echo "---"
done
printf -- '%s\n' "${bar[*]}" # print the column wise sorted ar
prints out the unsorted and sorted array
5 3 2 8
1 1 1 1
3 2 4 5
1 1 1 1
3 2 2 5
5 3 4 8

Resources