Iterate over json array of dates in bash (has whitespace) - arrays

I have an array that is being stored in a bash variable that is a json array of dates (it's currently being produced by a separate script). It looks like the following
["2019-09-19 03:13:29", "2019-09-19 20:20:18", "2019-09-19 18:19:50", "2019-09-19 06:07:17", "2019-09-19 11:53:25"]
I want to iterate over these dates and preserve the white space in the dates so I can use them with the gdate command. I'm currently using jq but that is splitting up the date part from the time part. i.e.
dates=$(python date_producing_script.py | jq -c -r '.[]')
for date in ${dates[#]}; do
echo $date
#end goal is to do something here w/ gdate ex gdate -d $date ...
done
However this gives something like
2019-09-19
03:13:29
2019-09-19
20:20:18
2019-09-19
18:19:50
2019-09-19
06:07:17
2019-09-19
11:53:25
Where I'm looking for something like the following
2019-09-19 03:13:29
2019-09-19 20:20:18
2019-09-19 18:19:50
2019-09-19 06:07:17
2019-09-19 11:53:25
(one confusing thing for me here is that if I just do the python date_producing_script.py | jq -c -r '.[]' command in a terminal it looks the way I want it to)
However ideally I would want something like. Is there anyway to get this result from the input in shell script

Your jq is fine; the problem is in the way you're looping through the bash array, as can be seen from this bash typescript:
$ ary=("a b" "c d")
$ for x in ${ary[#]} ; do echo "$x" ; done
a
b
c
d
$ for x in "${ary[#]}" ; do echo "$x" ; done
a b
c d
$
IOW
"${dates[#]}"

Here is one solution:
#!/bin/bash
#
IFS=','
dates='["2019-09-19 03:13:29", "2019-09-19 20:20:18", "2019-09-19 18:19:50", "2019-09-19 06:07:17", "2019-09-19 11:53:25"]'
for date in ${dates[#]}
do
# Cleanup $date
thedate=$(echo $date | tr -d '\[\]"' | sed -e 's/^[[:space:]]*//')
echo "-->$thedate<--"
done
IFS: sets the separator as ,
$dates is now separated on ,
$thedate is $date with [, ], " removed. The sed is to remove prefix spaces.
I put --> and <-- to show that $thedate does not include any spaces.
There are most probably solutions with awk too.

One of your issue is when you populate dates array ,
and the other is when you want to use the array for loop .
My proposal is :
change jq to generate a long string of date with = as a separator
tell bash to use = as separator instead of <space><tab><line return>
you must use the syntax "$dates[#]" instead of $dates[#]
Your script will be like :
OLDIFS=$IFS ;
IFS="=" ;
dates=( python date_producing_script.py | jq -c -j -r '.[]|(.,"=")') );
IFS=$OLDIFS
for d in "${dates[#]}" ;
do
echo $d ;
done

Related

how to parse json in bash script as an array? array value should have both key:value format

my json.file looks like
{
"price1" : "120.10",
"price2" : "110.30",
"price3" : "244.45"
}
I have used below sed command in my bash script to declare array that reads from json
array=( $(sed -n '/{/,/}/{s/[^:]*:[^"]*"\([^"]*\).*/\1/p;}' json.file) )
this gave me output for echo ${array[*]} values
120.10 110.10 244.45
I am looking for my array values to include the key name as well (key:value).
my desired output should be key:value format
price1:120.10 price2:110.10 price3:244.45
can someone please help or guide me?
Parsing json with sed is probably a bad idea. But let's assume it is a super-simple and regular kind of json format. You were not too far from a working solution:
$ array=($(sed -nr 's/.*"(.*)".*"(.*)".*/\1:\2/p' json.file))
$ echo ${array[*]}
price1:120.10 price2:110.30 price3:244.45
The p flag of the substitute command tells sed to print the result if there was a match. It is good to know. If you have only two quoted strings per line of interest it should work. The regular expression .*"(.*)".*"(.*)".* matches anything - .* - followed by a double quote, anything again, another double quote... The parentheses - (.*) - do not change what is matched. It just records the matched part between parentheses in one of nine buffers that can be used in the replacement string - \1:\2. So here, the replacement string corresponds to:
<first recorded match>:<second recorded match>
If you want to be more specific about the matching lines you can. For instance:
sed -nr 's/^ "(.*)" : "(.*)",?$/\1:\2/p' json.file
Also specifies that there are 2 leading spaces, that the colon is preceded and followed by one single space and that there is a optional comma after the last quote and before the end of line.
But there is also the much simpler:
sed -nr 's/[[:space:]",]//gp' json.file
that just removes all spaces, double quotes and commas, printing only the matching lines. And guess what? It is what you (apparently) want.
Anyway, remember that if your json files are more complex than what you show sed is definitely not the right tool.
Your solution just needs to extract the extra data:
array=( $(sed -n '/{/,/}/{s/"\([^"]*\)"[^:]*:[^"]*"\([^"]*\).*/\1:\2/p;}' <<< "$j") )
echo ${array[#]}
price1:120.10 price2:110.30 price3:244.45
Simpler looking solutions may work too, but I'm assuming that you wrote your pattern like that deliberately.
The key trick here is to enclose part of your pattern with escaped brackets \(...\) and then have a numbered substitution for each in the output part \1, \2, etc.
json='{"price1":"120.10","price2":"110.30","price3":"244.45"}'
array=($(jq -r 'to_entries[] | ( "\(.key):\(.value)")' <<< "$json"))
echo ${array[#]}
printf '%s\n' "${array[#]}"
declare -p array
echo
array=($(jq -r 'to_entries[] | ( "\(.key):\(.value)") | #sh' <<< "$json"))
echo ${array[#]}
printf '%s\n' "${array[#]}"
declare -p array
Output:
price1:120.10 price2:110.30 price3:244.45
price1:120.10
price2:110.30
price3:244.45
declare -a array='([0]="price1:120.10" [1]="price2:110.30" [2]="price3:244.45")'
'price1:120.10' 'price2:110.30' 'price3:244.45'
'price1:120.10'
'price2:110.30'
'price3:244.45'
declare -a array='([0]="'\''price1:120.10'\''" [1]="'\''price2:110.30'\''" [2]="'\''price3:244.45'\''")'
For bash scripting, you might want to use an Associative array:
declare -A prices
while IFS=$'\t' read -r key value; do
prices[$key]=$value
done < <(
jq -r 'to_entries[] | [.key, .value] | #tsv' json.file
)
Then, inspect the array
declare -p prices
declare -A prices='([price1]="120.10" [price3]="244.45" [price2]="110.30" )'
or iterate over it
for key in "${!prices[#]}"; do
printf '%s => %s\n' "$key" "${prices[$key]}"
done
price1 => 120.10
price3 => 244.45
price2 => 110.30

How do i echo specific rows and columns from csv's in a variable?

The below script:
#!/bin/bash
otscurrent="
AAA,33854,4528,38382,12
BBB,83917,12296,96213,13
CCC,20399,5396,25795,21
DDD,27198,4884,32082,15
EEE,2472,981,3453,28
FFF,3207,851,4058,21
GGG,30621,4595,35216,13
HHH,8450,1504,9954,15
III,4963,2157,7120,30
JJJ,51,59,110,54
KKK,87,123,210,59
LLL,573,144,717,20
MMM,617,1841,2458,75
NNN,234,76,310,25
OOO,12433,1908,14341,13
PPP,10627,1428,12055,12
QQQ,510,514,1024,50
RRR,1361,687,2048,34
SSS,1,24,25,96
TTT,0,5,5,100
UUU,294,1606,1900,85
"
IFS="," array1=(${otscurrent})
echo ${array1[4]}
Prints:
$ ./test.sh
12
BBB
I'm trying to get it to just print 12... And I am not even sure how to make it just print row 5 column 4
The variable is an output of a sqlquery that has been parsed with several sed commands to change the formatting to csv.
otscurrent="$(sqlplus64 user/password#dbserverip/db as sysdba #query.sql |
sed '1,11d; /^-/d; s/[[:space:]]\{1,\}/,/g; $d' |
sed '$d'|sed '$d'|sed '$d' | sed '$d' |
sed 's/Used,MB/Used MB/g' |
sed 's/Free,MB/Free MB/g' |
sed 's/Total,MB/Total MB/g' |
sed 's/Pct.,Free/Pct. Free/g' |
sed '1b;/^Name/d' |
sed '/^$/d'
)"
Ultimately I would like to be able to call on a row and column and run statements on the values.
Initially i was piping that into :
awk -F "," 'NR>1{ if($5 < 10) { printf "%-30s%-10s%-10s%-10s%-10s\n", $1,$2,$3,$4,$5"%"; } else { echo "Nothing to do" } }')"
Which works but I couldn't run commands from if else ... or atleaste I didn't know how.
If you have bash 4.0 or newer, an associative array is an appropriate way to store data in this kind of form.
otscurrent=${otscurrent#$'\n'} # strip leading newline present in your sample data
declare -A data=( )
row=0
while IFS=, read -r -a line; do
for idx in "${!line[#]}"; do
data["$row,$idx"]=${line[$idx]}
done
(( row += 1 ))
done <<<"$otscurrent"
This lets you access each individual item:
echo "${data[0,0]}" # first field of first line
echo "${data[9,0]}" # first field of tenth line
echo "${data[9,1]}" # second field of tenth line
"I'm trying to get it to just print 12..."
The issue is that IFS="," splits on commas and there is no comma between 12 and BBB. If you want those to be separate elements, add a newline to IFS. Thus, replace:
IFS="," array1=(${otscurrent})
With:
IFS=$',\n' array1=(${otscurrent})
Output:
$ bash test.sh
12
All you need to print the value of the 4th column on the 5th row is:
$ awk -F, 'NR==5{print $4}' <<< "$otscurrent"
3453
and just remember that in awk row (record) and column (field) numbers start at 1, not 0. Some more examples:
$ awk -F, 'NR==1{print $5}' <<< "$otscurrent"
12
$ awk -F, 'NR==2{print $1}' <<< "$otscurrent"
BBB
$ awk -F, '$5 > 50' <<< "$otscurrent"
JJJ,51,59,110,54
KKK,87,123,210,59
MMM,617,1841,2458,75
SSS,1,24,25,96
TTT,0,5,5,100
UUU,294,1606,1900,85
If you'd like to avoid all of the complexity and simply parse your SQL output to produce what you want without 20 sed commands in between, post a new question showing the raw sqlplus output as the input and what you want finally output and someone will post a brief, clear, simple, efficient awk script to do it all at one time, or maybe 2 commands if you still want an intermediate CSV for some reason.

Comment out items that do not match pattern in array

I have a log file I am trying to comment out lines that do not match my array. I did successfully learn how to create an array and I can echo out the array items but I am having trouble taking anything that doesn't match my array and adding something in front of it. Here is my code, if you have suggestions on another path or ways I can make it better:
for itsSaturday in $(find "$LOCATION" -mindepth 1 -maxdepth 1 -name "*.log" ); do
TEMPFILE="$itsSaturday.$$"
declare -a someArray=( "breakfast" "scrambled eggs" "Bloody Mary" )
theCall='some_additional_text_'
commentOn="## You_need_"
for arrayItem in "${someArray[#]}"; do
merged="$theCall$arrayItem"
if ! grep -q "$merged" "$itsSaturday"; then
sed -e '/$merged/! s:$commentOn$theCall::g' "$itsSaturday" > $TEMPFILE && mv $TEMPFILE "$itsSaturday"
fi
done
done
file:
some_additional_text_breakfast
some_additional_text_bacon
some_additional_text_scrambled eggs
some_additional_text_Bloody Mary
some_additional_text_orange juice
some_additional_text_breakfast
file into:
some_additional_text_breakfast
## You_need_some_additional_text_bacon
some_additional_text_scrambled eggs
some_additional_text_Bloody Mary
## You_need_some_additional_text_orange juice
some_additional_text_breakfast
How can I add a variable before items that do not match my array?
I don't like doing this using bash and sed, but I think the following might be enough:
#! /bin/bash
declare -a someArray=( "breakfast" "scrambled eggs" "Bloody Mary" )
theCall='some_additional_text_'
commentOn="## You_need_"
OIFS="$IFS"
IFS='|' mergedLines="${someArray[*]/#/$theCall}"
IFS="$OIFS"
for i in *.txt
do
TEMPFILE="$i.$$"
sed -r "/$mergedLines/!s/^/$commentOn/" "$i" >> "$TEMPFILE"
done
I shifted the array and other constants out of the loop.
"${someArray[*]/#/$theCall}" uses bash string substitution to append the contents of $theCall to every element in the array.
IFS='|' mergedLines="${someArray[*]} is a convenient trick to combine the elements of an array into a pipe-separated string.
Combined, (2) and (3) get me
some_additional_text_breakfast|some_additional_text_scrambled eggs|some_additional_text_Bloody Mary
in mergedLines.
Then it's just a matter of using extended regular expressions in sed (for |) and replacing non-matching lines.
Your sed pattern used single quotes, so the variables within were not expanded.
Try replacing the inner for-loop with:
PROG=$(printf '%s\n' "${COMMENT[#]}" | while read comment ; do
/bin/echo -n '$0 !~ /'"$comment"'$/ && '
done
echo '1 { printf commentOn } ; { print }')
awk -v commentOn="$commentOn" "$PROG" $itsSaturday > $TEMPFILE && mv $TEMPFILE $itsSaturday
On each file, this creates an awk program that does the work.

Accessing a JSON object in Bash - associative array / list / another model

I have a Bash script which gets data in JSON, I want to be able to convert the JSON into an accessible structure - array / list / or other model which would be easy to parse the nested data.
Example:
{
"SALUTATION": "Hello world",
"SOMETHING": "bla bla bla Mr. Freeman"
}
I want to get the value like the following: echo ${arr[SOMETHING]}
[ Different approach is optional as well. ]
If you want key and value, and based on How do i convert a json object to key=value format in JQ, you can do:
$ jq -r "to_entries|map(\"\(.key)=\(.value|tostring)\")|.[]" file
SALUTATION=Hello world
SOMETHING=bla bla bla Mr. Freeman
In a more general way, you can store the values into an array myarray[key] = value like this, just by providing jq to the while with the while ... do; ... done < <(command) syntax:
declare -A myarray
while IFS="=" read -r key value
do
myarray[$key]="$value"
done < <(jq -r 'to_entries|map("(.key)=(.value)")|.[]' file)
And then you can loop through the values like this:
for key in "${!myarray[#]}"
do
echo "$key = ${myarray[$key]}"
done
For this given input, it returns:
SALUTATION = Hello world
SOMETHING = bla bla bla Mr. Freeman
Although this question is answered, I wasn't able to fully satiate my
requirements from the posted answer. Here is a little write up that'll help any
bash-newcomers.
Foreknowledge
A basic associative array declaration
#!/bin/bash
declare -A associativeArray=([key1]=val1 [key2]=val2)
You can also use quotes (', ") around the declaration, its keys, and
values.
#!/bin/bash
declare -A 'associativeArray=([key1]=val1 [key2]=val2)'
And you can delimit each [key]=value pair via space or newline.
#!/bin/bash
declare -A associativeArray([key1]=value1
['key2']=value2 [key3]='value3'
['key4']='value2' ["key5"]="value3"
["key6"]='value4'
['key7']="value5"
)
Depending on your quote variation, you may need to escape your string.
Using Indirection to access both key and value in an associative array
example () {
local -A associativeArray=([key1]=val1 [key2]=val2)
# print associative array
local key value
for key in "${!associativeArray[#]}"; do
value="${associativeArray["$key"]}"
printf '%s = %s' "$key" "$value"
done
}
Running the example function
$ example
key2 = val2
key1 = val1
Knowing the aforementioned tidbits allows you to derive the following snippets:
The following examples will all have the result as the example above
String evaluation
#!/usr/bin/env bash
example () {
local arrayAsString='associativeArray=([key1]=val1 [key2]=val2)'
local -A "$arrayAsString"
# print associative array
}
Piping your JSON into JQ
#!/usr/bin/env bash
# Note: usage of single quotes instead of double quotes for the jq
# filter. The former is preferred to avoid issues with shell
# substitution of quoted strings.
example () {
# Given the following JSON
local json='{ "key1": "val1", "key2": "val2" }'
# filter using `map` && `reduce`
local filter='to_entries | map("[\(.key)]=\(.value)") |
reduce .[] as $item ("associativeArray=("; . + ($item|#sh) + " ") + ")"'
# Declare and assign separately to avoid masking return values.
local arrayAsString;
# Note: no encompassing quotation (")
arrayAsString=$(jq --raw-output "${filter}" <<< "$json")
local -A "$arrayAsString"
# print associative array
}
jq -n / --null-input option + --argfile && redirection
#!/usr/bin/env bash
example () {
# /path/to/file.json contains the same json as the first two examples
local filter filename='/path/to/file.json'
# including bash variable name in reduction
filter='to_entries | map("[\(.key | #sh)]=\(.value | #sh) ")
| "associativeArray=(" + add + ")"'
# using --argfile && --null-input
local -A "$(jq --raw-output --null-input --argfile file "$filename" \
"\$filename | ${filter}")"
# or for a more traceable declaration (using shellcheck or other) this
# variation moves the variable name outside of the string
# map definition && reduce replacement
filter='[to_entries[]|"["+(.key|#sh)+"]="+(.value|#sh)]|"("+join(" ")+")"'
# input redirection && --join-output
local -A associativeArray=$(jq --join-output "${filter}" < "${filename}")
# print associative array
}
Reviewing previous answers
#Ján Lalinský
To load JSON object into a bash associative array efficiently
(without using loops in bash), one can use tool 'jq', as follows.
# first, load the json text into a variable:
json='{"SALUTATION": "Hello world", "SOMETHING": "bla bla bla Mr. Freeman"}'
# then, prepare associative array, I use 'aa':
unset aa
declare -A aa
# use jq to produce text defining name:value pairs in the bash format
# using #sh to properly escape the values
aacontent=$(jq -r '. | to_entries | .[] | "[\"" + .key + "\"]=" + (.value | #sh)' <<< "$json")
# string containing whole definition of aa in bash
aadef="aa=($aacontent)"
# load the definition (because values may contain LF characters, aadef must be in double quotes)
eval "$aadef"
# now we can access the values like this: echo "${aa[SOMETHING]}"
Warning: this uses eval, which is dangerous if the json input is from unknown source (may contain malicious shell commands that eval may execute).
This could be reduced to the following
example () {
local json='{ "key1": "val1", "key2": "val2" }'
local -A associativeArray="($(jq -r '. | to_entries | .[] |
"[\"" + .key + "\"]=" + (.value | #sh)' <<< "$json"))"
# print associative array
}
#fedorqui
If you want key and value, and based on How do i convert a json object to key=value format in JQ, you can do:
$ jq -r "to_entries|map(\"\(.key)=\(.value|tostring)\")|.[]" file
SALUTATION=Hello world
SOMETHING=bla bla bla Mr. Freeman
In a more general way, you can store the values into an array myarray[key] = value like this, just by providing jq to the while with the while ... do; ... done < <(command) syntax:
declare -A myarray
while IFS="=" read -r key value
do
myarray[$key]="$value"
done < <(jq -r "to_entries|map(\"\(.key)=\(.value)\")|.[]" file)
And then you can loop through the values like this:
for key in "${!myarray[#]}"
do
echo "$key = ${myarray[$key]}"
done
For this given input, it returns:
SALUTATION = Hello world
SOMETHING = bla bla bla Mr. Freeman
The main difference between this solution and my own is looping through the
array in bash or in jq.
Each solution is valid and depending on your use case, one may be more useful
then the other.
Context: This answer was written to be responsive to a question title which no longer exists..
The OP's question actually describes objects, vs arrays.
To be sure that we help other people coming in who are actually looking for help with JSON arrays, though, it's worth covering them explicitly.
For the safe-ish case where strings can't contain newlines (and when bash 4.0 or newer is in use), this works:
str='["Hello world", "bla bla bla Mr. Freeman"]'
readarray -t array <<<"$(jq -r '.[]' <<<"$str")"
To support older versions of bash, and strings with newlines, we get a bit fancier, using a NUL-delimited stream to read from jq:
str='["Hello world", "bla bla bla Mr. Freeman", "this is\ntwo lines"]'
array=( )
while IFS= read -r -d '' line; do
array+=( "$line" )
done < <(jq -j '.[] | (. + "\u0000")')
This is how can it be done recursively:
#!/bin/bash
SOURCE="$PWD"
SETTINGS_FILE="$SOURCE/settings.json"
SETTINGS_JSON=`cat "$SETTINGS_FILE"`
declare -A SETTINGS
function get_settings() {
local PARAMS="$#"
local JSON=`jq -r "to_entries|map(\"\(.key)=\(.value|tostring)\")|.[]" <<< "$1"`
local KEYS=''
if [ $# -gt 1 ]; then
KEYS="$2"
fi
while read -r PAIR; do
local KEY=''
if [ -z "$PAIR" ]; then
break
fi
IFS== read PAIR_KEY PAIR_VALUE <<< "$PAIR"
if [ -z "$KEYS" ]; then
KEY="$PAIR_KEY"
else
KEY="$KEYS:$PAIR_KEY"
fi
if jq -e . >/dev/null 2>&1 <<< "$PAIR_VALUE"; then
get_settings "$PAIR_VALUE" "$KEY"
else
SETTINGS["$KEY"]="$PAIR_VALUE"
fi
done <<< "$JSON"
}
To call it:
get_settings "$SETTINGS_JSON"
The array will be accessed like so:
${SETTINGS[grandparent:parent:child]}
To load JSON object into a bash associative array efficiently (without using loops in bash), one can use tool 'jq', as follows.
# first, load the json text into a variable:
json='{"SALUTATION": "Hello world", "SOMETHING": "bla bla bla Mr. Freeman"}'
# then, prepare associative array, I use 'aa':
unset aa
declare -A aa
# use jq to produce text defining name:value pairs in the bash format
# using #sh to properly escape the values
aacontent=$(jq -r '. | to_entries | .[] | "[\"" + .key + "\"]=" + (.value | #sh)' <<< "$json")
# string containing whole definition of aa in bash
aadef="aa=($aacontent)"
# load the definition (because values may contain LF characters, aadef must be in double quotes)
eval "$aadef"
# now we can access the values like this: echo "${aa[SOMETHING]}"
Warning: this uses eval, which is dangerous if the json input is from unknown source (may contain malicious shell commands that eval may execute).
Building on #HelpNeeder's solution (nice one btw)
His solution wasn't really working with integers, so i made some additions. Extended amount of condition checks, so it's fair to say some performance is sacrificed.
This version works with integers and also floating point values.
SOURCE="$PWD"
SETTINGS_FILE="./test2.json"
SETTINGS_JSON=`cat "$SETTINGS_FILE"`
declare -A SETTINGS
get_settings() {
local PARAMS="$#"
local JSON=`jq -r "to_entries|map(\"\(.key)=\(.value|tostring)\")|.[]" <<< "$1"`
local KEYS=''
if [ $# -gt 1 ]; then
KEYS="$2"
fi
while read -r PAIR; do
local KEY=''
if [ -z "$PAIR" ]; then
break
fi
IFS== read PAIR_KEY PAIR_VALUE <<< "$PAIR"
if [ -z "$KEYS" ]; then
KEY="$PAIR_KEY"
else
KEY="$KEYS:$PAIR_KEY"
fi
res=$(jq -e . 2>/dev/null <<< "$PAIR_VALUE")
exitCode=$?
check=`echo "$PAIR_VALUE" | grep -E ^\-?[0-9]*\.?[0-9]+$`
# if [ "${res}" ] && [ $exitCode -eq "0" ] && [[ ! "${PAIR_VALUE}" == ?(-)+([0-9]) ]] ALTERNATIVE, works only for integer (not floating point)
if [ "${res}" ] && [ $exitCode -eq "0" ] && [[ "$check" == '' ]]
then
get_settings "$PAIR_VALUE" "$KEY"
else
SETTINGS["$KEY"]="$PAIR_VALUE"
fi
done <<< "$JSON"
}
get_settings "$SETTINGS_JSON"
Solution: use jq( it's a lightweight and flexible command-line JSON processor.).
In bash I'd rather assign JSONs object to a variable and use jq in order to access and parse the right result from it. It's more convenient than parse this structure with arrays and it comes out of the box with multiple functionalities and features such as accessing nested and complex objects, select methods, builtin operators and functions, regex support ,comparisons etc...
example:
example='{"SALUTATION": "Hello world","SOMETHING": "bla bla bla Mr. Freeman"}'
echo $example | jq .SOMETHING
# output:
"bla bla bla Mr. Freeman"

Append elements of an array to the end of a line

First let me say I followed questions on stackoverflow.com that relate to my question and it seems the rules are not applying. Let me show you.
The following script:
#!/bin/bash
OUTPUT_DIR=/share/es-ops/Build_Farm_Reports/WorkSpace_Reports
TODAY=`date +"%m-%d-%y"`
HOSTNAME=`hostname`
WORKSPACES=( "bob" "mel" "sideshow-ws2" )
if ! [ -f $OUTPUT_DIR/$HOSTNAME.csv ] && [ $HOSTNAME == "sideshow" ]; then
echo "$TODAY","$HOSTNAME" > $OUTPUT_DIR/$HOSTNAME.csv
echo "${WORKSPACES[0]}," >> $OUTPUT_DIR/$HOSTNAME.csv
sed -i "/^'"${WORKSPACES[0]}"'/$/'"${WORKSPACES[1]}"'/" $OUTPUT_DIR/$HOSTNAME.csv
sed -i "/^'"${WORKSPACES[1]}"'/$/${WORKSPACES[2]}"'/" $OUTPUT_DIR/$HOSTNAME.csv
fi
I want the output to look like:
09-20-14,sideshow
bob,mel,sideshow-ws2
the sed statements are supposed to append successive array elements to preceding ones on the same line. Now I know there's a simpler way to do this like:
echo "${WORKSPACES[0]},${WORKSPACES[1]},${WORKSPACES[2]}" >> $OUTPUT_DIR/$HOSTNAME.csv
But let's say I had 30 elements in the array and I wanted to appended them one after the other on the same line? Can you show me how to loop through the elements in an array and append them one after the other on the same line?
Also let's say I had the output of a command like:
df -m /export/ws/$ws | awk '{if (NR!=1) {print $3}}'
and I wanted to append that to the end of the same line.
But when I run it I get:
+ OUTPUT_DIR=/share/es-ops/Build_Farm_Reports/WorkSpace_Reports
++ date +%m-%d-%y
+ TODAY=09-20-14
++ hostname
+ HOSTNAME=sideshow
+ WORKSPACES=("bob" "mel" "sideshow-ws2")
+ '[' -f /share/es-ops/Build_Farm_Reports/WorkSpace_Reports/sideshow.csv ']'
And the file right now looks like:
09-20-14,sideshow
bob,
I am happy to report that user syme solved this (see below) but then I realized I need the date in the first column:
09-7-14,bob,mel,sideshow-ws2
Can I do this using syme's for loop?
Okay user syme solved this too he said "Just add $TODAY to the for loop" like this:
for v in "$TODAY" "${WORKSPACES[#]}"
Okay now the output looks like this I changed the elements in the array btw:
sideshow
09-20-14,bob_avail,bob_used,mel_avail,mel_used,sideshow-ws2_avail,sideshow-ws2_used
Now below that the next line will be populated by a , in the first column skipping the date and then:
df -m /export/ws/$v | awk '{if (NR!=1) {print $3}}
which equals the value of available space on bob in the first iteration
and then:
df -m /export/ws/$v | awk '{if (NR!=1) {print $2}}
which equals the value of used space on bob in the 2nd iteration
and then we just move on to the next value in ${WORKSPACE[#]}
which will be mel and do the available and used as we did with bob or $v above.
I know you geniuses on here will make child's play out of this.
I solved my own last question on this thread:
WORKSPACES2=( "bob" "mel" "sideshow-ws2" )
separator="," # defined empty for the first value
for v in "${WORKSPACES2[#]}"
do
available=`df -m /export/ws/$v | awk '{if (NR!=1) {print $3}}'`
used=`df -m /export/ws/$v | awk '{if (NR!=1) {print $2}}'`
echo -n "$separator$available$separator$used" >> $OUTPUT_DIR/$HOSTNAME.csv # append, concatenated, the separator and the value to the file
done
produces:
sideshow
09-20-14,bob_avail,bob_used,mel_avail,mel_used,sideshow-ws2_avail,sideshow-ws2_used
,470400,1032124,661826,1032124,43443,1032108
echo -n permits to print text without the linebreak.
To loop over the values of the array, you can use a for-loop:
echo "$TODAY,$HOSTNAME" > $OUTPUT_DIR/$HOSTNAME.csv # with a linebreak
separator="" # defined empty for the first value
for v in "${WORKSPACES[#]}"
do
echo -n "$separator$v" >> $OUTPUT_DIR/$HOSTNAME.csv # append, concatenated, the separator and the value to the file
separator="," # comma for the next values
done
echo >> $OUTPUT_DIR/$HOSTNAME.csv # add a linebreak (if you want it)

Resources