I want to transform JSON file into bash array of strings that i will later be able to iterate over. My JSON structure is as follows:
[
{
"USERID": "TMCCP",
"CREATED_DATE": "31/01/2020 17:52"
},
{
"USERID": "TMCCP",
"CREATED_DATE": "31/01/2020 17:52"
}
]
And this is my bash script:
test_cases=($(jq -c '.[]' data.json))
echo ${test_cases[0]}
echo ${test_cases[1]}
echo ${test_cases[2]}
echo ${test_cases[3]}
As you can see it returns array with 4 elements instead of 2. Output:
{"USERID":"TMCCP","CREATED_DATE":"31/01/2020
17:52"}
{"USERID":"TMCCP","CREATED_DATE":"31/01/2020
17:52"}
For some reason having whitespace in date field causes some parsing issues. Any idea how to get over this?
Use readarray instead.
$ readarray -t test_cases < <(jq -c '.[]' file)
$ declare -p test_cases
declare -a test_cases=([0]="{\"USERID\":\"TMCCP\",\"CREATED_DATE\":\"31/01/2020 17:52\"}" [1]="{\"USERID\":\"TMCCP\",\"CREATED_DATE\":\"31/01/2020 17:52\"}")
And read can be used as shown below where readarray is unavailable.
IFS=$'\n' read -d '' -a test_cases < <(jq -c '.[]' file)
Use readarray to populate the array, rather than using an unquoted command substitution; bash doesn't care about JSON quoting when it splits the result into separate words.
readarray -t test_cases < <(jq -c '.[]' data.json)
In bash 3.2 (which is what you appear to be stuck with), you need something slightly more unwieldy
while IFS= read -r line; do
test_cases+=("$line")
done < <(jq -c '.[]' data.json)
Related
How to parse and convert string list to JSON string array in shell command?
'["test1","test2","test3"]'
to
test1
test2
test3
I tried like below:
string=$1
array=${string#"["}
array=${array%"]"}
IFS=',' read -a array <<< $array;
echo "${array[#]}"
Any other optimized way?
As bash and jq are tagged, this solution relies on both (without summoning eval). The input string is expected to be in $string, the output array is generated into ${array[#]}. It is robust wrt spaces, newlines, quotes, etc. as it uses NUL as delimiter.
mapfile -d '' array < <(jq -j '.[] + "\u0000"' <<< "$string")
Testing
string='["has spaces\tand tabs","has a\nnewline","has \"quotes\""]'
mapfile -d '' array < <(jq -j '.[] + "\u0000"' <<< "$string")
printf '==>%s<==\n' "${array[#]}"
==>has spaces and tabs<==
==>has a
newline<==
==>has "quotes"<==
eval "array=($( jq -r 'map( #sh ) | join(" ")' <<<"$json" ))"
This is how I am grabbing all the NVME volumes:
all_nvme_volumes=$(sudo nvme list -o json | jq .Devices[].DevicePath)
This how the output looks like:
"/dev/nvme0n1" "/dev/nvme1n1" "/dev/nvme2n1" "/dev/nvme3n1" "/dev/nvme4n1" "/dev/nvme6n1"
How do I loop thru them process them individually?
I tried for r in "${all_nvme_volumes[#]}"; do echo "Device Name: $r"; done but the output is Device Name: "/dev/nvme0n1" "/dev/nvme1n1" "/dev/nvme2n1" "/dev/nvme3n1" "/dev/nvme4n1" "/dev/nvme6n1"
which is one string instead of each element of array:
Populating a bash array with mapfile from null delimited raw output from jq:
#!/usr/bin/env bash
mapfile -d '' all_nvme_volumes < <(
sudo nvme list --output-format=json |
jq --join-output '.Devices[].DevicePath + "\u0000"'
)
A solution for bash < 4.4:
#!/bin/bash
IFS=$'\t' read -r -a all_nvme_volumes < <(
sudo nvme list -o json | jq -r '[ .Devices[].DevicePath ] | #tsv'
)
note: device paths shouldn't be escaped by #tsv, so you won't need to unescape the values, but in case you use this trick for other purposes, you can unescape a value with printf -v value '%b' "$value"
How do I loop thru them process them individually?
Well, once you have the array, you can loop though its elements with:
for nvme_volume in "${all_nvme_volumes[#]}"
do
# process "$nvme_volume"
done
But, if you only need to loop though the nvme volumes without storing them then you can use #LéaGris null delimiter method with a while loop:
#!/bin/bash
while IFS='' read -r -d '' nvme_volume
do
# process "$nvme_volume"
done < <(sudo nvme list -o json | jq -j '.Devices[].DevicePath + "\u0000"')
This question already has answers here:
How to split a string into an array in Bash?
(24 answers)
Closed 1 year ago.
Can somebody help me out. I want to split TEXT(variable with \n) into array in bash.
Ok, I have some text-variable:
variable='13423exa*lkco3nr*sw
kjenve*kejnv'
I want to split it in array.
If variable did not have new line in it, I will do it by:
IFS='*' read -a array <<< "$variable"
I assumed the third element should be:
echo "${array[2]}"
>sw
>kjenve
But with new line it is not working. Please give me right direction.
Use readarray.
$ variable='13423exa*lkco3nr*sw
kjenve*kejnv'
$ readarray -d '*' -t arr < <(printf "%s" "$variable")
$ declare -p arr
declare -a arr=([0]="13423exa" [1]="lkco3nr" [2]=$'sw\nkjenve' [3]="kejnv")
mapfile: -d: invavlid option
Update bash, then use readarray.
If not, replace separator with zero byte and read it element by element with read -d ''.
arr=()
while IFS= read -d '' -r e || [[ -n "$e" ]]; do
arr+=("$e")
done < <(printf "%s" "$variable" | tr '*' '\0');
declare -p arr
declare -a arr=([0]="13423exa" [1]="lkco3nr" [2]=$'sw\nkjenve' [3]="kejnv")
You can use the readarray command and use it like in the following example:
readarray -d ':' -t my_array <<< "a:b:c:d:"
for (( i = 0; i < ${#my_array[*]}; i++ )); do
echo "${my_array[i]}"
done
Where the -d parameter defines the delimiter and -t ask to remove last delimiter.
Use a ending character different than new line
end=.
read -a array -d "$end" <<< "$v$end"
Of course this solution suppose there is at least one charecter not used in your input variable.
In Bash, how can I get the strings between acolades (without the '_value' suffix) from for example
"\\*\\* ${host_name_value}.${host_domain_value} - ${host_ip_value}\\*\\*"
and put them into an array?
The result for the above example should be something like:
var_array=("host_name" "host_domain")
The string could also contain other stuff such as:
"${package_updates_count_value} ${package_updates_type_value} updates"
The result for the above example should be something like:
var_array=("package_updates_count" "package_updates_type")
All variables end with _value. There could 1 or more variables in the string.
Not sure what would be the most efficient way and how I'd best handle this. Regex? Sed?
input='\\*\\* ${host_name_value}.${host_domain_value} \\*\\*'
# would also work with cat input or the like.
myarray=($(echo "$input" | awk -F'$' \
'{for(i=1;i<=NF;i++) {match($i, /{([^}]*)_value}/, a); print a[1]}}'))
Split your line(s) on $. Check if a column contains { }. If it does, print what's after { and before _value}. (If not, it will print out the empty string, which bash array creation will ignore.)
If there are only two variables, this will work.
input='\\*\\* ${host_name_value}.${host_domain_value} \\*\\*'
first=$(echo $input | sed -r -e 's/[}].+//' -e 's/.+[{]//')
last=$(echo $input | sed -r -e 's/.+[{]//' -e 's/[}].+//')
output="var_array=(\"$first\" \"$last\")"
Maybe not very efficient and beautiful, but it works well.
Starting with a string variable:
$ str='\\*\\* ${host_name_value}.${host_domain_value} - ${host_ip_value}\\*\\*'
Use grep -o to print all matching words.
$ grep -o '\${\w*_value}' <<< "$str"
${host_name_value}
${host_domain_value}
${host_ip_value}
Then remove ${ and _value}.
$ grep -o '\${\w*_value}' <<< "$str" | sed 's/^\${//; s/_value}$//'
host_name
host_domain
host_ip
Finally, use readarray to safely read the results into an array.
$ readarray -t var_array < <(grep -o '\${\w*_value}' <<< "$str" | sed 's/^\${//; s/_value}$//')
$ declare -p var_array
declare -a var_array=([0]="host_name" [1]="host_domain" [2]="host_ip")
How can I get just the filenames into an array using the cat command?
How I've been trying:
array=()
while IFS= read -r -d $'\0'; do
array+=("$REPLY")
done < <(cat /proc/swaps | grep "swap")
This either grabs all the information from the output into an array, or just doesn't work. How can I successfully get my expected output of [/swapfile, /dev/hda1, /some/other/swap] into an array form using the cat command?
readarray array < <(awk '/swap/{print $1}' /proc/swaps)
Bash introduced readarray in version 4 which can take the place of the while read loop. readarray is the solution you want.
here is the syntax
readarray variable < inputfile
echo "${variable[0]}" ' to print the first element in array