I need to put the json keys of a file as a array in a shell script how can i do that?
{
"employee4" : {
"aliases" : { }
},
"employee3" : {
"aliases" : { }
},
"employee" : {
"aliases" : { }
},
"employee2" : {
"aliases" : { }
}
}
I need to have a array like keys["employee", "employee2", "employee3", "employee4"]
If there is more keys the array need to find them
The jq keys function returns a list of keys. So with your example data in data.json, we see:
$ jq 'keys' data.json
[
"employee",
"employee2",
"employee3",
"employee4"
]
To get rid of the JSON list, we run:
$ jq -r 'keys[]' data.json
employee
employee2
employee3
employee4
And to get that into a bash array:
myarray=( $(jq -r 'keys[]' data.json) )
As #glennjackman mentions in a comment, the above construct will have problems if your keys contain whitespace or shell special characters. For example, given this data:
{
"employee*" : {
"aliases" : { }
}
}
If your directory contains files named employee1 and employee2, then you'll get, effectively:
myarray=( employee1 employee2 )
...which is not what you want. You can fix this by using the mapfile builtin (also known as readarray, which makes its purpose more obvious):
mapfile -t myarray < <(jq -r 'keys[]' data.json)
The JSON-parser xidel can do what you want.
To return the JSON keys / attributes:
$ xidel -s input.json -e '$json()' # or -e 'map:keys($json)'
employee4
employee3
employee
employee2
To create the Bash array:
$ mapfile -t myArr < <(xidel -s input.json -e '$json()')
Alternatively, xidel can do this too with --output-format=bash:
$ eval "$(xidel -s input.json -e 'myArr:=$json()' --output-format=bash)"
In both cases this will result in:
$ printf '%s\n' "${myArr[*]}" ${#myArr[*]} ${myArr[0]} ${myArr[1]}
employee4 employee3 employee employee2
4
employee4
employee3
Related
I want to transform JSON file into bash array of strings that i will later be able to iterate over. My JSON structure is as follows:
[
{
"USERID": "TMCCP",
"CREATED_DATE": "31/01/2020 17:52"
},
{
"USERID": "TMCCP",
"CREATED_DATE": "31/01/2020 17:52"
}
]
And this is my bash script:
test_cases=($(jq -c '.[]' data.json))
echo ${test_cases[0]}
echo ${test_cases[1]}
echo ${test_cases[2]}
echo ${test_cases[3]}
As you can see it returns array with 4 elements instead of 2. Output:
{"USERID":"TMCCP","CREATED_DATE":"31/01/2020
17:52"}
{"USERID":"TMCCP","CREATED_DATE":"31/01/2020
17:52"}
For some reason having whitespace in date field causes some parsing issues. Any idea how to get over this?
Use readarray instead.
$ readarray -t test_cases < <(jq -c '.[]' file)
$ declare -p test_cases
declare -a test_cases=([0]="{\"USERID\":\"TMCCP\",\"CREATED_DATE\":\"31/01/2020 17:52\"}" [1]="{\"USERID\":\"TMCCP\",\"CREATED_DATE\":\"31/01/2020 17:52\"}")
And read can be used as shown below where readarray is unavailable.
IFS=$'\n' read -d '' -a test_cases < <(jq -c '.[]' file)
Use readarray to populate the array, rather than using an unquoted command substitution; bash doesn't care about JSON quoting when it splits the result into separate words.
readarray -t test_cases < <(jq -c '.[]' data.json)
In bash 3.2 (which is what you appear to be stuck with), you need something slightly more unwieldy
while IFS= read -r line; do
test_cases+=("$line")
done < <(jq -c '.[]' data.json)
I actually have 2 arrays in bash that contains string values.
Something like that :
Array1=(Kevin Paul)
Array2=(OK DANGER)
I would like to create a json with 2 attributes, something like that if possible
{
"results":[
{
"nom":"Kevin",
"status":"OK"
},
{
"nom":"Paul",
"status":"Danger"
}
]
}
I read a lot speaking about JQ that i alreay use for my arrays, but no one speak about something like i want :(
One of my test ( that does not respond to what i would like ) :
declare -a test_array
declare -a test_array2
test_array=(apple orange lemon)
test_array2=(DANGER OK WARNING)
echo ${test_array[0]}
echo '['
printf '{"CVEC": "%s", "LVL" : "%s"},\n' "${test_array[#]}, ${test_array2[#]}" | sed '$s/,$//'
echo ']'
Display
[
{"CVEC": "apple", "LVL" : "orange"},
{"CVEC": "lemon, DANGER", "LVL" : "OK"},
{"CVEC": "WARNING", "LVL" : ""}
]
Using a template engine: perl's Template::Toolkit command line tool: tpage:
Files
header:
{
"results":[
footer:
]
}
file.tpl (template):
{
"nom": "[% x1 %]",
"status": "[% x2 %]"
}[% sep %]
Bash script
#!/bin/bash
arr1=( Kevin Paul )
arr2=( OK danger )
{
cat header
for i in "${!arr1[#]}"; do
((i==${#arr1[#]}-1)) && sep='' || sep=','
tpage --define x1="${arr1[i]}" \
--define x2="${arr2[i]}" \
--define sep=$sep file.tpl
done
cat footer
} | tee file.json
Validation
$ jq . file.json
{
"results": [
{
"nom": "Kevin",
"status": "OK"
},
{
"nom": "Paul",
"status": "danger"
}
]
}
Package
For debian and debian like:
apt install libtemplate-perl
Via CPAN:
cpan -i Template::Toolkit
Check http://www.template-toolkit.org/docs/tools/tpage.html
One way:
paste <(printf "%s\n" "${Array1[#]}") <(printf "%s\n" "${Array2[#]}") |
jq -nRc '{ results: [inputs] | map(split("\t") | { nom: .[0], status: .[1] }) }'
produces
{"results":[{"nom":"Kevin","status":"OK"},{"nom":"Paul","status":"DANGER"}]}
This assumes that the elements of your arrays do not have tabs or newlines in them. It uses paste to generate pairs of corresponding array elements, separated by tabs, one pair per line, and then uses jq to create the JSON output from that.
If the objective is to stick with a non-jq solution - and chepner's comments about needing to validate the array entries is not an issue for this situation - one idea would be to loop through the array indices.
Test data:
$ declare -a test_array=(apple orange lemon)
$ typeset -p test_array
declare -a test_array=([0]="apple" [1]="orange" [2]="lemon")
$ declare -a test_array2=(DANGER OK WARNING)
$ typeset -p test_array2
declare -a test_array2=([0]="DANGER" [1]="OK" [2]="WARNING")
A simple loop through the indices (0,1,2):
sfx=',' # default printf suffix is a comma
for (( i=0 ; i<${#test_array[#]} ; i++ ))
do
(( ${i} == ( ${#test_array[#]} - 1 ) )) && sfx='' # clear suffix for last pass through loop
printf '{"CVEC": "%s", "LVL" : "%s"}%s\n' "${test_array[${i}]}" "${test_array2[${i}]}" "${sfx}"
done
Which generates the following:
{"CVEC": "apple", "LVL" : "DANGER"},
{"CVEC": "orange", "LVL" : "OK"},
{"CVEC": "lemon", "LVL" : "WARNING"}
I parsed a json file with jq like this :
# cat test.json | jq '.logs' | jq '.[]' | jq '._id' | jq -s
It returns an array like this : [34,235,436,546,.....]
Using bash script i described an array :
# declare -a msgIds = ...
This array uses () instead of [] so when I pass the array given above to this array it won't work.
([324,32,45..]) this causes problem. If i remove the jq -s, an array forms with only 1 member in it.
Is there a way to solve this issue?
We can solve this problem by two ways. They are:
Input string:
// test.json
{
"keys": ["key1","key2","key3"]
}
Approach 1:
1) Use jq -r (output raw strings, not JSON texts) .
KEYS=$(jq -r '.keys' test.json)
echo $KEYS
# Output: [ "key1", "key2", "key3" ]
2) Use #sh (Converts input string to a series of space-separated strings). It removes square brackets[], comma(,) from the string.
KEYS=$(<test.json jq -r '.keys | #sh')
echo $KEYS
# Output: 'key1' 'key2' 'key3'
3) Using tr to remove single quotes from the string output. To delete specific characters use the -d option in tr.
KEYS=$((<test.json jq -r '.keys | #sh')| tr -d \')
echo $KEYS
# Output: key1 key2 key3
4) We can convert the comma-separated string to the array by placing our string output in a round bracket().
It also called compound Assignment, where we declare the array with a bunch of values.
ARRAYNAME=(value1 value2 .... valueN)
#!/bin/bash
KEYS=($((<test.json jq -r '.keys | #sh') | tr -d \'\"))
echo "Array size: " ${#KEYS[#]}
echo "Array elements: "${KEYS[#]}
# Output:
# Array size: 3
# Array elements: key1 key2 key3
Approach 2:
1) Use jq -r to get the string output, then use tr to delete characters like square brackets, double quotes and comma.
#!/bin/bash
KEYS=$(jq -r '.keys' test.json | tr -d '[],"')
echo $KEYS
# Output: key1 key2 key3
2) Then we can convert the comma-separated string to the array by placing our string output in a round bracket().
#!/bin/bash
KEYS=($(jq -r '.keys' test.json | tr -d '[]," '))
echo "Array size: " ${#KEYS[#]}
echo "Array elements: "${KEYS[#]}
# Output:
# Array size: 3
# Array elements: key1 key2 key3
To correctly parse values that have spaces, newlines (or any other arbitrary characters) just use jq's #sh filter and bash's declare -a. (No need for a while read loop or any other pre-processing)
// foo.json
{"data": ["A B", "C'D", ""]}
str=$(jq -r '.data | #sh' foo.json)
declare -a arr="($str)" # must be quoted like this
$ declare -p arr
declare -a arr=([0]="A B" [1]="C'D" [2]="")
The reason that this works correctly is that #sh will produce a space-separated list of shell-quoted words:
$ echo "$str"
'A B' 'C'\''D' ''
and this is exactly the format that declare expects for an array definition.
Use jq -r to output a string "raw", without JSON formatting, and use the #sh formatter to format your results as a string for shell consumption. Per the jq docs:
#sh:
The input is escaped suitable for use in a command-line for a POSIX shell. If the input is an array, the output will be a series of space-separated strings.
So can do e.g.
msgids=($(<test.json jq -r '.logs[]._id | #sh'))
and get the result you want.
From the jq FAQ (https://github.com/stedolan/jq/wiki/FAQ):
𝑸: How can a stream of JSON texts produced by jq be converted into a bash array of corresponding values?
A: One option would be to use mapfile (aka readarray), for example:
mapfile -t array <<< $(jq -c '.[]' input.json)
An alternative that might be indicative of what to do in other shells is to use read -r within a while loop. The following bash script populates an array, x, with JSON texts. The key points are the use of the -c option, and the use of the bash idiom while read -r value; do ... done < <(jq .......):
#!/bin/bash
x=()
while read -r value
do
x+=("$value")
done < <(jq -c '.[]' input.json)
++ To resolve this, we can use a very simple approach:
++ Since I am not aware of you input file, I am creating a file input.json with the following contents:
input.json:
{
"keys": ["key1","key2","key3"]
}
++ Use jq to get the value from the above file input.json:
Command: cat input.json | jq -r '.keys | #sh'
Output: 'key1' 'key2' 'key3'
Explanation: | #sh removes [ and "
++ To remove ' ' as well we use tr
command: cat input.json | jq -r '.keys | #sh' | tr -d \'
Explanation: use tr delete -d to remove '
++ To store this in a bash array we use () with `` and print it:
command:
KEYS=(`cat input.json | jq -r '.keys | #sh' | tr -d \'`)
To print all the entries of the array: echo "${KEYS[*]}"
I have one json e.g
var bbview ={
"tbl_am_api":[
{
"Modified_User":"user1",
"Modified_Time":"04-Jul-2018 01:40:05",
"Line_Number":"SS001",
"Service_Type":"BB3",
"Status":"Yes",
"ID":3144526000014337832,
"Added_Time":"04-May-2018 11:37:29"
},
{
"Modified_User":"user2",
"Modified_Time":"04-Jul-2018 01:40:05",
"Line_Number":"SS002",
"Service_Type":"BB2",
"Status":"Yes",
"ID":3144526000014337832,
"Added_Time":"04-May-2018 11:37:29"
},
{
"Modified_User":"user3",
"Modified_Time":"04-Jul-2018 01:40:05",
"Line_Number":"SS004",
"Service_Type":"BB1",
"Status":"No",
"ID":3144526000014337832,
"Added_Time":"04-May-2018 11:37:29"
}
]
};
I want to compare this json data and array. Primary key as Line Number.
arrayA = {[{Line_Number : SS001, Service_Type : BB3; Status : Yes}]}
arrayA have Line_Number SS001. Find this Line_Number in json and compare Service_Type value and Status value are same or not. I want to write with Shell Script in bash file. I am not proficient in shell script. Please help me.
Update:
I tried with following bash code. But still fail. Please advice me
echo "Download FMS AM API File"
rm -rf tbl_am_api_Report?authtoken=da84d49f334c33b88d30dd2c947a4ff0 && wget -q https://creator.zoho.com/api/json/fixed-management-system/view/tbl_am_api_Report?authtoken=da84d49f334c33b88d30dd2c947a4ff0&scope=creatorapi&zc_ownername=tmlbroadband < /dev/null
cat > tbl_api_Report?authtoken=da84d49f334c33b88d30dd2c947a4ff0 //read json file
for row in $(echo "${apiview}" | jq -r '.[] | #base64'); do
_jq() {
echo ${row} | base64 --decode | jq -r ${1}
}
echo $(_jq '.name') >> info.txt
done
mail -s "Test email" aa#gmail.com -A info.txt < /dev/null
The value of arrayA is not JSON so I am going to let you figure out how to extract the values of Line_Number and Status from arrayA (but see below). Once those values are available, one could proceed as illustrated here:
#!/bin/bash
bbview='...' # as above
echo "$bbview" |
jq --arg Line_Number SS001 --arg Status Yes '
.tbl_am_api
| map(select(.Line_Number==$Line_Number and .Status==$Status)) '
Output
[
{
"Modified_User": "user1",
"Modified_Time": "04-Jul-2018 01:40:05",
"Line_Number": "SS001",
"Service_Type": "BB3",
"Status": "Yes",
"ID": 3144526000014338000,
"Added_Time": "04-May-2018 11:37:29"
}
]
true/false
Under a different reading of the question, the following variant might be relevant:
echo "$bbview" |
jq --arg Line_Number SS001 --arg Status Yes '
.tbl_am_api
| map(select(.Line_Number==$Line_Number) | .Status==$Status) '
arrayA
If you are using a version of bash that supports associative arrays, you could define arrayA as an associative array, like so:
declare -A arrayA
arrayA=([Line_Number]=SS001 [Service_Type]=BB3 [Status]=Yes)
Then to retrieve the value associated with Line_Number, you would write: ${arrayA[Line_Number]}; etc.
I have a json array that looks like this:
{
"StackSummaries": [
{
"CreationTime": "2016-06-01T22:22:49.890Z",
"StackName": "foo-control-eu-west-1",
"StackStatus": "UPDATE_COMPLETE",
"LastUpdatedTime": "2016-06-01T22:47:58.433Z"
},
{
"CreationTime": "2016-04-13T11:22:04.250Z",
"StackName": "foo-bar-testing",
"StackStatus": "UPDATE_COMPLETE",
"LastUpdatedTime": "2016-04-26T16:17:07.570Z"
},
{
"CreationTime": "2016-04-10T01:09:49.428Z",
"StackName": "foo-ldap-eu-west-1",
"StackStatus": "UPDATE_COMPLETE",
"LastUpdatedTime": "2016-04-17T13:44:04.758Z"
}
]
}
I am looking to create text output that looks like this:
foo-control-eu-west-1
foo-bar-testing
foo-ldap-eu-west-1
Is jq able to do this? Specifically, what would the jq command line be that would select each StackName in the array and output each key one per line?
jq --raw-output '.StackSummaries[].StackName'
$ jq -r '[.StackSummaries[] | .StackName] | unique[]' input.json
foo-bar-testing
foo-control-eu-west-1
foo-ldap-eu-west-1
The -r option strips the quotation marks from the output. You might not want the call to 'unique'.
For reference, if you wanted all the key names:
$ jq '[.StackSummaries[] | keys[]] | unique' input.json
[
"CreationTime",
"LastUpdatedTime",
"StackName",
"StackStatus"
]
Here is another solution
jq -M -r '..|.StackName?|values' input.json