Related
I'm trying to filter and output from JSON with jq.
The API will sometime return an object and sometime an array, I want to catch the result using an if statement and return empty string when the object/array is not available.
{
"result":
{
"entry": {
"id": "207579",
"title": "Realtek Bluetooth Mesh SDK on Linux\/Android Segmented Packet reference buffer overflow",
"summary": "A vulnerability, which was classified as critical, was found in Realtek Bluetooth Mesh SDK on Linux\/Android (the affected version unknown). This affects an unknown functionality of the component Segmented Packet Handler. There is no information about possible countermeasures known. It may be suggested to replace the affected object with an alternative product.",
"details": {
"affected": "A vulnerability, which was classified as critical, was found in Realtek Bluetooth Mesh SDK on Linux\/Android (the affected version unknown).",
"vulnerability": "The manipulation of the argument reference with an unknown input leads to a unknown weakness. CWE is classifying the issue as CWE-120. The program copies an input buffer to an output buffer without verifying that the size of the input buffer is less than the size of the output buffer, leading to a buffer overflow.",
"impact": "This is going to have an impact on confidentiality, integrity, and availability.",
"countermeasure": "There is no information about possible countermeasures known. It may be suggested to replace the affected object with an alternative product."
},
"timestamp": {
"create": "1661860801",
"change": "1661861110"
},
"changelog": [
"software_argument"
]
},
"software": {
"vendor": "Realtek",
"name": "Bluetooth Mesh SDK",
"platform": [
"Linux",
"Android"
],
"component": "Segmented Packet Handler",
"argument": "reference",
"cpe": [
"cpe:\/a:realtek:bluetooth_mesh_sdk"
],
"cpe23": [
"cpe:2.3:a:realtek:bluetooth_mesh_sdk:*:*:*:*:*:*:*:*"
]
}
}
}
Would also like to to use the statement globally for the whole array output so I can parse it to .csv and escape the null, since sofware name , can also contain an array or an object. Having a global if statement with simplify the syntax result and suppress the error with ?
The error i received from bash
jq -r '.result [] | [ "https://vuldb.com/?id." + .entry.id ,.software.vendor // "empty",(.software.name | if type!="array" then [.] | join (",") else . //"empty" end )?,.software.type // "empty",(.software.platform | if type!="array" then [] else . | join (",") //"empty" end )?] | #csv' > tst.csv
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 7452 0 7393 100 59 4892 39 0:00:01 0:00:01 --:--:-- 4935
jq: error (at <stdin>:182): Cannot iterate over null (null)
What I have tried is the following code which i tried to demo https://jqplay.org/ which is incorrect syntax
.result [] |( if .[] == null then // "empty" else . end
| ,software.name // "empty" ,.software.platform |if type!="array" then [.] // "empty" else . | join (",") end)
Current output
[
[
"Bluetooth Mesh SDK"
],
"Linux,Android"
]
Desired outcome
[
"Bluetooth Mesh SDK",
"empty"
]
After fixing your input JSON, I think you can get the desired output by using the following JQ filter:
if (.result | type) == "array" then . else (.result |= [.]) end \
| .result[].software | [ .name, (.platform // [ "Empty" ] | join(",")) ]
Where
if (.result | type) == "array" then . else (.result |= [.]) end
Wraps .result in an array if type isn't array
.result[].software
Loops though the software in each .result obj
[ .name, (.platform // [ "Empty" ] | join(",")) ]
Create an array with .name and .platform (which is replaced by [ "Empty" ] when it doesn't exist. Then it's join()'d to a string
Outcome:
[
"Bluetooth Mesh SDK",
"Linux,Android"
]
Online demo
Or
[
"Bluetooth Mesh SDK",
"Empty
]
Online demo
I would like in JQ to substitute the values in the first array with the names in the second array based on the key "size" while keeping the length of the first array unchanged.
[
[{\"size\":0},{\"size\":1},{\"size\":2},{\"size\":3},{\"size\":4},{\"size\":5},{\"size\":6},{\"size\":7},{\"size\":8},{\"size\":9},{\"size\":0},{\"size\":1},{\"size\":2},{\"size\":3},{\"size\":4},{\"size\":5},{\"size\":6},{\"size\":7},{\"size\":8},{\"size\":0},{\"size\":1},{\"size\":2},{\"size\":3},{\"size\":4},{\"size\":5},{\"size\":6},{\"size\":7},{\"size\":8}]
[{\"size\":0,\"name\":\"6M\"},{\"size\":1,\"name\":\"6.5M\"},{\"size\":2,\"name\":\"7M\"},{\"size\":3,\"name\":\"7.5M\"},{\"size\":4,\"name\":\"8M\"},{\"size\":5,\"name\":\"8.5M\"},{\"size\":6,\"name\":\"9M\"},{\"size\":7,\"name\":\"9.5M\"},{\"size\":8,\"name\":\"10M\"},{\"size\":9,\"name\":\"11M\"}]
]
So the wanted result would look like this:
[{\"size\":6M},{\"size\":6.5M},{\"size\":7M},{\"size\":7.5M},{\"size\":8M},{\"size\":8.5M},{\"size\":9M},{\"size\":9.5M},{\"size\":10M},{\"size\":11M},{\"size\":6M},{\"size\":6.5M},{\"size\":7M},{\"size\":7.5M},{\"size\":8M},{\"size\":8.5M},{\"size\":9M},{\"size\":9.5M},{\"size\":10M},{\"size\":6M},{\"size\":6.5M},{\"size\":7M},{\"size\":7.5M},{\"size\":8M},{\"size\":8.5M},{\"size\":9M},{\"size\":9.5M},{\"size\":10M}]
I used this input to obtain the above arrays jq "[ .DATA.product.traits|([.colors.colorMap[]| {size:.sizes[]}]), ( [ .sizes.sizeMap[]| {size:.id, name}])]" and then tried using if-then-else statement (|if (.[0:-1][][].size == .[-1][].size) then [.[-1][].name ] else null end") but I was unable to reach the desired output..
Given the input
[
[
{"size":0}, {"size":1}, {"size":2}, {"size":3}, {"size":4},
{"size":5}, {"size":6}, {"size":7}, {"size":8}, {"size":9},
{"size":0}, {"size":1}, {"size":2}, {"size":3}, {"size":4},
{"size":5}, {"size":6}, {"size":7}, {"size":8}, {"size":0},
{"size":1}, {"size":2}, {"size":3}, {"size":4}, {"size":5},
{"size":6}, {"size":7}, {"size":8}
],
[
{"size":0,"name":"6M"}, {"size":1,"name":"6.5M"},
{"size":2,"name":"7M"}, {"size":3,"name":"7.5M"},
{"size":4,"name":"8M"}, {"size":5,"name":"8.5M"},
{"size":6,"name":"9M"}, {"size":7,"name":"9.5M"},
{"size":8,"name":"10M"}, {"size":9,"name":"11M"}
]
]
you can generate an INDEX over the second item, and use it for reference in a map on the first
(.[1] | INDEX(.size)) as $map | .[0] | map($map[.size | tostring])
to get
[
{"size":0,"name":"6M"}, {"size":1,"name":"6.5M"},
{"size":2,"name":"7M"}, {"size":3,"name":"7.5M"},
{"size":4,"name":"8M"}, {"size":5,"name":"8.5M"},
{"size":6,"name":"9M"}, {"size":7,"name":"9.5M"},
{"size":8,"name":"10M"}, {"size":9,"name":"11M"},
{"size":0,"name":"6M"}, {"size":1,"name":"6.5M"},
{"size":2,"name":"7M"}, {"size":3,"name":"7.5M"},
{"size":4,"name":"8M"}, {"size":5,"name":"8.5M"},
{"size":6,"name":"9M"}, {"size":7,"name":"9.5M"},
{"size":8,"name":"10M"}, {"size":0,"name":"6M"},
{"size":1,"name":"6.5M"}, {"size":2,"name":"7M"},
{"size":3,"name":"7.5M"}, {"size":4,"name":"8M"},
{"size":5,"name":"8.5M"}, {"size":6,"name":"9M"},
{"size":7,"name":"9.5M"}, {"size":8,"name":"10M"}
]
Demo
So, I'm trying to set up check_json.pl in NagiosXI to monitor some statistics. https://github.com/c-kr/check_json
I'm using the code with the modification I submitted in pull request #32, so line numbers reflect that code.
The json query returns something like this:
[
{
"total_bytes": 123456,
"customer_name": "customer1",
"customer_id": "1",
"indices": [
{
"total_bytes": 12345,
"index": "filename1"
},
{
"total_bytes": 45678,
"index": "filename2"
},
],
"total": "765.43gb"
},
{
"total_bytes": 123456,
"customer_name": "customer2",
"customer_id": "2",
"indices": [
{
"total_bytes": 12345,
"index": "filename1"
},
{
"total_bytes": 45678,
"index": "filename2"
},
],
"total": "765.43gb"
}
]
I'm trying to monitor the sized of specific files. so a check should look something like:
/path/to/check_json.pl -u https://path/to/my/json -a "SOMETHING" -p "SOMETHING"
...where I'm trying to figure out the SOMETHINGs so that I can monitor the total_bytes of filename1 in customer2 where I know the customer_id and index but not their position in the respective arrays.
I can monitor customer1's total bytes by using the string "[0]->{'total_bytes'}" but I need to be able to specify which customer and dig deeper into file name (known) and file size (stat to monitor) AND the working query only gives me the status (OK,WARNING, or CRITICAL). Adding -p all I get are errors....
The error with -p no matter how I've been able to phrase it is always:
Not a HASH reference at ./check_json.pl line 235.
Even when I can get a valid OK from the example "[0]->{'total_bytes'}", using that in -p still gives the same error.
Links pointing to documentation on the format to use would be very helpful. Examples in the README for the script or in the -h output are failing me here. Any ideas?
I really have no idea what your question is. I'm sure I'm not alone, hence the downvotes.
Once you have the decoded json, if you have a customer_id to search for, you can do:
my ($customer_info) = grep {$_->{customer_id} eq $customer_id} #$json_response;
Regarding the error on line 235, this looks odd:
foreach my $key ($np->opts->perfvars eq '*' ? map { "{$_}"} sort keys %$json_response : split(',', $np->opts->perfvars)) {
# ....................................... ^^^^^^^^^^^^^
$perf_value = $json_response->{$key};
if perfvars eq "*", you appear to be looking for $json_reponse->{"{total}"} for example. You might want to validate the user's input:
die "no such key in json data: '$key'\n" unless exists $json_response->{$key};
This entire business of stringifying the hash ref lookups just smells bad.
A better question would look like:
I have this JSON data. How do I get the sum of total_bytes for the customer with id 1?
See https://stackoverflow.com/help/mcve
Need to write a regex to get 3 groups from strings-
<whatever text including new lines optional -group 1>/command <text until \n or </p> is encountered- group 2><whatever text including new lines optional -group 3>
what I tried is-
Pattern pattern1 = Pattern.compile('(.*?)[/]command (.*?)\n?(.*?)');
It should give the following output for string-
some\nthing/command cmdtext/nasdfjaklsdjf\nfgskdlkfg\ndgsdfgsdfgsdfg
group 1 - some\nthing
group 2 - cmdtext
group 3 - asdfjaklsdjf\nfgskdlkfg\ndgsdfgsdfgsdfg
What I am not getting is how to get the occurrence of </p> and .* is not considering the group. Although this is working for me-
String a = '\na\na\n\n\n\n\n\naaa';
Pattern pattern2 = Pattern.compile('\n(?s:.)*');
Matcher mchr = GiphyPattern.matcher(a);
system.assert (mchr.matches());
This regular expression should match what you need:
'([\\s\\S]*)/command (.*?)(?:\n|</p>)([\\s\\S]*)'
You cannot match \n with .* So I am using \\s\\S instead (which is actually \s\S but with Apex escaped backslashes).
Please help me on the below code :
I have an array with 155 elements and i have a file which has some elements of array inside it , i need all values of the array elements which are found in the file and also i need the array element to be printed as zero if the array element is not found in the file .
Thanks in advance, this is what i have tried.
args=("C9" "DP10" "DP11" "DP20" "DP21" "DP30" "DP31" "DP50" "FR31" "G128" "G402" "G602" "GA" "GI" "GT08" "GT14" "GT17" "GT25" "GT37" "GT67" "H6" "H7" "IL" "IM" "J6" "JD05" "JD09" "JD14" "JD25" "JD37" "K1" "K2" "L100" "L106" "L116" "L150" "L202" "L7" "L8" "L9" "LD11" "LD21" "LE09" "LE26" "LP11" "LP21" "LP31" "LP55" "LQ11" "LQ21" "LQ31" "LS07" "LT09" "LT10" "LT12" "LT15" "LT20" "LT22" "LT24" "LT25" "LT30" "LT38" "LT42" "LT43" "LT44" "LT48" "LT50" "LT59" "LT60" "LT65" "M395" "OV04" "OV07" "OV14" "OV18" "OV23" "OV27" "OV35" "OV39" "OV40" "OV79" "Q15" "Q150" "Q19" "QD11" "QD21" "QD31" "QD65" "QE11" "QE21" "QE31" "QF50" "QM25" "QP10" "QP15" "QP20" "QP30" "QP31" "QP50" "QT25" "QT50" "R39" "R40" "r57" "R9" "rc23" "RC27" "RC39" "rc7" "rc79" "S1" "S101" "S117" "S118" "S13p" "S18" "S202" "S317" "S318" "S319" "S40" "S408" "S67" "S76" "S82" "S99" "SD11" "SD12" "SD14" "SD17" "SD29" "SD3" "SD5" "SD98" "SF20" "SF74" "SR07" "SV19" "SV6p" "T402" "T602" "TG00" "TG17" "TG43" "TG8" "TG92" "WD09" "WD14" "WD17" "WD24" "WD29" "WD37" "WD43" "WWE1" "XR91")
MY CODE :
for loop i have used to traverse the elements search inside a file .
for i in ${args[#]}; do
grep $i file.txt
if [ $? -ne 0 ]; then
echo $i"","""0"
fi
done >> output.txt
TOTAL FILE:
C9,5015319
DP10,36870732
DP11,188
DP20,18728254
DP21,341182
DP30,8415555
DP31,2390000
DP50,12371853
FR31,24541
G128,49780
G402,2000
G602,2000
GA,879888
GT08,1580384
GT17,1968192
GT25,4104
GT37,21550
GT67,24770
H6,660652
IL,137651
JD05,1518400
JD14,325800
JD25,828600
JD37,357100
K1,261549
K2,4715330
L100,284
L116,80000
L7,200847
L8,3158
L9,5054495
LE09,75776
LE26,343410
LP11,1030
LP21,492
LP31,113
LP55,3
LQ11,6776000
LQ21,3543600
LQ31,4525600
LT09,682800
LT12,5715
LT15,568873
LT22,236077
LT24,702800
LT25,4600
LT38,28990
LT65,300125
M395,29600
OV14,462
OV18,86300
OV40,217899
Q150,678
QD11,1000022
QD31,50
QF50,58575
QM25,57900
QP10,1792153
QP15,953400
QP20,770000
QP30,179450
QP31,163223
QP50,8
QT50,66340
R39,62440
R40,18807
r57,3456
rc23,3370
RC27,2809
RC39,2570
rc7,7137
rc79,1296
S1,25007
S117,1000000
S13p,52313
S18,75000
S317,289148
S318,3046
S319,30000
S40,300
S408,4967
S76,28
S82,103238
S99,480
SD11,6719
SD12,23123
SD14,22595
SD17,100000
SD29,252392
SD3,20000
SD5,14090
SD98,653
SF20,1000
SF74,7330
SV19,26461
SV6p,154994
T402,2000
T602,2000
TG17,2031
TG8,2964
TG92,1759
WD17,131194
WD24,94589
WD29,202198
WD37,101794
WD43,112942
WWE1,9600
XR91,70000
EXPECTED OUTPUT :
The output should contain the values which are present in the file for each array element.
If not present the output should contain the array element as zero. For eg:
c9 is not present in the file
output of c9 should be
c9,0
Your approach is not bad. I just would use
^$i,
as a grep-pattern. With your current file data, it's not necessary, but maybe one day your file will contain things like
X,2354
XA,1234
and suddenly your algorithm will fail, if args contain the element X.
Also, the echo statement is unnecessarily complex. I would write it simply as
echo $x,0
You can also simplify the if, by combining it with the grep
if ! grep ^$i, file.txt
but this is mere cosmetics and a matter of taste.