I have a bash script that defines a function that returns dynamic JSON as a result:
# inside bash script
numInstances=getInstances();
Where an example of the JSON returned by getInstances() is:
[
[
{
"id": 193932,
"name": "foobaz"
},
{
"id": 28348,
"name": "fizzbuzz"
}
]
]
It's important to note that although the JSON is dynamic, it will always be a nested JSON array of the form:
[
[
<json list here; this is what I need a count of>
]
]
How can I pipe jq to the output of getInstances() so that it will parse the JSON and return the number of instances inside the nested JSON array? For example, in the above output, there are 2 "instances" inside the nested JSON array. So I would want numInstances to be set to 2, etc.
.[0]|length
would suffice, but is not very robust against a violation of assumptions.
Related
I have a use case for importing coordinates into a system. The coordinates are provided to me in a JSON format with each point having latitude, longitude, and elevation (sometimes). I'm using #jq to format the json file and remove everything but the coordinates. I have tried and googled methods to cycle through the array and remove any elevations, but no luck. I'm currently manually removing them using vim and reading through the coordinates. I would like to script it so I can use the system's API to fully automate receiving the coordinates and applying them to the system.
TIA.
The data looks like this when it arrives:
{ [ 48.2725225, 12.6538725, 595.2270812 ], [ 48.2725226, 12.6654544 ] }
I need it to be formatted like this without the elevations:
{ [ 48.2725225, 12.6538725 ], [ 48.2725226, 12.6654544 ] }
I've run through multiple data filters and wrote loops to iterate through each element of the array and remove the 3rd number.
Your input
{ [ 48.2725225, 12.6538725, 595.2270812 ], [ 48.2725226, 12.6654544 ] }
is not valid JSON. Curly object braces {} demand a comma-separated list of key-value pairs with a colon : in between.
If instead you had an array of arrays (both enclosed within square brackets []) like this:
[ [ 48.2725225, 12.6538725, 595.2270812 ], [ 48.2725226, 12.6654544 ] ]
you could use map on the outer array to individually process each inner array. That processing could then be to slice the array, reducing it to its first two elements .[:2]:
map(.[:2])
[ [48.2725225,12.6538725], [48.2725226,12.6654544] ]
Demo
If, however, you only have a stream of arrays like this (notice the lack of braces and commas compared to the outer array from above):
[ 48.2725225, 12.6538725, 595.2270812 ] [ 48.2725226, 12.6654544 ]
you can directly proceed with the slicing, no outer mapping is needed:
.[:2]
[48.2725225,12.6538725] [48.2725226,12.6654544]
Demo
Also notice that the output format provided here matches the input format (outer array or stream). Of course, you can convert one into another if you want (see the --slurp option or the item iterator .[] for that).
If your input format is neither, you'd have to apply other means to turn it into one of these.
[
{
"id": 4738245,
"project_id": 25486,
"sha": "871c0484c3a7e72deea96fbcd48djdgh2",
"ref": "",
"status": "success",
"created_at": "2022-01-06T04:44:40.372Z",
"updated_at": "2022-01-06T04:50:36.663Z",
"web_url": ""
}
]
The above is the json response I get from an gitlab API. I am trying to use jq '.id' to parse the id from the json response.
The error I get is :jq: error (at <stdin>:0): Cannot index array with string "id"
I understand that it should be jq '.<arrayName> .id' to parse it. However, the response does not give any arrayName so I cannot use that method.
Evidently the element id is nested within an object such like the other ones by wrapper curly braces, then call that as
jq '.[].id'
in order to get the value 4738245
I need to loop through this optional array (it's only the sectional of JSON I have trouble with).
As you can see from the code:
The optional bullseye has an array rings. rings has arrays of expansionCriteria and expansionCriteria may or may not have actions.
How do I iterate and get all type, threshold in expansionCriteria? I also need to access all skillsToRemove under actions, if available.
I am rather new to Logic Apps, so any help is appreciated.
"bullseye": {
"rings": [
{
"expansionCriteria": [
{
"type": "TIMEOUT_SECONDS",
"threshold": 180
}
],
"actions": {
"skillsToRemove": [
{
"name": "Claims Foundation",
"id": "60bd469a-ebab-4958-9ca9-3559636dd67d",
"selfUri": "/api/v2/routing/skills/60bd469a-ebab-4958-9ca9-3559636dd67d"
},
{
"name": "Claims Advanced",
"id": "bdc0d667-8389-4d1d-96e2-341e383476fc",
"selfUri": "/api/v2/routing/skills/bdc0d667-8389-4d1d-96e2-341e383476fc"
},
{
"name": "Claims Intermediate",
"id": "c790eac3-d894-4c00-b2d5-90cd8a69436c",
"selfUri": "/api/v2/routing/skills/c790eac3-d894-4c00-b2d5-90cd8a69436c"
}
]
}
},
{
"expansionCriteria": [
{
"type": "TIMEOUT_SECONDS",
"threshold": 5
}
]
}
]
}
Please let me know if you need more info.
To generate the schema, you can remove the name of the object at the top of the code: "bullseye":
Thank you pramodvalavala-msft for posting your answer in MS Q&A for the similar thread .
" As you are working with a JSON Object instead of an Array, unfortunately there is no built-in function to loop over the keys. There is a feature request to add a method to extract keys from an object for scenarios like this, that you could up vote for it gain more traction.
You can use the inline code action to extract the keys from your object as an array (using Object.keys()). And then you can loop over this array using the foreach loop to extract the object that you need from the main object, which you could then use to create records in dynamics."
For more information you can refer the below links:
. How to loop and extract items from Nested Json Array in Logic Apps .
.Nested ForEach Loop in Workflow. .
Im trying to load a JOSN file where some of the arrays are empty.
{"house_account_payable":"0.00","house_account_receivable":"0.00","gift_sales_payable":"0.00","gift_sales_receivable":"0.00","store_credit_sales_payable":"0.00","percentage_row":null,"sales_per_period":[["02:00AM - 02:59AM",{"amount":0,"qty":0}],["03:00AM - 03:59AM",{"amount":0,"qty":0}]],"revenue_centers":[],"tax_breakdowns":[]}
This is giving the error:
rror while reading table: test2, error message: Failed to parse JSON: No object found when new array is started.; BeginArray returned false; Parser terminated before end of string
Could somebody help me on this?
Are you trying to load data from your local machine or GCS? Please, remember about exporting in JSONL(Newline delimited JSON):
{"open_orders_ids": []}
{"unpaid_orders_ids": []}
The output:
Take a look for documentation about nested and repeated columns.
EDIT:
Your JSON schema should look like this:
{
"items": [
{
"house_account_payable": "0.00",
"house_account_receivable": "0.00",
"gift_sales_payable": "0.00",
"gift_sales_receivable": "0.00",
"store_credit_sales_payable": "0.00",
"percentage_row": "",
"sales_per_period": [
{
"AM02_00_AM02_59": {
"amount": "0",
"qty": "0"
}
},
{
"AM03_00_AM03_59": {
"amount": "0",
"qty": "0"
}
}
]
}
]
}
Regarding to Felipe Hoffa's post, run following commands:
jq -c .items[] <FILENAME>.json > <FILENAME>.jq.json
bq load --source_format NEWLINE_DELIMITED_JSON --autodetect <DATASET_ID>.<TABLENAME> <FILENAME>.jq.json
The schema:
Let me know if this is what you are looking for.
There's no problem with the null arrays.
The problem lies in this shorter json:
{"sales_per_period":[["02:00AM - 02:59AM",{"amount":0,"qty":0}],["03:00AM - 03:59AM",{"amount":0,"qty":0}]]}
The arrays there hold elements of different types, and to bring it into a structured table, a different schema is needed.
For example:
{"sales_per_period":[{"a":"02:00AM - 02:59AM","b":{"amount":0,"qty":0}},{"a":"03:00AM - 03:59AM","b":{"amount":0,"qty":0}}]}
Now this loads easily into BigQuery:
bq load --source_format=NEWLINE_DELIMITED_JSON --autodetect temp.short delete.short.json
Can you change this source JSON easily outside BigQuery? Otherwise load it raw into BigQuery, and parse it with a JS UDF inside BigQuery.
I have some data in Groovy's objects which I need to convert to a JSON string array. The final result should be..
[
{
"keys":{
"passcode": "12345"
},
"values":{
"EmailAddress": "john#doe.com",
"message": "Hello, is it me you are looking for?"
}
}
]
I found this post but the accepted answer which uses JSON Builder didn't make sense and the second answer using JSON converter didn't work...
def deJson = [
keys: [
passcode: secretCode
],
values: [
EmailAddress:emailData.to[0],
message: content.message
]
] as grails.converters.JSON
This created a JSON object when I needed an array.
Can anyone suggest how I can use JSON Builder or JSON converter to create an array string like I have above?
JSON is specific to Grails (as mentioned in one of the comments in the answer from the previous post). You should have followed the question itself in the post, that has the answer.
Using groovy.json.JsonBuilder, the expected way to serialize would be:
def jsonObj = [
keys: [
passcode: secretCode
],
values: [
EmailAddress:emailData.to[0],
message: content.message
]
]
// In order to get a json array, use a list containing the object
new JsonBuilder([jsonObj]).toPrettyString()