I have this JSON file and want to delete an element from an array:
{
"address": "localhost",
"name": "local",
"vars": {
"instances": [
"one",
"two"
]
}
}
I am using this command:
jq 'del(.vars.instances[] | select(index("one")))' data.json
The output is:
{
"address": "localhost",
"name": "local",
"vars": {
"instances": [
"two"
]
}
}
So it works as expected, but only with jq v1.6. With jq v1.5 I get this error:
jq: error (at data.json:20): Invalid path expression near attempt to access element 0 of [0]
So what am I doing wrong? Is this a bug or a feature of v1.5? Is there any workaround to get the same result in v1.5?
Thanks in advance
Vince
One portable to work with on both versions would be,
.vars.instances |= map(select(index("one")|not))
or if you want to still use del(), feed the index of the string "one" to the function as below, where index("one") gets the index 0 which then gets passed to delete as del(.[0]) meaning to delete the element at zeroth index.
.vars.instances |= del(.[index("one")])
The implementation of del/1 has proven to be quite difficult and indeed it changed between jq 1.5 and jq 1.6, so if portability across different versions of jq is important, then usage of del/1 should either be restricted to the least complicated cases (e.g., no pipelines) or undertaken with great care.
Related
I have a JSON similar to:
{
"orders":{
"678238": {
"orderId": 678238,
"itemName": "Keyboard"
},
"8723423": {
"orderId": 8723423,
"itemName": "Flash Drive"
}
}
}
I am trying JSON path to get first orderId. When I try $..orderId I get an array listing both orderId, then I tried $..[0].orderId to get first item from that array (following JsonPath - Filter Array and get only the first element). But it does not work. I am confused.
try this
console.log(jsonPath(json,"$['orders'].[orderId]")[0]); //678238
You're almost there. You need to combine the two things you've done.
$..orderId[0]
The ..orderId recursively searches for orderId properties, giving you all of their values, as you mentioned. Taking that result, you just need to apply the [0].
Be careful, though. Because your data is an object, keys are unordered, so the first one in the JSON text may not be the first one encountered in memory. You'll want to do some testing to confirm your results are consistent with your expectations.
Your JSON doesn't even have an array and you are expecting to get first item from the array which is why it's not working.
Suppose, if the structure of JSON is modified like this
{
"orders": [{
"orderId": 678238,
"itemName": "Keyboard"
},
{
"orderId": 8723423,
"itemName": "Flash Drive"
}
]
}
then you can use the query to get the first order.
$.orders[0].orderId
i'm writing a script that needs to parse incoming json into line-by-line data, taking information from the json at multiple different levels. i'm using jq to parse the data.
the incoming json is an array of 'tasks'. each task [i.e. each element of the array] is an object that looks like this :
{
"inputData": {
"transfers": [
{
"source": {
"directory": "/path/to/source",
"filename": "somefile.mp3"
},
"target": {
"directory": "/path/to/target",
"filename": "somefile.mp3"
}
},
{
"source": {
"content": "<?xml version=\"1.0\" encoding=\"UTF-8\"?><delivery>content description</delivery>",
"encoding": "UTF-8"
},
"target": {
"directory": "/path/to/target",
"filename": "somefile.xml"
}
}
]
},
"outputData": {
"transferDuration": "00:00:37:10",
"transferLength": 187813298,
},
"updateDate": "2020-02-21T14:37:18.329Z",
"updateUser": "bob"
}
i want to read all of the tasks and, for each one, output a single line composed of the following fields :
task[n].inputData.transfers[].target.filename, task[n].outputData.transferLength, task[n].updateDate
i've got my filter chain to where it will choose the appropriate fields correctly, even to where it will pick the 'correct' single entry from amongst the multiple entries in the task[].inputData.transfers[] array, but when i try to get the output of more than a single element, the chain iterates over the array three times, and i get
task[0].inputData.transfers[].target.filename
task[1].inputData.transfers[].target.filename
task[2].inputData.transfers[].target.filename
...
task[n].inputData.transfers[].target.filename
then the results of the outputData.transferLength field for all elements,
then the results of the updateDate field for all elements.
here is my filter chain :
'(.tasks[].inputData.transfers[] | select(.target.filename | match("[Xx][Mm][Ll]$")).target.filename), .tasks[].outputData.transferLength, .tasks[].updateDate'
i'm thinking there must be some efficient way to group all of these multi-level elements together for each element of the array ; something like a 'with ...' clause, like with tasks[] : blablabla, but can't figure out how to do it. can anyone help ?
The JSON example contained a superfluous , that jq won't accept.
Your example filter chain appears to operate on .tasks[] even though the example appears to be only a single task. So it is not possible to rewrite what you have got into a functioning state. So rather than provide an exact answer to an inexact question, here is the first of the three parts of your filter chain revised:
.inputData.transfers | map(select(.target.filename | match("xml$"; "i")))
See this jqplay snippet.
Rather than write [ .xs[] | select(p) ], just write .xs | map(select(p)).
i finally found the answer. the trick was to pipe the .tasks[] into an expression where the parens were placed around the field elements as a group, which apparently will apply whatever is inside the parens to each element of the array individually, in sequence. then using #dmitry example as a guide, i also placed the elements inside right and left brackets to recreate array elements that i could then select, which could then be output onto 1 line each with | #csv. so the final chain that worked for me is :
.task[] | ([.inputData.transfers[].target.filename, .outputData.transferLength, .updateDate]) | [(.[0],.[2],.[3])] | #csv'
unfortunately i couldn't get match() to work in this invocation, nor sub() ; each of these caused jq to offer a useless error message just before it dumped core.
many thanks to the people who replied.
I've learned so much using SoapUI, but, I'm just stuck on this one thing. I have the following payload returned:
[
{
"#c": ".CreditPaymentInfo",
"supplementalInfo": null,
"date": "06/30/2015 17:03:50",
"posTxCode": "107535",
"amt": 2.56,
"transactionId": 235087,
"id": 232163,
"cardType": "CREDIT",
"cardHolderName": "SMITH2/JOE",
"expMonthYear": "0119",
"lastFourDigits": "4444",
"approvalCode": "315PNI",
"creditTransactionNumber": "A71A7DB6C2F4"
},
{
"#c": ".CreditPaymentInfo",
"supplementalInfo": null,
"date": "07/01/2015 15:53:29",
"posTxCode": "2097158",
"amt": 58.04,
"transactionId": 235099,
"id": 232176,
"cardType": "CREDIT",
"cardHolderName": "SMITH2/JOE",
"expMonthYear": "0119",
"lastFourDigits": "4444",
"approvalCode": "",
"creditTransactionNumber": null
}
]
I would like to count how many nodes are returned... so, in this case, I would expect that 2 nodes be returned whenever I run this test step in SoapUI.
I was attempting to get this done using the JsonPath Count assertion, but, I just can't see to format it correctly.
Any help would be greatly appreciated!
I have not used JsonPath, but you can do this with XPath ... which works for all older versions too.
Internally SoapUI represents everything as XML. So you could use XPath assertion to check for:
${#ResponseAsXml#count(//*:id)}
and make sure it comes back as
2
Counted successfully with 'JsonPath count' using one of the following (assuming my top level object is an array) :
$
$.
$[*]
If you need to be more specific on the objects you're counting, you can rely only on the 3rd syntax, specifying one of the redundant field. One of the following worked for me :
$[*].fieldName
$[*].['fieldName']
Should return 2 in your case with one of the following :
$[*].#c
$[*].['#c']
$[*].id
$[*].['id']
And so on
This is a JSON format. Just use JsonPath Count. Use $ at the top and 2 at the bottom.
$[index].your.path.here
thus
$[0].date would return "06/30/2015 17:03:50"
and
$[1].date would return "07/01/2015 15:53:29"
Suppose I had three arrays nested in a document as follows and I knew all the indices. How would I add another element to the third array, in this case "array3"? I wouldn't be able to use the positional operator but is there a way to update/push when you know the indices?
"name1": "foo1",
"array1": [
{
name2: "foo2",
"array2" : [
{
"name3" : "foo3",
"array3": [
{
data : "ImData"
}
]
}
]
]
Well. I figured it out. I somehow missed just directly referencing it via its index. I figured it possible but just about every example showed positional references and I couldn't find the syntax to do it directly. It's quite simple. To add to array 3 above when knowing the index you'd:
db.collection('productlist').update({name1:'foo1'},{'$push': { "array1.0.array2.0.array3": {"data": "yay"}}}
Inspired by this question in Server Fault
https://serverfault.com/questions/459042/mongoexport-csv-output-array-values
I'm using mongoexport to export some collections into CSV files, however when I try to target fields which are the last members of an array I cannot get it to export correctly.
Command I'm using
mongoexport -d db -c collection -fieldFile fields.txt --csv > out.csv
One item of my collection:
{
"id": 1,
"name": "example",
"date": [
{"date": ""},
{"date": ""},
],
"status": [
"true",
"false",
],
}
I can access to the first member of my array writing the fields like the following
name
id
date.0.date
status.0
Is there a way to acess the last item of my array without knowing the lenght of the array?
Because the following doesn't work:
name
id
date.-1.date
status.-1
Any idea of the correct notation? Or if it's simply not possible?
It's not possible to reference the last element of the array without knowing the length of the array, since the notation is array_field.index where the index is in [0, length - 1]. You could use the aggregation framework to create the view of the data that you want to export, save it temporarily into a collection with $out, and then mongoexport that. For example, for your documents you could do
db.collection.aggregate([
{ "$unwind" : "$date" },
{ "$group" : { "_id" : "$_id", "date" : { "$last" : "$date" } } },
{ "$out" : "temp-for-csv" }
])
in order to get just the last date for each document and output it to the collection temp-for-csv.
You can return just the last elements in an array with the $slice projection operator, but this isn't available in aggregation and mongoexport only takes a query specification, not a projection specification, since the --fields and --fieldFile option are supposed to suffice. Might be a good feature request to ask for using a query with a projection for mongoexport.