I'm using the AWS CLI command aws iam list-users in Powershell 7.2.5 and trying to convert the output from JSON to a Powershell Object.
The JSON output of the above command looks like this (actual values omitted):
{
"Users": [
{
"Path": "/",
"UserName": "username1",
"UserId": "userid1",
"Arn": "arnid1",
"CreateDate": "createddate"
},
{
"Path": "/",
"UserName": "username2",
"UserId": "userid2",
"Arn": "arnid2",
"CreateDate": "createddate"
},
{
"Path": "/",
"UserName": "username3",
"UserId": "userid3",
"Arn": "arnid3",
"CreateDate": "createddate"
}
]
}
When I try and run the following code to create an array of Powershell Objects the output comes out as one Object.
$users = ConvertFrom-Json -InputObject $usersJson
Users
-----
{#{Path=/; UserName=username1; UserId=userid1; Arn=arnid1; CreateDate=createddate}, #{Path=/; UserName=username2; UserId=userid2; Arn=arnid2; CreateDate=createddate}, #{Path=/; UserName=username3; UserId=userid3; Arn=arnid3; CreateDate=createddate}}
Have read over countless posts and am now at a loss of what to do.
Any help would be greatly appreciated.
Santiago's comment was correct:
$users.Users shows all the entries as individual objects.
Related
I have pulled a large amount of data from a website via REST and converted it to JSON.
I have extracted the specific entry in JSON I need to edit. See below
$variables = contains all the data converted to JSON
$dacpacvariable = contains the specific entry I need to edit (which is below)
{
"Id": "c1f4fe9b-3c4d-8j02-0e7x-0a6528bn192c",
"Name": "Variable1",
"Value": "abc123",
"Description": null,
"Scope": {
"Machine": [
"Machines-1"
]
},
}
I need to edit the scope section to look like the following:
"Scope": {
"Machine": [
"Machines-1",
"Machines-2"
]
},
And then add the whole entry with the edited scope back to the larger JSON.
Any ideas?
You might simply replace the Attribute "Machine" with its new value like:
$dacpacvariable = $dacpacvariable.Scope.Machine = $dacpacvariable.Scope.Machine + "Machines-2"
Do you mean something like this:
$json = #'
{
"Id": "c1f4fe9b-3c4d-8j02-0e7x-0a6528bn192c",
"Name": "Variable1",
"Value": "abc123",
"Description": null,
"Scope": {
"Machine": [
"Machines-1"
]
}
}
'# | ConvertFrom-Json
$json.Scope.Machine += 'Machines-2'
$json | ConvertTo-Json # -Depth 99 unsure how many nestings there are in the complete json
Output:
{
"Id": "c1f4fe9b-3c4d-8j02-0e7x-0a6528bn192c",
"Name": "Variable1",
"Value": "abc123",
"Description": null,
"Scope": {
"Machine": [
"Machines-1",
"Machines-2"
]
}
}
Say I have a sample.json file that looks like this. Notice the objects are not separated by a comma. I am aware that this is not a JSON file without the commas.
[
{
"type": "message",
"user": "U024HFHU5",
"text": "hey there",
"ts": "1385407681.000003"
}
{
"type": "message",
"user": "U024HGJ4E",
"text": "right back at you",
"ts": "1385407706.000006"
}
]
How do I make it look like this (example from I got the example from https://thoughtbot.com/blog/jq-is-sed-for-json.)
[
{
"type": "message",
"user": "U024HFHU5",
"text": "hey there",
"ts": "1385407681.000003"
},
{
"type": "message",
"user": "U024HGJ4E",
"text": "right back at you",
"ts": "1385407706.000006"
}
]
I tried doing },{ as some have suggested but it does not work.
I think it's probably something like this in jq on the command line but am not sure. Here, my hope is that sample2.json outputs the desired format.
jq -r '. | join(", ")' sample.json > sample2.json
If the square brackets were not in the input, jq could process it like normal as a stream of objects. You could then throw them into an array or whatever.
You'll just need to invoke jq twice in this case. First read it in raw (-Rs) and trim out the leading and trailing brackets, output it raw (-r), then process in jq like normal.
$ jq -Rsr 'gsub("^\\s*\\[|\\]\\s*$"; "")' input.json | jq -n '[inputs]'
Using human JSON utility to recover your malformed JSON file:
hjson -j file
[
{
"type": "message",
"user": "U024HFHU5",
"text": "hey there",
"ts": "1385407681.000003"
},
{
"type": "message",
"user": "U024HGJ4E",
"text": "right back at you",
"ts": "1385407706.000006"
}
]
The option -j outputs the file with correct JSON syntax, by adding the missing comma between objects.
This question already has answers here:
Unexpected ConvertTo-Json results? Answer: it has a default -Depth of 2
(2 answers)
Closed 3 years ago.
I have the following JSON
{
"method": "exec",
"params": [
{
"url": "/sys/login/user",
"data": [
{
"user": "MyUsername",
"passwd": "MyPassword"
}
]
}
],
"id": 1,
"ver": "2.0"
}
I'm trying to build this JSON using Powershell, but the output is not correct, below is my code.
$fullJson=#{}
$params=#()
$paramsdata=#()
$paramsdata+=#{"user"="mailapi"}
$paramsdata+=#{"passwd"="**********"}
$params+=#{"url"="/sys/login/user"}
$params+=#{"data"=$paramsdata}
$fullJson.Add("method", "exec")
$fullJson.Add("params",$params)
$fullJson.Add("id", "1")
$fullJson.Add("ver", "2.0")
$JsonBody=$fullJson | ConvertTo-Json
$x=Invoke-WebRequest -Uri https://10.10.10.10/jsonrpc -Body $JsonBody -Method Post
The output is the below
{
"method": "exec",
"params": [
{
"url": "/sys/login/user"
},
{
"data": "System.Collections.Hashtable System.Collections.Hashtable"
}
],
"id": "1",
"ver": "2.0"
}
The problem is DATA properties is not correct format, it should be an a nested array inside the first one, but it seems that its being added as a hashtable.
This problem is the data array should be built like this one below
"params": [
{
"url": "/sys/login/user",
"data": [
{
"user": "MyUsername",
"passwd": "MyPassword"
}
]
But with my code, its being built like this
"params": [
{
"url": "/sys/login/user"
},
{
"data": "System.Collections.Hashtable System.Collections.Hashtable"
}
],
Any help in updating this.
Thanks
Use the -Depth option in ConvertTo-Json :
Specifies how many levels of contained objects are included in the
JSON representation. The default value is 2.
Your desired depth is 4 (object -> params -> data -> username/password) :
$JsonBody=$fullJson | ConvertTo-Json -Depth 4
I have the below format of JSON file which is having issues[] array and I tried to use it for Kibana. But unfortunately Kibana doesn't support nested objects and array and there is a plugin to utilize so that I need to downgrade which I can't do right now because in that case I will lose all my data.
Sample data:
{
"expand": "schema,names",
"startAt": 0,
"maxResults": 50,
"total": 4,
"issues": [{
"expand": "operations,versionedRepresentations,editmeta,changelog,renderedFields",
"id": "1999875",
"self": "https://amazon.kindle.com/jira/rest/api/2/issue/1999875",
"key": "KINDLEAMZ-67578",
"fields": {
"summary": "contingency is displaying for confirmed card.",
"priority": {
"name": "P1",
"id": "1"
},
"created": "2019-09-23T11:25:21.000+0000"
}
},
{
"expand": "operations,versionedRepresentations,editmeta,changelog,renderedFields",
"id": "2019428",
"self": "https://amazon.kindle.com/jira/rest/api/2/issue/2019428",
"key": "KINDLEAMZ-68661",
"fields": {
"summary": "card",
"priority": {
"name": "P1",
"id": "1"
},
"created": "2019-09-23T11:25:21.000+0000"
}
},
{
"expand": "operations,versionedRepresentations,editmeta,changelog,renderedFields",
"id": "2010958",
"self": "https://amazon.kindle.com/jira/rest/api/2/issue/2010958",
"key": "KINDLEAMZ-68167",
"fields": {
"summary": "Test Card",
"priority": {
"name": "P1",
"id": "1"
},
"created": "2019-09-23T11:25:21.000+0000"
}
}
]
}
So I just planned to restructure this payload like all the issues[] into as an object and write down in separate file. So that I can avoid that issue.
Expected output:
For that above sample data I have 4 records in issues[].length so I just want to create 4 different files with below format:
File1.json:
{
"key": "KINDLEAMZ-67578",
"summary": "contingency is displaying for confirmed card.",
"name": "P1",
"created": "2019-09-23T11:25:21.000+0000"
}
The same way I want to looping the other arrays and get the values as like above and write down in File2.json, File3.json, and File4.json.
Since the data is dynamic and so I just want this file creation happen based of length of issues[] array.
Is there anyway to achieve this by using shell script? Or any CLI library.
Please advise me.
Specify -c/--compact-output flag to make jq put each entity on a single, separate line, then use awk to write each line to a separate file.
jq -c '.issues[] | {
key,
summary: .fields.summary,
name: .fields.priority.name,
created: .fields.created
}' file | awk '{
f = ("file" NR ".json")
print > f
close(f)
}'
Using GNU awk and extension gawk-json:
awk '
#load "json"
{
lines=lines $0
if(json_fromJSON(lines,data)==1){
for(i in data["issues"]) {
out["key"] = data["issues"][i]["key"]
out["summary"] = data["issues"][i]["fields"]["summary"]
out["created"] = data["issues"][i]["fields"]["created"]
out["name"] = data["issues"][i]["fields"]["priority"]["name"]
file="file" i ".json"
print json_toJSON(out) > file
close(file)
delete out
}
}
}' file.json
Output:
$ cat file1.json | jq '.' # useless use of cat but used to emphasize
{
"created": "2019-09-23T11:25:21.000+0000",
"key": "KINDLEAMZ-67578",
"summary": "contingency is displaying for confirmed card.",
"name": "P1"
}
I have the following data structure as a result of aws logs get-query-results:
{
"status": "Complete",
"statistics": {
"recordsMatched": 2.0,
"recordsScanned": 13281.0,
"bytesScanned": 7526096.0
},
"results": [
[
{
"field": "time",
"value": "2019-01-31T21:53:01.136Z"
},
{
"field": "requestId",
"value": "a9c233f7-0b1b-3326-9b0f-eba428e4572c"
},
{
"field": "logLevel",
"value": "INFO"
},
{
"field": "callerId",
"value": "a9b0f9c2-eb42-3986-33f7-8e450b1b72cf"
}
],
[
{
"field": "time",
"value": "2019-01-25T13:13:01.062Z"
},
{
"field": "requestId",
"value": "a4332628-1b9b-a9c2-0feb-0cd4a3f7cb63"
},
{
"field": "logLevel",
"value": "INFO"
},
{
"field": "callerId",
"value": "a9b0f9c2-eb42-3986-33f7-8e450b1b72cf"
}
],
]
}
The AWS CLI support JMESPath language for filtering output. I need to apply a query string, to filter among the returned "results" the objects that contain the "callerId" as a "field", retrieve the "value" property and obtain the following output:
[
{
callerId: "a9b0f9c2-eb42-3986-33f7-8e450b1b72cf"
},
{
callerId: "a9b0f9c2-eb42-3986-33f7-8e450b1b72cf"
}
]
The first step I do is flatter the results array with the query string: results[]
This will get read of the other root properties (status, statistics) and return only one big array with all of the {field: ..., value: ...} alike objects. But after this I can't manage to properly filter for those objects that match field=="callerId". I tried, among others, the following expressions without success:
'results[][?field=="callerId"]'
'results[][*][?field=="callerId"]'
'results[].{ callerId: #[?field=="callerId"].value }'
I'm not an expert in JMESPath and I was doing the tutorials of the jmespath.org site but couldn't manage to make it work.
Thanks!
Using jq is a good thing because it's more complete language, but if you want to do it with JMES Path here the solution:
results[*][?field=='callerId'].{callerId: value}[]
to get:
[
{
"callerId": "a9b0f9c2-eb42-3986-33f7-8e450b1b72cf"
},
{
"callerId": "a9b0f9c2-eb42-3986-33f7-8e450b1b72cf"
}
]
I'm not able to reproduce fully since I don't have the same logs in my log stream but I was able to do this using jq and putting the sample JSON object in a file
cat sample_output.json | jq '.results[][] | select(.field=="callerId") | .value'
OUTPUT:
"a9b0f9c2-eb42-3986-33f7-8e450b1b72cf"
"a9b0f9c2-eb42-3986-33f7-8e450b1b72cf"
you could pipe the output from the aws cli to jq.
I was able to get pretty close with the native JMESPath query and using the built in editor in this site
http://jmespath.org/examples.html#filtering-and-selecting-nested-data
results[*][?field==`callerId`][]
OUTPUT:
[
{
"field": "callerId",
"value": "a9b0f9c2-eb42-3986-33f7-8e450b1b72cf"
},
{
"field": "callerId",
"value": "a9b0f9c2-eb42-3986-33f7-8e450b1b72cf"
}
]
but I'm not sure how to get callerId to be the key and the value to be the value from another key.