I am trying read a JSON array in Postgresql with the below mentioned JSON content but I couldn't achieve it. I am able to read the JSON objects like "select data ->> 'items' from events". Please let me know an approach where I can read an array, objects inside arrays & inner arrays etc.
JSON Content:
{"apiVersion":"2.0",
"data":{
"updated":"2010-01-07T19:58:42.949Z",
"totalItems":800,
"startIndex":1,
"itemsPerPage":1,
"items":[
{"id":"hYB0mn5zh2c",
"uploaded":"2007-06-05T22:07:03.000Z",
"updated":"2010-01-07T13:26:50.000Z",
"uploader":"GoogleDeveloperDay",
"category":"News",
"title":"Google Developers Day US - Maps API Introduction",
"description":"Google Maps API Introduction ...",
"tags":[
"GDD07","GDD07US","Maps"
],
"thumbnail":{
"default":"http://i.ytimg.com/vi/hYB0mn5zh2c/default.jpg",
"hqDefault":"http://i.ytimg.com/vi/hYB0mn5zh2c/hqdefault.jpg"
},
"player":{
"default":"http://www.youtube.com/watch?vu003dhYB0mn5zh2c"
},
"content":{
"1":"rtsp://v5.cache3.c.youtube.com/CiILENy.../0/0/0/video.3gp",
"5":"http://www.youtube.com/v/hYB0mn5zh2c?f...",
"6":"rtsp://v1.cache1.c.youtube.com/CiILENy.../0/0/0/video.3gp"
},
"duration":2840,
"aspectRatio":"widescreen",
"rating":4.63,
"ratingCount":68,
"viewCount":220101,
"favoriteCount":201,
"commentCount":22,
"status":{
"value":"restricted",
"reason":"limitedSyndication"
},
"accessControl":{
"syndicate":"allowed",
"commentVote":"allowed",
"rate":"allowed",
"list":"allowed",
"comment":"allowed",
"embed":"allowed",
"videoRespond":"moderated"
}
}
]
}
}
just use one arrow
# select blob->'data'->'items'->0->'title' from so_test;
?column?
----------------------------------------------------
"Google Developers Day US - Maps API Introduction"
(1 row)
exchange blob with column name
Related
In ADF I am calling an API source that's returning nested/complex JSON that I need to flatten out into csv files.
The Copy Activity wont work for me. It will only read the first record from the nested data.
I need to be able to call and then flatten the aliases array
Here is an example of the response from the API:
{
"items": [
{
"title_no": 12345,
"booking_xref_title_no": 45305,
"edi_no": "2495",
"title_global_id": "TTL-11",
"aliases": [
{
"source_name": "123A",
"title_alias_global_id": "ABC1234"
},
{
"source_name": "123B",
"title_alias_global_id": "ABC5678"
I need to get the following into my output csv:
source_name
title_global_id
123A
ABC1234
123B
ABC5678
Json output given in question is incomplete. So I added few braces to it.
JSON response should be as given below:
{
"items":[
{
"title_no":12345,
"booking_xref_title_no":45305,
"edi_no":"2495",
"title_global_id":"TTL-11",
"aliases":[
{
"source_name":"123A",
"title_alias_global_id":"ABC1234"
},
{
"source_name":"123B",
"title_alias_global_id":"ABC5678"
}
]
}
]
}
With above JSON response, you can flatten it and save output to CSV file as shown in below steps.
Use JSON response as Source to DataFlow Activity in Azure data factory.
Select document form as Single document.
Now select Flatten transformation. Select Unroll by = items.aliases and do the mapping as shown in below screenshot.
Data preview after Flatten transformation.
Now add CSV file as Sink.
Output:
I'm using a LogicApp triggered by an HTTP call. The call posts a JSON message which is a single row array. I simply want to extract the single JSON object out of the array so that I can parse it but have spent several hours googling and trying various options to no avail. Here's an example of the array:
[{
"id": "866ef906-5bd8-44d8-af34-0c6906d2dfd7",
"subject": "Engagement-866ef906-5bd8-44d8-af34-0c6906d2dfd7",
"data": {
"$meta": {
"traceparent": "00-dccfde4923181d4196f870385d99cb84-52b8333f100b844c-00"
},
"timestamp": "2021-10-19T17:01:06.334Z",
"correlationId": "866ef906-5bd8-44d8-af34-0c6906d2dfd7",
"fileName": "show.xlsx"
},
"eventType": "File.Uploaded",
"eventTime": "2021-10-19T17:01:07.111Z",
"metadataVersion": "1",
"dataVersion": "1"
}]
Examples of what hasn't worked:
Parse JSON on the array, error: InvalidTemplate when specifiying an array as the schema
For each directly against the http output, error: No dependent actions succeeded.
Any suggestions would be gratefully received.
You have to paste the example that you have provided to 'Use sample payload to generate schema' in the Parse JSON Connector and then you will be able to retrieve each individual object from the sample payload.
You can extract a single JSON object from your array by using its index in square brackets. E.g., in the example below you'd need to use triggerBody()?[0] instead of triggerBody(). 0 is an index of the first element in the array, 1 - of the second, and so on.
Result:
I need to loop through this optional array (it's only the sectional of JSON I have trouble with).
As you can see from the code:
The optional bullseye has an array rings. rings has arrays of expansionCriteria and expansionCriteria may or may not have actions.
How do I iterate and get all type, threshold in expansionCriteria? I also need to access all skillsToRemove under actions, if available.
I am rather new to Logic Apps, so any help is appreciated.
"bullseye": {
"rings": [
{
"expansionCriteria": [
{
"type": "TIMEOUT_SECONDS",
"threshold": 180
}
],
"actions": {
"skillsToRemove": [
{
"name": "Claims Foundation",
"id": "60bd469a-ebab-4958-9ca9-3559636dd67d",
"selfUri": "/api/v2/routing/skills/60bd469a-ebab-4958-9ca9-3559636dd67d"
},
{
"name": "Claims Advanced",
"id": "bdc0d667-8389-4d1d-96e2-341e383476fc",
"selfUri": "/api/v2/routing/skills/bdc0d667-8389-4d1d-96e2-341e383476fc"
},
{
"name": "Claims Intermediate",
"id": "c790eac3-d894-4c00-b2d5-90cd8a69436c",
"selfUri": "/api/v2/routing/skills/c790eac3-d894-4c00-b2d5-90cd8a69436c"
}
]
}
},
{
"expansionCriteria": [
{
"type": "TIMEOUT_SECONDS",
"threshold": 5
}
]
}
]
}
Please let me know if you need more info.
To generate the schema, you can remove the name of the object at the top of the code: "bullseye":
Thank you pramodvalavala-msft for posting your answer in MS Q&A for the similar thread .
" As you are working with a JSON Object instead of an Array, unfortunately there is no built-in function to loop over the keys. There is a feature request to add a method to extract keys from an object for scenarios like this, that you could up vote for it gain more traction.
You can use the inline code action to extract the keys from your object as an array (using Object.keys()). And then you can loop over this array using the foreach loop to extract the object that you need from the main object, which you could then use to create records in dynamics."
For more information you can refer the below links:
. How to loop and extract items from Nested Json Array in Logic Apps .
.Nested ForEach Loop in Workflow. .
I'm building a Rails 5+ API with JSON as the format between front and back ends.
I want the ability to create a single record or multiple records, depending if an array of JSON objects is sent.
Note that I'm NOT using JSON:API spec, but rather the JSON objects are coming as root attributes.
# Create a single child object for the associated parent
POST api/v1/parents/1/children PARAMS: { child_name: "Alpha" }
# Create multiple children objects for the associated parent
POST api/v1/parents/1/children PARAMS: [{ child_name: "Alpha" }, { child_name: "Bravo" }]
In the controller I have to differentiate if a single object or an array is being sent. It seems that Rails converts JSON data to a params["_json"] key automatically if the Content-Type="application/json" header is set, and I'm using this to tell if an array was passed or not.
class ChildrenController < ApplicationController
def create
#parent = Parent.find(params[:parent_id])
if params["_json"] && params["_json"].is_a?(Array)
#children = []
params["_json"].each do |child_attributes|
#children << #parent.children.create!(child_attributes)
end
render json: #children
else
#child = #parent.children.create!(child_params)
render json: #child
end
end
def child_params
params.permit(:child_name)
end
end
Questions
Is using params["_json"] a standard way to tell if an array was passed or not? It seems hacky, but I'm not sure of a better way.
If an array of JSON objects is passed, how can I still use the StrongParameters child_params method? Currently, if the user passes an array of JSON objects, they can put whatever attributes they want and I'm not filtering them out.
Is there a better way to implement this functionality? I don't have to use a single endpoint for both single and multiple creation, I just thought it would be more convenient to have a single API endpoint that can handle single or multiple objects.
I also plan on creating a single endpoint for the update action that can also accept single or multiple objects. Is this a bad practice?
In the case you send a JSON array (like [{ child_name: "Alpha" }, { child_name: "Bravo" }]) to an endpoint, Rails stores it into _json of the params hash. The reason for doing this is, that unlike a single JSON object, an array cannot be extracted into a hash, without introducing another key.
Let's say, that we post { child_name: "Alpha" } to an endpoint. The params will look like:
{"child_name"=>"Alpha", "format"=>:json, "controller"=>"api/children", "action"=>"create"}
Now if we post the array [{ child_name: "Alpha" }, { child_name: "Bravo" }] and it would be blindly extracted into the hash like a JSON object the result would look like:
{[{ child_name: "Alpha" }, { child_name: "Bravo" }], "format"=>:json, "controller"=>"api/children", "action"=>"create"}
This is not a valid hash! So Rails wraps your JSON array into _json and it becomes:
{"_json" => [{ child_name: "Alpha" }, { child_name: "Bravo" }], "format"=>:json, "controller"=>"api/children", "action"=>"create"}
This happens here in the actionpack. Yes, it seems a bit hacky, but the only and maybe cleaner solution would be to wrap all data from the request body into one key like data.
So to your actual question: How can I manage a JSON object and JSON array at a single endpoint?
You can use strong params like ..
def children_params
model_attributes = [:child_name]
if params.key? '_json'
params.permit(_json: model_attributes)['_json']
else
params.permit(*model_attributes)
end
end
.. and in the create action would look like ..
def create
result = Children.create!(children_params)
if children_params.kind_of? Array
#children = result
render :index, status: :created
else
#child = result
render :show, status: :created
end
end
Of course you would need to adapt it a bit for your specific use case. From the design perspective of your API, I think it's okay doing it like that. Maybe this should be asked within a separate question.
Let's say I have the following document schema in a collection called 'users':
{
name: 'John',
items: [ {}, {}, {}, ... ]
}
The 'items' array contains objects in the following format:
{
item_id: "1234",
name: "some item"
}
Each user can have multiple items embedded in the 'items' array.
Now, I want to be able to fetch an item by an item_id for a given user.
For example, I want to get the item with id "1234" that belong to the user with name "John".
Can I do this with mongoDB? I'd like to utilize its powerful array indexing, but I'm not sure if you can run queries on embedded arrays and return objects from the array instead of the document that contains it.
I know I can fetch users that have a certain item using {users.items.item_id: "1234"}. But I want to fetch the actual item from the array, not the user.
Alternatively, is there maybe a better way to organize this data so that I can easily get what I want? I'm still fairly new to mongodb.
Thanks for any help or advice you can provide.
The question is old, but the response has changed since the time. With MongoDB >= 2.2, you can do :
db.users.find( { name: "John"}, { items: { $elemMatch: { item_id: "1234" } } })
You will have :
{
name: "John",
items:
[
{
item_id: "1234",
name: "some item"
}
]
}
See Documentation of $elemMatch
There are a couple of things to note about this:
1) I find that the hardest thing for folks learning MongoDB is UN-learning the relational thinking that they're used to. Your data model looks to be the right one.
2) Normally, what you do with MongoDB is return the entire document into the client program, and then search for the portion of the document that you want on the client side using your client programming language.
In your example, you'd fetch the entire 'user' document and then iterate through the 'items[]' array on the client side.
3) If you want to return just the 'items[]' array, you can do so by using the 'Field Selection' syntax. See http://www.mongodb.org/display/DOCS/Querying#Querying-FieldSelection for details. Unfortunately, it will return the entire 'items[]' array, and not just one element of the array.
4) There is an existing Jira ticket to add this functionality: it is https://jira.mongodb.org/browse/SERVER-828 SERVER-828. It looks like it's been added to the latest 2.1 (development) branch: that means it will be available for production use when release 2.2 ships.
If this is an embedded array, then you can't retrieve its elements directly. The retrieved document will have form of a user (root document), although not all fields may be filled (depending on your query).
If you want to retrieve just that element, then you have to store it as a separate document in a separate collection. It will have one additional field, user_id (can be part of _id). Then it's trivial to do what you want.
A sample document might look like this:
{
_id: {user_id: ObjectId, item_id: "1234"},
name: "some item"
}
Note that this structure ensures uniqueness of item_id per user (I'm not sure you want this or not).