Google Cloud Datastore JSON for an Array of Keys - arrays

I tried this JSON at the Google Datastore Console to create a property as an array of keys (one-to-many relationship) but it doesn't work:
{
"values": [
{
"keyValue": key(user, 1234567890123456)
},
{
"keyValue": key(user, 6544567890123456)
},
]
}


See JSON format: https://cloud.google.com/datastore/docs/reference/rest/v1/Key
The JSON will be like:
{
"values": [
{
"keyValue": {
"path": [
{
"kind": "user",
"id": "1234567890123456"
}
]
}
},
{
"keyValue": {
"path": [
{
"kind": "user",
"id": "6544567890123456"
}
]
}
}
]
}

Related

Transform mongo arrarys

I have a collection, which contains documents like
{
"_id": ObjectId("60f561eb0d022a4c614966c1"),
"vehicleId": 288,
"startTime": [
ISODate("2021-06-19T13:00:19Z"),
ISODate("2021-06-19T13:00:40Z")
],
"odo": [0, 1116.0746443123298],
"location": [
{ "type": "Point", "coordinates": [75.759973, 26.977889] },
{ "type": "Point", "coordinates": [75.771209, 26.97858] }
]
}
I want to change the location array so that the document looks like
{
"_id": ObjectId("60f561eb0d022a4c614966c1"),
"vehicleId": 288,
"startTime": [
ISODate("2021-06-19T13:00:19Z"),
ISODate("2021-06-19T13:00:40Z")
],
"odo": [0, 1116.0746443123298],
"locations": [
{ loc: { "type": "Point", "coordinates": [75.759973, 26.977889] } },
{ loc: { "type": "Point", "coordinates": [75.771209, 26.97858] } }
]
}
that is, each entry becomes an object. I may want to do the same thing to odo and starttime.
I'm experimenting with aggregation pipelines, but I haven't found a way except using application code to read and transform the object.
I'd rather do it via a mongo-only way and not have use application and parse document by document, so any help with a mongo-only way would be much appreciated.

Would be this one:
db.collection.aggregate([
{
$set: {
location: "$$REMOVE",
locations: {
$map: {
input: "$location",
in: { loc: "$$this" }
}
}
}
}
])

parsing the array of objects in JSON and converting it to flat JSON using JOLT transform

My input looks like the below;
{
"family": [
{
"person": {
"personId": {
"value": "12345"
},
"employeeAuthCd": {
"code": "AUTH_12345"
},
"employeeTypeCd": {
"code": "cd"
},
"status": {
"code": "New"
}
}
}
]
}
Desired Output
{
"Person_ID":"12345",
"employeeAuthCd":"AUTH_1345",
"employeeTypeCd":"cd",
"status":"New"
}
Can anyone help me out with the Jolt spec, I have tried many possible specs but couldn't reach the desired output, like the above, JSON have multiple array of objects those I need to convert those into flat JSON

This spec should work for you:
[
{
"operation": "shift",
"spec": {
"family": {
"*": {
"person": {
"personId": {
"value": "Person_ID"
},
"employeeAuthCd": {
"code": "employeeAuthCd"
},
"employeeTypeCd": {
"code": "employeeTypeCd"
},
"status": {
"code": "status"
}
}
}
}
}
}
]

How to add Single-field Exemptions in Firestore Index Config File

Please let me know how to add Index exemptions for Single-field, in the firebase-indexes.json file, in order to deploy through CLI.
Currently, below is my index config in the file firebase-indexes.json, able to deploy through CLI, but it is creating an index of type Composite, not as a Single-field Exemption.
{
"indexes": [
{
"collectionGroup": "comments",
"queryScope": "COLLECTION",
"fields": [
{
"fieldPath": "id",
"order": "ASCENDING"
},
{
"fieldPath": "id",
"order": "DESCENDING"
}
]
}
]
}
Thanks in advance.

Assuming that your collection is called "comments" and your exemption field is called "field", you will add a new property to your firestore.indexes.json called "fieldOverrides", like this:
{
"indexes": [
// your indexes here
],
"fieldOverrides": [
{
"collectionGroup": "comments",
"fieldPath": "field",
"indexes": [
{
"order": "ASCENDING",
"queryScope": "COLLECTION"
},
{
"order": "DESCENDING",
"queryScope": "COLLECTION"
},
{
"arrayConfig": "CONTAINS",
"queryScope": "COLLECTION"
},
{
"order": "ASCENDING",
"queryScope": "COLLECTION_GROUP"
},
{
"order": "DESCENDING",
"queryScope": "COLLECTION_GROUP"
},
{
"arrayConfig": "CONTAINS",
"queryScope": "COLLECTION_GROUP"
}
]
}
]
}

How to get object with certain property in arraylist in Jolt

I want to do a Jolt transformation with a json array, and i only want the ones with certain attribute.
For example:
Input:
{
"characteristic": [
{
"name": "BrandId",
"value": "b"
},
{
"name": "status",
"value": "SENT"
},
{
"name": "statusTxt",
"value": "sent"
}
]
}
I want output to be
{
"status":"SENT",
"statusTxt":"sent"
}

This will lead to the output you want based on your input:
[
{
"operation": "shift",
"spec": {
"characteristic": {
"*": {
"name": {
"status": {
"#(2,value)": "status"
},
"statusTxt": {
"#(2,value)": "statusTxt"
}
}
}
}
}
}
]

Google Datastore Use Select Query by ID using API in GAS

I wish the Google API documentation was a little more newbie-proof.
I've worked my way through Selecting all entities, Updating an entity, inserting, and deleting. Now I would like to start selecting specific entities by criteria. The API https://datastore.googleapis.com/v1/projects/project-id-5200707080506492774:runQuery is for this purpose, and if I provide the payload of "query: {}" I get all entities. I can also filter by Kind. But I cannot figure out how to filter by a property. I try to get an entity by name with this JSON stringified payload:
var payload =
{
"query": {
"kind": [
{
"name": "Test"
}
],
"filter": {
"propertyFilter": {
"property": {
"name": "id"
},
"op": "EQUAL",
"value": {
"stringValue": "5634472569470976"
}
}
}
}
}
But I get the 200 results batch:
{
"batch": {
"entityResultType": "FULL",
"endCursor": "CgA=",
"moreResults": "NO_MORE_RESULTS"
}
}
AKA: Nothing was found.
Could someone enlighten me with regards to how to select by the entity's name/id or other field of data?
EDIT:
Here is the file structure of my entities. They are organized under the kind Test:
{
"batch": {
"entityResultType": "FULL",
"entityResults": [
{
"entity": {
"key": {
"partitionId": {
"projectId": "project-id-5200707080506492774"
},
"path": [
{
"kind": "Test",
"id": "5634472569470976"
}
]
},
"properties": {
"test": {
"stringValue": "Hi it is me"
}
}
},
"cursor": "CjsSNWogc35wcm9qZWN0LWlkLTUyMDA3MDcwODA1
MDY0OTI3NzRyEQsSBFRlc3QYgICAgN6QgQoMGAAgAA==",
"version": "1503343869436000"
},
{
"entity": {
"key": {
"partitionId": {
"projectId": "project-id-5200707080506492774"
},
"path": [
{
"kind": "Test",
"id": "5639445604728832"
}
]
},
"properties": {
"test": {
"stringValue": "testtesttest"
}
}
},
"cursor": "CjsSNWogc35wcm9qZWN0LWlkLTUyMDA3MDcwODA1MDY0OTI3NzRyEQsSBFRlc3QYgICAgLyhggoMGAAgAA==",
"version": "1503343008992000"
},
{
"entity": {
"key": {
"partitionId": {
"projectId": "project-id-5200707080506492774"
},
"path": [
{
"kind": "Test",
"id": "5649391675244544"
}
]
},
"properties": {
"test": {
"stringValue": "testtest"
}
}
},
"cursor": "CjsSNWogc35wcm9qZWN0LWlkLTUyMDA3MDcwODA1MDY0OTI3NzRyEQsSBFRlc3QYgICAgPjChAoMGAAgAA==",
"version": "1503342946693000"
},
{
"entity": {
"key": {
"partitionId": {
"projectId": "project-id-5200707080506492774"
},
"path": [
{
"kind": "Test",
"id": "5659313586569216"
}
]
},
"properties": {
"test": {
"stringValue": "testtesttest"
}
}
},
"cursor": "CjsSNWogc35wcm9qZWN0LWlkLTUyMDA3MDcwODA1MDY0OTI3NzRyEQsSBFRlc3QYgICAgNrjhgoMGAAgAA==",
"version": "1503343059530000"
},
{
"entity": {
"key": {
"partitionId": {
"projectId": "project-id-5200707080506492774"
},
"path": [
{
"kind": "Test",
"id": "5715999101812736"
}
]
},
"properties": {
"test": {
"stringValue": "hello world"
}
}
},
"cursor": "CjsSNWogc35wcm9qZWN0LWlkLTUyMDA3MDcwODA1MDY0OTI3NzRyEQsSBFRlc3QYgICAgLzVkwoMGAAgAA==",
"version": "1503343819165000"
}
],
"endCursor": "CjsSNWogc35wcm9qZWN0LWlkLTUyMDA3MDcwODA1MDY0OTI3NzRyEQsSBFRlc3QYgICAgLzVkwoMGAAgAA==",
"moreResults": "NO_MORE_RESULTS"
}
}
Edit:
I completed a filter query to check against my test field, as requested, and got the below 200 response:
{
"batch": {
"entityResultType": "FULL",
"entityResults": [
{
"entity": {
"key": {
"partitionId": {
"projectId": "project-id-5200707080506492774"
},
"path": [
{
"kind": "Test",
"id": "5715999101812736"
}
]
},
"properties": {
"test": {
"stringValue": "hello world"
}
}
},
"cursor":
"CjsSNWogc35wcm9qZWN0LWlkLTUyMDA3MDcwODA
1MDY0OTI3NzRyEQsSBFRlc3QYgICAgLzVkwoMGAAgAA==",
"version": "1503343819165000"
}
],
"endCursor": "CjsSNWogc35wcm9qZWN0LWlkLTUyMDA3MDcwODA1MDY0OTI3NzRyEQsSBFRlc3QYgICAgLzVkwoMGAAgAA==",
"moreResults": "NO_MORE_RESULTS"
}
}

At least in the ndb python datastore library id is not allowed as a property name. Which might be the reason your query is not working (as you expect) either.
After all you don't see any property named id within the properties structures of your entities, it is actually part of the key -> path structure.
Just to confirm, try using a valid property (i.e. one listed inside properties), for example:
"filter": {
"propertyFilter": {
"property": {
"name": "test"
},
"op": "EQUAL",
"value": {
"stringValue": "hello world"
}
}
}
If you have the entity key ids you don't need to perform queries to get the entities (you might not even be allowed to do that inside transactions), you can directly pull the entities by keys, using the projects.lookup method. I think something along these lines:
{
"keys": [
{
"path": [
{
"kind": "Test",
"id": "5634472569470976"
}
]
}
],
}

Resources