Querying a PickleProperty in NDB - google-app-engine

I am trying to code a Twitter-like microblogging application. In order to do that I'm using Google App Engine and the Datatstore.
I have two classes. The Tweet class which is the parent :
class Tweet(EndpointsModel):
_message_fields_schema = ('sender','body', 'name')
sender = ndb.StringProperty()
body = ndb.TextProperty()
created = ndb.DateTimeProperty(auto_now=True)
...
And the TweetIndex class, which is the child that contains all the receivers for a tweet.
class TweetIndex(EndpointsModel):
...
receivers = ndb.PickleProperty(indexed=True,repeated=True)
created = ndb.DateTimeProperty(auto_now=True)
...
#TweetIndex.method(request_fields=('receivers',),
path='mymodels', name='mymodel.list')
def MyModelList(self, query):
if not query.from_datastore:
raise endpoints.NotFoundException('MyModel not found.')
return query
I'm trying to query the TweetIndex entities with the API method MyModelList which purpose is to return the TweetIndex entity if a given id is included in the receivers array.
Example of a receivers array :
["13911772075915175317","1855429131779793831", ... ]
Which looks something like this in the datastore (stored as a blob):
["gAJYEwAAADY1NjU2NDM3MzA1NDI2NDU5ODlxAS4=","gAJYEwAAADU4MDM3MjE4OTEyNDgzNzgyNjNxAS4=",...]
However, when executing the following API request:
POST https://myapi/.../v1/mymodels
{
"receivers": [
"13911772075915175317"
]
}
The following is returned:
404
- Show headers -
{
"error": {
"errors": [
{
"domain": "global",
"reason": "notFound",
"message": "MyModel not found."
}
],
"code": 404,
"message": "MyModel not found."
}
}
"13911772075915175317" does exist at least once in a receivers array.
I also tried to input the blobs in the request (but with no surprise) did not work either.
How am I supposed to query this array correctly?

You'll need to query your receivers array with the pickled version. Given that a PickledProperty is binary data, it won't be easy to query. You are probably better off changing from PickledProperty to whatever type you actually want, in this case IntegerProperty.

Related

Azure logic apps: Nullable JSON values not available as dynamic content

I'm building a logic app that pulls some JSON data from a REST API, parses it with the Parse JSON block, and pushes it to Azure Log Analytics. The main problem I'm hitting is an important JSON field can either be an object or null. Per this post I changed the relevant part of my JSON schema to something like this
"entity": {"type": ["object", "null"] }
While this works, I'm now no longer to access entity later in the logic app as dynamic content. I can access all other fields parsed by the Parse JSON block downstream in the logic (that don't have nullable field). If I remove the "null" option and just have the type set to object, I can access entity in dynamic content once again. Does anyone know why this might be happening and/or how to access the entity field downstream?
Through the test, if we use "entity": {"type": ["object", "null"] }, we really cannot directly select entity in dynamic content.
But we can use the following expression to get the entity:
body('Parse_JSON')?['entity']
The test results seem to be no problem:
For a better understanding, let me cite a few more examples:
1. If your json is like this:
{
"entity": {
"testKey": "testValue"
}
}
Your expression is like this:
body('Parse_JSON')?['entity']
2. If your json is like this:
{
"test": {
"entity": {
"testKey": "testValue"
}
}
}
Your expression should like this:
body('Parse_JSON')?['test']?['entity']

Does Flask map JSON Array object to a Python string?

In my Flask server I am receiving a JSON-encoded parameter which is being sent via an HTTP POST from the client application.
Here is an example of what the JSON object looks like. For simplicity, I have kept only the first 2 entries, but the full object contains many more such entries.
[
{
"id": 1,
"start": 7.85,
"end": 9.813,
"text": "Θέλω να",
"words": [
"Θέλω",
"να"
],
"isBeingEditedByUser": false,
"translatedText": "I want to"
},
{
"id": 2,
"start": 9.898,
"end": 13.055,
"text": "Από κάτι το πήραν πολύ άσχημα ο οπαδός του Ολυμπιακού",
"words": [
"Από",
"κάτι",
"το",
"πήραν",
"πολύ",
"άσχημα",
"ο",
"οπαδός",
"του",
"Ολυμπιακού"
],
"isBeingEditedByUser": false,
"translatedText": "Something very bad for Olympiacos fan"
}
]
My understanding is that this JSON structure corresponds to an Array (in Javascript) or a List in Python. In this case, it is an array containing two elements, where each element is itself an object.
However, when I try to use the object on the Flask side, it seems that it has been mapped to a string (rather than a List). Is this normal behavior? I have not been able to find any documentation which states that this is the normal mapping. I would have expected the JSON object to be mapped to a Python List object instead, but this is not happening.
I know that I can use python.loads() myself to convert the string into the appropriate List structure, but I did not expect to have to do this and want to make sure that I am not misunderstanding something here.
Here is a snippet of code which shows the relevant portion in my Flask function:
#app.route('/update_SRT_file', methods=['POST'])
def update_SRT_file():
# Validate the request body contains JSON
if request.is_json:
json_obj = request.get_json()
eprint("update_SRT_file: received JSON object: ")
eprint("Type of received object is", type(json_obj));
else:
eprint("update_SRT_file: Request was not JSON ")
Here is what gets printed out:
23:12:12.771824 update_SRT_file: received JSON object:
23:12:12.771878 Type of received object is **<class 'str'>**
After more investigation, the problem was occuring because the client was JSON encoding the data twice. Upcon removing the additional encoding, it was found that now Flask correctly maps the incoming JSON Array to a python List structure.
Your POST request is sending the JSON as raw text. Flask then receives it as raw text in the request body. Some web application frameworks might automatically parse the text into a JSON-like object or data structure, but Flask does not, at least not out of the box.

Adding a field mapping to 'base64Encode' an index field by calling REST endpoint not working: "A resource without a type name was found"

I'm trying to update a search index (using the Update Indexer) by sending a PUT request to https://searchservicename.search.windows.net/indexes/indexName?api-version=2017-11-11 and it's not working.
If I make the exact same request with the list of fields, the request works as expected and I receive a 200. As soon as I try to add fieldMappings as well, I get an error.
My Json I'm adding to the request as "application/json":
{
"name": "indexName",
"fields": [
<List of Valid Fields w/ Valid Types>
],
"fieldMappings": [
{
"sourceFieldName": "fieldName",
"targetFieldName": "fieldName",
"mappingFunction": {
"name": "base64Encode"
}
}
]
}
When calling the API, the error I'm getting is:
{Search request failed: {"error":{"code":"","message":"The request is invalid. Details: index : A resource without a type name was found, but no expected type was specified. To allow entries without type information, the expected type must also be specified when the model is specified.\r\n"}}
I expect the request to return 200 and have the field mapping added.
The error I get seems to be related to the list of fields but as mentioned before, the request works as expected with the same body minus the field mappings.
Let me know if you need any other information from me - Thanks.
Field mappings should be added to an indexer, not an index. Based on your request, you are trying to update an index.

Gmail API history request returning Drafts

I posted a question abt the api returning invalid history ids. I'm trying to figure this out. I think the ids are just not valid in a messages get request, since these are not real messages, rather drafts. I don't know why history list is returning drafts for a messagesAdded request. Can somebody tell me if this is the expected behavior?
{
"history": [
{
"id": "10946109",
"messages": [
{
"id": "15cc8cd840c2945a",
"threadId": "15cc5ccf65733c7f"
}
],
"messagesAdded": [
{
"message": {
...
"labelIds": [
"SENT"
]
}
}
]
},
{
"id": "10975146",
"messages": [
{
...
}
],
"messagesAdded": [
{
"message": {
...
"labelIds": [
"DRAFT"
]
}
}
]
}
If I need to filter for actual messages - not drafts, do I just do labelIds does not contain DRAFT?
Your first question:
Can somebody tell me if this is the expected behavior?
Yes this is expected behavior (replicated). Check this Document regarding History List:
Users.history: list
Lists the history of all changes to the given mailbox. History results
are returned in chronological order (increasing historyId).
Your second question:
If I need to filter for actual messages - not drafts, do I just do labelIds does not contain DRAFT?
Yes there is an actual filter. You can change the "labelId" parameter to anything except "DRAFT" so it would not return draft results in the history.
Below is a simple guide on how to properly filter your messages without returning Draft label types:
To check your list of labelId's, try this Label API Test Link to see your list of labels just to be sure that you will be using a valid "labelId" later in step 3 by executing the API.
Get the value of the "historyId" by executing the Message List API, retrieving a list of message then get one id then use the Message Get API
by entering the ID to retrieve the "historyId". Make sure that the labelId is not a "DRAFT" type or you have to get another id from the list just to avoid returning a "DRAFT" type.
Then execute History API Test Link. Enter your "userId" and the "startHistoryId"(make sure to subtract the value of the "startHistoryId" by 1) of your message and change the "labelId" by using one from the list of labels you retrieved from your GET API in step 2, change the "historyTypes" to "messagesAdded" then click execute.
It should return a list of message under the "labelId" being inputted and not a "DRAFT" type.

Empty Array being returned

I have created an ionic app and I am currently stuck trying to retrieve an array back from MongoDB through Go. This is what the data in MongoDB looks like.
{
"_id": {
"$oid": "58a86fc7ad0457629d64f569"
},
"name": "ewds",
"username": "affe#dsg.com",
"password": "vdseaff",
"email": "fawfef",
"usertype": "Coaches",
"Requests": [
"test#t.com"
]
}
I am currently trying to get back the Requests field one of the ways I tried was trying to receive the whole document using the following code.
//this is the struct being used.
type (
User struct {
Name string
Username string
Password string
Email string
UserType string
Requests []string
}
)
results := User{}
err = u.Find(bson.M{"username": Cname}).One(&results)
This only returns the following with an empty array.
{ewds affe#dsg.com vdseaff fawfef Coaches []}
In your data the Requests field has a capital R. The bson library that converts the mongo document to your struct type has this to say
https://godoc.org/gopkg.in/mgo.v2/bson#Unmarshal
The lowercased field name is used as the key for each exported field, but this behavior may be changed using the respective field tag.
So your options are to either add a tag to your Requests field or change your data to use lowercase requests. If you choose the tag option it would be like
Requests []string `bson:"Requests"`

Resources