Google App Engine and Restlet Response differ from localhost - google-app-engine

I have a restlet Application on google app engine, when testing on my local machine, I get the normal json response in the form I desire, but when deployed on the live appspot, the response is somehow mixed into some type of object.
localhost:
{"status":"404"}
appspot:
//OK[1,["{\"status\":\"404\"}"],0,7]

I figured this out, when declaring #GET, #PUT, #POST, etcc... You need to specify the content type you will be returning, despite the fat that your function may be a String, Rep function, I believe the default is a representation, and that is why the extra junk is added on. If you simply have #Get("txt") you will get your response as plain text.

Related

App engine - raw request body - stripe webhooks

I'm trying to run my node.js app app engine and I am having trouble with stripe webhooks - with the constructEvent, that I need to give a request raw body. Worked on virtual machine but not on app engine.
event = stripe.webhooks.constructEvent(req.rawBody, sig, stripeKeys.webhookPaymentIntent);
Says:
No signatures found matching the expected signature for payload. Are you passing the raw request body you received from Stripe? https://github.com/stripe/stripe-node#webhook-signing
Just looking at the code that you posted here, I wonder if the last parameter is indeed the value that you wanted to pass to the constructEvent function. it reads webHookPaymentIntent. I wonder if this should really be the webhook signature secret? It may be that it really is the webhook secret value, but just named a bit misleadingly.
Maybe this is something though you can verify? A simple test would really to be to pass the string literal here to see if that would work first. Of course make sure not to commit that to any source control.
The stripe-node method params are listed here for reference: https://github.com/stripe/stripe-node/blob/1d6207e34f978d8709d42d8a05d7d7e8be6599c7/lib/Webhooks.js#L11

How does Google App Engine modify HTTP request/response encoding?

I'm trying to send JSON HTTP request from Google App Engine application and retrieve response, and while this works great locally, it suddenly breaks when I deploy it to GAE.
To be more precise, the HTTP response body that is returned to my application ends up looking like this instead of being simple JSON:
�\bD�[��8��ʖϣ�M�M$ �\\�bA` #!r���~pvk�cR]�_7E�
I did find one set of circumstances where I get correct response on GAE which might give some insight at this behavior - if the response doesn't have content-type header it goes through fine, but as soon as there is content-encoding header set to "gzip" present I get the incorrect garbage above as a response body.
Unfortunately I don't have control over the service I'm calling. So the only choice I have is to fix this somehow on my side, but to fix it I'm trying to understand the difference between what Google does to response. Does anyone know?
I understand that Google does some things to HTTP traffic. Is it forcing gzip on my responses as well?
I've also tried playing with encodings, trying to read response as utf-8, and setting utf-8 as default encoding for my GAE application as recommended here, but with no effect. I've ruled out incorrect processing of response in my code or anything I'm using, at least I think so, otherwise I'd have the same problems locally. I'm trying to understand what exactly is happening with hope that this will give me some idea how to prevent it.
EDIT: I figured it out and did a workaround but it's still a workaround, not a solution. So my GAE app calls another web service from outside GAE which sometimes gzips response and sometimes doesn't. If it does, GAE strips away content-type header from response, thus preventing my app from correctly decoding response body. My workaround so far is to get response bytes and test if response is valid JSON, unzipping it manually if it isn't. Would still want to know if stripping content-type can be prevented...
As explained at https://cloud.google.com/appengine/kb/#compression , the application should not supply the content-encoding header: "Google App Engine does its best to serve gzipped content to browsers that support it. Taking advantage of this scheme is automatic and requires no modifications to applications".
I believe that the origin of this architecture (that content encoding is not controlled by the application side of things -- on App Engine, your application code -- but by the server/gateway side) originated with WSGI, the Python standard interface of applications to web servers/gateways (which App Engine uses on the Python runtime), but the architecture makes enough sense that we generalized it -- as the above page puts it, "This approach avoids some well-known bugs with gzipped content in popular browsers.".
The client is far from powerless, and in fact, if it so chooses, can control the content encoding -- still quoting, "To force gzipped content to be served, clients may supply 'gzip' as the value of both the Accept-Encoding and User-Agent request headers. Content will never be gzipped if no Accept-Encoding header is present.".

httplib.HTTPConnection in Google AppEngine

I use httplib.HTTPConnection within my app. Do I really need to provide host parameter in httplib.HTTPConnection constructor? If so, why? (I mean, I know that it's a mandatory parameter, but I wonder if I could specify None or empty string) And is there any global constant in Google AppEngine and in development server which I can use within my app in order to omit explicitly defined host.
If you leave it out of the constructor, how will the other methods know where to send their messages?
The address/name of the server you're connecting to is the parameter for the HTTPConnection, the URI on that server is what goes into request.
From the python documentation (which is the basis for AppEngine)
h1 = httplib.HTTPConnection(host[, port[, strict[, timeout[, source_address]]]])
h1.request(method, url[, body[, headers]])
[edit]
Remember, it may not always be you who is responsible for this code. Also, why complicate things by including so much more information in the URI when you're (for example) making calls to numerous URI's on the same website?
[/edit]

Getting a forwarded URL from thousands of different domains in Google App Engine

I actually asked this question before, but I cannot get my account details back, so I'm asking again:
I have a series of different domain names that I would like to all point (via URL forwarding from my domain host) to a google app engine application that reads what the forwarding URL is. So if the domain typed in was original XYZ.com, then when I am forwarded to my application, I can return what that original domain name was. I'm using the python variant. How best can I do this without coding for each and every variant?
So for example I might have aaa.com and bbb.com and ccc.com that all should point to the same appspotdomain, and I wish to somehow determine what the referring URL was. I have thousands of domains and I have URL forwarding set-up. So unless I put something in the header is there a smart way to pull out the referring URL. I have tried the os.environ["SERVER_NAME"] route but this just gives the app-engine domain.
Try
os.environ['HTTP_REFERER']
or
self.request.headers['Referer']
Be careful though, it might not always be available.

Google Geocoding API - REQUEST_DENIED

Apparently I do not need a 'Maps API key' to use the 'Google Geocoding API' according to:
http://code.google.com/apis/maps/documentation/geocoding/index.html
However, I obtain this:
{
"status": "REQUEST_DENIED",
"results": [ ]
}
Does this mean that my IP is blocked? What can I do to overcome this?
Until the end of 2014, a common source of this error was omitting the mandatory sensor parameter from the request, as below. However since then this is no longer required:
The sensor Parameter
The Google Maps API previously required that you include the sensor parameter to indicate whether your application used a sensor to determine the user's location. This parameter is no longer required.
Did you specify the sensor parameter on the request?
"REQUEST_DENIED" indicates that your request was denied, generally because of lack of a sensor parameter.
sensor (required) — Indicates whether or not the geocoding request comes from a device with a location sensor. This value must be either true or false
Remove the API key parameter and its value.
eg. https://maps.googleapis.com/maps/api/geocode/json?address=[YOUR ADDRESS]&sensor=true
I found that in my case, calling to the service without secure protocol (meaning: http), after adding the key=API_KEY, cause this issue.
Changing to https solved it.
I've noticed that you also get REQUEST_DENIED for some addresses if you don't properly URL encode your address. For example, in
123 Main St #B, Mytown, CA 94110
the '#' character needs to be encoded as %23
For those who are looking this page in 2017 or beyond, like me
Sensor is not required anymore, I tried and got the error:
SensorNotRequired
I just needed to activate my Google Maps Geocoding API, that seems to be necessary nowadays.
Hope it helps someone like me.
If you just copy&paste the example URL that Google gives in their website
http://maps.google.com/maps/api/geocode/json?address=1600+Amphitheatre+Parkway,+Mountain+View,+CA&sensor=true_or_false
it will fail because of the wrong parameter of the sensor. You should change it to true or false and not the one that they wrote.
Maybe is the error that you have had, like it happened to me...
I had this problem and I realized that I was assuming that Geocoding came with the JS maps API. However, it is a separate API which I hadn't enabled in the cloud console. Enabling it fixed it right away.
I got this problem as well using the drupal 7 Location module. Autofilling all empty locations resulted in this error. Executing one of the requests to the location api manually resulted in this error in the returned JSON:
"Browser API keys cannot have referer restrictions when used with this API."
Resolving the problem then was easy: create a new key without any restrictions and use it only for Geocoding.
Note for those new to google api keys: by restrictions they mean limiting requests using an api key to specific domains / subdomains. (eg. only request from http://yourdomain.com are allowed).
It's suck Google don't let you that your service is not enabled by this account. Try to enable it first.
Go here https://console.developers.google.com/project
and create a new project with place service activated this may solve your problem.
If you not have configured a billing account with your credit card, the API do not work. The Google is now very hungry for money and not open more your products "free". Obviously that him offer a free limited access number of consults and over this acces, the billing is very high.
https://cloud.google.com/maps-platform/pricing
As you say, this can mean that your IP address has been blocked. I'd make sure that you specify the key parameter on the query string for the Geocoding API request.
https://maps.googleapis.com/maps/api/geocode/xml?sensor=false&address=Placename&key=XXxxxXXxXxxxxXXxx
Also make sure that if you've set up IP Address Restrictions within the Developer Console, you've allowed the correct IP address, just click the project within the list and you'll see the allowed IPs.
If you're still running into issues, you might want to look into printing out the values of the status and error_message elements from the response from Google, you'll see something like this:
REQUEST_DENIED - This IP, site or mobile application is not authorized to use this API key. Request received from IP address 123.4.5.678, with empty referer
If it doesn't mention an IP address restriction, it may well give you enough information about the problem to Google a fix.
For anyone struggling with this issue, I just found out that the Geocoding API can't be used with API keys that have referrer restrictions. Just remove all your referrer restrictions and you should be good.
If you're using any other APIs that do allow keys with referrer restrictions (like the Maps JS API), it's probably best to create a 2nd key with no restrictions to use exclusively for geocoding, because other APIs might display your key publicly and someone else could start using it on their own site.
Google is returning a very useful error message, which helps to correct the issue!
Dim Request As New XMLHTTP30
Dim Results As New DOMDocument30
Dim StatusNode As IXMLDOMNode
Request.Open "GET", "https://maps.googleapis.com/maps/api/geocode/xml?" _
& "&address=xxx", False
Request.Send
Results.LoadXML Request.responseText
Set StatusNode = Results.SelectSingleNode("//status")
Select Case UCase(StatusNode.Text)
Case "REQUEST_DENIED"
Debug.Print StatusNode.NextSibling.nodeTypedValue
...
Error Message Examples
Message 1:
Requests to this API must be over SSL. Load the API with "https://"
instead of "http://".
Message 2:
Server denied the request: You must use an API key to authenticate each request to Google Maps Platform APIs. For additional information, please ...
If none of given solutions fixed the error, the issue probably about Google Cloud Billing settings. You must enable Billing on the Google Cloud Project at billing/enable.
Learn more
{
"error_message" : "You must enable Billing on the Google Cloud Project at https://console.cloud.google.com/project/_/billing/enable Learn more at https://developers.google.com/maps/gmp-get-started",
"results" : [],
"status" : "REQUEST_DENIED"
}
I created another Credential(New API) only for Geocoding, with "Key restrictions" 'None' and "API restrictions" 'Restrict key'
Selected APIs:
Directions API
Geocoding API
Geolocation API
Maps JavaScript API
Places API
and then it's worked.
I just ENABLED my geocoding API, geolocation API and places API on my google cloud platform (where I had generated my API key I was using) and it worked.

Resources