Is there any way to forward an incoming request to GAE, kind of like a reverse proxy, to another server, but without urlfetch? URLFetch is very buggy, and during an attack, I run out of resources, and hence the server crashes.
You can use the Python standard libraries urllib, urllib2 or httplib to make HTTP requests. When running in App Engine, these libraries perform HTTP requests using App Engine's URL fetch service, which runs on Google's scalable HTTP request infrastructure.
So whatever you use, it's all App Engine's URL Fetch service anyway. So I'd say no.
Fetch Overview
There are a lot of ways to do a forward or redirect. You can use a cname in your DNS to create an alias for you new address or a meta refresh in HTML:
<meta http-equiv="refresh" content="0;URL='http://www.example.com/'" />
It all depends on your needs and the kind of request.
Related
I have an App Engine Standard (1st Gen) Python app, set up with serverless VPC access. I use requests 2.3 (I have issues with later versions on GAE) to do HTTP requests.
When I try doing an HTTP request to an internal IP address (10.x), it refuses to connect. However, serverless VPC access is working properly: I tested connecting to the same IP address on the same port using a non-HTTP client library (e.g. redis) from App Engine, and this works fine. It's just HTTP requests that fail.
I suspect the URL Fetch service is trying to do the HTTP requests, and fails on internal addresses. If so, is there a way to use requests without triggering the URL Fetch service? Or do you have any other clue what might be going wrong?
I think I found the answer by looking at the SDK source code:
from python_std_lib import httplib
gets you an HTTPLib that uses sockets, without having to override the global httplib (causing sockets to be used everywhere).
I'd like to send requests against the Google Directions API. Google provides a Node.JS client library for the API. However, this AP is server-side only. Attempting to use it from a browser script results in a CORS failure. Multiple past answers (such as this one) indicate that this library simply can't be used in this way.
The alternative is to use the client-side JavaScript API. However, this requires adding a <script> tag to the document root. That's the wrong level of abstraction for my needs. I'd like to use a method from a library or dot-js file instead.
Following the advice given here, I'd like to ask: is there a module available through npm I can use to query the Google Directions API client-side?
It's not naively possible to access the Google Maps Directions API from the client side. Web browsers implement the Single-Origin Policy, which requires that any requests to a domain come from the same domain. Requires between domains are disallowed by default. Cross-domain requests can be enabled at the server lever by setting the right CORS headers on the endpoint, but the Google Maps servers choose not to do this.
There are two ways of working around this. One is to wrap the request using the Google API Auth library. However, I could not get this to work.
What did work was using a reverse proxy. This workaround is actually mentioned in the Google Directions API intro page (albeit obliquely). You will need to set up a server which forwards any requests to an API request, then returns that API request to the original requester. Since this is now a server-side request, SOP will not apply, and you will be good to go.
For an example implementation check out this repository on GitHub.
https://developers.google.com/maps/documentation/directions
This is the Directions API web service. It does not require adding a <script> tag.
You can make direct requests to the service as per the example:
https://maps.googleapis.com/maps/api/directions/json?origin=75+9th+Ave+New+York,+NY&destination=MetLife+Stadium+1+MetLife+Stadium+Dr+East+Rutherford,+NJ+07073&key=YOUR_API_KEY
once you have generated an API key and replaced YOUR_API_KEY in the request with your own key.
I have some appengine modules in my project.
I am building a "Cloud Endpoints" that will works like a API Gateway. Both in them same project.
The endpoints will receive a request and forward to another appengine module, so, when the module process the request, the endpoints will return the response to frontend.
The main reponsibility this API Gateway will be validate permissions and log informations.
The frontend sends: GET,PUT and POST methods.
I read about URLFetch to do it.
I would like to know, Is it fast to use URLFetch to to do it?
Should I use other framework to to id?
If you're on App Engine, external requests should use URL fetch regardless of if you use it directly or use your language level networking primitives. It should be relatively fast, though you should benchmark this for yourself to see if it's an acceptable latency.
I am sending HTTP requests from GAE to MongoLab in the form of:
https://api.mongolab.com/api/1/databases/rival_testdb/collections/testCollection?apiKey=someKey
If this request was coming directly from client side, I would be very concerned. But since this request is coming from inside a servlet in GAE, is this a security concern? How could I make this more secure if possible?
I would like to use MongoLab if possible, but their API only allows for putting the API key directly in the requesting URL. So I am stuck using a URL like the one above for all API requests.
As long as you can secure the API key on the server you should be good. You do not have to worry about it being sniffed on the network as all the communications are over HTTPS thus encrypting the request string and the apiKey along with it.
-will
This is a valid concern for most public services.
Specifically for MongoLab, this discussion is present over here: https://support.mongolab.com/entries/20269612-REST-api-permissions-and-security-best-practice and it does not look like the security additions that they are talking about has been released in public.
Do check out to the link.
Having said, I am assuming that the only way that you are accessing the Mongolab API is from within your GAE Application? That at least makes it more difficult for anyone to sniff out the url from your application.
A good solution would be for MongoLab to allow a HTTP Header to be inserted before your make the REST call. Here you would some ID that identifies that it is your GAE App. And then MongoLab should be doing a check for that before permitting the call to do its work.
I'm testing a Google App Engine app on my Windows machine, running locally on localhost:8084. Fiddler2 shows all my activity when I navigate around my app, but when requesting an external url with urlfetch.fetch() it doesn't show up in Fiddler at all, even when using an http, not an https address, and with a successful status code 200 in the response.
What do I need to do to get the urlfetch.fetch() request from Google App Engine to show up in Fiddler2?
My understanding is that Fiddler2 runs as an HTTP proxy; browser requests go through this proxy instead of directly to the internet resource. This allows Fiddler2 to capture information about the request and the response.
According to the Fiddler2 docs, "You can configure any application which accepts a HTTP Proxy to run through Fiddler so you can debug its traffic". So I think you would need to change the URLFetch API call to use a proxy, supplying the Fiddler URL and port. However, the URLFetch documentation doesn't specify exactly how to do this. You might be able to use urllib2 as specified in this question.
Irussell is generally right, but I'd like to make the answer more specific.
As proxies aren’t supported within Google AppEngine production environment, it’s not directly supported by development engine either. It seems that the only way to overcome this limitation is to modify the code of AppEngine development server.
You'll have to modify the urlfetch_stub.py file, by adding the following lines:
connection = connection_class('127.0.0.1', 8888)
and
full_path = protocol + "://" + host + full_path
You may find the detailed explanation in my blog post Use Fiddler to debug urlfetch requests in Google AppEngine