I need to make application that needs to poll server often, but GAE has limitations on requests, so making a lot of requests could be very costly. Is it possible to use long polling and make requests wait for the maxium 30 seconds for changes?
Google AppEngine has a new feature Channel API, with that you have
a possibility to build a good realtime application.
Another solution is to use a third part comet server like mochiweb
or twisted with a iframe pattern.
Client1, waiting a event:
client1 --Iframe Pattern--> Erlang/Mochiweb(HttpLongPolling):
Client2, sending a message:
client2 --XhrIo--> AppEngine --UrlFetch--> Erlang/Mochiweb
To use mochiweb with comet pattern, Richard Jones has written a good
topic (on google: Richard Jones A Million-user Comet Application).
We've tried implementing a Comet-like long-polling solution on App Engine, with mixed results.
def wait_for_update(request, blob):
"""
Wait for blob update, if wait option specified in query string.
Otherwise, return 304 Not Modified.
"""
wait = request.GET.get('wait', '')
if not wait.isdigit():
return blob
start = time.time()
deadline = start + int(wait)
original_sha1 = blob.sha1
try:
while time.time() < deadline:
# Sleep one or two seconds.
elapsed = time.time() - start
time.sleep(1 if elapsed < 7 else 2)
# Try to read updated blob from memcache.
logging.info("Checking memcache for blob update after %.1fs",
elapsed)
blob = Blob.cache_get_by_key_name(request.key_name)
# Detect changes.
if blob is None or blob.sha1 != original_sha1:
break
except DeadlineExceededError:
logging.info("Caught DeadlineExceededError after %.1fs",
time.time() - start)
return blob
The problem I'm seeing is that requests following a long-polling one, are getting serialize (synchronized) behind the long-polling request. I can look at a trace in Chrome and see a timeline like this:
Request 1 sent. GET (un-modified) blob (wait until changed).
Request 2 sent. Modify the blob.
After full time-out, Request 1 returns (data unmodified).
Request 2 gets processed on server, and returns success.
I've used wireshark and Chrome/timeline to confirm that I AM sending the modification request to the server on a distinct TCP connection from the long-polling one. So this snychronization must be happing on the App Engine production server. Google doesn't document this detail of the server behavior, as far as I know.
I think waiting for the channel API is the best hope we have of getting good real-time behavior from App Engine.
I don't think long polling is possible. The default request timeout for google appengine is 30 seconds.
In long polling if the message takes more than 30 secs to generate, then it will fail.
You are probably better off using short polling.
Another approach is to "simulate" long polling withing the 30 sec limit. To do this if a message does not arrive within, say 20 sec, the server can send a "token" message instead of a normal message, requiring the client to consume it and connect again.
There seems to be feature request (and its accepted) on google appengine for long polling
Related
Google describes basic scaling like this:
I don't really have any other choice, as I'm using a B1 instance, so automatic scaling is not allowed.
That raises the question though, if I have an endpoint that takes a variable amount of time (could be minutes, could be hours), and I basically have to set an idle_timeout, will App Engine calculate idle_timeout from the time the request was made in the first place or when the app is done handling the request?
If the former is right, then it feels a bit unfair to have to guess how long requests will take when thread activity is a usable indicator of whether or not to initiate shutdown of an app.
Here you are mixing up two different terms.
idle_timeout is the time that the instance will wait before shutting down after receiving its last request
Request timeout is the amount of time that App Engine will wait for the return of a request from your app
As per the documentation:
Requests can run for up to 24 hours. A manually-scaled instance can
choose to handle /_ah/start and execute a program or script for many
hours without returning an HTTP response code. Task queue tasks can
run up to 24 hours.
I have an application where a user can upload a PDF using angular-file-upload.js
This library does not support file chunking: https://github.com/nervgh/angular-file-upload/issues/41
My elastic load balancer is configured to have an idle timeout of 10 seconds and other parts of the application depend on keeping this parameter.
The issue is if the file upload takes longer than 10 seconds the user receives a 504 Gateway Timeout in the browser and an error message. However, the file still reaches the server after some time.
How can I ignore or not show the user this 504 Gateway Timeout that comes from the ELB? Is there another way around this issue?
The issue you have is that an ELB is always going to close the connection unless it gets some traffic back from your server. See below from AWS docs. It's the same behaviour for an ALB or a Classic load balancer.
By default, Elastic Load Balancing sets the idle timeout to 60 seconds
for both connections. Therefore, if the instance doesn't send some
data at least every 60 seconds while the request is in flight, the
load balancer can close the connection. To ensure that lengthy
operations such as file uploads have time to complete, send at least 1
byte of data before each idle timeout period elapses, and increase the
length of the idle timeout period as needed.
So to get around this, you have two options:
Change the server processing to start sending some data back as soon as the connection is established, on an interval of less than 10 seconds.
Use another library for doing your uploads, or use vanilla javascript. There are plenty of examples out there, e.g. this one.
Edit: Third option
Thanks to #colde for making the valid point that you can simply work around your load balancer altogether. This has the added benefit of freeing up your server resources which get tied up with lengthy uploads. In our implementation of this we used pre-signed urls to securely achieve this.
I have a Google App Engine program that calls BigQuery for data.
The query usually takes 3 - 4.5 seconds and is fine but sometimes takes over five seconds and throws this error:
DeadlineExceededError: The API call urlfetch.Fetch() took too long to respond and was cancelled.
This article shows the deadlines and the different kinds of deadline errors.
Is there a way to set the deadline for a BigQuery job to be above 5 seconds? Could not find it in the BigQuery API docs.
BigQuery queries are fast, but often take longer than the default App Engine urlfetch timeout. The BigQuery API is async, so you need to break up the steps into API calls that each are shorter than 5 seconds.
For this situation, I would use the App Engine Task Queue:
Make a call to the BigQuery API to insert your job. This returns a JobID.
Place a task on the App Engine task queue to check out the status of the BigQuery query job at that ID.
If the BigQuery Job Status is not "DONE", place a new task on the queue to check it again.
If the Status is "DONE," then make a call using urlfetch to retrieve the results.
Note I would go with Michael's suggestion since that is the most robust. I just wanted to point out that you can increase the urlfetch timeout up to 60 seconds, which should be enough time for most queries to complete.
How to set timeout for urlfetch in Google App Engine?
I was unable to get the urlfetch.set_default_fetch_deadline() method to apply to the Big Query API, but was able to increase the timeout when authorizing the big query session as follows:
from apiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
credentials = ServiceAccountCredentials.from_json_keyfile_dict(credentials_dict, scopes)
# Create an authorized session and set the url fetch timeout.
http_auth = credentials.authorize(Http(timeout=60))
# Build the service.
service = build(service_name, version, http=http_auth)
# Make the query
request = service.jobs().query(body=query_body).execute()
Or with an asynchronous approach using jobs().insert
query_response = service.jobs().insert(body=query_body).execute()
big_query_job_id = query_response['jobReference']['jobId']
# poll the job.get endpoint until the job is complete
while True:
job_status_response = service.jobs()\
.get(jobId=big_query_job_id).execute()
if job_status_response['status']['state'] == done:
break
time.sleep(1)
results_respone = service.jobs()\
.getQueryResults(**query_params)\
.execute()
We ended up going with an approach similar to what Michael suggests above, however even when using the asynchronous call, the getQueryResults method (paginated with a small maxResults parameter) was timing out on url fetch, throwing the error posted in the question.
So, in order to increase the timeout of URL Fetch in Big Query / App Engine, set the timeout accordingly when authorizing your session.
To issue HTTP requests in AppEngine you can use urllib, urllib2, httplib, or urlfetch. However, no matter what library you choose, AppEngine will perform HTTP requests using App Engine's URL Fetch service.
The googleapiclient uses httplib2. It looks like httplib2.Http passes it's timeout to urlfetch. Since it has a default value of None, urlfetch sets the deadline of that request to 5s no matter what you set with urlfetch.set_default_fetch_deadline.
Under the covers httplib2 uses the socket library for HTTP requests.
To set the timeout you can do the following:
import socket
socket.setdefaulttimeout(30)
You should also be able to do this but I haven't tested it:
http = httplib2.Http(timeout=30)
If you don't have existing code to time the request you can wrap your query like so:
import time
start_query = time.time()
<your query code>
end_query = time.time()
print(end_query - start_query)
This is one way to solve bigquery timeouts in AppEngine for Go. Simply set TimeoutMs on your queries to well below 5000. The default timeout for bigquery queries is 10000ms which is over the default 5 second deadline for outgoing requests in AppEngine.
The gotcha is that the timeout must be set both in the initial request: bigquery.service.Jobs.Query(…) and the subsequent b.service.Jobs.GetQueryResults(…) which you use to poll the query results.
Example:
query := &gbigquery.QueryRequest{
DefaultDataset: &gbigquery.DatasetReference{
DatasetId: "mydatasetid",
ProjectId: "myprojectid",
},
Kind: "json",
Query: "<insert query here>",
TimeoutMs: 3000, // <- important!
}
queryResponse := bigquery.service.Jobs.Query("myprojectid", query).Do()
// determine if queryResponse is a completed job and if not start to poll
queryResponseResults := bigquery.service.Jobs.
GetQueryResults("myprojectid", res.JobRef.JobId).
TimeoutMs(DefaultTimeoutMS) // <- important!
// determine if queryResponseResults is a completed job and if not continue to poll
The nice thing about this is that you maintain the default request deadline for the overall request (60s for normal requests and 10min for tasks and cronjobs) while avoiding setting the deadline for outgoing requests to some arbitrary large value.
I have an application hosted on the Google App Engine platform. The application is mostly I/O intensive and involves a high number of upload and download operations to the app engine server by an HTTP client.
My question is: what does the instance hour comprise of in this case ? Does it include the total time taken by the HTTP client to upload the request data ? Or does the instance hour calculation begin when the entire request data is uploaded and processing of the request starts ?
Example results from the application:
An HTTP client sent an upload request to the app engine server, request data size 1.1 MB
Time taken for request to complete on the client side - 78311 ms
Corresponding server log entry:
- - [Time] "POST / HTTP/1.1" 200 127 - "Apache-HttpClient/UNAVAILABLE (java 1.4)" "" ms=3952 cpu_ms=1529 api_cpu_ms=283 cpm_usd=0.154248 instance=
An HTTP client sent a download request to the app engine server.
Time taken for request to complete on the client side - 8632 ms
Corresponding server log entry:
- - [Time] "POST / HTTP/1.1" 200 297910 - "Apache-HttpClient/UNAVAILABLE (java 1.4)" "" ms=909 cpu_ms=612 api_cpu_ms=43 cpm_usd=0.050377 instance=
Which of these figures contributes towards the instance hour utilization - is it a) ms, b) cpu_ms or c) the time taken for request to complete on the client side ?
Please note that the HTTP client uses a FileEntity while uploading data, therefore I assume that data is sent over by the client to the server in a single part.
Incoming requests are buffered by the App Engine infrastructure, and the request is only passed to an instance of your app once the entire request has been received. Likewise, outgoing requests are buffered, and your app does not have to wait for the user to finish downloading the response. As a result, upload and download time are not charged against your app.
To understand numbers in log look at log breakdown, a bit more readable here.
None of the options you presented (a. b. c.) are directly billed. It used to be that GAE counted CPU time as a unit of cost, but that changed Nov 2011. Now you pay for instance uptime, even if instance is not handling any requests. Instances stop being billed after 15 min of inactivity.
(This does not mean that GAE actually shuts instances down after they stop billing for them - see "Instances" graph in your dashboard.)
How many instances are up depends on your app's performance settings.
Since your app is IO intensive it will help to enable concurrent requests (Java, Python 2.7). This way instance can run multiple parallel requests which are mainly waiting for IO - in our app I'm seeing about 15-20 requests being served in parallel on one instance.
Update:
This is what first link says about ms=xyz log entry:
This is the actual time (hence 'wallclock' time) taken to return a response to
the user, not including the time it took for the user to send their request or
the time it takes to send the response back - that is, just the time spent
processing by your app.
Note that Nick Johnson is an engineer on GAE team, so this can be taken as authoritative answer.
I am connecting to a 3rd party server on the following wsdl
http://webservices.ticketvala.com/axis2/services/WSTicketvala?wsdl
I am using JAX-WS to generate client code and call relevent method on 3rd party server. 3rd party server may take time between 15-25 seconds to send response.
It works fine on tomcat.
Now when i deploy this to GAE 1.5.3, often i get ScocketTimeoutException in less than 10 seconds. Sometimes it is succesfull taking even 20 seconds. I want to know why it fails many times. And any workaround to increase this response deadline time / to avoid this ScoketTimeOutException forever.
Similarly,
I have another RESTfull service at http://ticketgoose.com/bookbustickets/TGWSStationNameResponseAction.do?" +
"event=getStationDetails&password=123456&userId=ctshubws
I am connecting it through java.net.URL and many times i get TimeoutException. How can i raise this timeout limit to more than 30 seconds?
Thanks
Deepak
No user-initiated request can take more than 30 seconds to complete in Google App Engine:
http://code.google.com/intl/en/appengine/docs/java/runtime.html#The_Request_Timer
And a HTTP request to a external URL in a user-initiated request can't take more than 10 seconds to complete:
http://code.google.com/intl/en/appengine/docs/java/urlfetch/overview.html#Requests
If you need to do the overall work in more than 30 seconds and can do it in background (not needing to return the response directly via HTTP), use Task Queues. Note that the simplest way to do background work with tasks is to use the DeferredTask. And the best part: HTTP requests to external URLs on tasks can take up to 10 minutes to complete.