I was in a place without Internet access for 3 weeks and just came back to find out that one of my apps since January 18 started to reach a quota limit (Datastore Read Operations) after around the 18 hours.
I don't see any increase in traffic from either users or crawlers.
This is the error in the logs:
"The API call datastore_v3.RunQuery() required more quota than is available."
It seems very strange since this application has been running for some years and I'm memcaching most the datastore requests.
Please help - This is affecting my bottom line!
Thanks.
I found a subset of pages in the site that had got a sudden interest from several crawlers and some of the requests that those pages made to the Datastore were not being memcached, so that was it...problem solved.
Thanks.
Related
I'm maintaining a blog app(blog.wokanxing.info, it's in Chinese) for myself which was built upon Google app engine. It's been like two or three years since first deployment and I've never met any quota issue because its simpicity and small visit count.
However since early last month, I noticed that from time to time the app reported 500 server error, and in admin panel it shows a mysterious fast consumption of free datatstore read operation quota. Within a single hour about 10% of free read quota (~5k ops) are consumed, but I'm counting only a dozen requests that involve datastore read ops, 30 tops, which means an average 150 to 200 read op per request, which sounds impossible to me.
I've not commited any change to my codebase for months, and I'm not seeing any change in datastore or quote policy either. Despite that, it also confuses me how such consumption can be made. I use memcache a lot, which leaves first page the biggest player, which fetch the first threads using Post.all.order('-date').fetch(10, offset). Other request merely fetch a single model using Post.get_by_key_name and iterates post.comment_set.
Sorry for my poor English, but can anyone give me some clues? Thanks.
From Admin console check your log.
Do not check for errors only, rather check all types of messages inside the log.
Look for the requests made by robots/web crawlers. In most cases, you can detect such "users" by words "robot" or "bot" (well, if they are honest...).
The first thing you can do is to edit your "robot" file. For more detail read How to identify web-crawler? . Also, GAE has help for use of "robot" file.
If that fails, try to detect IP address used by bot/bots. Using GAE Admin console put such addresses in blacklist and check your quota consumption again.
We have a PHP application running on GAE. It connects to Cloud Datastore using the Google PHP library (v0.6.7).
Google introduced in the last days a new version of App Engine, v1.9.0 (not oficially released), which apparently was running fine, just as 1.8.9 was. However, we have been experiencing some issues related to Cloud Datastore. Sometimes, all the operations regarding to entities updating are just ignored. All the queries used to retrieve information work perfectly, however if we want to create a new entity of update any property, no action is performed. I have been checking for some errors in the response returned by the Cloud Api, but there is no errors or warnings at all.
This situation happened for the first time the 31st of January, and it is also happening today. It started to fail at 3am (GMT +1) and according to the instance log, at the same time the latency times of all the requests suffered an important increase (from 1-3 secs to 5-10 secs). The first time after a few hours the system started to work properly again, but now this problem is lasting much more.
Has anyone experienced anything similar?
Thank you for the report, we're investigating the issue now.
Update: We've addressed the issue. Please join the Google Cloud Datastore downtime notify mailing list for future updates.
https://groups.google.com/forum/?fromgroups=#!topic/gcd-downtime-notify/sNXCFJYFNQU
For future reports about production issues, please refer to the Contact support section of our documentation.
Almost a week ago I migrated my (paid) app from Master/Slave to HRD. Since that time my app has been restricted to 100 emails/day with a warning indicating "Resource is currently experiencing a short-term quota limit". I know it mentions a limitation until the first successful billing so I was hoping the limit would disappear once that happened - but alas it has not! I have also filled out the "request additional resources" form hoping that might help.
Anyone encountered this problem migrating from master/slave? Any suggestions of who I can contact or how I can recover from this limit? The migration process was relatively smooth - except for this problem which has become a significant impact to my customers.
I am having the same problem right now. I just finished my HRD migration yesterday and only today realized that my Mail API requests are failing because the mail quota on my new HRD app is only at 100 messages per day. I did not have this limitation before, and I find it pretty disappointing that such a triviality is causing my users trouble despite a successful migration. I submitted a quota increase request and hoping I don't also have to wait a week before it's applied.
Anyone who is waiting until the last minute to migrate their app to HRD be warned: make sure you apply for a mail quota increase on your new app because your old setting will not carry over, even if you have billing enabled on both apps.
Greetings,
I'm creating a multi-player android game and thought it would be a interesting idea to have App Engine handle the server work.
The game consists of 4 players, each phone requests an update every 0.5 seconds.
These requests are very simple and lightweight so i shouldn't be over reaching any free quotas.
The problem i found was that App Engine only handles 500 requests per second, i would only be able to
have around 60 game sessions active before App Engine will start ignoring new requests?
"App Engine's quota system allows for efficient applications with billing enabled to scale to around 500 queries per second (qps) or more than 40 million queries per day."
Or should i just not use this platform because it is not made for this kind of usage?
I sent this same question to the discussion groups on google but after 4 hours it hasn't been posted, there was no response on whether it was a bad question or anything. Hopefully someone here can give me some advice.
Thank you kindly, i'm looking forward to an answer and or advice.
Greetings,
Rohan C
That's an interesting question, considering the only page where I can find that quote contains the answer in the same paragraph.
http://code.google.com/support/bin/request.py?contact_type=AppEngineCPURequest
App Engine's quota system allows for
efficient applications with billing
enabled to scale to around 500 queries
per second (qps) or more than 40
million queries per day. This is a
substantial amount of traffic and
should easily suffice for even the
heaviest of Slashdottings. But if you
expect your application will need to
handle even higher qps, please
complete this form so we can assist
you.
Is anyone else having this problem? Sometime last night after app engine's server maintenance, i have only successfully been able to geocode a few addresses -- most responses are 620 errors. ive been running the same app with no problems for 3 months, so i think i think its a problem on google's end. one other person on the google groups discussion was able to confirm, but i want to be sure because im very skeptical given that more people arent complaining.
here's the explaination of the 620 status code.
"The given key has gone over the requests limit in the 24 hour period or has submitted too many requests in too short a period of time. If you're sending multiple requests in parallel or in a tight loop, use a timer or pause in your code to make sure you don't send the requests too quickly."
And a post on the subject.
Hope it help!