I'm almost at the limit of 200. I thought there used to be a form to request an increase of quota beyond 200, but it looks like that link is gone now. All I see is a generic feature request link on this page: https://developers.google.com/appengine/docs/quotas#Datastore
Related
I am trying to use google search engine via python script. It was working fine at the beginning but later I have got "HTTP Error 403: Forbidden". I thing it is due to 100 requests per day limitation. But I have payed account and billing support told me that everyting is ok. Is it possible somehow to solve this problesm?
Thanks, Ilia
There are lot of reasons for which API will throw Error 403. You can find them all here : GOOGLE DOC FOR 403 Forbidden
Your 403 error might indicate that the API key you are using may have lost it's permissions, or that the quota of your API requests may have been exceeded and the new quota (The one you have when you already have a billing account) might not have been updated.
I assume you are using this API; If not, don't worry, the following advice might also help you.
You first need to see if your API key is still working or you might want to generate new ones. You can do this here. Your code should make reference to the new API Key file.
If it is not the case then you should wait a little bit until your Billing account is updated and the quota gets readjusted. You can read a bit more about how to monitor the requests to your APIs here.
You can always find more help with your particular issue if you describe a little the symptoms and your attempts or hypothesis to solve it.
At last you may find more info on the community page of the API.
There's a hard limit on the socket connect count for GAE https://cloud.google.com/appengine/quotas. The number 3M per day seems relatively low for any largish scale project. I'm calling into Google Drive APIs (https://developers.google.com/drive/api/v3/reference/) from java and hitting against the limit for my project. Is there a workaround for this?
You can ask for a quota increase request to Google but keep in mind that your account must not be a free trial to do it and that in some regions this is not possible. In your request you must to explain the reason why you need the quota increase.
Once you submit the request, it will be take until 3 business days to apply the change.
You can take a look at this link Request a quota increase to know how to request a quota increase. You only must to replace ‘GPU’ for ‘Socket receive count per day’
I am using the Youtube Data API to get search results for a given query.
Example Request (# avg. rate: 1-2 per minute):
https://content.googleapis.com/youtube/v3/search?q=The+Time+(Dirty+Bit)+The+Black+Eyed+Peas+music+video&maxResults=5&part=snippet&key=XXX
Here is my quota usage for the 2 occasions I have tried this. https://imgur.com/yoEKmft
Here is a chart of requests I have made today: https://imgur.com/pke4TMO
On both days I have exceeded my quota of 10,000 while only making 200ish requests.
These numbers do not match up and I can't understand why. I would expect the number of requests to equal quota usage?
I've checked my code and the number of requets being made by my application matches the number of requets on the dashboard.
Any help or guidance would be greatly appreciated.
For search resource will take 100 cost from quota per request.
And you can also use this. https://developers.google.com/youtube/v3/determine_quota_cost
To calculate quota cost that will be use per request.
I'm getting "Over Quota
This application is temporarily over its serving quota. Please try again later." on my GAE app. It's not billing-enabled. I ran a security scan against it today, which presumably triggered the over quota, but I can't explain why based on the information in the console.
Note that 1.59G has been used responding to 4578 requests. That's an average of about 347k per request, but none of my responses should ever be that large.
By filtering my logs I can see that there was no request today whose response size was greater than 25k. So although the security scan generated a lot of small requests over its 14 minute run, it couldn't possibly account for 1.59G. Can anyone explain this?
Note: mostly suppositions ...
The Impact of Security Scanner on logs section mentions:
Some traces of the scan will appear in your log files. For instance,
the security scanner generates requests for unlikely strings such as
"~sfi9876" and "/sfi9876" in order to examine your application's error
pages; these intentionally invalid page requests will show up in your
logs.
My interpretation is that some of the scan requests will not appear in the app's logs.
I guess it's not impossible for some of the scanner's requests to similarly not be counted in the app's request stats, which might explain the suspicious computation results you reported. I don't see any mention of this in the docs to validate or invalidate this theory. However...
In the Pricing, costs, and traffic section I see:
Currently, a large scan stops after 100,000 test requests, not
including requests related to site crawling. (Site crawling requests
are not capped.)
A couple of other quotes from Google Cloud Security Scanner doc:
The Google Cloud Security Scanner identifies security vulnerabilities
in your Google App Engine web applications. It crawls your
application, following all links within the scope of your starting
URLs, and attempts to exercise as many user inputs and event handlers
as possible.
Because the scanner populates fields, pushes buttons, clicks links,
and so on, it should be used with caution. The scanner could
potentially activate features that change the state of your data or
system, with undesirable results. For example:
In a blog application that allows public comments, the scanner may post test strings as comments on all your blog articles.
In an email sign-up page, the scanner may generate large numbers of test emails.
These quotes suggest that, depending on your app's structure and functionality, the number of requests can be fairly high. Your app would need to be really basic for the quoted kinds of activities to be achieved in 4578 requests - kinda supporting the above theory that some scanner requests might not be counted in the app's stats.
Current procedure to serve image is as follows:
Store image on google cloud storage
Get blob_key: google.appengine.ext.blobstore.create_gs_key(filename)
Get url: google.appengine.api.images.get_serving_url(blob_key,size=250,secure_url=True)
To remove the image, after retrieving the blob_key:
Delete serving url:
google.appengine.api.images.delete_serving_url(blob_key)
Delete google cloud storage file: 'cloudstorage.delete(filename)'
Issue
The issue is that the url is still serving for an undefined amount of time, even though the underlying image does not exist on google cloud storage anymore. Most of the time the url returns 404 in ~24hrs, but have also seen 1 image still serving now (~2wks).
What are the expectations about the promptness of the delete_serving_url call? Any alternatives to delete the url faster?
I can address one of your two questions. Unfortunately, it's the less helpful one. :/
What are the expectations about the promptness of the delete_serving_url call?
Looking at the Java documentation for getServingUrl, they clearly spell out to expect it to take 24 hours, as you observed. I'm not sure why the Python documentation leaves this point out.
If you wish to stop serving the URL, delete the underlying blob key. This takes up to 24 hours to take effect.
The documentation doesn't explain why one of your images would still be serving after 2 weeks.
It is also interesting to note that they don't reference deleteServingUrl as part of the process to stop serving a blob. That suggests to me that step (1) in your process to "delete the image" is unnecessary.