My service sends messages to users, and I want to throttle on the following criteria; Saying a user A belongs to group G.
max 5 messages within last hour to user A
max 100 messages with last hour to members belong to group A
The number of messages and floating window size are configurable.
Any recommended options?
Is Cloud Endpoints an option?
Apigee is too expensive for our service.
More Background:
use app engine standard environment python2.7
the request is internal; a app engine service send request to another app engine service
Thanks in advance.
You can use API Gateway with App Engine instead of Apigee or Cloud Endpoints API Gateway quotas, it provides support for Python 2.7 and supports quotas like the ones you need.
Related
I have a couple of running Cloud functions that fetch data from an external API and then post the received data into an internal API which is running on Google App Engine. I've come up with this approach because the external API has a rate limit that I sometimes need to bypass.
In order to calculate the monthly cost of this setup I was looking at the Google Docs and saw this:
Outbound data to other Google APIs in the same region is free, as is inbound data. Where the Google API you are using is global (i.e. not region-specific), it is considered to be the same region.
My question: What does Google mean with "Google APIs"? Is an App Engine endpoint (e.g. xxx.appspot.com) a Google API?
Google APIs generally refer to the APIs and services provided as part of Google Cloud, as well as other Google services such as SMTP provided by Gmail and recaptcha.
I have a Python GAE flex app, and I received an email from Google stating:
We are writing to let you know that the v0.1 and v1beta1 endpoints of
the Compute Engine Metadata Server will be shut down on January 15,
2020. From January 15, 2020, requests to the v0.1 and v1beta1 endpoints will no longer be supported, and may return HTTP 404 NOT
FOUND responses.
The email also stated that my app used these old metadata endpoints in the last 90 days.
I had never heard of the Compute Engine Metadata Server before so it is not something that I have intentionally used. I suppose it is possible that one of my dependencies (Flask etc.) but it seems unlikely.
Any idea where my GAE Flex app might be making a call to the Compute Engine Metadata Server? Is this a false alarm from Google?
No, it is not false alarm. As it writes in the public documentation:
v1beta1 server and v0.1 metadata server endpoints are deprecated and scheduled for shutdown. Ensure that you update all requests to use v1. For more information, see Transitioning to the v1 metadata server endpoint.
The metadata server is the place on where the GCE instances are storing their metadata. Your App Engine Flex app is running on compute instances. You can use the Metadata server to query information about the instance like : ip, instance id, service account info etc..
App Engine Flex uses the metadata server in the back scene, so, even though you did not use the Metadata Server explicitly, app engine does it. Although App Engine flex should do all the updates by itself and it is less probable for you to face any issues,it is recommended to migrate from v0.1 to v1 metadata server endpoint just to be sure. Here
there is a guide in the official documentation on how to achieve that.
What is the maximum number of services that an App Engine Flex app can have?
I found this site, under limits is the max number of services 5. Is this correct?
The Maximum services per application that the documentation you linked refers to are Backend services. The limit is the number of backend services available to the GAE flex application. This applies to running versions of GAE.
Also, regarding the last part of the limits documentation:
If you expect a notable upcoming increase in usage, you can proactively request adjustments from the App Engine Quotas page in the GCP Console.
This means that you can request an increase for the maximum services your application have access to by going to the GCP Quotas page and selecting Compute Engine API as Service. In the list of quotas you'll find the Backend services quota. With the current usage and the current limit. Select it and click Edit quotas to request an increase.
I'm working on an project on GAE which contains several high traffic endpoints and one endpoint that has low traffic but need authentication. My question is, can I enable Cloud Endpoints on the low traffic endpoint for authentication, but not on those high traffic endpoints, because Endpoints is expensive?
My understanding of how Cloud Endpoints work with GAE is that once the swagger file is deployed, and you specify a Endpoints version in app.yaml file, all traffic that goes through the Endpoints proxy server before reaching the application server. Also because Endpoints is charged based on # of requests, so in my case the entire application will take tens of millions request per day, but only a tiny fraction of that request will goes to a specific endpoint in the application that requires authentication.
In order to avoid spending thousands of dollars per month on Endpoint, It would be way cheaper if I can only let those privileged request go through Cloud Endpoints, and be charged only on those request.
You can have only one Endpoint (thus only one charged Endpoint pricing), if you separate it from the other URLs. You can do that by separating the Services or Projects
Google Cloud Storage is accessed by the App Engine UrlFetch feature
Is this always the case? No matter what I use to connect to Google Cloud Storage from AppEngine - All client libraries, any language, will always use UrlFetch feature to access Google Cloud Storage
Also
This is an excerpt from App engine - cloud storage client document
There are no bandwidth charges associated with making Google Cloud Storage client library calls to Cloud Storage. However, there are operations charges; because the calls count against your URL fetch quota, as the library uses the URL Fetch service to interact with Cloud Storage. And There are operations and storage charges
https://cloud.google.com/appengine/docs/java/googlecloudstorageclient/setting-up-cloud-storage#pricing_quotas_and_limits
Does this means that out of 3 quotas of URL Fetch (call/request count, outgoing bandwidth and incoming bandwidth) only call/request count is added against the url fetch quota. Aren't Url Fetch Incoming and Outgoing bandwidth quota considered and added too?
Sorry, my GCP free billing account got expired recently and I'm planning to renew it in next month. So till then not able to try out and verify the second part of the question.
Thanks.
Kiran.
Yes, all 3 quotas of URL fetch count in this case.
Instead of finding a way around these limitations, you could maybe think about redesigning your application.
As Nick from Cloud Platform Support puts it:
look into the use of Datastore to store the temporary results of your
process, since this will have better rate-limiting quotas than cloud
storage, which isn't really meant for rapid writes such as this. You
could also look into BigTable, or any number of distributed databases
such as memcached to resolve your issue of temporary file storage.