I'm trying to run a simple Ktor server on Google App Engine Standard. If I run their sample code on github it works well, but fails as soon as I call any App Engine API with the following message:
Can't make API call memcache.Set in a thread that is neither the original request thread nor a thread created by ThreadManager
All App Engine APIs trigger similar error messages. Is this related to how Ktor handles requests, and is there a way to make them work?
I think that the Google Cloud java library doesn't have this threading limitation, but unfortunately I can't use it. I need to access the App Engine Memcache service, which is only available through the App Engine APIs.
Related
I can't find any documentation on how gcp scheduler works under the hood. An App Engine is needed in the project, so I assume that the Http calls or Pub/Sub messages are started from the App Engine.
Currently I can use a cloud scheduler even without an App Engine in the project. Apparently a compute engine that also contains a permanently running VM is also sufficient. Could someone confirm my assumptions please or does anyone have sources on this?
I can't tell you how work Cloud Scheduler under the hood. I can just tell you that works well!
I'm sure there is a VM, or a cluster of VM, on Google serverless environment, and your Cloud Scheduler job is set on it. It's serverless, the under the hood doesn't matter, it works, and it's what I want!
Now, the relation with App Engine can be confusing. In fact, there is no longer relation between the product now, but you need the App Engine API activated on your project to use Cloud Scheduler. This strange things is normal if you have been using Google Cloud for a while. At the beginning, only App Engine existed, and Datastore, Cloud Task, Cloud Scheduler was all "modules" of App Engine. Years, after years, google has refactored and extracted these modules to create independent products, as you can see them today. However, some relations are still present, like the API activation.
I have a Google Cloud project which has the following resources:
App Engine Standard Environment, which hosts the frontend (basically react-app).
A private Google Cloud function
Now, I though of the following scenarios to call the function from the react-app
Set Allow only internal traffic to Cloud function: Only App Engine flexible environments can call the function, so ruled out
Authentication using CORS: The clients can still call the function.
making the cloud function public: Would let everyone invoke the function, which is not recommended for private API
Invoking function using Google Cloud API: This would expose my Google Cloud API keys in the frontend.
So is there a work around to call a Cloud function from an App Engine Standard environment
You can't call a Cloud Functions (or Cloud Run) with ingress = internal only with App Engine. See my answer here
To secure your cloud functions, you can rely on the IAM service. If your App Engine service call your cloud functions (I mean you have backend code that run on App Engine, not static JS file that run in the user browser), you can grant only the App Engine service account (you can now customize the service account to run with your App Engine service) the permission to call your cloud function.
Of course, your cloud functions is publicly exposed but only the authenticated and authorised traffic will be routed to your Cloud Functions. All the bad traffic (from internet or from other unauthorised services) will be rejected automatically by GFE (Google Front End) and before invoking your Cloud Functions. Therefore you will pay only for the valid traffic, all the bad traffic is discarded by google.
If the code runs on the user browser (because your App Engine serves only static files), you can use a proxy layer to add security filtering, like API Gateway.You can have a look to this answer
Is there an API to inspect a task queue's backlog from within app engine flex? App Engine Standard has the com.google.appengine.api.taskqueue.Queue.fetchStatistics API but that seems to fail in App Engine Flex with error:
com.google.apphosting.api.ApiProxy$CallNotFoundException: Can't make API call taskqueue.FetchQueueStats in a thread that is neither the original request thread nor a thread created by ThreadManager
The Task Queue API was deprecated and is currently only available for App Engine Standard 1st generation runtime.
For App Engine Flex, Standard 2nd gen and other applications, you should use the Cloud Tasks API
Previously the Firebase Admin SDK would not run on App Engine with automatic scaling. Probably it will create some threads and the Thread class or something like that is not whitelisted.
Now that Google has given us support for Java 8 on App Engine (and removed the whitelist), would this problem be gone? Will the Firebase Admin SDK work on App Engine with automatic scaling?
Firebase Admin SDK still won't work in the AppEngine Java 8 runtime without manual scaling, for several reasons:
The threads support available in the Java 8 runtime comes with some limitations. For instance, any thread not created using the AppEngine's ThreadManager interface cannot call AppEngine APIs. This means these threads cannot make outgoing network calls.
The database client code in the SDK spawns long-running threads. The only way to do that in the AppEngine environment is via background thread support. The SDK specifically looks for this feature when deployed in AppEngine. I've tried to run the SDK with this requirement relaxed, but it doesn't work. It seems request-scoped threads cannot outlive the request even in the new Java 8 runtime.
However, you should be able to use the FirebaseAuth interface (used for custom token minting, ID token verification and user management) in the AppEngine, without manual scaling. This part of the SDK supports running on request-scoped threads.
Im creating a Node.js website that probably won't have loads of traffic, and was looking into cheap solutions to host the site. Came across Google cloud services offering free usage for their services with limits. A f1-mirco is more than enough for my needs, but I will happily pay for some usage if it goes over by any chance.
I wanted to setup a linux centOS 7 on GCE (which I already did), and run my application and REST API on it. Now here comes the problem.
I tried to use Google's datastore service, but it sprung an app engine instance and without it datastore won't work.
Is datastore entirely relying on app engine to function?? In the docs, it said if you use any of the client API, it requires app engine. What can I do to not use the client api and query data then? Don't want to use the app engine at the moment or datastore is just not for me then?
Thanks for any help!
Some of the underlying infrastructure of Cloud Datastore and App Engine are still tied together for creation, etc. So while creating an Cloud Datastore database also defines an App Engine instance for the project, it doesn't require you to use it. You don't get charged for App Engine either, unless you decide to deploy an App using it.
You should be totally fine use the Google Cloud Node client library on the f1 micro instance.