I have a Quarkus application already deployed on Google Cloud Run.
It depends on MySQL, hence there is an instance started on Cloud SQL.
Next step in my deployment process is to add keycloak. From what I've read the best option seems to be Google App Engine.
The approved answer in this question gave me some good insight of what needs to be done ... mostly.
What I did was:
Locally I made a sub-directory in the main project.
In that directory I added the app.yaml and the Dockerfile (as described here for instance).
There I executed the said two commands: gcloud init and gcloud app deploy.
I had my doubts about this set up and they were backed up by the error I got eventually:
ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: The first service (module) you upload to a new application must be the 'default' service (module). Please upload a version of the 'default' service (module) before uploading a version for the 'morph-keycloak-service' service (module).
I understand my set up breaks the overall structure of the project but I'm not sure how to mix those two application with the right services.
I understand keycloak is a stateful application, hence cannot live on Cloud Run (by the way the intention is for keycloak to use the same database instance shared with the application).
So does any one know a more sensible set up, or what can I move in mine in order to fix it?
In short:
The answer really is in reading the error message (thanks #gaefan) - about the error itself it explains enough. So I just commented out the service: my-keycloak-service line in the app.yaml (thus leaving gcloud to implicitly mark it as the default one) and the deployment continued.
Eventually keycloak didn't connect to the database but if I don't manage to adjust the configurations that would probably be a subject to a different question.
On the point of project structure and functionality:
First off, thanks #NoCommandLine and #guillaume-blaquiere for your input!
#NoCommandLine the application on Cloud Run is sort of a headless REST API enabled backend. Most of the API calls are secured by keycloack. A next step in the deployment process would be to port an existing UI (React) client on the Firebase hosting (or on another suitable service - I'm still not completely sure which approach is best) and in order for the users to work with this client properly they must make an SSO through keycloak first.
I'm quite new to GCP and the number and variants of the available options are still overwhelming to me - one must get familiar with the nuances but I guess it takes time. So I'm still taking suggestions on how to adjust my project structure to fit better the services stack. Thanks!
I am hosting some Web Applications in Google Cloud Platform using App Engine and those are for internal purpose only. One month ago I got a mail from Google Cloud Team, saying one of my apps needs verification. By based on their response I did some research and finally migrated all apps to the Organisation level as they mentioned in documentation (below link for reference). https://support.google.com/googleapi/answer/7394288#gsuite-app
But, yesterday also I got another notification regarding the same.
May I ignore this notification, or are there any further steps I need to complete?
As stated in this other documentation page:
If you're creating an internal web app for which [...] your project is
associated with a Cloud Organization that your users belong to, you
don't need to go through verification. Internal users of your
application won't see the unverified app screen.
If your application will only be used by internal users belonging to the same organization as where your project is located, you can ignore this message. It was probably triggered by the fact that your application is indeed not verified (although you do not need to do so).
So if that is the case, you will only need, as stated in the link you shared, to create an Organization and then migrate your existing project to that organization (then make sure that the users who will be accessing the app belong to the same organization).
In my app (Google App Engine Standard Python 2.7) I have some flags in global variables that are initialized (read values from memcache/Datastore) when the instance start (at the first request). That variables values doesn't change often, only once a month or in case of emergencies (i.e. when google app engine Taskqueue or Memcache service are not working well, that happened not more than twice a year as reported in GC Status but affected seriously my app and my customers: https://status.cloud.google.com/incident/appengine/15024 https://status.cloud.google.com/incident/appengine/17003).
I don't want to store these flags in memcache nor Datastore for efficiency and costs.
I'm looking for a way to send a message to all instances (see my previous post GAE send requests to all active instances ):
As stated in https://cloud.google.com/appengine/docs/standard/python/how-requests-are-routed
Note: Targeting an instance is not supported in services that are configured for auto scaling or basic scaling. The instance ID must be an integer in the range from 0, up to the total number of instances running. Regardless of your scaling type or instance class, it is not possible to send a request to a specific instance without targeting a service or version within that instance.
but another solution could be:
1) Send a shutdown message/command to all instances of my app or a service
2) Send a restart message/command to all instances of my app or service
I use only automatic scaling, so I'cant send a request targeted to a specific instance (I can get the list of active instances using GAE admin API).
it's there any way to do this programmatically in Python GAE? Manually in the GCP console it's easy when having a few instances, but for 50+ instances it's a pain...
One possible solution (actually more of a workaround), inspired by your comment on the related post, is to obtain a restart of all instances by re-deployment of the same version of the app code.
Automated deployments are also possible using the Google App Engine Admin API, see Deploying Your Apps with the Admin API:
To deploy a version of your app with the Admin API:
Upload your app's resources to Google Cloud Storage.
Create a configuration file that defines your deployment.
Create and send the HTTP request for deploying your app.
It should be noted that (re)deploying an app version which handles 100% of the traffic can cause errors and traffic loss due to:
overwriting the app files actually being in use (see note in Deploying an app)
not giving GAE enough time to spin up sufficient instances fast enough to handle high income traffic rates (more details here)
Using different app versions for the deployments and gradually migrating traffic to the newly deployed apps can completely eliminate such loss. This might not be relevant in your particular case, since the old app version is already impaired.
Automating traffic migration is also possible, see Migrating and Splitting Traffic with the Admin API.
It's possible to use the Google Cloud API to stop all the instances. They would then be automatically scaled back up to the required level. My first attempt at this would be a process where:
The config item was changed
The current list of instances was enumerated from the API
The instances were shutdown over a time period that allows new instances to be spun up and replace them, and how time sensitive the config change is. Perhaps close on instance per 60s.
In terms of using the API you can use the gcloud tool (https://cloud.google.com/sdk/gcloud/reference/app/instances/):
gcloud app instances list
Then delete the instances with:
gcloud app instances delete instanceid --service=s1 --version=v1
There is also a REST API (https://cloud.google.com/appengine/docs/admin-api/reference/rest/v1/apps.services.versions.instances/list):
GET https://appengine.googleapis.com/v1/{parent=apps/*/services/*/versions/*}/instances
DELETE https://appengine.googleapis.com/v1/{name=apps/*/services/*/versions/*/instances/*}
GAE recently got some updates including "organisations" which seems to have also resetted many other settings - e.g. super unclean migration.
Question: how can I select the deploy region/location for google-app-engine apps?
Unfortunately it is not anymore possible to do so as described here: How to choose Google App Engine servers' location
When I create a new project there is no "advanced" settings. If I still just "blindly" deploy (after ignoring the annoying new tutorial / quick start that is forced into ones throat) it will use europe-west as region.
In the Google Cloud Console -> Menu -> Compute -> App Engine
You should then see a box with title "Your first app", follow the steps and at some point it will prompt you to select the server location (cannot be changed later once selected). After selecting server location you don't need to continue with the Hello World app, just deploy your code as normal.
I believed something has been changed recently and there's no clear documentation yet.
Is it possible to delete an GAE application after it has been created?
I made a mistake while typing the name and now have a dummy application that I haven't been able to remove.
With the new Google Cloud console, you can still disable GAE applications as before (App Engine --> Settings --> Disable). They cannot currently be deleted. However you can delete the entire project by going to IAM --> Settings --> Shut Down. This button is in the header and a bit tricky to spot. It looks like this:
As of AppEngine SDK 1.2.6 it's possible to delete apps completely. But beware, the app-id won't be usable again.
As most of the answers are outdated or contradictive and this is an important question I decided to clarify current possible solutions when intending to delete an application in Google App Engine or having related issues.
Currently, there is no way to delete an existing app in GAE. Once created it cannot be removed, nor its initial settings can be changed (like the region where it was deployed). The only possible workaround is starting a new project and deploying a new application. There were feature requests in Google Issue Tracker regarding these issues: deleting an app and changing zone/region. You can still delete the whole project as described in Steve Armstrong's answer, but bear in mind that this will remove everything you created there (like GCE, GKE etc.), not only GAE.
However, it all depends on why you would like to delete your app. If you would simply like to stop it from serving requests or you don't want it to incur further costs, you can disable the app as described in the GCP docs here.
This feature is already logged, please star it:
http://code.google.com/p/googleappengine/issues/detail?id=335
To disable /delete your application:
In the Administration Console, click your application to make it the active application.
Click Application Settings on the left side under Administration.
Click Disable Application.
Click Disable Application Now.
If you want to delete your app:
If billing is enabled for your app, disable billing. You aren't allowed to delete before you do this.
Click Request Permanent Deletion. The application will be deleted in approximately 72 hours.
To re-enable your disabled application, click Re-Enable Application.
source
There currently isn't a way to delete a GAE application.
Beyond disabling the App Engine application you can:
Disable it's API permission under APIs & Services
Remove the App Engine related files from Storage
Delete App Engine permissions under IAM & Admin
Delete the App Engine Service account
This will freeze all App Engine related billing charges for the undeletable disabled App engine application. At least it worked for me :)
I was evaluating if we could use AppEngine and ran the standard tutorial which created a test app for me under my the default project. When I tried to delete the App I was shocked that it can't be done ! The only way is to delete the project which would delete all other GKE and any other services under that account.
After a bunch of research and calling product support of Google this what they suggested: To upgrade to Silver Support for 150$/month and send them an email to delete the app.
Here is the chat session with Google Support. If you were considering using Google AppEngine I would think again.
This issue has been fixed; see the docs here:
https://cloud.google.com/appengine/docs/standard/python/console/?csw=1#delete_app
Carlos, you're right that the issue has been fixed, and I up-voted you for that. However, your link is a little outdated and an updated link is listed below.
https://developers.google.com/appengine/docs/adminconsole/applicationsettings#Disable_or_Delete_Your_Application
open https://console.cloud.google.com/cloud-resource-manager?organizationId=0 ,select the project(or application) to be deleted,then click delete
I wanted to delete some legacy Google App Engine applications I made years ago, but when I tried to delete them from the new Google Cloud Platform (like this: https://support.google.com/cloud/answer/6251787#shut-down-a-project) I kept getting "You do not have permission" errors. The solution I found was to sign up for a free trial of Google Cloud Platform, then I was able to delete them.
I couldn't find to delete the default app-engine, however if you navigate to App Engine > Settings , there is a button to Disable it and it stop serving. when you click on the button type the project name in the prompt.