When I first time setup(do not recall details) Appengine somehow mapped default service to datastore_admin. I was adding additional service and gcloud forced me to update default.
ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: The first service (module) you
upload to a new application must be the 'default' service (module). Please
upload a version of the 'default' service (module) before uploading a version
for the 'datastore-export' service (module).
I updated but it seems I messed up. Now it does not give me the option to rollback default service map to datastore_admin. For educational perspective any thoughts how can I do it? I could not find clear guidance on this.
The message simply indicates that the app needs a default service. See:
Why do I need to deploy a "default" app before I can deploy multiple services in GAE?
What purpose does the default service serve in Google's app engine
Assuming your app was indeed running with just a service called datastore_admin (maybe from before the mandatory default service check was in place, these days it shouldn't be possible to make such deployment) - you just need to "make" one of the services the default now, to pass the check. It doesn't really matter which service or what the service does, as long as it doesn't interfere with the named services, of course.
You can even make it not do anything or just return errors for the url patterns it is responsible for, if you want to, but you'd be just wasting instance hours - the default service gets all the garbage traffic that other services aren't prepared to handle. So if you have one service which would normally be accessible via a web browser - make that the default one - it typically has to be prepared for garbage requests anyways.
Related
I have a Quarkus application already deployed on Google Cloud Run.
It depends on MySQL, hence there is an instance started on Cloud SQL.
Next step in my deployment process is to add keycloak. From what I've read the best option seems to be Google App Engine.
The approved answer in this question gave me some good insight of what needs to be done ... mostly.
What I did was:
Locally I made a sub-directory in the main project.
In that directory I added the app.yaml and the Dockerfile (as described here for instance).
There I executed the said two commands: gcloud init and gcloud app deploy.
I had my doubts about this set up and they were backed up by the error I got eventually:
ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: The first service (module) you upload to a new application must be the 'default' service (module). Please upload a version of the 'default' service (module) before uploading a version for the 'morph-keycloak-service' service (module).
I understand my set up breaks the overall structure of the project but I'm not sure how to mix those two application with the right services.
I understand keycloak is a stateful application, hence cannot live on Cloud Run (by the way the intention is for keycloak to use the same database instance shared with the application).
So does any one know a more sensible set up, or what can I move in mine in order to fix it?
In short:
The answer really is in reading the error message (thanks #gaefan) - about the error itself it explains enough. So I just commented out the service: my-keycloak-service line in the app.yaml (thus leaving gcloud to implicitly mark it as the default one) and the deployment continued.
Eventually keycloak didn't connect to the database but if I don't manage to adjust the configurations that would probably be a subject to a different question.
On the point of project structure and functionality:
First off, thanks #NoCommandLine and #guillaume-blaquiere for your input!
#NoCommandLine the application on Cloud Run is sort of a headless REST API enabled backend. Most of the API calls are secured by keycloack. A next step in the deployment process would be to port an existing UI (React) client on the Firebase hosting (or on another suitable service - I'm still not completely sure which approach is best) and in order for the users to work with this client properly they must make an SSO through keycloak first.
I'm quite new to GCP and the number and variants of the available options are still overwhelming to me - one must get familiar with the nuances but I guess it takes time. So I'm still taking suggestions on how to adjust my project structure to fit better the services stack. Thanks!
Whenever I deploy a new version in Google App Engine, and I transfer traffic to it, the previous version still consumes messages from our message broker. How can I make sure only the newly deployed version will consume messages without shutting down the old instances?
If you have multiple versions deployed while traffic is being migrated, you can check the current version using the Modules API and compare that with the default version.
Your check might look something like this:
from google.appengine.api import modules
def default_version = modules.get_default_version()
def instance_version = modules.get_current_version_name()
# you may additionally want to query the instances of the default version
# to make sure they've booted up and are actively serving traffic.
if default_version != instance_version:
# don't consume messages
In the code example above, the default version is the version traffic is being migrated to, and the current version is the version of the instance.
See also Using the Modules API.
Note: Services were formerly known as modules and the API methods still reflect that naming.
During the deployment of your service, you can use --promote and --stop-previous-version options.
https://cloud.google.com/sdk/gcloud/reference/app/deploy
However, generally it's best to gradually migrate the traffic. This is the case for both users and backend services. Since you can't deploy two services at the exact same time, imagine your GAE deployment is delayed by a few seconds. Do you expect the messages to be consumed by your running service that is getting replaced? So, it shouldn't matter if a few messages still got routed to the old instance while the traffic is migrating. That would be the right design.
In my app (Google App Engine Standard Python 2.7) I have some flags in global variables that are initialized (read values from memcache/Datastore) when the instance start (at the first request). That variables values doesn't change often, only once a month or in case of emergencies (i.e. when google app engine Taskqueue or Memcache service are not working well, that happened not more than twice a year as reported in GC Status but affected seriously my app and my customers: https://status.cloud.google.com/incident/appengine/15024 https://status.cloud.google.com/incident/appengine/17003).
I don't want to store these flags in memcache nor Datastore for efficiency and costs.
I'm looking for a way to send a message to all instances (see my previous post GAE send requests to all active instances ):
As stated in https://cloud.google.com/appengine/docs/standard/python/how-requests-are-routed
Note: Targeting an instance is not supported in services that are configured for auto scaling or basic scaling. The instance ID must be an integer in the range from 0, up to the total number of instances running. Regardless of your scaling type or instance class, it is not possible to send a request to a specific instance without targeting a service or version within that instance.
but another solution could be:
1) Send a shutdown message/command to all instances of my app or a service
2) Send a restart message/command to all instances of my app or service
I use only automatic scaling, so I'cant send a request targeted to a specific instance (I can get the list of active instances using GAE admin API).
it's there any way to do this programmatically in Python GAE? Manually in the GCP console it's easy when having a few instances, but for 50+ instances it's a pain...
One possible solution (actually more of a workaround), inspired by your comment on the related post, is to obtain a restart of all instances by re-deployment of the same version of the app code.
Automated deployments are also possible using the Google App Engine Admin API, see Deploying Your Apps with the Admin API:
To deploy a version of your app with the Admin API:
Upload your app's resources to Google Cloud Storage.
Create a configuration file that defines your deployment.
Create and send the HTTP request for deploying your app.
It should be noted that (re)deploying an app version which handles 100% of the traffic can cause errors and traffic loss due to:
overwriting the app files actually being in use (see note in Deploying an app)
not giving GAE enough time to spin up sufficient instances fast enough to handle high income traffic rates (more details here)
Using different app versions for the deployments and gradually migrating traffic to the newly deployed apps can completely eliminate such loss. This might not be relevant in your particular case, since the old app version is already impaired.
Automating traffic migration is also possible, see Migrating and Splitting Traffic with the Admin API.
It's possible to use the Google Cloud API to stop all the instances. They would then be automatically scaled back up to the required level. My first attempt at this would be a process where:
The config item was changed
The current list of instances was enumerated from the API
The instances were shutdown over a time period that allows new instances to be spun up and replace them, and how time sensitive the config change is. Perhaps close on instance per 60s.
In terms of using the API you can use the gcloud tool (https://cloud.google.com/sdk/gcloud/reference/app/instances/):
gcloud app instances list
Then delete the instances with:
gcloud app instances delete instanceid --service=s1 --version=v1
There is also a REST API (https://cloud.google.com/appengine/docs/admin-api/reference/rest/v1/apps.services.versions.instances/list):
GET https://appengine.googleapis.com/v1/{parent=apps/*/services/*/versions/*}/instances
DELETE https://appengine.googleapis.com/v1/{name=apps/*/services/*/versions/*/instances/*}
I'm wondering if I can configure a GAE Flexible "service" to use a specific service account ID instead of the default service account ID, which all services run as.
Why? I want to isolate each service's permissions, but also simplify the service code by allowing it to still use the Application Default credentials method of calling Google APIs.
Is there an app.yaml configuration/environment variable where I can control this?
I didn't see the answer here https://cloud.google.com/appengine/docs/flexible/java/migrating
or
https://cloud.google.com/appengine/docs/flexible/java/authorizing-apps
I also found this https://cloud.google.com/compute/docs/access/create-enable-service-accounts-for-instances#using which tells me how to do what I want at the compute engine level, but I don't see how to apply it at the GAE Flex managed level.
If this is not possible, what is the next best alternative?
This isn't possible at the moment. The best way to go forward with this would be to open a feature request here.
I was unable to access my Backend Instance at custom domain.
For example, I have an app and I access the Normal Instance sucessfully at:
http://www.[my_app_id].appspot.com or http://[my_app_id].appspot.com
And I have a backend config name=test and I accessed Backend Instance successfully at:
http://test.[my_app_id].appspot.com
In admin interface, the "Instances" link show the instances of Backend and Normal Instance separately. The content show is the same, but is easy to see when a request go to the Backend Instance and when go to Normal Instance.
Then I configured the wildcard "test" in Google Apps to access my Backend Instance at a custom URL:
I continue access the Normal Instance sucessfully at:
http://www.[my_domain].com or http://[my_domain].com
But request at
http://test.[my_domain].com
redicted to the Normal Instance instead of Backend Instance.
The doc's said it should work but I cann't at this moment and I need uses custom domain because my app is multitenancy.
What I do wrong?
Your backed is really supposed to be accessed by the front end, as I understand it.
So when your application front end makes a request to it's back end (e.g. via a URL), it'll work as it's all done internally.
Have you set your back end to be publicly accessible?
https://developers.google.com/appengine/docs/python/backends/overview#Public_and_Private_Backends
Backends are private by default, since they typically function as a component inside an application, rather than acting as its public face. Private backends can be accessed by application administrators, instances of the application, and by App Engine APIs and services (such as Task Queue tasks and Cron jobs) without any special configuration. Backends are not primarily intended for user-facing traffic, but you can make a backend public for testing or for interacting with an external system.
I don't know why the redirection is not working, but perhaps you should modify your question to show what problem it is you are trying to solve here and get an answer to that instead?