I need to start my google app engine project locally. It works normally on google server, but debugging becomes a pain, since deployment on each fix takes a lot of time. I've almost managed to start it locally, but I don't understand how to create queues, which are necessary. My steps:
run dev_appserver.py app.yaml, following using local development server guide. All works fine except of queues:
_, err := taskqueue.Add(u.Ctx(), task, queueName)
exceptions.ThrowIfErr(true, "err_msg", err)
Throws
Panic! UserMessage: <err_msg>, Error: API error 1 (taskqueue: UNKNOWN_QUEUE),...
I can easily create queues on remote server (using creating push and creating pull guides):
gcloud app deploy queue.yaml
For queue.yaml:
queue:
- name: Pull-Data-Queue
mode: pull
- name: Push-Data-Queue
mode: push
rate: 1/s
I can open http://localhost:8000/datastore and see some created data.
I can open http://localhost:8000/taskqueue and see the only one default push queue. No tools to add new queues here.
Google guide says queues can't be created dynamically from code, only by yaml or xml. But how to create them in local environment. Is it even possible? gcloud app deploy queue.yaml works only for remote deployment as far as I understand.
If the queue.yaml file is side-by-side with the app.yaml file the development server detects it automatically. It is possible to use symlink instead of copying. It fixed the problem.
Related
I have a Quarkus application already deployed on Google Cloud Run.
It depends on MySQL, hence there is an instance started on Cloud SQL.
Next step in my deployment process is to add keycloak. From what I've read the best option seems to be Google App Engine.
The approved answer in this question gave me some good insight of what needs to be done ... mostly.
What I did was:
Locally I made a sub-directory in the main project.
In that directory I added the app.yaml and the Dockerfile (as described here for instance).
There I executed the said two commands: gcloud init and gcloud app deploy.
I had my doubts about this set up and they were backed up by the error I got eventually:
ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: The first service (module) you upload to a new application must be the 'default' service (module). Please upload a version of the 'default' service (module) before uploading a version for the 'morph-keycloak-service' service (module).
I understand my set up breaks the overall structure of the project but I'm not sure how to mix those two application with the right services.
I understand keycloak is a stateful application, hence cannot live on Cloud Run (by the way the intention is for keycloak to use the same database instance shared with the application).
So does any one know a more sensible set up, or what can I move in mine in order to fix it?
In short:
The answer really is in reading the error message (thanks #gaefan) - about the error itself it explains enough. So I just commented out the service: my-keycloak-service line in the app.yaml (thus leaving gcloud to implicitly mark it as the default one) and the deployment continued.
Eventually keycloak didn't connect to the database but if I don't manage to adjust the configurations that would probably be a subject to a different question.
On the point of project structure and functionality:
First off, thanks #NoCommandLine and #guillaume-blaquiere for your input!
#NoCommandLine the application on Cloud Run is sort of a headless REST API enabled backend. Most of the API calls are secured by keycloack. A next step in the deployment process would be to port an existing UI (React) client on the Firebase hosting (or on another suitable service - I'm still not completely sure which approach is best) and in order for the users to work with this client properly they must make an SSO through keycloak first.
I'm quite new to GCP and the number and variants of the available options are still overwhelming to me - one must get familiar with the nuances but I guess it takes time. So I'm still taking suggestions on how to adjust my project structure to fit better the services stack. Thanks!
I am trying to use task queues in GAE Java 8, but somehow it does not seem to deploy correctly via the queue.xml file. I can also not see the task queues in the Cloud Tasks console (which is where I get redirected from the app engine console).
I get an error java.lang.IllegalStateException: The specified queue is unknown : xxxxx when running the app.
The app runs fine locally. I can see the task queues appearing locally in the admin page.
Does this mean that I cannot deploy task queues via queue.xml anymore?
You should be aware that the queue configuration is not a per-version config (or even a per-service one!), it is a global, per-application config. Or per-project if you want - considering that there can only be one GAE application per GCP project.
This single queue configuration is shared by all versions of all services of your application, so:
if/when services/versions need different queue configs all of them need to be merged into a single file for deployment.
pay attention at deployment not to overwrite/negatively impact existing services/versions
While in some cases the queue.xml file might be deployed automatically when you deploy your application code it is not always the case. The official recommended deployment method is using the deployment command dedicated for the queue configuration, which can be performed independently from deploying application/service code. From Deploying the queue configuration file:
To deploy the queue configuration file without otherwise altering the
currently serving version, use the command:
appcfg.sh update_queues <application directory>
replacing <application directory> with the path to your application
main directory.
Pay extra attention if you have:
other non-java standard environment services in your app - they use the queue.yaml queue configuration file, managing the same content in 2 different files/formats can be tricky
other services managing queue via using Cloud Tasks. See Using Queue Management versus queue.yaml.
I created a Nodejs program that is intended to be executed as a long-running process unattended. The program works on localhost when running npm start. I need to be able to 'tail -f' its stdout on demand. There is no port it listens to. Does Google App Engine support this use case? If so, how to configure it?
I have deployed the app with following app.yaml file but it doesn't seem to work. At least the stdout of the instance is empty/unexpected. I believe the reason is App Engine is designed for web server and this process failed to pass health check of some sort.
runtime: nodejs
vm: true
btw, on Heroku this use case is clearly supported because it has two types of dyno: web and worker. Obviously worker dyno is the choice.
In Google App Engine Go SDK sometimes we can fill the local datastore with a bunch of test data. It's tiring to delete thousands of records 20 at a time using the web interface. Is there a command the erases the local datastore?
Simply provide the --clear_datastore command line parameter when starting:
goapp serve --clear-datastore
Documented at: The Go Development Server: Using the Datastore.
To clear the local datastore for an application, use the --clear_datastore=yes option when you start the web server:
Note that the documentation was "copied" from the Python section, you need to use it as presented above (you get an error if you try to execute goapp serve --clear_datastore=yes).
I don't know about Go, but in Java you can simply delete local_db.bin from WEB-INF/appengine-generated folder. An empty file will be created next time you run the app.
I am trying to follow the instructions for running a simple new Google App Engine web application locally (without Google Web Toolkit, just the Web App) named "tunes".
I am following these instructions. Step one is to make a run configuration. I made one using all the defaults; I checked that under the Server tab the "Run built-in Server" box is checked. However, when I Run the app, I get the Console; right below the console tab it says
<terminated> tunes [Web Application] C:\Program Files (x86)\Java\jre7\bin\javaw.exe (Feb 26, 2014, 5:35:44 PM)
then below the line is what looks like a classic Unix "Usage" error message in the console, in red type, whose first line is
Usage: <dev-appserver> [options] <app directory>
followed by a list of options, and then nothing else happens that I can see.
I tried pointing a browser at http://localhost:8888/tunes as suggested by the documentation, but Firefox could not find a server active at that port.
Under the Arguments tab in the Run Configuration is the following:
"-codeServerPort 9997 --port=8888 org.tunes.gaeproject.Tunes C:\Users\cdf\java\eclipse4.3.1workspace\tunes\war"
I can successfully deploy the web application to the Google App Engine site and run it there.
What do I have misconfigured?
Google AppEngine application is not meant to run on local server, and neither you could create its docker image etc to deploy it anywhere you wish.
Instead I suggest you to port your application to Google Compute Engine (GCE) first within your deployment setup, which might require minor code refactorings, and Kubernetes kinda auto-scaling deployment will functional equivalent to what you have now. But with approach you may port the application easily to local server setup, or a docker image to run from any virtualization environment