I have a module that works fine when I push it to app engine. When it works it logs stuff nicely and the logs are accessible in the console logs viewer. But then after a while it just stops working and when I try to access any url give me 500 server errors with no info (it just says waiting 30 seconds might be a good idea). When this happens nothing gets logged for requests.
If I restart the module (by pushing my code to app engine) then it works for a little while again.
The module is running a Pyramid app and the configuration file looks a little something like:
application: my_app
module: my_module
version: dev
runtime: python27
api_version: 1
threadsafe: false
instance_class: B2
basic_scaling:
max_instances: 2
idle_timeout: 10m
handlers:
- url: /actions/.*
script: my_module.application
login: admin
- url: /.*
script: my_module.application
builtins:
- appstats: off
libraries:
- name: webob
version: latest
- name: setuptools
version: latest
includes:
- mapreduce/include.yaml
I think what is happening is that it's hitting the idle timeout and shutting down. I need requests to the module to turn it back on again. How do I do that?
Let me know if you need more info, I'm an app engine noob at this stage. Any help would be greatly appreciated.
When a module start, App Engine call the url /_ah/start. You mustn't handle this request.
In you my_module.application you need to add in the handler who match this request :
def get(self):
# Let module start
if "X-Appengine-Cron" in self.request.headers or "X-AppEngine-TaskName" in self.request.headers or "X-Appengine-Failfast" in self.request.headers:
return
Even if you hit the idle timeout, AppEngine will spin a new instance when new request is coming in.
Use Cloud Debugger to inspect the state of your application. The debugger makes it easier to view the application state and understand what happens after your app has been running for while.
Check the docs on startup state https://cloud.google.com/appengine/docs/python/modules/#Python_Instance_states
you current default handler /.* should be able deal with a /_ah/start if youe handler can gracefully deal with a 404.
Thats how I handle startups. Goes through the main handler which can deal with non existent url requests using the default pyramid not found.
I have a config.add_notfound_view(notfound) registered.
Related
I am still working to get my old style appengines to work under at least go111 (go112 would not work due to dependencies on memcache). I am now stumbling over app.yaml configuration issues with my static files, I used a completely static directory layout before and just specified a few dynamic handlers in the root like this:
runtime: go111
handlers:
- url: /_ah/.*
script: auto
login: admin
secure: always
- url: /dynamic
script: auto
secure: always
- url: /admin/.*
script: auto
login: admin
secure: always
- url: (.*)/
static_files: html\1/index.html
upload: html/index.html
secure: always
- url: /(.*\.map)
mime_type: application/json
static_files: html/\1
upload: html/(.*\.map)
secure: always
- url: /
static_dir: html
secure: always
dev_appserver.py will never call my dynamic entry point. In production this does work, but I am still working on the conversion and would like to test locally. Any hints how to convince dev_appserver.py to let me do this? By the way my gcloud tools are updated as of today.
Your question is why your application works well when you deploy it to production but doesn't work when you use dev_appserver.py to run it locally, and how could you run it with dev_appserver.py. The answer to that is:
You won't be able to run it locally properly using dev_appserver.py, since it does not support runtime Go 1.11. Look at the Local Development Server Options documentation, there's only a link for "Go 1.9".
(as you can see, the links for "Go 1.11" and "Go 1.12" are disabled, which translates to: not supported.)
Documentation for Go 1.11 on App Engine Standard states that in order to test your application locally you would have to use "go run" command (notice how it doesn't mention dev_appserver.py tool). The command would be something like this:
go run [build flags] [-exec xprog] package [arguments...]
For more information about the command go here.
I'm sure you might have already read this but, to know more about the migrations process from Go1.9 to Go1.11 read this documentation.
You have stated that "go run" command wouldn't work for your case. So, a workaround for that would be to test your application directly into App Engine but without migrating the traffic.
When deploying your test version use:
gcloud app deploy --no-promote
To access to it go to:
http://VERSION_ID.default.YOUR_PROJECT_ID.appspot.com
If everything turned out great, you can migrate the traffic on the Cloud Console UI selecting the version you just deployed and clicking "Migrate traffic".
# [START app_yaml]
runtime: nodejs
env: flex
service: 'frontend'
env_variables:
DEPLOY_ENV: 'PRODUCTION'
handlers:
- url: /.*
script: IGNORED
secure: always
resources:
cpu: 2
memory_gb: 4
# [END app_yaml]
Autoscaling is working but continuously 502 coming on Google Flexible Environment (NodeJs). See logs, response time is only 0, 1 and sometimes it's more than that. Any help would be very appreciated.
Are all your requests getting 502s or only some of them? Did you have it working before? What changes did you make?
If all your requests are getting 502s and it's never worked before, then it's likely you don't have it setup properly. Please make sure your app is listening to port 8080 and it should work.
If it worked before and it's continue to work now without any code change, then any transient issue should be investigated on our end. Please file a issue on our Public Issue Tracker, thanks.
Using Google App Engine, I have an application myapp as a default service that adds a task to a task queue and launch a background service worker called optimize. Although myapp is running fine, unfortunately I always see a POST 404 error in the myapp log when the AppEngine task queue tries to launch the URL /optimize-dot-myapp.appspot.com/index.php/optimize. Of course because of the 404 error the task queue keeps retrying. My current optimize.yaml file contains the following. Any thoughts?
# optimize.yaml configuration for Google App Engine
# Full details at: https://cloud.google.com/appengine/docs/php/config/appref
runtime: php55
api_version: 1
service: optimize
handlers:
# Serve php scripts.
- url: /index.php/optimize
script: index.php/optimize
The default app.yaml file contains the following:
# app.yaml configuration for Google App Engine
# Full details at: https://cloud.google.com/appengine/docs/php/config/appref
runtime: php55
api_version: 1
handlers:
# Serve php scripts.
- url: /(.+\.php).*
script: \1
- url: /
script: index.php
# All URLs beginning with /assets are treated as paths to
# static files in the assets/ directory.
- url: /assets
static_dir: assets
In case it's useful, the optimize worker is started in the default task queue with the following PHP:
// Start the background worker
// API details: https://cloud.google.com/appengine/docs/php/refdocs/classes/google.appengine.api.taskqueue.PushTask
$url = '/optimize-dot-myapp.appspot.com/index.php/optimize';
$task = new PushTask($url, $param);
$task_name = $task->add();
You have collisions in the URL path patterns. The /index.php/optimize path matches both the /index.php/optimize url pattern from optimize.yaml and the /(.+\.php).* pattern from app.yaml. Probably the request ends up in the default service instead of the optimize one. Easy to confirm: check the app logs, you can select a specific service and you'll see which service got the request.
I would add a dispatch.yaml file to clarify things and eliminate the possibility of ambiguous routing (no need to specify the default module, anything not matching dispatch rules is sent to the default module):
application: my_app
dispatch:
- url: "*/optimize/*"
module: optimize
Then adjust the url patterns accordingly in optimize.yaml (they should all start with /optimize):
- url: /optimize/index.php
script: index.php
Note: the index.php file mentioned above would be in the optimize service dir, not in the default service one. Assuming here that each service has its own dir, as mentioned in Can a default service/module in a Google App Engine app be a sibling of a non-default one in terms of folder structure?
And in the task enqueueing code the url should only contain the request path, not the hostname (which is interpreted as part of the path, thus causing a mismatch with the handler's url pattern). You want:
$url = '/optimize/index.php';
In Google App Engine, applications define task queues in a configuration file called queue.yaml. You can use queue.yaml to configure both push queues and pull queues.
The following a basic example that defines a named queue and overrides the default processing rate:
queue:
- name: my-push-queue
rate: 1/s
The following is a more complex example of a queue.yaml configuration that demonstrates setting up task retries and modifying the default processing rate.
queue:
- name: fooqueue
rate: 1/s
retry_parameters:
task_retry_limit: 7
task_age_limit: 2d
- name: barqueue
rate: 1/s
retry_parameters:
min_backoff_seconds: 10
max_backoff_seconds: 200
max_doublings: 0
- name: bazqueue
rate: 1/s
retry_parameters:
min_backoff_seconds: 10
max_backoff_seconds: 200
max_doublings: 3
Please read this documentation for further details
I am following the Quickstart for Cloud Endpoints Frameworks on App Engine in standard environment. I have deployed the sample API. When I open https://[my-project].appspot.com/ I get the error message:
Error: Not Found. The Requested URL / was not found on this server
The logs show the message:
No Handlers matched this url
The app.yaml handlers are the what came with the endpoints-frameworks-v2/echo sample:
handlers:
# The endpoints handler must be mapped to /_ah/api.
- url: /_ah/api/.*
script: main.api
I was having great difficulty generating the OpenAPI configuration file in a previous step of the quickstart. I got it to work by updating the system variable path for the SDK but I did get this error:
No handlers could be found for logger "endpoints.apiserving"
WARNING:root:Method echo.echo_path_parameter specifies path parameters buy you are
not using a ResourceContainer. This will fail in future releases; please
switch to using ResourceContainer as soon as possible.
I have no idea if this error is relavant to the current problem.
Any help would be much appreciated.
Regarding the "No handlers could be found for logger..." you need to do this:
http://excid3.com/blog/no-handlers-could-be-found-for-logger
The other issue is a known issue:
What are ResourceContainers and how to use them for Cloud Endpoints?
You need a url handler for / if that is a valid url:
handlers:
# The endpoints handler must be mapped to /_ah/api.
- url: /_ah/api/.*
script: main.api
- url: /.* # catchall for all other urls
script: main.api # or wherever you handle the request for `/` and others
I'm refactoring an existing codebase. I switched from using the appcfg.py to using the gcloud command, which seemed to go fine. Our entire codebase was running on one default frontend instance, which I'm now trying to break into services. To start, I created one "worker" backend service, and I'm using a cron job to test.
I can see the worker in the console, but no instance is started. The logs for that service are rapidly flooded with 404's to /_ah/start. I've tried manual and basic scaling. The documentation states that it's okay not to have a startup script, and that a 404 at that endpoint is considered success. However, the instance is not starting.
Logs
worker.yaml
service: worker
runtime: python27
api_version: 1
instance_class: B2
manual_scaling:
instances: 1
threadsafe: false
handlers:
- url: /work/.*
script: worker.app
secure: always
login: admin
worker.py
import webapp2
import handlers
config = {
#...
}
app = webapp2.WSGIApplication([
webapp2.Route(
'/work/test<:/?>',
handlers.Test,
methods=['GET'],
),
], debug=True, config=config)
dispatch.yaml
dispatch:
- url: "*/work/*"
module: worker