I try to download the logs from my Google App Engines with the command:
appcfg.py request_logs [directory] mylogs.txt
However, I encountered the server error as below:
C:\Program Files (x86)\Google\google_appengine>appcfg.py request_logs E:\PhD\cod
e\python\webDevelop\similarAPI\similarAPI/ mylogs.txt
05:22 PM Application: similarapi
05:22 PM Host: appengine.google.com
05:22 PM Downloading request logs for app similarapi version 1.
Error 500: --- begin server output ---
Server Error (500)
A server error has occurred.
--- end server output ---
I have looked at the official document and related Stack Overflow questions, but no clues are found. Actually I can get the logs in Dec, 2016 but not now. And I tried it for one week without the success, so it should not be the error of Google's infrastructure.
Does anyone know how to solve that problem? Thanks in advance.
Just a workaround, not a direct answer.
The GAE SDK might be getting a bit less attention compared to the cloud SDK (which is the recommended one lately). I, for one, can't even get as far as you got since appcfg.py request_logs can't properly detect my multi-service app from its directory and was not happy with any syntax attempt I made to specify it via args.
But I have the cloud SDK installed as well and I was able to get the logs for my app using gcloud app logs read (after setting up the gcloud context for my app)
FWIW, this isn't yet enough to make me switch, I'm still primarily a GAE SDK user. YMMV.
Finally, it has be proved that it is a bug of Google's infrastructure from 22 Mar to 10 Apr, 2017. Three related bug reports can be seen in the Google Issue Tracker 36637246, 37192011, 36969442.
Now the command appcfg.py request_logs [directory] mylogs.txt works well.
Related
After some time I needed to do changes on GAE Python (2.7) First Generation app, but when I develop and run locally, it's basically nonusable due to a very slow start.
These are printed logs:
INFO 2019-10-18 07:56:35,533 devappserver2.py:278] Skipping SDK update check.
INFO 2019-10-18 07:56:35,595 api_server.py:275] Starting API server at: http://localhost:36159
INFO 2019-10-18 07:56:35,599 dispatcher.py:270] Starting module "default" running at: http://127.0.0.1:8080
INFO 2019-10-18 07:56:35,600 admin_server.py:152] Starting admin server at: http://localhost:8000
INFO 2019-10-18 08:01:01,644 instance.py:294] Instance PID: 28496
What I notice that the last line (instance.py) is printed after ~5 minutes and only after that app responds to requests, not before. Interesting that the admin server (localhost:8000) is available right away. Of course, when I do some code change it automatically reloads and it repeats again.
Things I tried/found out:
it behaves like that on my all GAE projects Python First Gen.
tried to create a bare minimal version (webapp2 with one URL), clean virtualenv, still the same behavior
tried to reinstall Google Cloud SDK. delete the whole google-cloud-sdk folder and install again, no changes
tried to install older version of Cloud SDK
used clean VM and it works ok!!!, so it looks like there could be something wrong with my system (outside of SDK), but I'm not sure what.
It's interesting that the pause between the last two log lines is always about 5 minutes, not sure why exactly that time.
Python 2.7.14
OS: OpenSuse Leap 15.0
I'm running out of ideas so any advice would be appreciated.
I solved this accidentally.
I wanted to run Jupyter notebook, but I got the error:
error: [Errno 99] Cannot assign requested address
after debugging in /tornado/netutil.py, I noticed that it tries to work with IP 192.168.1.50 which I wasn't sure where did that come from, (probably I set it since I was playing with my home network some time ago), but when I deleted it from /etc/hosts, Jupyter, as well as GAE, works ok.
What a coincidence :)
not sure if Google App Engine changed something, but I was able to download logs until recently to parse the logs for generating custom analytics on usage etc. Now the log is getting exported but the issue is that the last log is from Nov 23 while I see logs as recent as 12/18 in the console. The commandline version I use is as below.
appcfg.cmd -n 90 --severity=1 request_logs <appDirPath> <full_log_file_path>
Has anything changed recently with logs export with GAE ? Does anyone know a workaround for this issue. If it matters, I am using appengine-java-sdk-1.9.64.
I finally figured out the issue. Posting here in case anyone else runs into similar silly issue. After upgrading to google-cloud-sdk and Java 8, I am getting the following log in console
Configuration Warning : <application>/<version> XML elements and --application/--version should not be specified when staging
The following parameters will be scrubbed from app.yaml
application : appId
version : chosenVersion
Since google app engine was ignoring this, it was creating a new version with dateTimeStamp with new deployment. But the command line in my post points to the appDirPath that has the appengine-web.xml listing a version and so it was downloading the logs for that version only. Since that version was overridden with a new dynamic version on Nov 24, the command line was giving only logs till Nov 23. It took me a while to figure out so posting for anyone else who run into this issue.
Currently the GAE Managed VMs feature is broken for any project with a name longer than 27 characters.
The underlying issue is that Docker restricts image namespace to between 4-30 chars. This has been fixed (https://github.com/docker/docker/issues/10392) but is still awaiting a release at time of writing.
It seems when deploying a Managed VM to GAE that the namespace is automatically generated from your project name plus _m_ prefix. This leads to error when attempting to deploy the vm:
DEBUG: "POST /v1.10/images/gcr.io/_m_<my project name>/<my project name>.default.20150330t140211/push HTTP/1.1" 500 111
INFO: Exception 500 Server Error: Internal Server Error ("Invalid namespace name (_m_<my project name>). Cannot be fewer than 4 or more than 30 characters.") thrown in ProgressHandler. Retrying.
The obvious solution would be for GAE gcloud tools to respect the underlying limit via some auto-truncation or hashing scheme.
Does anyone know a way around this? Or I have to wait for Google to fix or Docker to release a new version and Google to update?
We're aware of the issue and we're working on a long-term fix. For now, you can switch to an old version of gcloud. You can do this by setting this variable to point to an old version (0.9.51):
gcloud config set --scope=installation component_manager/fixed_sdk_version 0.9.51
then run "gcloud components update"
Then run "gcloud config set app/hosted_registry false"
and you should be able to deploy. I'll update this answer when we've fixed the naming issue.
UPDATE:
The naming issue has been fixed as of this week's release (0.9.57).
I'm attempting to use appcfg.py migrate_traffic, but I get a 500 error when using this with a module. The documentation states that a module can be specified:
The 'migrate_traffic' command gradually gradually sends an increasing fraction
of traffic your app's traffic from the current default version to another
version. Once all traffic has been migrated, the new version is set as the
default version.
app.yaml specifies the target application, version, and (optionally) module; use
the --application, --version and --module flags to override these values.
Can be thought of as an enhanced version of the 'set_default_version'
command.
If I try this with a module, I get the following error
Error 500: --- begin server output ---
Server Error (500)
A server error has occurred.
--- end server output ---
The source at https://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/tools/appcfg.py?r=396 (MigrateTraffic) doesn't seem to use the module at all. Is this a bug in appcfg.py or a missing feature of app engine?
I filed a ticket with google support. The answer is that modules are not supported with migrate_traffic yet. No ETA on when they might be supported. I think the current version of appcfg.py doesn't mention modules as prominently in the help for migrate_traffic as well.
I saw this link - App engine CPU times when downloading logs that refers to a question about how to get the performance information when downloading logs from google app engine with --include_all option.
I've tried it with the Java command line options and read the documetnation and it is not mentioned there at all!
How can I get the performance infromation such as cpu time when I download the logs from app engine?
The command I'm currently using (it works) is:
appcfg.cmd --num_days=3 --severity=0 request_logs . logs.txt
In the admin dashboard you can see this information:
"<my_app_name>.appspot.com" ms=13 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.000102
I want to be able to get this information in the logs as well.
thanks,
Li
The Java version of appcfg doesn't currently support that flag. You can create an app.yaml and use the Python version of the SDK to download the logs, though.