In the browser cloud console for my Google App Engine app, I can choose to see the logs for /dev/log and stderr which gives me all of the log entries that I'm expecting to see.
However, when I use the command line gcloud app logs tail to stream the logs in my terminal, I can't get it to give me the /dev/log entries.
The docs say the default list of logs include: stderr,stdout,crash.log,nginx.request,request_log
So the /dev/log must be represented by some other identifier, but I can't find any docs on what it might be. I've tried a few guesses, but none work.
How can I can the terminal to stream the same logs I'm getting in my browser?
You can use command like gcloud logging read to interact with Stackdriver logging & get a non streamed version of those logs. Setup the Stackdriver GUI with the logs you wish to see, then convert to an advanced filter. You can then paste the advanced filter as is, in quotations after gcloud logging read. Examples in gcloud logging read documentation. I will get back to you in a comment on this post as to whether you can get the /dev/log logs with the gcloud app logs tail command. I will update on Saturday
Related
I run my application in AppEngine (Java).
I want to use gcloud app logs command to view my application logs:
gcloud app logs tail
However, the output is just few data as below:
....
2019-10-04 12:46:43 default[api-v1-0-0] "GET / HTTP/1.1" 200
2019-10-04 12:46:43 default[api-v1-0-0] "GET / HTTP/1.1" 200
....
Actually, I expect gcloud app logs will show logs that my application printed to stdout and stderr. However, it does not.
My question is: How to get/view application stdout/stderr logs by using gcloud command?
From gcloud app logs tail:
DESCRIPTION
Streams logs for App Engine apps.
The command is designed as a blocking call primarily watching for the app logs being generated after the command is invoked (though at startup it may also display a few historical logs, generated before the command is issued).
To focus primarily on the historical logs you want to use the gcloud app logs read command:
DESCRIPTION
Display the latest log entries from stdout, stderr and crash log for
the current Google App Engine app in a human readable format.
Note that the presence of the stdout/stderr in the logs depends on the environment and runtime your app uses. For example in the 1st generation standard environment (at least for the python runtime) the messages printed directly to stdout/stderr do not show up in the logs, one has to specifically use the python logging calls.
I suspect the same might be true for Java, as I see stdout/stderr mentioned in the logging docs for Java11 (and flex environment), but not in the Java8 one.
I also could not use
> gcloud app logs tail
but i found out you can use:
gcloud logging logs list
This will return a list of log names like:
NAME
projects/intan1/logs/appengine.googleapis.com%2Frequest_log
projects/intan1/logs/cloudaudit.googleapis.com%2Factivity
Then you can call:
gcloud beta logging read logName=projects/intan1/logs/appengine.googleapis.com%2Frequest_log
This will return a log, with a lot of extra info, but maybe this can help you!
I created a DotNet Core App
Deployed it on the Google App Engine (Custom / Flex)
I opened the logging tab
I noticed the following entry: _ah/vm_health
It is not there once or twice, it is there very large number of times
Questions:
How can I exclude this one from the logs, I know the system is checking if everything is healthy, and this is good, I just don’t want it logged.
How can I exclude anything from the logs? For example, there is an entry, and it is sending things to the logs, and wanted ignored by the log system.
You can't disable logging of the health checks: they're still requests hitting your app and they're logged like any other request.
In the StackDriver Logs Viewer you might be able to use the Advanced Logs Filters to filter out and prevent displaying of the undesired logs. I can't give an actual example, though, as I didn't yet use this facility. Just to be clear - this just prevents displaying the logs when the filter is applied, the logs are not ignored by the logging system.
I like that I can use the Logs API (described here: https://cloud.google.com/appengine/docs/java/logs/) to programatically access and display app & request logs as I see fit--it's great.
Now that I'm using Managed VMs on AppEngine you can see on the Admin Console Logs Viewer that there are a ton of additional logs--including in my case a custom log which I found I could include in the viewer (decribed here: https://cloud.google.com/appengine/docs/managed-vms/custom-runtimes#logging).
My question is: Is there any way I can use the Logs API (or other pipelines already built?) to access these logs? My Managed VM module includes several components which could produce logs that I want to view:
App logs -- I can get these! No problem here.
Custom log files created by background processes I kick off in _ah/start (like "my_custom_1.log" in the screenshot)
STDERR & STDOUT from my background processes
Relevant Managed VM logs (e.g. for when an instance was restarted due to bad health... other system events like normal restarts?)
Basically I want "the total picture" at the instance level. Anyone tried to tame Managed VMs in this way with success? I'm not looking forward to rolling my own solution. And I wouldn't even know where to start on the problem of capturing STDERR and STDOUT. Any help appreciated.
There is a difference between App Engine logging and Google Cloud logging. Some of the Managed VM logs go to both, but much of it only goes to cloud logging.
Until recently there was not an API to read Cloud logs, only to write them. However, there is a new v2 beta API: https://cloud.google.com/logging/docs/api/introduction_v2
To do things at an instance level, entries in Cloud logging should have metadata set to denote which VM they came from. Both of these values seem to vary on logs from my VMs:
compute.googleapis.com/resource_name
compute.googleapis.com/resource_id
I am running a Java Microengine on GAE.
I have my own html pages for data input and output. When there is a data error and the engine cannot complete its execution (crashes) - the microengine spits out the "Response" as Server not available, please try later.
In order to debug, I run the dataset in the dev environment - as a Java application to identify the error in the console output.
Is there a way to capture the "error" (the console output equivalent when run as a Java application) - as an output string and send it as a content of the servlet response from the deployed Application in GAE..
thanks,
assuming you are using App Engine Managed VM, and a logging framework, you should also forward the log entry into a file, e.g. /var/log/app_engine/custom_logs/app.log
https://cloud.google.com/appengine/docs/managed-vms/custom-runtimes#logging
Subsequently, you'll be able to read the output from Google Cloud Logging.
I know you can download the raw access logs with appcfg.py, but I'm really interested in all the information around a specific request like python logging statements, exceptions and api statistics (just like the online log viewer). Does anyone know if there is a way to get that information another way then having to build it yourself?
If case anyone is wondering, we want to do some continuos statistical analyzing for problems and displaying them on a large screen on a wall in the office.
Sure - just pass the --severity flag to appcfg.py:
$ appcfg.py help request_logs
Usage: appcfg.py [options] request_logs <directory> <output_file>
Write request logs in Apache common log format.
The 'request_logs' command exports the request logs from your application
to a file. It will write Apache common log format records ordered
chronologically. If output file is '-' stdout will be written.
Options:
-h, --help Show the help message and exit.
-q, --quiet Print errors only.
-v, --verbose Print info level logs.
--noisy Print all logs.
-s SERVER, --server=SERVER
The server to connect to.
--insecure Use HTTP when communicating with the server.
-e EMAIL, --email=EMAIL
The username to use. Will prompt if omitted.
-H HOST, --host=HOST Overrides the Host header sent with all RPCs.
--no_cookies Do not save authentication cookies to local disk.
--passin Read the login password from stdin.
-A APP_ID, --application=APP_ID
Override application from app.yaml file.
-V VERSION, --version=VERSION
Override (major) version from app.yaml file.
-n NUM_DAYS, --num_days=NUM_DAYS
Number of days worth of log data to get. The cut-off
point is midnight UTC. Use 0 to get all available
logs. Default is 1, unless --append is also given;
then the default is 0.
-a, --append Append to existing file.
--severity=SEVERITY Severity of app-level log messages to get. The range
is 0 (DEBUG) through 4 (CRITICAL). If omitted, only
request logs are returned.
--vhost=VHOST The virtual host of log messages to get. If omitted,
all log messages are returned.
--include_vhost Include virtual host in log messages.
--end_date=END_DATE End date (as YYYY-MM-DD) of period for log data.
Defaults to today.
This is what works for us really well:
appcfg.py --append --num_days=0 --include_all request_logs /path/to/your/app/ /var/log/gae/yourapp.log
Anyway, the line above will get all your log records and append them to a log file if you've executed this before, if not, it will create a new log file. It actually looks at your existing log (if it's there) and it will not get any duplicates. You can run this without --append if you want, but use it if you are automating log downloads.
The key here is the --include_allflag which seems to be undocumented. This flag will get all the data that you see if you use GAE's web log viewer. So, you will get fields such as: ms=71 cpu_ms=32 api_cpu_ms=12 cpm_usd=0.000921... etc.
OK, I hope that helps someone.
BTW, we wrote up a blog post on this, check it out here.
I seem to be running into 100M limit with appcfg. I ended up using logservice API to get the logs
Here's the code - https://github.com/manasg/gae-log-fetcher
Here is a way to access raw logs so you can further processing without custom parsing (also for me request_logs is not downloading all the data for specified time frame).
Here is an app which runs in the appengine itself:
https://gaelogapp.appspot.com/
You can easily add this functionality to your app with updates to app.yaml and copy logs.py:
https://github.com/okigan/gaelogapp