My python application on the Google App Engine (standard environment) writes some logs to stdout.
I used to be able to see them on the Logs Viewer but not anymore.
The default options under the "Log name" drop down menu do not show stdout, and I have tried to manually add logName="projects/airlib-main/logs/stdout" into the Query Builder but it doesn't find anything.
I solved my problem by NOT using basicConfig to configure the logs and rather instantiate my own loggers and use them throughout the application. The python documentation says that < This function does nothing if the root logger already has handlers configured > and maybe the Google App Engine does that now. My application used to produce logs properly so there might have been a change recently in the way the Google App Engine works in this environment.
I would recommend you to take a look at the post from the Community:
How to group related request log entries GAE python 3.7 standard env. This post provides you with possible solutions to your stdout logging.
For example, as mentioned on this answer here - that it's in Python language - there are some methods that you can use to set the stdout as the pattern for logs. So, checking this post should assist you in configuring the use of stdout on your App Engine.
Let me know if the information helped you!
Related
I've recently been migrating an App Engine (Go) app from Go 1.9 to 1.11. I've followed the steps in the
Migrating your App Engine app from Go 1.9 to Go 1.11 document, excluding the optional ones. I plan on doing the optional tasks later once I get this working.
I get the app to build and I can deploy it just fine. Most of everything works fine, with the exception of one API that is used to look up some info on a Google Play IAP. When I do that, I get the following error:
Error 403: Insufficient Permission: Request had insufficient authentication scopes., insufficientPermissions
I've checked, and the scope that I'm using is https://www.googleapis.com/auth/androidpublisher, which is still the scope listed in the Google Play Developer API documentation.
I'm using the App Engine default service account for the client by calling DefaultClient from the golang.org/x/oauth2/google library, which returns a client without an error
When I test the same code with Go 1.9, there are no authentication issues at all, and the API works. I'm guessing that these is something in the authentication setup which has changed but I can't find any documentation on it, nor on what I should do differently.
I have to imagine that a lot of people have had to do this migration, and I can't find any posts with this problem, so I'm lost as to why I'm getting it.
I think the issue is in the differences of runtime in Go 1.9 and 1.11. It doesn't seem to allow you to use Application Default Credentials anymore, you have to set them via a JSON file in 1.11. I found someone who had a very similar issue to you and they used a workaround by uploading a key and using that to get a Client.
Have a look here
Let me know.
I have NodeJS 10 based Standard code hosted on google-app-engine. Until last update, stdout and stderr logs were being imported automatically to the stackdriver logging. However after last deployment, the stdout/stderr completely stopped appearing in stackdriver. I tried to migrate traffic to the earlier known version but the logs still won't appear. The service appears to be working without any issues, just the logging functionality is gone. Any pointers?
You are correct, there is nothing wrong with your service versioning, the logging functionality for stdout and stderr is deprecated and no longer write any logs to Stackdriver. Instead, you can use the Stackdriver Logging API.
Here's a step-by-step guide to configure the Stackdriver Logging API for Node.js that you can follow, and you should be good to go.
Let me know if this helps!
So it seems from a few SO questions I've seen that this is a problem among other users. Recently one of our head dev's left and I inherited a lot of his projects. One of which, is a website that what seems like lives on an app engine from google cloud platforms. From the App Engine documentation, to download source code you use the appcfg.py download_app command. Which I did, however the only results I get back from that call is:
Fetching file list...
Fetching files...
And then it just ends. No error message or any kind of message at all, and of course, it did not download the source code into the output dir I specified.
Scratching my head and looking at various SO posts, someone mentioned something about going into the google cloud vm directly and doing the same command, and to my surprise finding the same exact behavior that I did in my local terminal.
This made me realize it must be something else at play. I took a look at my versions tab in the App Engine dashboard on GCP. I see my instance running, it correctly says Serving and if I click the link it brings me to the website which loads fine. However, under Size it says 0 B which made me think perhaps this is why the download_app isn't downloading anything, because the version is 0 B?
What I'm trying to figure out is why it says 0 B for the version, when clearly the site runs fine and how I can get the source code for this. Here's a screenshot for reference
And screenshot of my terminal (local). Obviously I omitted the -A and -V flags, but they are correctly set and if I purposely make them incorrect I do indeed get an error message.
EDIT
Just so everyone is aware, I also made sure my user had the correct permissions. Owner, App Engine Owner... and some others. I don't think that's the problem.
When you deploy an App Engine Flexible application, the source code is uploaded to Cloud Storage on your project in a bucket named staging.<project-id>.appspot.com. You can navigate in this bucket and download the source code for a specific version as a .tar file.
Alternatively, you can find the exact Cloud Storage URL for your source code by going to Dev Console > Container Registry > Build History and select the build for your version. You'll find the link to your source code under Build Information.
One thing to note however is that the staging... bucket is created by default with a Lifecycle rule that deletes files older than 15 days automatically. You can delete this rule if you want so that all versions' source code is kept indefinitely.
In your case I believe that may not have helped since files may have been deleted already but it's worth knowing you can get the source code from there (source code isn't pushed to Source Repository by default, your developer had to configure it manually).
Posting this since none of the listed methods on the web didn't take me to the code (by June 2021)
Note: appcfg.py is deprecated by Google
You could try accessing your source code through;
Google Cloud Platform > Debugger > choosing the version of the
Application from combo at top.
This will list the files of that version on the left pane. There is no way to download code automatically but you can copy-paste the code.
Advice: Push your code to a Git repository to avoid this hassle next time.
Hope you will find this helpful.
In the developer console you can select the respective project and check:
on the Services page - which services, AKA modules - as they used to be (and still are) called in various places, you app has deployed
on the Versions page - which versions for each of the services are deployed
This information is what appcfg.py download_app expects. See also:
the various appcfg.py options using its --help flag
How do I download a specific service's source code off of AppEngine?
You can also access the deployed source code live (if everything else fails it could still be a last resort method to get the code, but tedious), see my answer to Google Cloud DataStore automatic indexing
Update:
I just now noticed in your screenshot that it's a flexible environment app. The appcfg.py docs are in the standard environment section, I suspect it's not applicable to the flexible environment, for which what's deployed is actually a docker image built during the deployment operation. From Deploying your application:
Deploy your app to App Engine using the gcloud app deploy
command. This command automatically builds a container image by using
the Container Builder service and then deploys that image to the
App Engine flexible environment. The container will include any local
modifications that you've made to the runtime image.
It might be possible to access the code on the actual GCE instance running the app, by connecting to the running instance and starting a shell in your app container, see Connecting to the instance
I'm working on a project with Google App Engine. I am using continuous integration via Travis, and wish to be able to deploy directly from it. Due to a bug that will not be resolved directly, I can't rely on Travis' built-in GAE deployment, so I basically have to use mvn appengine:update manually. This requires me navigate to a generated URL and manually paste to the terminal an authentication code, which I can't do in automated builds.
It was suggested to me, however, that I do some Unix magic instead. While I can easily pick out the URL I need to navigate to from grep, I still need to log in to Google with my credentials in order to actually get the authentication code (which I can then grep out and pipe to the deployment program).
Given that, how do I log in to Google with my credentials, using only curl or similar command-line utilities?
I've accomplished similar things in the past using Service Accounts. These are likely a good fit for your problem.
Service Accounts will allow you to authenticate and upload your app without manual intervention.
Overview
A Service Account will allow you to do "passwordless" authentication like you may already do with ssh, and git, etc. by setting up your keys. This will remove the requirement that you log in manually, or follow the road to madness by trying to do a "manual" login automatically.
There are basically two steps:
Create your service account and key (with the right permissions)
Use that credential instead of what you're doing now
Resources
I think it's better to give a list of resources than concrete instructions since it's basically impossible to express concisely (even though it's a simple process, there's bound to be a lot of little things that annoy), everyone's requirements will be slightly different, and Google is likely to change the process at some point.
Using the Google Cloud Platform Console for App Engine | Permissions
Using OAuth 2.0 for Server to Server Applications
Setting up OAuth 2.0 | Service Accounts
gcloud auth activate-service-account
Hopefully that's enough to get you headed in the right direction.
Note
You'll likely have to spend some time looking at your .appcfg_oauth2_tokens_java and sorting out a variety of other annoyances, but I believe that this approach is the best way to solve your problem.
It sounds like you have a pretty straight-forward setup and that a Service Account alone will get you there, but if you need to get a little weird, the App Engine Admin API is always there.
I'm trying to implement wowza streaming engine with google cloud using this tutorial.
The problem I am having is that when i try to add the Wowza Streaming Engine image via the developer console it fails saying " Required 'read' permission for 'rawDisk.source' ". I have also tried adding the image via the command-line tool with no success.
From my research on this error, all I have been able to surmise is that maybe this particular image is no no longer publicly shared, but this seem highly unlikely.
So I tried exactly what you said, followed the same tutorial and got the same error. While the tutorial is somewhat out-dated (as certain commands are no longer the same and so on), the image they provide should still be fine. I wasn't able to create my image via the Web UI but it WORKED via CLI.
Here's what you can do as a workaround:
gcloud compute images create [IMAGE_NAME] --project=[YOUR_PROJECT] --source-uri http://storage.googleapis.com/wowzamediasystems/wse/WowzaStreamingEngine-4.1.0.v2.image.tar.gz
As long as you're properly authenticated (for example gcloud auth login) the above command should work. I tested, and was also able to create my instance using the image it creates. SSHed to it without a problem.