My app is deployed on the python 3.7 Google App Engine standard environment. It has a bug that causes it to silently fail when a certain library function gets called. I'd like to set some log points inside the library function, but I'm not seeing a way to do that. Is it possible to set log points inside external dependencies using Stackdriver? If so, how?
You can use Stackdriver debugger. It allows you to inspect the state of an application, at any code location, without stopping or slowing down the running app and you can configure your Python application to use Stackdriver Debugger.
Also Stackdriver Trace is a distributed tracing system that collects latency data from your applications and displays it in the Google Cloud Platform Console.
Related
My python application on the Google App Engine (standard environment) writes some logs to stdout.
I used to be able to see them on the Logs Viewer but not anymore.
The default options under the "Log name" drop down menu do not show stdout, and I have tried to manually add logName="projects/airlib-main/logs/stdout" into the Query Builder but it doesn't find anything.
I solved my problem by NOT using basicConfig to configure the logs and rather instantiate my own loggers and use them throughout the application. The python documentation says that < This function does nothing if the root logger already has handlers configured > and maybe the Google App Engine does that now. My application used to produce logs properly so there might have been a change recently in the way the Google App Engine works in this environment.
I would recommend you to take a look at the post from the Community:
How to group related request log entries GAE python 3.7 standard env. This post provides you with possible solutions to your stdout logging.
For example, as mentioned on this answer here - that it's in Python language - there are some methods that you can use to set the stdout as the pattern for logs. So, checking this post should assist you in configuring the use of stdout on your App Engine.
Let me know if the information helped you!
I open a new project on Google Cloud, and I start a default Standard Env App Engine. I don't do anything else, and I start to get mysterious non-zero traffic to my project.
I specifically keep getting a alternating 0.033/s and 0.017/s request per seconds on the project. This is not specific to this project but to ALL my google cloud projects. Also I get 100% errors rate. I trying to investigate but I didn't even know where to start.
Where can I get more information on what this APIs requests are specifically? And why this happens?
So it seems from a few SO questions I've seen that this is a problem among other users. Recently one of our head dev's left and I inherited a lot of his projects. One of which, is a website that what seems like lives on an app engine from google cloud platforms. From the App Engine documentation, to download source code you use the appcfg.py download_app command. Which I did, however the only results I get back from that call is:
Fetching file list...
Fetching files...
And then it just ends. No error message or any kind of message at all, and of course, it did not download the source code into the output dir I specified.
Scratching my head and looking at various SO posts, someone mentioned something about going into the google cloud vm directly and doing the same command, and to my surprise finding the same exact behavior that I did in my local terminal.
This made me realize it must be something else at play. I took a look at my versions tab in the App Engine dashboard on GCP. I see my instance running, it correctly says Serving and if I click the link it brings me to the website which loads fine. However, under Size it says 0 B which made me think perhaps this is why the download_app isn't downloading anything, because the version is 0 B?
What I'm trying to figure out is why it says 0 B for the version, when clearly the site runs fine and how I can get the source code for this. Here's a screenshot for reference
And screenshot of my terminal (local). Obviously I omitted the -A and -V flags, but they are correctly set and if I purposely make them incorrect I do indeed get an error message.
EDIT
Just so everyone is aware, I also made sure my user had the correct permissions. Owner, App Engine Owner... and some others. I don't think that's the problem.
When you deploy an App Engine Flexible application, the source code is uploaded to Cloud Storage on your project in a bucket named staging.<project-id>.appspot.com. You can navigate in this bucket and download the source code for a specific version as a .tar file.
Alternatively, you can find the exact Cloud Storage URL for your source code by going to Dev Console > Container Registry > Build History and select the build for your version. You'll find the link to your source code under Build Information.
One thing to note however is that the staging... bucket is created by default with a Lifecycle rule that deletes files older than 15 days automatically. You can delete this rule if you want so that all versions' source code is kept indefinitely.
In your case I believe that may not have helped since files may have been deleted already but it's worth knowing you can get the source code from there (source code isn't pushed to Source Repository by default, your developer had to configure it manually).
Posting this since none of the listed methods on the web didn't take me to the code (by June 2021)
Note: appcfg.py is deprecated by Google
You could try accessing your source code through;
Google Cloud Platform > Debugger > choosing the version of the
Application from combo at top.
This will list the files of that version on the left pane. There is no way to download code automatically but you can copy-paste the code.
Advice: Push your code to a Git repository to avoid this hassle next time.
Hope you will find this helpful.
In the developer console you can select the respective project and check:
on the Services page - which services, AKA modules - as they used to be (and still are) called in various places, you app has deployed
on the Versions page - which versions for each of the services are deployed
This information is what appcfg.py download_app expects. See also:
the various appcfg.py options using its --help flag
How do I download a specific service's source code off of AppEngine?
You can also access the deployed source code live (if everything else fails it could still be a last resort method to get the code, but tedious), see my answer to Google Cloud DataStore automatic indexing
Update:
I just now noticed in your screenshot that it's a flexible environment app. The appcfg.py docs are in the standard environment section, I suspect it's not applicable to the flexible environment, for which what's deployed is actually a docker image built during the deployment operation. From Deploying your application:
Deploy your app to App Engine using the gcloud app deploy
command. This command automatically builds a container image by using
the Container Builder service and then deploys that image to the
App Engine flexible environment. The container will include any local
modifications that you've made to the runtime image.
It might be possible to access the code on the actual GCE instance running the app, by connecting to the running instance and starting a shell in your app container, see Connecting to the instance
It seems to me that LiteIDE is a good environment to get started with Go...
But in order to debug, it's asking me for the following:
What should I set these to if I'm developing a Google App Engine app?
I need the ability to detect and give hints to compile-time errors in the IDE. That should also be the case when using GAE-specific APIs (Channel API, etc.)
Though for runtime errors, I would assume that those are better left to visiting the dev site.
In other words- the way Google App Engine does things if I understand correctly, is you launch dev_appserver.py and then you don't actually recompile/relaunch again, the dev server does all of that and you simply visit http://localhost:8080 to test it, and it will give runtime errors descriptively in the browser.
I'm currently building a live web application based upon the PubSubHubBub protocol. However, I encountered several issues.
First, I'm in search of a hub application that I can run on my server. There are several applications, but most of them are not mature yet, or they don't support the 0.3 spec. The official google hub runs on the Google App Engine and can even be executed locally. Unfortunately, "Tasks will not run automatically. Push the 'Run' button to execute each task." This behaviour is useful for debugging and understanding the workflow, but in some live tests, it would be nice not to invoke all tasks manually. Is there a way to tweak the local app engine due automatically run tasks?
Next, I have a question concerning the spec itself. The Google reference implementation provides the initial publish method bound to the outpoint uri + /publish. But this is not reflected in the specs.
So are there any mature hubs that can be run locally for debugging? Or are there ways to configure the offical google app engine hub to run locally and to execute tasks directly?
Thanks in advance
The new 1.3.4 App Engine SDK automatically runs tasks. Just upgrade your SDK to take advantage of it.
Next, I have a question concerning the spec itself. The Google reference implementation provides the initial publish method bound to the outpoint uri + /publish. But this is not reflected in the specs.
/publish is the only endpoint provided by the hub. As per the discovery section of the spec, that is the one you advertise.
Btw, https://github.com/pubsubhubbub/PubSubHubbub/wiki/Hubs lists all currently known hubs and software.
I also made my own hub that implements the 0.4 spec; see https://github.com/cweiske/phubb