I have an issue with our Python 2.7 Google App Engine project where after a lot of research it appears as though App Engine is repeatedly sending through the same request to our servers. Is anyone else experiencing this issue?
EDIT:
It's a standard request with all the same parameters as it was first sent with (some of which are 15 days old now). I'm wandering if it's a similar situation to this https://groups.google.com/forum/#!topic/google-appengine-downtime-notify/9fAYP7UyppQ
Typically App Engine doesn't send requests, it's the framework you use to handle requests.
If you're getting duplicate requests coming in, you should look to see where it's coming from. If it's coming from an app engine server to your own external server, then somebody is running some sort of process on their app engine account that's accessing data from your server during its execution.
The details, including what you would do about it, are all situation-dependent. There's no catch-all response. Just be aware that, yes, any computer on the Internet can make web requests to any other. And that includes app engine (in both directions).
Related
I have been working with Google Cloud Platform & Flask for the first time - my client wanted me to deliver a solution on it in 2 weeks.
I have been successful so far in creating a Flask application and wanted to productionize it through Google App Engine. However, the runtime of my function is a little over 2.5 minutes and I get a "504 Gateway Timeout" error. In the code piece, I am accessing bigQuery, google spreadsheets and GCS buckets. Should I switch to a new GCP service or can some tweaks in my code/yaml file suffice? My yaml config is -
runtime: python37
liveness_check:
check_interval_sec: 300
timeout_sec: 299
failure_threshold: 10
success_threshold: 10
initial_delay_sec: 500
readiness_check:
app_start_timeout_sec: 1800
I would be very, very grateful to anyone who can help me resolve this issue.
Thank you!
Edit: Just to give a brief about the application - this is a forecasting application that reads data from bigquery, GCS buckets, and Google Spreadsheets, processes it and runs ML models on it. The results are written back to Google Spreadsheets within the application itself (i.e. no response needed from the application per see). I'm triggering the application using google AppsScript.
The best option is to restructure how your website works. You can then stick with GAE standard. Even if you switch to GAE flexible, it will work better this way.
Here is the sequence of operations:
A user submits a request from web page.
Your website returns immediately with a page that indicates that you are working on it.
The page you return includes Javascript that will poll your website to ask if the task is complete.
When the task is complete, the Javascript will update the page and present the results to the user.
On the server side, you can use cloud tasks for doing the processing which I believe has a time limit of 10 minutes.
This way, the user is always viewing a web page and isn't sitting there waiting and looking at a blank screen while waiting for the server to return something.
I open a new project on Google Cloud, and I start a default Standard Env App Engine. I don't do anything else, and I start to get mysterious non-zero traffic to my project.
I specifically keep getting a alternating 0.033/s and 0.017/s request per seconds on the project. This is not specific to this project but to ALL my google cloud projects. Also I get 100% errors rate. I trying to investigate but I didn't even know where to start.
Where can I get more information on what this APIs requests are specifically? And why this happens?
I followed this guide: Quickstart for Python. After launching the "hello, world" app to app engine (flex) I went to [project].appspot.com and noticed that it is very slow. I tried testing it in different devices and network conditions and I still have the same issue. I went to Cloud Trace and can't build a report due to a lack of traces. It is also slow in both http and https. I deployed to us-central and I am in Texas.
I have attached some logs from Logging and a snippet from Google Chrome's Dev Tools to show the slowness.
Logs from Logging:
Chrome Dev Tools:
The images don’t show anything especially unexpected. Response time will vary depending on the location of the client and its distance to the region of the App Engine Flex instance. Responses that take an especially long time are likely due to a cold boot.
You probably use a free instance of app engine. Because it's free the lifespan is very short, therefor it shuts down after a short amount of time without requests. When you send a new request after some time, the instance has to set up and then process the request, which takes time. You can keep pinging the app to keep the instance alive. Similiar question is anwered here.
I have had a Python-based Google App Engine app working great using Cloud Endpoints 1.0 for several years without incident. I have had nothing but trouble migrating to Cloud Endpoints 2.0.
Currently I'm in the following state after already clearing many previous hurdles described in other similar questions:
I have one version of my service called gce1 which uses Endpoints 1.0 and is set as the default service receiving 100% of my traffic. I can point API clients and the APIs Explorer to both gce1-dot-myservice.appspot.com and the default myservice.appspot.com and everything works fine. I can verify in the logs that anything that goes through here is using GCE 1.0.
I have a second version of my service called gce2 which is not receiving any traffic by default, but if I point an API client or the APIs Explorer to gce2-dot-myservice.appspot.com it works just fine, and I can verify in the logs that anything that goes through here is using GCE 2.0.
Great, right? So it would seem that all I need to do is migrate all my traffic to gce2 and I'm done.
But... when I do that everything breaks! The default myservice.appspot.com serves up 405 POST Method Not Allowed responses to my existing clients, and if I look at the APIs Explorer, suddenly it now shows a bunch of obsolete methods that I think are from years ago and are no longer used in my current API. I can't tell where those are coming from (they are nowhere in my code, and haven't been for years), and I can't get the default service to serve the GCE 2.0 API no matter what I do.
The biggest problem is that I have thousands of users in the wild that all point to the default API URL, so it isn't so easy to just have them start pointing to gce2-dot-myservice, and besides, it doesn't make sense that I can't make the new default the new default. I've been working on this migration to GCE 2.0 for months, the deadline for getting off GCE 1.0 is getting closer by the day, and Google Support has not responded since late last year on this topic.
I should also mention I have tried:
Pushing a new service with the GCE2.0 code directly to default
Pushing a new service with no API at all (to maybe clear a cache or something)
Pushing services with all different sorts of version names
None of these have worked, although I haven't done any of them allowing a long delay since I'm working on a live service with real users.
This issue is now resolved, so for most people it should no longer occur. However, in my specific case, I had a legacy API that was getting in the way and had to be deleted, which did require specific attention from a Google engineer.
If you have similar issues, visit issuetracker.google.com/issues/76031966 and comment there.
Thanks to #saiyr for help tracking this down.
My Google App Engine application returns 503 codes to clients when they should try later to receive data from the server. This appears to cause app engine to think the application is failing and after several such responses the instance is restarted, adding to average latency. Is there a way to prevent app engine from restarting instances just because you manually return 503 or other non-200 http response codes? TIA!
Edit 1:
Here's a screenshot of how it typically goes (with some app-specific stuff ablated due to the sensitive nature of my app). Note that all the [I]nformational and [D]ebug messages are generated by my code, while the [W]arning about the restart is obviously GAE itself. The only thing that distinguishes the times this happens is when I return a 503.
App Engine doesn't base decisions to terminate or restart instances based on the status codes you return, or on what you log.
It seems App Engine now does base instancing decisions on your return code. To the best of my knowledge, there's no way to return a 5xx code, but tell App Engine that nothing's really wrong.