I'm trying to add timeouts to GWT sessions, by using the following code to check if a session is alive:
public boolean isSessionAlive() {
return System.currentTimeMillis() - getThreadLocalRequest().getSession()
.getLastAccessedTime() < timeout;
}
I based this code on many examples I saw on web for GWT sessions, such as this.
The above code works great while running on a local web server, but after deploying the project to App Engine it doesn't. The following always returns 0 on App Engine:
getThreadLocalRequest().getSession().getLastAccessedTime()
As far as I understand, the last accessed time is updated on each RPC call.
I made several calls, but this value still remains zero and incorrect result is returned.
Does anybody know how to fix this issue?
Things will change after deployed on GAE
Just today attended the session on app engine by #roman irani .
remember that App Engine is a distributed architecture so a difference from Java EE is that you are never guaranteed the same application server instance during request processing as the previous request. While the object is being serialized correctly in memcache, you still have to call setAttribute() every time due to the fact that memory is not shared.
Clear cut picture here to handle the session
I have found a workaround. Adding the following code in war/WEB-INF/web.xml will cause the session to expire after 30 minutes:
<!-- timeout in minutes -->
<session-config>
<session-timeout>30</session-timeout>
</session-config>
Reference: Session Timeouts with GWT RPC calls.
Related
I successfully deployed a twitter screenshot bot on Google App Engine.
This is my first time deploying.
First thing I noticed was that the app didn't start running until I clicked the link.
When I did, the app worked successfully (replied to tweets with screenshots) as long as the tab was loading and open.
When I closed the tab, the bot stopped working.
Also, in the cloud shell log, I saw:
Handling signal: term
[INFO] Worker exiting (pid 18)
This behaviour surprises me as I expect it to keep running on google server indefinitely.
My bot works by streaming with Twitter api. Also the "worker exiting" line above surprises me.
Here is the relevant code:
def get_stream(set):
global servecount
with requests.get(f"https://api.twitter.com/2/tweets/search/stream?tweet.fields=id,author_id&user.fields=id,username&expansions=author_id,referenced_tweets.id", auth=bearer_oauth, stream=True) as response:
print(response.status_code)
if response.status_code == 429:
print(f"returned code 429, waiting for 60 seconds to try again")
print(response.text)
time.sleep(60)
return
if response.status_code != 200:
raise Exception(
f"Cannot get stream (HTTP {response.status_code}): {response.text}"
)
for response_line in response.iter_lines():
if response_line:
json_response = json.loads(response_line)
print(json.dumps(json_response, indent=4))
if json_response['data']['referenced_tweets'][0]['type'] != "replied_to":
print(f"that was a {json_response['data']['referenced_tweets'][0]['type']} tweet not a reply. Moving on.")
continue
uname = json_response['includes']['users'][0]['username']
tid = json_response['data']['id']
reply_tid = json_response['includes']['tweets'][0]['id']
or_uid = json_response['includes']['tweets'][0]['author_id']
print(uname, tid, reply_tid, or_uid)
followers = api.get_follower_ids(user_id='1509540822815055881')
uid = int(json_response['data']['author_id'])
if uid not in followers:
try:
client.create_tweet(text=f"{uname}, you need to follow me first :)\nPlease follow and retry. \n\n\nIf there is a problem, please speak with my creator, #JoIyke_", in_reply_to_tweet_id=tid, media_ids=[mid])
except:
print("tweet failed")
continue
mid = getmedia(uname, reply_tid)
#try:
client.create_tweet(text=f"{uname}, here is your screenshot: \n\n\nIf there is a problem, please speak with my creator, #JoIyke_", in_reply_to_tweet_id=tid, media_ids=[mid])
#print(f"served {servecount} users with screenshot")
#servecount += 1
#except:
# print("tweet failed")
editlogger()
def main():
servecount, tries = 1, 1
rules = get_rules()
delete = delete_all_rules(rules)
set = set_rules(delete)
while True:
print(f"starting try: {tries}")
get_stream(set)
tries += 1
If this is important, my app.yaml file has only one line:
runtime: python38
and I deployed the app from cloud shell with gcloud app deploy app.yaml
What can I do?
I have searched and can't seem to find a solution. Also, this is my first time deploying an app sucessfully.
Thank you.
Google App Engine works on demand i.e. when it receives an HTTP(s) request.
Neither Warmup requests nor min_instances > 0 will meet your needs. A warmup tries to 'start up' an instance before your requests come in. A min_instance > 0 simply says not to kill the instance but you still need an http request to invoke the service (which is what you did by opening a browser tab and entering your Apps url).
You may ask - since you've 'started up' the instance by opening a browser tab, why doesn't it keep running afterwards? The answer is that every request to a Google App Engine (Standard) app must complete within 1 - 10 minutes (depending on the type of scaling) your App is using (see documentation). For Google App Engine Flexible, the timeout goes up to 60 minutes. This tells you that your service will timeout after at most 10 minutes on GAE standard or 60 minutes on GAE Flexible.
I think the best solution for you on GCP is to use Google Compute Engine (GCE). Spin up a virtual server (pick the lowest configuration so you can stick within the free tier). If you use GCE, it means you spin up a Virtual Machine (VM), deploy your code to it and kick off your code. Your code then runs continuously.
App Engine works on demand, i.e, only will be up if there are requests to the app (this is why when you click on the URL the app works). As well you can set 1 instance to be "running all the time" (min_instances) it will be an anti-pattern for what you want to accomplish and App Engine. Please read How Instances are Managed
Looking at your code you're pulling data every minute from Twitter, so the best option for you is using Cloud Scheduler + Cloud Functions.
Cloud Scheduler will call your Function and it will check if there is data to process, if not the process is terminated. This will help you to save costs because instead of have something running all the time, the function will only work the needed time.
On the other hand I'm not an expert with the Twitter API, but if there is a way that instead of pulling data from Twitter and Twitter calls directly your function it will be better since you can optimize your costs and the function will only run when there is data to process instead of checking every n minutes.
As an advice, first review all the options you have in GCP or the provider you'll use, then choose the best one for your use case. Just selecting one that works with your programming language does not necessarily will work as you expect like in this case.
I have a small service discovery service running on Google App Engine free tier. It queries Google Cloud Datastore with simplest queries on the data that is virtually static. Recently we had incidents that the service was returning empty results. That was going on for 12 hours. After that results came back to normal. We only recently noticed it. In the logs I see at least 3 incidents like that.
I logged into a console and saw:
0 app errors
0 server errors
all green GCE status
max used quota is just 5%
intervals of time where every response is 204
I see absolutely no reason to receive an empty response and yet they are sometimes empty. I see no way to notify Google that there is cloud-side problems, since this is a free tier account with no support.
So, is there anything I might have missed?
UPD: As I look into the code, the only way to get 204+empty body is to have exception in getting an instance of javax.jdo.PersistenceManager or a new instance of javax.jdo.Query. So it is less likely that Cloud Datastore is at fault, because if the result would be empty, app should answer 200+[], if there was an error during a query, app should answer 204+<h1>Exception</h1>....
But again, I don't see how request could work now, but not work 5 minutes ago.
UPD2: The app was stable for more than two years.
I have been struggling with a problem in Google App Engine, using Java, for several days.
Many times (about 50% of the time) when I try to request the connection to a Cloud Sql instance, the connection returns a null value, resulting in several NullPointerException messages when trying to invoke Cloud Sql queries (when invoking .prepareCall(stored_proc)).
I have the latest App Engine Java SDK, in a project service, shared with other services built in Python which consume this Java backend.
Could it be possible that after certain time the instance/s could crash (I am just testing at this point, so I am using default scalation)?
This is the code that returns null:
Class.forName("com.mysql.jdbc.GoogleDriver");
url = "jdbc:google:mysql://project:instance/database?user=root";
log.info(url);
return DriverManager.getConnection(url);
This is part of my configuration file:
<application>app</application>
<module>mod</module>
<version>1</version>
<threadsafe>true</threadsafe>
<use-google-connector-j>true</use-google-connector-j>
I tried several suggestions from other posts, but with no success at all.
Any suggestion will be welcome, thanks in advance.
I was facing the same problem while using Google Cloud SQL and App engine.
I solved the problem by managing the connection pool my self. I realised that when you request a new connection for each request and close it on completion of the thread. The other requests would get back a null resulting to NullPointException.
I decided to do the following and it work for me for like 2 years now.
Open a connection and keep it to a static class that has a number of connections;
Every time i want to find a connection to the database, i would first check if the is an available connection for me to use.
Incase a Query killed the connection, thus means i needed to request another extra connection just to for the sack of connection drops.
I will add this as an answer, since it is not exactly what Chrispinus mentioned, although he gave me a good idea for teh solution.
I went deeper in the code and found that some of the methods were not closing the database connection. I had assumed all of them were doing that, but looking at each method, I found I was wrong.
So, although it sounds obvious, check connections are being closed (or managed, as Chrispinus says) properly.
I'm attempting to create a microservice on Google App Engine that is not intended to handle HTTP requests.
Instead, I was hoping to have a continuously running Python script that monitors a remote queue--RabbitMQ, to be precise--and sends out an api-call to another service as tasks are pushed to the queue.
I was wondering, firstly, is it possible to run a script upon deployment--one that did not originate with a user action/request?
Secondly, how would I accomplish this?
Thanks in advance for your time!
You can deploy your "script" as a manually scaled module -- see https://cloud.google.com/appengine/docs/python/modules/ -- with exactly one instance. As the docs say, "When you start a manual scaling instance, App Engine immediately sends a /_ah/start request to each instance"; so, just set that module's handler for /_ah/start to the handler you want to run (in the module's yaml file and the WSGI app in the Python code, using whatever lightweight framework you like -- webapp2, falcon, flask, bottle, or whatever else... the framework won't be doing much for you in this case save the one-off routing).
Note that the number of free machine hours for manual scaling modules is limited to 8 hours per day (for the smaller, B1 instance class; proportionally fewer for larger instance classes), so you may need to upgrade to paid-app status if you need to run for more than 8 hours.
Like #brant said, App Engine is designed to handle HTTP requests. It's not a perfect fit for background jobs, unless you try to wrap your logic into one http request.
Further, App Engine will emit an error when the response timeout, depending on your scaling settings. If you want to try it, consider basic or manual scaling.
For this type of workload, I would suggest you use a VM.
I think there are a few problems with this design.
First, App Engine is designed to be an HTTP request processor, not a RabbitMQ message processor. GAE is intended for many small requests, not one long-running process.
Second, "RabbitMQ should not be exposed to the public internet, it wasn't created for such use case."
I would recommend that you keep the RabbitMQ clients on the same internal network as the RabbitMQ broker, and have the clients send HTTP requests to App Engine.
I've recently started having coldstarts on nearly every call to my appengine app. Initially I thought this was an issue with Cloud Endpoints, however now I believe it is an appengine issue, or something else in my code.
This started on my most recent deployment. I'm at a loss right now as to what is going on. It has made my app useless. I have tried 1.7.4 and 1.7.5 and both have this problem.
The requests work other than being extremely slow! Any help would be greatly appreciated, as I can not continue with 10-15 request times!
Update: By looking at my running instances I see NO instances running even after making a request. Previously instances would remain running after requests were made. It appears when a request is made an instance is spun up, serves the request, and then dies. There are no errors in my logs. No changes have been made to my app settings or billing. This app does have billing enabled.
Update 2: I have adjusted my idle instance settings(which up to this point have worked and have been left untouched). I set to a min of 1 and max of 2. The instances are staying alive and serving requests as normal. Previously it was set to automatic-1. Not sure what is going on here, perhaps Google adjusting the request scheduler or something. Not COOL!
I have found an open issue on code.google.com. Apparently this is a real appengine issue that is effecting some, if not all apps at the moment. I do not have a solution at this time other than setting the idle minimum instances to 1 (or greater). Even doing this, the new dynamic instances that are spun up die almost immediately after serving a request. Just waiting on a fix from google at this point.
http://code.google.com/p/googleappengine/issues/detail?id=8844