Configuration:
We have iPlanet web server which sits before WebSphere portal 6.1 cluster (2) deployed in Linux machines.
When user tries to copy a 10 GB file across file systems (NFS mounted), we are using java run time to copy the file across to a different NFS mount, hoping that it would be faster than using any other java libraries.
proc = rt.exec("cp " + fileName + " " + outFileName);
Application deployed is a JSF portlet application.
a) session timeout is 60 mins on the app server and the application
b) we have an Ajax call from the client page to keep the session alive
User receives HTTP 500 within 3 minutes, while our logs show that file is still copying. Not sure why WebSphere is sending HTTP 500?
After 10 minutes are so file is copied, and when he clicks on refresh he can proceed.
Not sure what is causing this HTTP 500.
WebContainer threads are not supposed to be used for long tasks.
He's getting 500 after 3 minutes because that is the time WebSphere decides the thread is hung.
What you should be doing is using a WorkManager to perform that long task and the client can poll to check the status of the task.
If you consider upgrading to WAS v8/v8.5 in the near future a good idea will be to use Asynchronous Servlets for that
The reason that your client receives an HTTP 500 error after a few minutes can happen for a few reasons. Without a stack trace and some relevant logging, it is impossible to know which component within WebSphere "woke up" after 3 minutes and stopped everything. It might be WebSphere's timeout setting for the Web Container thread pool, or it can be some other timeout - should be easily concluded from the logs.
To fix this, you can do one of the following:
Adjust the relevant timeout value (depending, again, on which timeout it is exactly).
Change your design so long-running tasks are executed in the background. You can use WebSphere's Work Manager API for that, or asynchronous beans / servlets.
Related
I have an application where a user can upload a PDF using angular-file-upload.js
This library does not support file chunking: https://github.com/nervgh/angular-file-upload/issues/41
My elastic load balancer is configured to have an idle timeout of 10 seconds and other parts of the application depend on keeping this parameter.
The issue is if the file upload takes longer than 10 seconds the user receives a 504 Gateway Timeout in the browser and an error message. However, the file still reaches the server after some time.
How can I ignore or not show the user this 504 Gateway Timeout that comes from the ELB? Is there another way around this issue?
The issue you have is that an ELB is always going to close the connection unless it gets some traffic back from your server. See below from AWS docs. It's the same behaviour for an ALB or a Classic load balancer.
By default, Elastic Load Balancing sets the idle timeout to 60 seconds
for both connections. Therefore, if the instance doesn't send some
data at least every 60 seconds while the request is in flight, the
load balancer can close the connection. To ensure that lengthy
operations such as file uploads have time to complete, send at least 1
byte of data before each idle timeout period elapses, and increase the
length of the idle timeout period as needed.
So to get around this, you have two options:
Change the server processing to start sending some data back as soon as the connection is established, on an interval of less than 10 seconds.
Use another library for doing your uploads, or use vanilla javascript. There are plenty of examples out there, e.g. this one.
Edit: Third option
Thanks to #colde for making the valid point that you can simply work around your load balancer altogether. This has the added benefit of freeing up your server resources which get tied up with lengthy uploads. In our implementation of this we used pre-signed urls to securely achieve this.
I planning to create a NodeJS program, that work 24/7, that ping and make requests to an external server (outside of google cloud) every minute. Just to see that it the external services are are live.
If there is any error it will notify me by SMS & Email.
I don't need any front-end for this app, and no one needs to connect to it. Just simple NodeJS program.
The monitoring and configuration will be by texts files.
Now the questions:
It looks like it will cost me just $1.64. It sounds very cheap. Am I missing something?
It needs to work around the clock, I will request it to start it once, and it need to continue working, (by using setInterval). Is it will be aborted?
What it is exactly mean buy 1 instance. What an instance can do? Only respond to one request or what?
I tried to search in Google: appengine timeout, but didn't found anything that helps.
Free Quota
If you write your application in Python, PHP, Go or Java it can fit in free usage quota:
https://cloud.google.com/appengine/docs/quotas
So there will be absolutely no costs to run it on Google App Engine platform.
There are limit of 657,000 UrlFetch API Calls per day (more than 450 calls per minute in 24/7 mode) for free apps. 4GB traffic may also be sufficient for this kind of work.
Keep in mind there is no SMS sending services provided by Google App Engine and you will need to spend additional UrlFetch API calls to use external SMS services.
Email sending is also limited to 100 Emails per day (or 5000 Emails to admin address), so try not so send repeated notifications about same monitored server every minute, or you'll deplete your Email quote in 1.5 hours.
Scheduled Tasks
There is no way to run single process indefinitely without interruption on App Engine. But you don't have to!
You'll need to encapsulate all the work you're planning to execute in every iteration into single task and then schedule it to run every minute with Cron. See this documentation for Python: https://cloud.google.com/appengine/docs/python/config/cron
It is recommended to have some configuration page where you can set some internal configuration or see monitoring statistics, at least manage flag to temporarily pause tasks execution without redeploying your app.
I have a website deployed on appengine that has a API.
On my computer I have a node.js script that sends data to the website using a POST through the API.
The problem is that while sometimes the website processes the requests fast (average of 1 per second), some other times it is very slow (average of 1 request per minute).
After some digging, I've found out that when that happens, 4 requests are processes very fast, and then the website does nothing for 4 minutes, repeating the process all over again, making the average time of 1 request per minute like said before. What could be causing this?
I don't know if it is very relevant or not, but I have a free appengine account.
Looks like a bug in your node.js script.
The Appengine has a limit of 60 seconds for external requests - its not possible that one
request blocks an instance for more than these 60 seconds. (And even if the instance is blocked,
the gae would spawn another one)
Other guesses:
check the access logs for first/warmup requests (eg. slow instance startup)
check the access-log to ensure that the gae actually recived the requests
I am observing strange behavior of my tomcat server, it seems like tomcat is not writing response to the client fast enough. Here is what I am seeing:
When firing aound 200 requests at the same time at my tomcat server, my application logs shows that my servlet's doGet() finishes process the request in about 500ms. However, at the client side the average response time is about 30 seconds (which means client start seeing response from tomcat after 30 seconds)!
Does anyone have any idea about how come there are such long delay between the end of my servlet's process time and the time when client receives response?
My server is hosted on Rackspace VM.
Found the culprit. I observed that the hosting server was using abnormally high CPU usage for even for only few requests, so I attached JConsole to Tomcat and found that all my worker thread has high blocking count... and are constantly in blocking state. Looking at the stack trace the locking happened during JAXBContext instantiation. Digg further, the application creating JAXBContext, which is relatively expensive, for each request.
So in summary, the problem was caused by JAXBContext instantiation per thread. Solution was to ensure JAXBContext is created once per application.
I am connecting to a 3rd party server on the following wsdl
http://webservices.ticketvala.com/axis2/services/WSTicketvala?wsdl
I am using JAX-WS to generate client code and call relevent method on 3rd party server. 3rd party server may take time between 15-25 seconds to send response.
It works fine on tomcat.
Now when i deploy this to GAE 1.5.3, often i get ScocketTimeoutException in less than 10 seconds. Sometimes it is succesfull taking even 20 seconds. I want to know why it fails many times. And any workaround to increase this response deadline time / to avoid this ScoketTimeOutException forever.
Similarly,
I have another RESTfull service at http://ticketgoose.com/bookbustickets/TGWSStationNameResponseAction.do?" +
"event=getStationDetails&password=123456&userId=ctshubws
I am connecting it through java.net.URL and many times i get TimeoutException. How can i raise this timeout limit to more than 30 seconds?
Thanks
Deepak
No user-initiated request can take more than 30 seconds to complete in Google App Engine:
http://code.google.com/intl/en/appengine/docs/java/runtime.html#The_Request_Timer
And a HTTP request to a external URL in a user-initiated request can't take more than 10 seconds to complete:
http://code.google.com/intl/en/appengine/docs/java/urlfetch/overview.html#Requests
If you need to do the overall work in more than 30 seconds and can do it in background (not needing to return the response directly via HTTP), use Task Queues. Note that the simplest way to do background work with tasks is to use the DeferredTask. And the best part: HTTP requests to external URLs on tasks can take up to 10 minutes to complete.