I am running Tomcat 6 and have a connectionTimeout set to 3000 (3 seconds).
I am trying to find out, on the Tomcat server itself, is there any way for me to know how many connection timeouts are occurring?
In Tomcat 6 there isn't anything the explicitly counts connection timeouts. (In later versions you could enable debug logging which would provide details of every failure). What you can do is use JMX to look at the error count. Every failed request increments the error count. Be aware that any response code >=400 will increment the error count so there is lots of legitimate traffic that could trigger this.
In a JMX browser (e.g. JConsole) you need to look at the errorCount attribute of Catalina:type=GlobalRequestProcessor,name=http-8080 (assuming an http connector on port 8080).
For a more accurate figure you could use the access log to count up the number of requests with >=400 response codes. The errorCount less that figure should be close to the number of connection timeouts in 6.0.x. In later versions that won't work because the access logging was improved to capture more failed requests.
Related
I'm trying to debug 502 errors coming out of the nginx container with my AppEngine Flex setup.
I noticed that the logs indicate liveness and rediness checks being spammed very rapidly (see attached).
For clarification this is currently running a single instance in manual_scaling mode.
check_interval_sec is set for 30s on liveness_check and 5 sec on rediness_check.
Can anyone provide insight into what is going on here?
It looks like you setup readiness and liveness checks too aggressively in your app.yaml. Please keep in mind that the checks works for every instance so if you have a lot of instances, it will occur frequently.
If you only have one instance setup, then the behavior contradict what the documentation described. Please file an issue with us on the issue tracker.
I have configured a version of my default service on Google App Engine Standard (Java, though that shouldn't make any difference) to use basic scaling and run a single B2 instance:
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<application>${app.id}</application>
<version>tasks</version>
<threadsafe>true</threadsafe>
<runtime>java8</runtime>
<module>default</module>
<instance-class>B2</instance-class>
<basic-scaling>
<idle-timeout>60s</idle-timeout>
<max-instances>1</max-instances>
</basic-scaling>
<!-- Other stuff -->
</appengine-web-app>
Despite not getting any request for almost 28 minutes, the instance did not shutdown by itself (I manually shut it down with appcfg.cmd stop_module_version...):
There are no background threads.
Why doesn't this instance shutdown? Two days prior the instance ran almost the whole day, idle, with this configuration... so what's the issue?
The definition of idle is that no new requests are received in x amount of time. What if the last request is taking 20 minutes to execute? Shouldn't idle be defined as the time since the last request finished?
I posted this question on SeverFault (since this is not a programming question) but was told that StackOverflow would be the better site...
GCP Support here:
I tried to reproduce the same behaviour but either using 1m or 60s, instances would be shutdown after serving its last request.
Nonetheless, when I had any long lasting requests, threads, and/or Task queues running for minutes, the instance wouldn't be shutdown until this request was finished. You can also find this information here for both Manual/basic scaling :
Requests can run for up to 24 hours. A manually-scaled instance can
choose to handle /_ah/start and execute a program or script for many
hours without returning an HTTP response code. Task queue tasks can
run up to 24 hours.
In your case, it seems there was a request lasting for minutes prior being finished and thus, the instance was active (rather than idle) throughout until you manually stopped it. You may find this Instance life cycle documentation useful as well.
If you believe this behaviour isn't what you experienced back then, I'd suggest you to create a private issue tracker so we can investigate further. Make sure to provide the project number and all required details in there (fresh samples). Once created, share the issue tracker number so we can look into this.
I am using solr (v4) - I am sporadically getting the following exception:
Timeout occured while waiting response from server at: http://localhost:8983/solr
I am assuming that I can change the timeout parameter through a config file or via my code in solr (which I believe I reduced already a few weeks ago). Besides changing the timeout period in the config/code (and checking why my code or perhaps solr is taking so long for the connection, is there anything else I can look into to troubleshoot this issue)?
Update:
This seems to occur around the time when I try to commit a few documents to solr (which are well defined) - however, a few of them might already be in solr & I'm not sure if that is causing any issues with solr
Edit:
What I mentioned in my first edit seems to be be the case though I am not entirely certain.
Are you caching or pooling HttpClient or connection? If so, do you have by any chance a firewall between your client and Solr Server?
It might be a case of long running connection being shut down by firewall and all consequent attempts to use that connection will either get packets dropped (leading to timeout) or get RST packet back.
If you can replicate it, try running Wireshark on the client and seeing what happens to the connection. But if it is firewall, it is usually very hard to replicate (need to create a connection gap of X hours).
I am trying to use MongoDB on Ubuntu 11.10 with CakePHP 1.3 using the cakephp-mongodb driver from ichikaway, but I am seeing persistent connection problems.
My model is very simple. I am attempting a connect and a read
$this->loadModel('Mongo.Mpoint');
$data = $this->Mpoint->find('first');
However the result is inconsistent. A significant amount of the time the server returns
Error 324 (net::ERR_EMPTY_RESPONSE): The server closed the connection
without sending any data.
However issuing a refresh, or several refreshes in quick succession will eventually deliver the expected returned data. It feels like the server is going to sleep and needs to be woken up as repeatedly hitting return does not generate errors, but this is subjective. The crash occurs in the find, not the connection itself.
I have rockmongo installed on the server which never fails to connect. Also I see the same behaviour if I point the connection at a different sever (same version of mongo, but on centos) so I do not believe the issue is with mongodb itself.
I have attempted setting the connection to persistent and directly setting the timeout, all without success.
My colleague, who also has a copy of the app running directly on the centos server, says he saw this problem initially, but 'it went away'.
From what I can see therefore the issue is most likely in the cakePHP layer as connections across different servers yields the same result and a direct connection in PHP is trouble-free, but placing diagnostics does not reveal anything of immediate use. It is rather odd to see a complete PHP crash with nothing useful returned from the server. Has anyone else seen this behaviour before and fixed it?
Try updating the Mongodb php driver. I had the same problem and I just upgraded from 1.2.7 to 1.2.9 and It seems to have been solved.
Just for reference
I am using
Ubuntu 11.10 x86
Xampp(Lampp) 1.7.7 with php-mongo-driver 1.2.9
Mongodb 2.0.2
CakePHP 2.0.6
cakephp-mongodb driver from ichikaway (Branch:cake2.0)
When I access a wrong call to a sql server data into my application in classical ASP I get this message in my entire site: Service Unavailable. It stopped. My site is in a remote host. Don´t know what to do. What can I tell to the "support team" of them to fix that?
If you check out Administration Tools/Event Viewer - Application log you will probably see an error message.
This should give you more information as too why the application pool died or why IIS died.
If you paste this into your question we should be able to narrow things down a bit.
Whenever there are a number of subsequent errors in your asp.net page, the application pool may shut down. There's a tolerance level, typically 5 errors in 10 mins, or so. Beyond this level, IIS will stop the service. I've run into a lot of problem due to this error.
What you can do is either fix all your websites (will take time), or increase the tolerance level or just disable the auto shutdown system. Here's how
Run IIS
Right click on the node 'Application Pools' in your left sidebar.
Click on the tab 'Health'
Remove the check on 'Enable Rapid Fail Protection'
or change the tolerance level.
Hope that helped.
One reason you can get this is if the application pool has stopped.
Application pools can stop if they error. Usually after 5 errors in 5 minutes IIS shutsdown the AppPool. It is part of the Rapid-fail protection and it can be disabled for an AppPool otherwise the AppPool has to be restarted every time it happens.
These settings can be changed by the IIS administrator. It looks like you can setup a script to restart and app-pool so you should be able to set up a new web application (in a different app-pool) to restart your closed app-pool. Hoster might not like that though.
Best result for you would be to catch all the exceptions before they get out into IIS.
Could be a SQL exception in your Application_Start (or similar) method in Global.asx. If the application (ASP.NET worker process) can't start, it can't run, so the worker process has to shut down.