I am writing a multi-threaded app to handle incoming email. I have one thread reading the Inbox messages and then farming them out to other threads which process the message (thread per message). I am connecting to Exchange 2013 (v.15) with all current updates.
The Store connection is being closed and when I try to re-open it I am getting an exception which states "failed to create new store connection". This is happening on the folder.exists() call. The folder definately exists.
I am opening in R/W mode because I have to move the messages to other folders as they are being processed.
The app runs for a while and then this starts happening. We've bumped up every setting we could find in Exchange to allow as many connections as possible. I'm wonderng if Exchange is only allowing 1 Read_Write connection to a folder?
We are doing IMAP (SSL)
Note: we are not a spammer.
Exception:
javax.mail.StoreClosedException: failed to create new store connection
at com.sun.mail.imap.IMAPFolder.throwClosedException(IMAPFolder.java:3368)
at com.sun.mail.imap.IMAPFolder.doCommand(IMAPFolder.java:3497)
at com.sun.mail.imap.IMAPFolder.exists(IMAPFolder.java:560)
at com.myco.myapp.MessageHandler.getFolder(MessageHandler.java:68)
Turn on JavaMail session debugging. The debug output might provide more clues as to why JavaMail can't connect to the server, or why the server is refusing the login attempt.
Are you running any firewall or anti-virus software that might be interfering with the connection attempt?
It would be easy to write a program to test whether Exchange allows more than one read/write connection to the same folder.
Related
In our production environment there are sometimes peaks of ORA-12599 errors the connection works otherwise.
The description is not quite clear to me.
Cause: The data received is not the same as the data sent.
Action:
Attempt the transaction again. If error persists, check (and correct)
the integrity of your physical connection.
https://www.oraexcel.com/database-oracle-11gR2-ORA-12599
Does that mean there is a problem with my network cable? Or any other Hardware?
Because our DBA said the error can be caused by a configuration issue on the client side.
Edit:
It started without any changes on the client or server side. And there are spikes. 80 errors in an hour and then no errors for 4 hours while the load is more or less constant
It can mean a mismatch in how the client and the server are wishing to communicate (in encrypted fashion). For example, trying to connect to an 11g database with 10g JDBC drivers will get this because the encryption implementation changed between these versions.
Check your local (ie, client) sqlnet.ora file - you may need to opt for a different method or change the order of them.
The error can be typically be ignored (its informational) but using the latest drivers will normally resolve the issue.
We had sporadic ORA-12599 errors and also ORA-24300 ("bad value for mode") in a Python web app using Flask and uwsgi. It turned out to be caused by multiple worker processes writing and reading to/from the same TCP connection to the DB, giving chaotic communications on the socket.
In our setup this was caused by creating the TCP connections to the DB before uwsgi forked the workers, and the file descriptors being inherited. We fixed this by using uwgsi's lazy-apps mode.
See also this question.
I had a perfectly working SQL Server Service Broker this morning, until I tested how it recovers from crashing.
I forced a system shutdown on the sender during a messaging session between servers over a network. I was sending binary messages of about 5mb size. There are automatic procedures for sending, replying and receiving messages and ending conversations from both sides in place and my setup uses certificates for security.
I am now unable to send any messages from the server side.
Both sides of the messaging chain have queues on and it does not seem like poison message handling would be causing this. The sender side accepts new messages but is not sending them.
The sender side transmission queue has messages with transmission_status
The Service Broker endpoint cannot listen for connections due to the following error: '10013(An attempt was made to access a socket in a way forbidden by its access permissions.)'.
Running ALTER ENDPOINT myendpoint STATE = STARTED returns the same error as above.
Running select * from sys.endpoints shows the endpoint with state_desc = STARTED anyhow..
Running select state_desc from [sender_database].sys.conversation_endpoints shows state_desc = CONVERSING for all results.
Running SELECT COUNT(*) FROM dbo.sender_queue returns 0.
There is no other traffic to the port my endpoint is using, at least not any that is visible with netstat or the TCPView tool. The ports have rules to allow traffic from the firewall and sqlagent and sqlsrvr processes also have extra rules to be allowed.
Using ssbdiagnose tool with ssbdiagnose -level info configuration from service... from the sender side shows a (not new) error
The route for service sender_service is classified as REMOTE. This will result in the message being forwarded.
along with some other errors about certificates that have always been there even when messaging was working. Ssbdiagnose with RUNTIME flag shows nothing at all.
Ssbdiagnose from the target side now says an exception occurs during connection. The target database also has a couple of reply messages stuck in the transmission queue with an empty transmission_status.
Edit: Seems that occasionally the status on the target side changes to the error 10060 connection failed...
What more can I do to diagnose the problem and fix it?
Edit: I tried changing the port the endpoint uses but the same error is thrown.
Edit: I am able to ping the servers from each other. Ssbdiagnose with RUNTIME option on target side says it cannot find the connection to the SQL Server that corresponds to the routing address of my sender endpoint/database.
The Service Broker endpoint cannot listen for connections due to the following error: '10013(An attempt was made to access a socket in a way forbidden by its access permissions.)'
WSAEACCESS (10013) is a rather unusual socket listen error. I never encountered it before. A quick search reveals KB3039044: Error 10013 (WSAEACCES) is returned when a second bind to a excluded port fails in Windows which is an acknowledged bug in Windows Server 2008R2, 2012 and 2012R2 when excluding a range of ports (netsh ... add excludedportrange ...). So my first question is, are you on one of the affected server OSes and are you actually using a network port exclusion range?
I strongly urge you to open a Microsoft support case for this issue and follow up with them, making sure networking guys are involved (again, WSAEACCESS is rather unusual symptom). This is not one of the usual issues and it is difficult to diagnose over forums discussion.
I was assigned to implement an application (in C++) to evaluate pending submissions (a submission is a programming algorithm to a given problem). A site (in ASP.NET MVC) posts problems and allows the users to submit their answers, then marks the submissions as "pending to evaluation" on the database (SQL Server 2008R2) and that is when my work begins:
I'll have 3 (or maybe more) instances of my application running as services.
Each instance has to check if any pending submissions exists in the DB every 2 seconds.
If it exists I retrieve and compile it, after successful compilation I execute it and finally, after execution, check the correctness of the answer. Then I update that submission setting the results and deleting it from the pending table.
I need to specify in the DB the current status of the pending submission (compiling, running, judging).
The time to evaluate a submition is ~(1-3)s and the same instance never evaluates more that one submission at the same time.
My problem is: How to connect to the DB server?
I have 3 possibles solutions and I need to know what should be better (in order to increase efficiency) and why:
1 - Establish a connection to the DB once I instantiate the application and never close it (close it when I delete the instance or shut down the server, that theoretically never will happen.)
2 - Open a connection each 2s in order to get the pending submission (if any one exists) wait for the full evaluation process to end, sets the evaluations results and then close the connection.
3 - Same as 2, but closing the connection when I retrieve the submission, when the compilation finish, open it again and update pending submission's status, close it, when the execution finish, open it again and update pending submission's status, close it, finally when the judging finish open it and set the evaluation result.
You don't say what database access library you are using (ODBC, ado.net, other?). Opening and closing database connections is a relatively expensive operation. You should be using some sort of connection pooling scheme in your db access framework. A pool of connections is opened for a period of time, and when your app opens a connection it will get handed an already open connection from a pool. That will make it more efficient. Go read about connection pooling
for SQL Server
Can jdbc connections which are closed due to database un-availability be recovered.
To give back ground I get following errors in sequence. It doesn't look to be manual re-start. The reason for my question is that I am told that the app behaved correctly without
the re-start. So if the connection was lost, can it be recovered, after a DB re-start.
java.sql.SQLException: ORA-12537: TNS:connection closed
java.sql.SQLRecoverableException: ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
IBM AIX RISC System/6000 Error: 2: No such file or directory
java.sql.SQLRecoverableException: ORA-01033: ORACLE initialization or shutdown in progress
No. The connection is "dead". Create a new connection.
A good approach is to use a connection pool, which will test if the connection is still OK before giving it to you, and automatically create a new connection if needed.
There are several open source connection pools to use. I've used Apache's JDCP, and it worked for me.
Edited:
Given that you want to wait until the database comes back up if it's down (interesting idea), you could implement a custom version of getConnection() that "waits a while and tries again" if the database doesn't respond.
p.s. I like this idea!
The connection cannot be recovered. What can be done is to failover the connection to another database instance. RAC and data guard installations support this configuration.
This is no problem for read-only transactions. However for transactions that execute DML this can be a problem, especially if the last call to the DB was a commit. In case of a commit the client cannot tell if the commit call completed or not. When did the DB fail; before executing the commit, or after executing the commit (but not sending back the acknowledgment to the client). Only the application has this logic and can do the right thing. If the application after failing over does not verify the state of the last transaction, duplicate transactions are possible. This is a known problem and most of us experienced it buying tickets or similar web transactions.
For some time now our flagship application has been having mysterious errors. The error message is the generic
[DBNETLIB][ConnectionWrite (send()).]General network error. Check your network documentation.
This is reliably reproduced by leaving the app open for the night and resuming work in the morning. Since it's a backend server app this is a normal scenario.
The funny thing is - we've migrated from SQL Server 7 to 2000 to 2008 and the issue is present on all of them. But what seems to matter is the OS on which we run the app. On WinXP it works fine, on Vista/7 it fails. So the problem is at the client end.
The results of Google on the error message cover a very wide spectrum of different causes (since this is a very generic error) and none of the scenarios found there are similar to ours.
So perhaps someone around here will know what the problem is in our case?
You should be able to reproduce this error condition on demand by:
1. Opening a database connection (in your client application)
2. Unplugging the network cable
3. Plugging network cable back in (wait until the network connection is restored)
4. Using the previously opened connection to query the database
As far as I can tell from experience, client side ADO code is not able to consistently determine if an underlying network connection is actually valid or not. Checking if the database connection is open (in the client code) returns true. However, performing any operations on that connection results in a General network error.
The connection pool appears to be able to determine when a connection goes 'bad' so it never returns a bad connection to the application. It simply opens a new connection instead.
So, if a database connection is kept alive for a long time (used or unused) by the application, the underlying TCP/IP connectivity can get broken.
The bottom line is that database connections should be closed and returned back to the connection pool when not in use.
Edit
Also, depending on the number of clients connecting to the db, not using the connection pool can cause another issue. You may hit the maximum number of sockets open on the server side. This is from memory. Once a connection is closed on the client side, the connection on the server goes into a TIME_WAIT state. By default, the server socket takes about 4 minutes to close, so it is not available to other clients during that time. The bottom line is that there is a limited number of available sockets on the server. Keeping too many connections open can create a problem.
One project I worked on easily hit this socket limit with around 120 users. A new 'feature' was added that absolutely hammered the server, and after a few hours of using the app, things would suddenly slow to a crawl for everyone. SQL server was not closing enough sockets in time for new connection requests. Although there are 65K sockets altogether, only the first 5000 are made available to the ADO (this is a default registry setting thing, so can be changed).
The number of sockets in TIME_WAIT state would slowly build up until the OS would not allocate any more. So clients had to wait until server side sockets closed and a new connection could then be created.
Have you tried disabling SNP/TCP Chimneying?
Had a similar error. For me it was indirectly caused by mismatched calls to WSACleanup and WSAStartup.
The program called WSACleanup more times than WSAStartup. This would cause a reference counter (somewhere in the sockets library) to reach zero too early.
I think effectively from that moment on all sockets owned by the process are broken.
And this would also kill the SQL client since it uses sockets to 'talk' to the SQL server as well.