IIS 7 consuming connections and not releasing them - sybase

We have a asp.net website, hosted by IIS7. The website functions normally without any problem.
But after a while, like 1 or 2 months, the Sybase database which the website connects to will produce this error whenever accessed by any application : Maximum number of connections already opened
At first we didn't realize that it was the web site's application pool that causes this, so we restart the database and everything is back to normal.
But then the 2nd time, the 3rd time ... and we came to aware that we just need to restart the application pool for those unreleased connection to be released.
I checked the source code, there was only one place has database-connect-code and it was a very simple code which connects to the db, get the results then close it :
con.open()
x = con.getdata()
con.close
Btw, after checking the application's log, there wasn't any error or exceptions so I'm pretty sure that the con.close is probably reached and executed.
So if we could rule out the possibility that there were unclosed connections in the source code, is there any other explanation for this ?

Related

Network access was interrupted

The Access database just needs to be open and it will usually crash within the next 20-40mins, resulting in the following error message:
Your network access was interrupted. To continue, close the database, and then open it again.
More details:
The database is split, with the back end and front end on a server. The computers are then connected to the server via LAN (ethernet).
Although there are multiple computers connected to the server, the database only has one user at a time.
The database has been fine for almost a year, until this week where this error has started occurring.
We never have connectivity issues with the server.
I have seen several answers saying it is:
the databases fault, as it is starting to corrupt
the servers fault, as it broken, dropping my connection briefly
microsofts fault, they should patch it
I am hoping this is a problem with the database itself, as I am not responsible for the server.
Does anyone have a definitive solution?
I have recently experienced the same problem, and it all started when I moved my DB in an extrernal disk. The same db was working just fine in the local disk, or in the previous external disk. So, i am guessing is just a bug that has to do with the disk letter changing or something like this.
The problem sounds like an unstable LAN connection OR changes the LAN location (e.g. new hardware or changs to admin settings) causing increased latency.
If you have forms in the FE bound to BE tables the latency can cause the connection to be severed resulting in the error you see.
I'm not a network admin but the main culprits I've seen are:
Users connecting to the network using a VPN using an unstable connection (cell phones, crappy wifi, or just bad ISP service).
Network admins capping persistent connections to a share causing disconnects.
Unstable network hardware or bad hardware configuration.
"Switching" between wired and wireless LAN connections.
I don't think the issue is the database other than having bound forms to a BE database which is more of a fundemental design problem than anything else.
Good luck!
I use Access 2010. I had the same issue but solved it in the following ways.
On the external data ribbon, go to the Import & link group and click on Linked Table Manager.
Click on select all.
Click on Ok to refresh the links.
In cases where the path of the BackEnd database file has been changed, browse to the new location and select the new path. This will also refresh the links. This will solve the problem. It did for me.
You wrote, "The database has been fine for almost a year, until this week where this error has started occurring."
Clearly something has recently changed for this to be happening and without narrowing the field of possibilities it's anyone's guess. However, in my experience Jet DB crashes when two or more users are accessing and editing the same record(s) at the same time. So, if you've recently added new users this is a possibility.
Note: Jet is a file-server DB not a client server, which means the app was probably designed for a specific number of front-end users. Without knowing more I would start there.
I resolved my issue on this when I figured out that I had a offline directory setup and the sync was having an issue I turned off the sync and tested it and the error went away.

OData HTTP400 Timeout Error

This is one of the most bizarre problems I've come across since I started using OData for my mobile apps. The OData server I've developed is backed by SQL Express 2008 and this combination has been installed on 50 different servers and/or PCs over the last 15 months. All 50 servers have been running stable with consistent function for large amounts of data.
A couple of days ago one of my clients contacted me indicating that my client app (running on iOS7) was having an odd error come up when POSTing data to their server. The error had an HTTP code of 400 and the error text is "The operation couldn't be completed. (Timeout error 400.)". My first question is: why is a timeout error coming back with a 400 code? Generally when I get timeouts (due to firewall, etc) they're in the 100x range. There is no indication in the event logs on the server of ANY problems occurring. My own logs (stored in the SQL database) show no error (which is odd because I'm using the generic exception catching method in my OData service to log any problems). I haven't got to the step of adding logging of all requests as yet.
The error is only being raised when posting one particular set of data. All other posts from the device are functioning perfectly. I got the client to re-install the app (deleting all data) and then to download the data set that was causing the error. The download worked fine. We began making changes to the data to replicate what the data looked like when the error occurred in incremental changes, posting the change to the server and observing the result. Most of the incremental changes work fine but certain combinations cause the error to occur. One of the increments involves a large volume of changes and that posts fine, but subsequent alteration of any of the objects (sometimes altering as little as 6 characters in a text field) cause the error to occur. And yet in some circumstances altering objects that have already been posted to the server works without a problem.
I wiped the service components from the server and undertook a fresh install. I shifted TCP ports in case 443 had another listener causing problems. I reset the server. None of these change the behaviour of the error.
My last ditch solution is to completely re-install IIS and .NET Framework but I'd obviously like to avoid this as it's not my server... The server is overseas from my current location so debugging isn't really an option. Hoping someone has an idea as to what I can do diagnostically to try and determine the source of this bizarre 'gremlin'.
Have you tried a more thorough traffic analysis using a tool like Fiddler? The "timeout" error does indeed seem odd and what stood from you post was that your server was "overseas". Could there be something with the "times" that are being used/generated, e.g. server time, local time, etc?
Just to confirm, the "same" exact set of data always fails? Can you replicate this via a remote debugger or via localhost? If so, can you turn on "verbose errors"?

Network appears to be down when it isn't

A client is using our app over a LAN.
A VB6 front end accesses a Jet database via DAO.
The database connection is is kept open for the duration.
They are getting errors like as though the network is down when it isn't.
If I can explain:
Sometimes when there's a dodgy network connection a query will fail giving an error along the lines of "unable to find table A". As soon as the network is up again this instance will resume working properly again.
In this case however what happens is that an instance will go down in the manner described above but if a new instance is started it will behave properly (until after a random period it too hangs) meanwhile the previous instance will keep returning error messages suggestive of a network being down and never resume working properly.
Does anyone have an idea what this could be?
*UPDATE 08 Feb2011 The problem has been resolved *
The cause of this problem was the the computer storing the database had a static IP while the rest had dynamic IPs. The static IP address fell within the range from which the dynaic ones were selected.
Eventually a dynamic address was generated identical with the static one and caused the issues described above.
Thanks to everyone for their comments

Site update, testing was fine, after deployment, again fine, once user load increases, FAIL?

We are using ASP.NET MVC with LINQ to SQL. We added some features and tested them all to perfection on our QA box. We are using Windows Server 2003 and SQL Server 2005. So when we pushed out changes to the Live web server we also used Red Gate SQL Compare to push new database changes to the LIVE database. We tested again between the few of us, no problems. Time for bed.
The morning comes and users are starting to hit the app, and BOOM. We have no idea why this would happen as we have not been doing any new types of code things that we were not doing before. However we did notice that during the SQL Compare sync the names of all the foreign keys were different between the two databases, not the IDs in the tables, FK_AssetAsset_A0EB67 to FK_AssetAsset_B67EF8 (for example, don't remember the exact number of trailing mixed characters during the SQL Compare), we are not sure why but that is another variable in this problem.
Strangely once this was all pushed out we could then replicate the errors on QA, but not before everything was pushed to LIVE.
QA and LIVE databases are on the same SQL Server, but the apps are on different instances of Windows Server 2003.
Errors generated:
Index was outside the bounds of the array.
Invalid attempt to call FieldCount when reader is closed.
Server failed to resume the transaction.
There is already an open DataReader associated with this Command which must be closed first.
A transport-level error has occurred when sending the request to the server.
A transport-level error has occurred when receiving results from the server.
Invalid attempt to call Read when reader is closed.
Invalid attempt to call MetaData when reader is closed.
Count must be positive and count must refer to a location within the string/array/collection. Parameter name: count
ExecuteReader requires an open and available Connection. The connection's current state is connecting.
Any one have any idea what the heck could have happened?
EDIT: Since we were able to replicate the errors all of a sudden on QA, it might not be a user load issue... Needless to say we all feel really screwed here.
Concurrency always brings bugs out of the woodwork. I'd recommend you check for objects that could be shared among requests (such as static members and singletons) and refactor your code so that as little as possible is shared.
As far as specifics go, for the error "There is already an open DataReader associated with this Command which must be closed first," you may want to try adding MultipleActiveResultSets=True to your connection strings.
It sounds like you're crossing the streams a bit and trying to share DataContexts across requests. My suggestion would be to wire in a dependancy injection framework that creates a new instance of the dependancy for each request.
I use Castle's IoC and wire it into the controller factory so that when it sees a dependancy on a repository it creates a new instance of that repository for each request. If you go this route let me know and I can shoot you a few more resources.

How can I find why some classic asp pages randomly take a real long time to execute?

I'm working on a rather large classic asp / SQL Server application.
A new version was rolled out a few months ago with a lot of new features, and I must have a very nasty bug somewhere : some very basic pages randomly take a very long time to execute.
A few clues :
It isn't the database : when I run the query profiler, it doesn't detect any long running query
When I launch IIS Diagnostic tools, reqviewer shows that the request is in state "processing"
This can happen on ANY page
I can't reproduce it easily, it's completely random.
To have an idea of "a very long time" : this morning I had a page take more than 5 minutes to execute, when it normaly should be returned to the client in less than 100 ms.
The application can handle rather large upload and download of files (up to 2 gb in size). This is also handled with a classic asp script, using SoftArtisan FileUp. Don't think it can cause the problem though, we've had these uploads for quite a while now.
I've had the problem on two separate servers (in two separate locations, with different sets of data). One is running the application with good ol' SQL Server 2000 and the other runs SQL Server 2005. The web server is IIS 6 in both cases.
Any idea what the problem is or on how to solve that kind of problem ?
Thanks.
Sebastien
Edit :
The problem came from memory fragmentation. Some asp pages were used to download files from the server. File sizes could go from a few kb to more than 2 gb. These variations in size induced memory fragmentation. The asp pages could also take quite some time to execute (the time for the user to download the pages minus what is put in cache at IIS's level), which is not really standard for server pages that should execute quickly.
This is what I did to improve things :
Put all the download logic in a single asp page with session turned off
That allowed me to put that asp page in a specific pool that could be recycled every so often (download would now disturb the rest of the application no more)
Turn on LFH (Low Fragmention Heap), which is not by default on Windows 2003, in order to reduce memory fragmentation
References for LFH :
http://msdn.microsoft.com/en-us/library/aa366750(v=vs.85).aspx
Link (there is a dll there that you can use to turn on LFH, but the article is in French. You'll have to learn our beautiful language now!)
I noticed the same thing on a classic ASP + ajax application that I worked on. Using Timer, I timed the page load to be 153 milliseconds but in the firebug waterfall chart it randomly says 3.5 seconds. The Timer output is on the response and the waterfall chart claims that it's Firefox waiting for a response from the server. Because the waterfall chart also shows the response, I can compare the waterfall chart to the timer and there's a huge discrepancy 'every so often'
Can you establish whether this is a problem for all pages or a common subset of pages?
If a subset examine what these pages have in common, for example they all use a specific COM dll, that other pages don't.
Does this problem affect multiple clients or just a few?
IOW is there an issue with a specific browser OS version.
Is this public or intranet?
Can you reproduce the problem from a client you own?
Is there any chance there are some full-text search queries going on SQL Server?
Because if so, and if SQL Server has no access to internet, it may cause a 45-second delay every few hours or so when it tries to check the certifications (though this does not apply to SQL Server 2000).
For a detailed explanation of what I'm referring to, read this.
Are any other apps running on your web server? If so, is your problematic in the same app pool as any of them? If so, try creating a dedicated app pool for it. Maybe one of the other apps is having a problem and is adversely affecting yours.
One thing to watch out for is if you have server side debugging turned on in IIS, the web server will run in single threaded mode.
So if you try to load a page, and someone else has hit that url at the same time, you will be queued up behind them. It will seem like pages take a long time to load, but its simply because the server is doling out page requests in a single file line and sometimes you aren't at the front of the line.
You may have turned this on for debugging and forgot to turn it off for production.

Resources