I'm testing ELMAH and have deliberately turned off the database connection for the ELMAH log in my application to see what will happen in production if the DB isn't available.
It seems that ELMAH can't trap its own errors- the AXD file isn't available when the SQL databse log fails.
What is the intended behavior of ELMAH if the database isn't available?
How can I diagnose my errors if this occurs?
It seems that ELMAH can't trap its own
errors
ELMAH does trap its own errors to some extent. If the ErrorLogModule encounters an exception while attempting to log the error then the exception resulting from logging is sent to the standard .NET Framework trace facility. See line 123 from 1.0 sources. See also the following walk-through from the ASP.NET documentation for getting the standard .NET Framework tracing working with ASP.NET tracing:
Walkthrough: Integrating ASP.NET Tracing with System.Diagnostics Tracing
the AXD file isn't available when the
SQL databse log fails.
That is correct. SQL Server database connectivity must be functional to view errors stored in a SQL Server database when using SqlErrorLog.
What is the intended behavior of ELMAH
if the database isn't available?
If, for example, the SQL Server database is down, a SqlException will occur during logging. ELMAH will then send the SqlException object content to the standard .NET Framework trace facility.
How can I diagnose my errors if this
occurs?
The best option here is to also enable logging and e-mailing of errors. If the database is down, chances are good that the mail gateway is up and you will still get notified of errors. The errors will, in effect, get logged in some mailbox(es). This also has the added advantage that if the mail gateway is ever down then chances are that the database will be up and errors will get logged there. If both are down, however, then you will need to seriously review your production infrastructure and possibly take measures for monitoring health of your system via additional measures.
Not really sure about ELMAH but expected behaviour of such logging frameworks is not to throw any exceptions if something goes wrong with them. I.e. if ELMAH's database is down I'd assume it will just not log the errors to database.
As suggested above you can/should use alternative sinks - email or flat file.
You can always use the xml file option to log your errors.
I think you're mixing up contexts a bit.
ELMAH's behavior if the database isn't available is to not log errors to the database. If an exception is thrown on the server or if you raise an exception via an ErrorSignal, ELMAH is going to let that exception pass through to either a yellow screen or a custom errors page (your setting.)
Since Errors.axd page is only accessible to those that should be seeing it (ideally,) it is okay to present that error to the user.
The bottom line is that if the errors database is down you can't diagnose errors. For us, if that were the case, we'd have bigger problems since the error database sits with the production database.
I would also advocate against using XML logging for your primary logging source. SQL server is going to give you the best performance without having to manage the files. With XML logging that is not the case.
Related
I'm working on a .NET application migration from Oracle to SQL Server database. The application was developed in the 2000s by a third party, so we intend to modify it as little as possible in order to avoid introducing new bugs.
I replaced the Oracle references to SqlClient ones (OracleConnection to SqlConnection, OracleTransaction to SqlTransaction etc.) and everything worked fine. However, I'm having trouble with a logic that tries to reconnect to the DB in case of errors.
If a problem occurs when trying to read/write to the database, method TryReconnect is called. This method checks whether the Oracle exception number is 3114 or 12571; if so, it tries to reopen the connection.
I checked these error codes:
ORA-03114: Not Connected to Oracle
ORA-12571: TNS: packet writer failure
I searched for the equivalent error codes for SQL Server but I couldn't find them. I checked the MSSQL and .NET SqlClient documentation but I'm not sure that any of those is equivalent to ORA-3114 and ORA-12571.
Can somebody help me deciding which error numbers should be checked in this logic? I thought about checking for codes 0 (I saw it happen when I stopped the database to force an error and test this) and -2 (Timeout expired), but I'm not really sure about it.
The behavior is different. You can't base your SQL Server retry logic on Oracle semantics. For starters, SqlConnection will retry to connect even in the old System.Data.SqlClient library. Its replacement, Microsoft.Data.SqlClient includes configurable retry logic to handle connections to cloud databases from on-premise applications, eg an on-prem application connecting to Azure SQL. This retry logic is on by default in the current RTM version , 3.0.0.
You can also look at high-level resiliency libraries like Polly, a very popular resiliency package that implements recovery strategies like retries with backoff, circuit breakers etc. This article describes Cadru.Polly which contains strategies for handling several SQL Server transient faults. You could use this directly or you can handle the transient error numbers described in that article:
Exception Handling Strategy
Errors Handled
SqlServerTransientExceptionHandlingStrategy
40501, 49920, 49919, 49918, 41839, 41325, 41305, 41302, 41301, 40613, 40197, 10936, 10929, 10928, 10060, 10054, 10053, 4221, 4060, 12015, 233, 121, 64, 20
SqlServerTransientTransactionExceptionHandlingStrategy
40549, 40550
SqlServerTimeoutExceptionHandlingStrategy
-2
NetworkConnectivityExceptionHandlingStrategy
11001
Polly allows you to combine policies and specify different retry strategies for them, eg :
Using a cached response in some cases (lookup data?)
Retrying with backoff (even random delays) in other cases (deadlocks?). Random delays can be very useful if you run into timeouts because too many concurrent operations cause deadlocks or timeouts. Without it, all failing requests would retry at the same time, causing yet another failure
Using a circuit breaker to switch to a different service or server.
You could create an Oracle strategy so you can use Polly throughout your projects and handle all recoverable failures, not just database retries.
I have a SSIS project that I have created that has the DontSaveSensitive protection level and has happily deployed to the local server several times before today. I am now, however, getting the following error on deployment:
A .NET Framework error occurred during execution of user-defined
routine or aggregate "encrypt_binarydata":
System.IO.FileLoadException: Could not load file or assembly
'System.Core, Version=4.0.0.0, Culture=neutral,
PublicKeyToken=b77a5c561934e089' or one of its dependencies. Not
enough storage is available to process this command. (Exception from
HRESULT: 0x80070008) System.IO.FileLoadException: at
Microsoft.SqlServer.IntegrationServices.Server.Security.CryptoGraphy.CreateSymmetricKey(String
algorithm) at
Microsoft.SqlServer.IntegrationServices.Server.Security.CryptoGraphy.EncryptBinaryData(SqlString
algorithmName, SqlBytes key, SqlBytes IV, SqlBytes binaryData) .
(Microsoft SQL Server, Error: 6522)
I have had a google but come across nothing that specifically references encrypt_binarydata. There are a number of references to deploy_project_internal or untrusted assemblies but nothing on this particular issue.
The important part seems to be
Not enough storage is available to process this command
but I can't make head or tail of this as there are many gigabytes of RAM going spare and plenty of drive space to use so the resources shouldn't be problem.
Can anyone shed any light on what this error is referring to and ideally how I can resolve it?
Turns out this is a problem with permissions that gets very muddied deep down in the inner working of the SSISDB, between SQL and dll files. The error message in the original question is actually a bit of a red herring and the real problem was the same as the one solved in this excellent resolution.
For posterity's sake in case that answer ever disappears (as well as for lazy people that don't want to click another link), here is the referenced answer in full
Credit to Remus Rusanu
Assemblies with EXTERNAL_ACCESS are, through some convoluted path, falling under the EXECUTE AS path. The problem appears when the 'dbo' cannot be mapped to a valid login. dbo's login is the login with the SID the owner_sid value in sys.databases. Unless an AUTHORIZATION clause was used in CREATE DATABASE the owner_sid is the login sid of the principal issuing the CREATE DATABASE statement. Most times this is the Windows SID of the user logged in and issuing the CREATE DATABASE. With this knowledge in hand one can easily envision the problems that may arise:
copy database: CREATE DATABASE was issued on machine A by an user local to A (ie. MachineA\user or DomainA\user) then the database was copied to machine B (via backup/restore or via file copy). The owner_sid is preserved by file copy as well as by backup/restore, this on machine B the owner_sid is invalid. Everything requiring EXECUTE As fails, including loading assemblies from the database.
tombstoned account. CREATE DATABASE was issued by an user that has left the company. The AD account is deleted and all of the sudden EXECUTE AS mysteriously fails, including loading assemblies.
disconnected laptop. CREATE DATABASE was issues when the laptop was connected in the work network. At home you can log in using Windows cached credentials, but EXECUTE AS wants to connect to the unavailable AD and fails. Loading assemblies also fails. Problems mysteriously resolves itself next day at work, when you're again within reach of AD.
spotty AD connectivity. The EXECUTE AS does not uses system cached credentials and connects to the AD every time. If the AD connectivity has issues (timeout, errors) those issues manifest as similar timeouts and errors in EXECUTE AS, including loading assemblies
All these issues can be diagnosed by simply running: EXECUTE AS USER = 'dbo'; in the context of the problem db. It it fails with an error then the cause of your assembly load problems is the EXECUTE AS context of dbo.
The solution is trivial, simply force the owner_sid to a valid login. sa is the usually the best candidate:
ALTER AUTHORIZATION ON DATABASE::[<dbanme>] TO sa;
The funny thing is that the database may seem to be perfectly healthy; tables are available and you can run selects, updates, deletes, create and drop tables etc. Only certain components require EXECUTE AS:
code signing requires the code to have an EXECUTE AS clause
assembly validation
explicit EXECUTE AS in T-SQL code
Service Broker message delivery (including Query Notifications)
The latter is the most often seen culprit, as applications relying on SqlDependency all of a sudden seem to stop working, or have random problems. This article explains how SqlDependency ultimately depends on EXECUTE AS: The Mysterious Notification
I am trying to set up a merge replication using web synchronization between a publishing SQL Server 2012 standard and subscribing SQL Server 2012 Express. After following the instructions provided at Technet, I am stuck on this:
Source: Merge Process(Web Sync Server)
Number: -2147200985
Message: The subscription to publication 'MyMergePublication' has expired or does not exist.
I already verified that SSL certification are good, that I can browse to the publishing machine's URL https:\\mycomputer\replisapi.dll and get the expected output. I already verified that snapshot was set up and I took a giant hammer & use an administrator account to run the pool identity which is really bad security-wise but wanted to validate that it was not security that was tripping me up.
To further the mystery, when I try and fail to sync, the publisher acknowledges that a new subscriber has been registered, but it cannot get the snapshot at all and thus subscriber database is still empty.
On the replication monitor, there are no failed synchronization history, or any errors; all it has to say is that the subscriber is uninitialized, and no more.
Turning up the verbosity of the merge agent, I saw some sql being executed and tried replicating the sql and i found this was failing with same error:
{call sys.sp_MSgetreplicainfo(?,?,?,?,?,?,?,90)}
I called it with only the 3 mandatory parameters supplied and it would fail. That is despite the prior call sp_helpmergepublication does return a row for that publication. Oddly, the content of sp_helpmergepublication does not match what I configured for the subscription (e.g. it says web url is null when viewing the properties correctly shows the web url being set). Not sure that is significant.
The content of sp_MSgetreplicainfo contains a call to another system sprocs that I cannot run for some reason (says not found) so I'm not sure what is actually going on here.
Any clues would be greatly appreciated.
I am developing a WCF service, so far I have added only a few simple interfaces. Testing the service in the VS debug environment, all is well.
When I published the service to IIS 7 for further testing, all was fine, at least after I added db_datareader and db_datawriter permissions to the relevant databases for user IIS APPPOOL\ASP.NET v4.0.
Now the service is failing silently. Calling the service through a browser, I get the following message: "The server encountered an error processing the request. See server logs for more details." Fiddler says this is a result of an HTTP 400 error message (Bad Request).
I rolled back the code to return a hard-coded value, so I am sure that the issue is at the DB level, not the IIS 7 server installation.
The problem is that the Event Viewer shows no meaningful error messages. This despite the fact that all the code is surrounded by try/catch, with any exception caught going to the event viewer.
There is one message stating "Starting up database 'ReportServer$SQLEXPRESSTempDB'", but as far as I can tell that appears every 10 minutes, without reference to any attempts to access the database. Just to make sure, I gave the .NET 4 user R/W permission to access that DB as well.
In addition, I don't see any messages in the SQL Server Logs.
How can I debug this?
The problem was that db_datareader and db_datawriter permissions are not enough to run stored procedures. I upgraded the permissions and all was good.
The only thing I don't understand is why the exception wasn't written to the event log.
we have an application running on an IIS 6/ASP.NET 2.0 backed by a SQL Server 2005 STD edition X64. From time to time, the application crashes with some silly messages (some of the fileds are not found into a "select firled1, filed2.. from mytable"). I obtained a trace of the activity from the sql server taken while the application reported the errors. Note: the crash is encountered only during some heavy load on that server, like creating some reports simmultaneously by several users.
The question is: how can I use the trace file to solve the situation? How can I detect what goes wrong?
Thanks
I've got a video tutorial on getting started with Profiler at SQLServerPedia. In a nutshell, you'll want to export that trace file into a table, and then step through it in order looking at the errors that popped up. Profiler's trace files or trace tables by themselves won't say, "Here's what you need to fix in order to avoid this error" any more than a dump file will tell you "Here's the bad line of code" - you'll still need an experienced DBA to interpret the results. You may want to take the trace file to your local SQL Server User Group meeting and see if someone can help you.
OK, you need to get the trace data into a form that you can interrogate, i.e. a table.
Here are the details on how to load a trace file into a SQL Server table.
http://support.microsoft.com/kb/270599
You then need to identify the events that are responsible for your issue. Search the trace for events that occurred within the database that you are interested in and that also occurred around the time of the error you experienced.
The following link provides a good starting point for SQL Server Profiler/Trace information.
http://msdn.microsoft.com/en-us/library/ms187929.aspx
Have a read through this and if you get stuck let me know.
Cheers,