I am using CDC in SQL Server 2008 and triggering it via SSIS(In the form Stored procedures and calling commands like table-diff in EST). Recently i came across a different issue and I am simply not understanding what might be the reason. What exactly happened is CDC suddenly hanged while executing. It was in running state for almost around two days. After that we manually had to kill CDC process in order to run the next batch for the same. When I ran same CDC job next time it ran properly without any interruption (Completed in few minutes/No changes were made to package or anywhere before next execution).When that process was executing for around two days I checked changed records, except for 2-3 new Inserts/Updates nothing was there. After going through log file details I came to know my table-diff cmd took around 5 hours to execute which normally takes around 5 mins for my package everyday.I just want to know what might be the possible reason for such behaviour.
PS: Such behaviour occurred few times in past as well.
Related
Two years ago a SQL expert opened a SSMS and showed all queries as they were happening in real time. That way he saw which SQL statements were running fast and which took some time to be ran. I remember the queries to be displayed in a "CMD" look alike window. I can't remember if new queries were displayed at the top or were they shown at the bottom of the window.
For the past month I have been trying to figure out how he got this working. I looked everywhere in monitor, but I can't find anything similar to what he showed me than.
The results were similar to the "claymore eth miner window" ...
Can someone point me in the direction of getting this?
You can use sp_whoisactive to get the current running queries. It is very useful to see if currently there is blocking, locking, long running statements.
In order to get better picture of what was going on in the past, you can enable the Query Store. There are some predefined reports:
and various statistics and it is user friendly:
I have a web scraping job that needs to be executed each evening. Our company has a virtual machine with the "Windows Task Manager" application installed. So, I created a new task (i.e., an entry in task manager) to run every evening at 3 a.m.
Initially, the process did exactly as expected: it fetched the data and inserted them into our database. A few nights later, the website we were scraping from kept shutting down for maintenance, so I went into Task Manager, changed the time setting to start at 10:30 pm instead of 3:00 a.m., and waited until the next morning.
The script executed completely with no exception issues- but nothing was entered into the database! The task manager even said that the script ran completely, and it even took the usual amount of time to run, but alas, no new rows.
One might posit that there was no new data to fetch. However, When I execute the script manually from the command line (and kept the start date/end date the same), the script uploaded the usual ~10,000 rows into the database. So there is data- but it only gets written to the database when we launch the script manually during the day, and not when scheduled in the evening.
Does anyone know a potential reason as to why this happens?
Thank you in advance.
Edited to add:
I understand that this question might sound a little ridiculous, especially since from the surface, there doesn't seem to be a lone factor in determining the issue. If I could provide any further background information into the issue, feel free to ask.
We have a Practice Management system that is about 15 years old. I've been working on it for about 12, and I've never encountered this problem until just recently, and we can't figure it out.
It is written in VB6 which uses ADO/JET to access an Access .mdb file on the network. The application opens the connection when it starts, keeps it open while it's open, and closes it when it exits. It does a LOT of stuff with the DB - the system deals with Patient Accounts, Charges, Payments, Scheduling Appointments, and about a million other things. We have dozens of clients that use this program, each with their own DB and most of them 'offsite', where they have their own server, some number of workstations, between 1 and 20 users, pounding away at the system 24 hours a day, 7 days a week, and except for the occasional DB field bug or having to compact/repair DBs, it's pretty stable.
About 3 weeks ago, we started seeing a problem that we hadn't seen before.
We have an 'in-house' system setup for us to use: the DB is on our Server which houses maybe 10 other DBs, and only a couple of people connect in to this system.
We started noticing that if someone logged into the system, went straight to our Scheduler screen, and then sat idle for about 5-10 minutes, they might get "Disk or Network Error", or "Cannot find the input table or query X on database Y".
What's strange is that it SEEMS to happen only when 2 or more people are logged into that DB from different computers, and then one of them will get the error (Randomly?) but the other 2 users will be fine.
There is a Timer on the 'main' MDI Parent form of the system which wakes up about ever minute (there are some things which will change the interval to a shorter interval, and there are some things which disable the timer, but we don't think either of them are happening in this situation). It performs a pretty basic SQL Query on the DB: SELECT loggedin FROM Users WHERE UserId = 'DBUPDATER' THIS is the SQL Query that seems to be ALWAYS triggering one of those errors.
There is also a timer that runs about every 2 minutes while a user is logged in to check for emails and a few other things that would be running during this time.
And since they are on the Scheduler screen, there's a timer on there that runs every 30s or so which will check the DB to see if any changes to the scheduler have been made to see if it needs to refresh the screen or not.
There are some other strange things:
The DB and the Users table seem to be perfectly fine. When the person who gets the error logged into the system - usually only 5 minutes earlier - the system HAD to look at the Users table to mark them as Logged in, and I'm almost 100% sure that the query that it's dying on had been run at least once - probably 4-5 times before it dies.
Once a user gets this error, if they leave the Error's MessageBox on the screen (not quit the .exe), if they try to access that DB file in any way, they get "Disk or Network error" - this includes if they try to open that DB in Access. HOWEVER, they can still get to the Network just fine, and even open up other mdb files IN THE SAME FOLDER as the one they can't open. OTHER PEOPLE on other computers can open that mdb without any errors. Once the user acknowledges the Error, and allow the exe to close, they can open that DB file again just fine.
I'm told that this is not in anyway a Network issue. Our IT guy says he's run pings and traces and all sorts of tests to ensure that the network connection is not getting dropped.
I've also run some things on the DB to make sure it's not corrupted and it seems to be fine - and we get it to happen on other DBs.
If anyone has seen anything like this and knows a possible fix, I would greatly appreciate it! Thanks!
New Information (9/5/13 - 10:30am)
We noticed that at the same time we get this error, in the errorlog on the computer that gets the error is a Warning Event that says:
{Delayed Write Failed} Windows was unable to save all the data for the file \MEDTECHSERVER\MEDTECH DATA\VB\SCHEDTEST2\MEDTECH.MDB; the data has been lost. This error was returned by the server on which the file exists. Please try to save this file elsewhere.
I have an ASP.NET reporting interface that displays several values that are returned from a SQL Server backend. Once logged in, the browser page is never reloaded, but several screen areas are updated on a timer through AJAX calls.
My problem is that the screen areas are intermittently displaying values that are coming from previous AJAX calls. I have thoroughly and intensively investigated the problem for a number of days and I haven't been able to specifically pinpoint what is causing it, or how to completely overcome it. Currently the incorrect values are very infrequent (3 in 50,000, say), but I should be getting none whatsoever! These are some details about the setup:
the screen refresh timer runs every 30 seconds to update all screen areas
there is a 1.5 second lag between the screen area updates
the values used to decide which SQL stored proc to run to get the correct values from the database for each screen area are being passed in to the ASP.NET interface correctly - I know with 100% certainty which stored proc to run
the stored proc returns its values to a SQLDataReader
it is the reader that is sometimes yielding values that seem to be "buffered" from previous AJAX interactions, i.e. I am running the correct stored proc with the correct variables, but the values returned are not what I get if I ran that precise same command in a SQL query interface - they are results from a previous call
the SQL connection, command and reader are all created and instantiated afresh for each interaction and disposed of correctly after use through the IDisposable interface
I have swapped the reader for a dataset, with no difference in results
my AJAX calls are synchronous (async=false), so they should each complete before the next one is run, but I also have the 1.5 sec delay between screen area updates and 30 secs between cycles, so they shouldn't run into each other in any event.
What is frustrating is that the reader is running a SQL statement without throwing an exception, but seemingly returning results from a previous interaction - and then only very seldom, but one incorrect result is one too many.
I am not using ASP.NET state management at all - switched off in web.config.
What am I missing?
Profile the sql statements that are arriving at sql server with sql server profile. You can narrow the potential locations of the bug that way. next, start the fiddler http debugger and verify the http requests. and let us know what you found!
it is the reader that is sometimes
yielding values that seem to be
"buffered" from previous AJAX
interactions, i.e. I am running the
correct stored proc with the correct
variables, but the values returned are
not what I get if I ran that precise
same command in a SQL query interface
- they are results from a previous call the SQL connection, command and
reader are all created and
instantiated afresh for each
interaction and disposed of correctly
after use through the IDisposable
interface I have swapped the reader
for a dataset, with no difference in
results
It would help to show us your code here. If the data objects contain the wrong values, you have to demonstrate with certainty that the correct SPs are being run on the back end and, if applicable, that the correct parameters are being passed to the back end. You should be able to step through the call in the Visual Studio debugger.
When importing customizations to CRM 4.0, the import fails with a message "generic SQL error". Digging a bit deeper the error message is really that a timeout has occured. The same error occurs when trying to create a new entity.
I increased the timeout as suggested in the link below, but the timeout occured anyway - it just took longer time to happen.
Increasing the timeout:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSCRM\OLEDBTimeout
This value does not exist by default, and if it does not exist then the default value will be 30 seconds. To change it, add the registry value (of type DWORD), and put in the value you want (measured in seconds). Then you have to recycle the CrmAppPool application pool for the change to take effect
https://community.dynamics.com/product/crm/crmtechnical/b/crmdavidjennaway/archive/2008/09/04/sql-timeouts-in-crm-generic-sql-error.aspx
The SQL profiler displays a set of inserts and updates related to the metadata in CRM, and then a call to the stored procedure exec p_RecreateIndexes
This call is apparently the culprit and never completes in a timely fashion (30+ minutes now and not completed yet). This is an existing test instance of CRM and is quite extensively used and filled with lots of data. Creating new entities has never before taken this long. Just in case, I have run the asyncoperation cleaning scripts from MS. It did not have any visible effect.
Is there any way to find the reason for the delay in this procedure, or some other solution I can try?
Try splitting up your import into chunks. For example, import the first 20 entities, then the second 20 entities, and so on until you've imported all of them. Then publish. Then go back and try importing the entire customizations file at the same time and republish. Following this method exactly has been the only way we've found to import some customization files in particularly stubborn environments.
It sounds like the re-index operation is taking some time - which would be as expected if the data size is large, and the fragmentation is high. It also depends on exactly what that stored procedure does, and how many cores/CPUs you have - and how many SQL Server is allowed to use.
Does the app allow you to defer that operation? You'd be able to run it manually yourself through Management Studio - if that doesn't break the application.
You could be cheeky, and rename that procedure, and replace it with one of your own that does nothing.... and then rename back, and run. Again, it might break something.
Or just keep increasing the timeout until you get past this issue. Some of my re-index jobs on databases generally take hours....
Or contact the vendor if you have support?
If you ran that query through Management Studio, it would complete.... doing that would give you the approximate time required to put (temporarily) into the timeout setting.
I just experienced the exact same problem and I was only importing 2 entities.
I found your questions when I was googling for p_RecreateIndexes after seeing it in the Trace files.
I ended up running exec p_RecreateIndexes in SQL Server. After it completed - about 2 minutes - I reran the Entity import and it worked.