I am shipping transaction logs to another database that we will be using for certain reports that don't need real-time data. This works fine until we start directing traffic to it, then it lasts for a day or so and then just gets stuck in a Restoring state.
I am shipping transaction logs every 15 minutes from the production server and on the standby server, I have it set to to Standby mode with 'Disconnect users in the database when restoring backups' set to true (if I don't do this, the restore will be put off sometimes by a day or so, I am ok with killing the active sessions). Also on the standby server I have it set to run LSRestore every 10 minutes.
The problem is that I don't know what is causing the database to hang, nor do I know where I can even look to get some diagnostics that may tell me something.
Does anyone know where I can look?
It sounds like the Secondary cannot keep up with the processing of the restores of the t-log backups on the secondary. I would try decreasing the time of the backups, copies, and restores and see if you can find an optimal time interval where the secondary database does not stay in the restoring mode. I see you already have it down to every 15 minutes. Also, check this article and use the t-sql in it to dive down deep and try and see where the delays are occurring in more detail - https://www.sqlshack.com/monitor-transaction-log-shipping-using-t-sql-and-ssms/. I hope this helps.
Related
I've recently taken on the database administration of a few SQL servers varying from SQL Server 2005 to 2014, where many of the DB's are in Full recovery mode, however no good ongoing backup maintenance plans were ever setup.
It seems to me that the previous DBA would only deal with transaction log files when they got out of control and fill up the hard drive. So i'd like to change this and fix the issue once and for all. I've been doing some reading and think I have a decent understanding of what need to be done, so i'd like to validate my understanding as well as ask a few question to clarify a few points that I still don't fully get.
So based on my understanding to date i would need to create a maintenance plan which starts with a Full Backup. I still need to talk to management to figure out things like RTO, acceptable data loss, etc so let's assume for this example that we'll do Full Backup's on Sunday.
Next I would add to this maintenance plan Differential backups every night... so Monday to Saturday. I realize that this could also be Full backup's or run the differentials more frequently, but again this is just as an example to make sure i'm understanding things correctly.
Now as for the transaction log backups. I get that i would need to back these up and truncate the log file to prevent it from continually growing and getting out of control. I don't know if there are any specific recommendation for how often to back this up, but i've seen 15 minutes suggested. I guess this would more so fall under the acceptable data loss window. Is that correct?
So the other thing that i've discovered is that when you backup the transaction log file with truncation, that if the log file has already grown out of control that it doesn't shrink the file. I've also read that it isn't good to shrink these files, at least on regular basis because once you shrink it, it would need to grow again and this will cause fragmentation and performance issues.
Now since I am currently in a situation that the files have already grown out of control, i'd assume that i should in fact shrink the log files this one time once i've but my maintenance in place. Is that assumption correct?
Also once i Shrink the transaction log file this one time, are there any maintenance task that i should be running to avoid performance issue due to shrinking the log files?
One other question that i was wondering about would be with respect to point in time recovery. So let's say I take a full backup at 5:00 AM and i also take a transaction log backup every 15 minutes. I get alerted that at 6:18 AM something has gone wrong (let's say a table was deleted). So i know i can restore by Full backup that happened at 5:00 AM and leave it in NO RECOVERY mode and restore all of the Transaction log backup from 5:15 AM to 6:15 AM, but here is what i'm interested in...since i have my DB in full recovery mode, is it possible to somehow use my existing transaction log file (not the backups) to roll forward all transaction between 6:15 and 6:17 just before the table was deleted? If so how would you do this? I guess this obviously wouldn't work in the case of you loosing the hard drive with your transaction log files, or your server exploding...but in a case like i've outlined is it do able?
Thanks
I would recommend doing a full backup after everyone stopped working, e. g. at 10 p.m. (if that is the case), not in the morning shortly before people start working. Just in order to give it enough time to run.
Personally, I prefer doing daily full backups instead of incremental backups if the database is not too big to save backups for, say, 14 days. I feel better to rely on less files. If database and full backups are too big, incremental backups might be the better choice.
As you said: How many transaction log backups you create during the day depends on the acceptable data loss window. In an environment > 5 people working on the system (just as a gut feeling) I would configure them to run all 15 minutes, on very big systems maybe even more.
After the first transaction log backup, you might want to shrink the LOG file ONCE.
I think it's not necessary to run any optimizations after a log shrink.
As far as I know it's not possible to restore transactions between 06:15 and 06:17.
When activating the transaction log backups, keep in mind that the first transaction log backup will be quite big (around the size of the current, large log). Ensure to have enough space on disk until you shrink the log file and delete the first transaction log (usually done automatically within the maintenance plan, e. g. after 14 days.).
I have several SSIS packages jobs running, and some months ago my disk got full because the size of the SSISDB database.
I noticed that the cleanup_server_retention_window was set to 365 days, and I changed it to one day. (It is a development server, and at this point I really dont care about the history).
The (big) problem, obviously, is now that the transaction log grows a lot and fast.
To prevent this, I start performing a full backup every week and a transaction log backup every day, and the size of the database is now controlled.
However, some more experienced guys are telling me that this is not the best aproach to this issue, but I cant see any problem with it..
I would like to know if there is a better solution for this.
I tried just about everything including changing retention window; it was deleted the transactions but not reducing the log size. For me the allocated log file size grew to 75 GB. Nothing seemed to help.
The main issue has to do with the recovery model
of the SSIS DB that was set to 'Full'. Once I set this to 'Simple' and changed the initial log file size, all was fixed!
I have been monitoring this for the last couple of days just to make sure all is well and it looks fine to me so this operation is safe.
The current log file size is 512KBMB as opposed to 75GB!
The (big) problem, obviously, is now that the transaction log grows a lot and fast.
You will not see this everyday..The Cause of transaction log growth was changing cleanup_server_retention_window ..when you changed the value from 365 to 1,internally it has to do a lot of deletes
I start performing a full backup every week and a transaction log backup every day, and the size of the database is now controlled
I don't see an issue with backing up SSISD.In our instance ,we changed the recovery model to simple and do daily full backups
I fixed this in 3 ways:
adding some missing indexes on the SSISDB database; they should be there after installation of CU4
changed the parameter #delete_batch_size from 1000 to 25 in the stored procedure internal.cleanup_server_retention_window
changed the recoevry model from Full to Simple
Now, when running the SSISDB maintenance job, the transaction log no longer fills up beyond repair causing a database 'crash' / rollback
I have some very strange problems. I have an application running on Windows 2003 terminal server from multiple clients. The application uses SQL Server 2008 Express as its database.
Yesterday, I connected to the app, closed some sessions on the server that were not responding, and to my surprise, I saw that some data was missing from the database. After a futher search I found that all the database changes made from last week were lost.
It's like the database rolled back all the changes, and returned to the state of one week ago! I can confirm that all the changes were lost. In fact I have inserted a record into a table with identity_insert ON (to manually insert an ID on an autonumeric col) and that record is missing, so there is no way this is a program failure.
Does anyone have any idea of what could have happend here?
EDIT
I have a suspect: could a transaction initiated by a session stays in a unconfirmed state for one week, retain all the database changes and when I close the session rollback all the changes made?
EDIT II
Find this on log:
SQL Server never rolls back a database to a previous state (like this). The database was restored, or the entire disk/VM was rolled back, or DML was executed to create the impression that a rollback happened (but really didn't). Maybe someone executed a sync tool in the wrong direction.
The question does not have information that allows for finding the problem. But it certainly isn't SQL Server rolling back a database.
You can try examining the log using fn_dblog.
From the log it looks like the server has only just started up after a reboot or service restart.
If a database is not cleanly shut down then the database can be left with partially applied transactions. If this happens then the database is recovered on start up.
Any transactions that are incomplete are rolled back. Committed transactions that were not yet applied are rolled forwards. How long this recovery takes depends on the size of the transactions in the log that have not yet been applied to the database.
The transactions may not show up in the log after they have been rolled back following a crash. This depends upon their location in the log and the databases's recovery mode.
If the transaction is at the end of the log it is likely the log will just be rolled back and the transaction removed.
If the transaction is in the middle of the log you might see a LOP_ABORT_XACT in the log.
When using simple recovery there is a good chance the log will be cleared after recovery (since the logs are only kept until the transactions are committed).
See Are log records removed from ldf file for rollbacks? for more details.
On my SQL Server I have change tracking enabled, auto cleanup set to false or to true with a very high retention period like 364 days.
Using Microsoft Sync Framework I sync the first time successfully, the second time I try to sync I get the error
"System.Data.SqlClient.SqlException (0x80131904): SQL Server Change
Tracking has cleaned up tracking information for table 'TableName'. To
recover from this error, the client must reinitialize its local
database and try again".
Why is the tracking information being cleaned although the auto clean up is set to false?
I tried the solution posted by this article, but it did not work for me, when I run the reinitialize code it takes forever and the RAM usage keeps building until I get out of memory exception.
Please note that my database is big, over 400 MB, the total number of records synced is over 163,000 record, the first sync takes a long time so resorting to deleting the local CE database when this error happens is not an option.
I discovered that the problem occurs when I restore the server database from backup and before any change has happened on the database.
So the simple solution was that after I restore the database from backup I just go and update any column of any row in any tracked table and that fixes the problem!
Whenever I try to backup a database it goes until 90% and gets stuck there until I manually kill (because it doesn't stop if I try to stop it) the msftesql process.
That clearly means that something makes a conflict between the fulltext indexing and the backup process.
So, have you seen anything like this? If not, how would you go about debugging this problem?
The first and obvious debug point is to disable full text indexing and try backing up the database again. If it does backup, then you know that FTS is the problem. If it doesn't, then you have another issue to find.
I would also check both the SQL Logs and the Event Viewer to see if any useful information is there.
Finally, if you have actual, physical access to the server during the backup, listen and see if the disk is making any funny noises during the backup process to indicate a disk failure of some sort.
I can say that I've never had FTS stop a backup from happening, but that doesn't mean it couldn't happen.
What time do you happen to have the job that refreshes the Full text indexes running? Perhaps it is trying to repopulate those indesxes at the same time the backup is running.
I have the same problem.
The activity monitor shows that the Backup job has a wait type MSSEARCH
The index is manually populated when run it is hanging for days on end until I force-ably stop it or the service is restarted. it used to take minutes to populate.