i am facing issue in sql server Transactional replication and not able to get the root cause for it. First, let me tell you that i am not a DBA, so i may be dumb on few DBA concepts.
i am .Net developer and i have been given responsibility to setup the replication.
i have a Database in Headoffice and replicating few Tables to another server at retail Store.
First time, i configured the replication with selected articles.
the replication was continuous. it was running fine, but one Sunday night, it got failed with error "process could not execute 'sp_replcmds'".
after spending sometime on google, i couldn't find any solution. so, i rebuilt the Replication, but this time the replication was scheduled (every 15 Min), also i configured it as PULL instead of PUSH. it started, but again next Sunday night it got crashed.
So, i analyzed that in Sunday night, i had configured the Reindexing Job on the database, and Since, the recovery model was full, it was generating a very large TLOG and Repolication agent was not able to parse that.
Now, the third time, i again Rebuilt the Replication, and this time i scheduled the replication every 15 minutes but from 8:00 AM Morning to 11:30 PM, because after 11:30, no store do any transaction. Also, for Reindexing Job, i added 2 more steps. before Re-Indexing, i was changing the recovery model to simple and then Re-Indexing and after that i was changing the Recovery model back to Full. i was changing the recovery to Full, irrespective of the result from Re-Indexing step.
This setup was working fine and worked properly for around 2 Months.
Now, after 2 Months, again one Sunday night it got failed, with the same reason ("process could not execute 'sp_replcmds'"). Actually, i had scheduled the backup job, and i was taking Full Backup everyday and Log backup every 15 minutes, and no differential backup.
after, discovering that i had not configured the differential backup, i also configured the same (every 6 Hours). but, after configuring the Differential backup, in Sunday night Replication got failed.
Now, anybody, please help me with the recommended setup for my scenario.
my setup is
sql server - SQL Server 2008 R2 Enterprise on Windows Server 2008 R2
Distributor and Publisher are on same machine.
Subsriber is on the Retail Store server.
sp_replcmds is run by the log reader agent against the published database to get, well, replicated commands. According to the documentation, one needs to be at least db_owner to run that command. Make sure whatever account is running the log reader agent has at least db_owner in the published database.
Related
We run our daily/weekly backups from CA D-Series job scheduling tool. I perform daily differentials and weekly fulls on our SQL Server 2012 databases. I am finding every few days both the differential and full backup will run on our largest databases (the maintenance jobs are split into 4 streams going from smallest to largest databases). The full backup job does not look to be running from D-Series.
For instance, the last full ran on 7/29 and today 8/1 a differential ran at 1 am and a full executed at 3 am. I am trying to find where the job is running from. It filled up the disk. I looked at the application log / SQL Server log / Agent log and cannot find any location. It is driving me insane!
This is a production server so I cannot implement auditing due to the high activity. Any advice would be greatly appreciated!
It's a long story, but I work for the research department of a school system, and we have beef with our IT department. A while back the IT dept. issued me a server with SQL installed on it and basically told me that I was on my own - they will not help with maintenance of the server or the behind-the-scenes functioning of the SQL database. For the most part this suits me fine, but I am not a DB Admin and I've never been able to solve setting up database backups and transaction logging.
The issue I'm trying to solve for now is transaction logging. Each night I use SSIS to pull in a large amount of records from our master database into my database. I've chunked this process into 18 individual steps, and step 9 & 15 are to do transaction log backups. Unfortunately my process still crashes regularly due to the transaction log filling up -- sometimes even on Step 10 just after a backup. I was going to disable logging during my process but have been reading and see that you cannot turn off logging. I'm hoping there is a simple solution that I just don't know about...
I'm running:
Microsoft SQL Server 2012 (SP3-GDR) (KB3194721) - 11.0.6248.0 (X64)
Sep 23 2016 15:49:43
Copyright (c) Microsoft Corporation Enterprise Edition (64-bit)
on Windows NT 6.2 <X64> (Build 9200: ) (Hypervisor).
Honestly I don't even know what type of transaction logging I'm doing or how to check and find out.
Any help would be greatly appreciated
First, when you say,
Each night I use SSIS to pull in a large amount of records from our master database into my database
I presume this does not literally mean the system master database. If it does, there should not be user objects in there and it should not be processing or storing any user data there. If you are, migrate this to a user database.
Second, the short answer to the logging problem is this:
The default recovery model for a database is Full. As noted in the comments, this means that no logs are overwritten until a back up occurs. This model works for point in time recovery and makes sense for a transactional system. So there are a few options:
If this is a reporting system, it usually makes sense to put this into simple recovery model and do nightly backups. Especially if the data is only changing once per day.
If you do require up to the minute point in time recovery, log backups should be performed every 15 min. I'm guessing that the management and retention of these backups will make no sense for you and you should use option 1.
When processing data in an SSIS dataflow, each buffer is committed one at a time. If you have not fiddled with the defaults, this means you are committing 10k rows at a maximum. So everything is already getting nicely chunked up. The problem then, is not that the batch sizes are too great, it is that you are in the wrong recovery model or that you are not backing up your logs often enough.
I have a SQL database setup on Azure. I was trying to understand if it was automatically backed up or if it is something that I need to setup. Could I get some feedback on this please?
Azure SQL Databases are automatically backed up for you. You don't need to explicitly set anything up. As far as frequency with which the backup happens, from this link:
For local database backups, full database backups happen weekly,
differential database backups happen hourly, and transaction log
backups happen every five minutes. The first full backup is scheduled
immediately after a database is created. It usually completes within
30 minutes, but it can take longer when the database is of a
significant size.
Furthermore, the duration for which the backups are kept (so that go can go back in time and restore them) depends on the service tier of your database. From the same link:
Each SQL Database backup has a retention period that is based on the
service-tier of the database. The retention period for a database in
the:
Basic service tier is seven days.
Standard service tier is 35 days.
Premium service tier is 35 days.
I have inherited a SQL Server 2008 live server with a hot swap-able backup server which has transaction logs shipped every 15 minutes from multiple production servers. One of the production servers keeps getting out of sync, when I came it the last successful log restore on the backup was over a year ago... So clearly my predecessor wasn't watching this, I restored the database and ensured that the logs synced correctly at the next 15 minute interval. However, every couple of days or so (it is random sometimes an hour sometimes 3 days) it gets back out of sync and I have to spend 10 minutes of my morning restoring the database.
Basically I am wondering what I need to be looking at to figure out why these keep getting out of sync? I ran a query I found on sqlauthority.com which shows me the trn's and their license numbers for the primary database and when I try to just restore the transaction logs on the backup database with the file which is supposed to be the next license to get it back in sync it says the license number is too recent, so I try the next file and it says is out of sync and can't be restored.
Any help would be appreciated :)
Upon digging into the job history further I found that my LS_Restore job was succeeding with errors, which were that the sql server agent service account didn't have access to the folder the trn files were stored which it was trying to restore the database from.
I researched what the original developer had setup as the sql server agent service account and then granted that user the necessary permissions, the logs are now restoring as needed.
I hope this information helps someone in the future!!
I really need some help here.
I'm the owner of a SQL Server Database application that lost three days data! I can't understand how or why.
So here is the set-up.
SQL Server 2005 32bit standard edition database on Windows 2000 server. (Database B)
Database is in simple recovery mode
The database is connected as a subscriber to another database(SQL Server 2005 64bit enterprise edition on Win2k3 enterprise) using SQL Server continuous Merge Replication. (Database A)
DatabaseB was rebooted on night X as part of scheduled reboot. When the database came back up it was used as normal for a couple of days and data was created into it perfectly fine.
But then yesterday Day X + 4 it lost a lot of data.
Database B is on a server with another instance of SQL Server and they both started to run out of memory(conflicting with each other).
Here is the sequence of events from the event log when I think this happened.
AppDomain 2 (DatabaseB.dbo[runtime].1) is marked for unload due to memory pressure.
AppDomain 2 (DatabaseB.dbo[runtime].1) unloaded.
BACKUP LOG WITH TRUNCATE_ONLY or WITH NO_LOG is deprecated.
The simple recovery model should be used to automatically truncate the transaction log. (on DatabaseB)
AppDomain 3 (DatabaseB.dbo[runtime].2) created.
I know the data is missing because of my audit logs and that a user had taken a screen shot of some of the data before it was deleted.
So here is my dilema...how could this have happened?
How can several days data go missing from DatabaseB?? (it subsequently is missing from the publication db also!)
Did the truncate with the Appdomain down cause the data to be flushed from the log?
Any and all theories considered. If anyone needs more data I can add it.
Help!
This isn't the answer you want to hear, but in a nutshell, SQL Server doesn't "lose" data. Someone deleted it. If you had the database in full recovery mode, you could use a product like Quest LiteSpeed to read the logs and identify exactly how it was deleted, but in simple mode...sorry, sir, but you're out of luck.
Merge replication is implemented with triggers, so it doesn't need full recovery. Is it possible that someone disabled all triggers in the db? its easy to do DISABLE TRIGGER [database] This would at least account for the subscriber losing data.
Those appdomain lines in the log don't mean that much, its the SQL CLR telling you its unloading assemblies to free up some memory. & then reloading them later on.
Truncating the log removes inactive parts that have been committed to disk, having the recovery model set to simple means there's no point in truncating the log, as the message suggests.
None of this explains why data went missing on both the servers though. There has to be something else that caused this.
How did you verify that for the 4 days when everything was 'created perfectly fine' that it actually was? do you have backups from these days? can you see records with time stamps from those days?
Is it possible there's a ghost in the machine that did a restore without telling you?